Python Cookbook 3rd Edition David Beazley download

pdf

https://ebookultra.com/download/python-cookbook-3rd-edition-david-
beazley/

Visit ebookultra.com today to download the complete set of

ebook or textbook!
Here are some recommended products for you. Click the link to
download, or explore more at ebookultra.com

Python cookbook Third Edition Beazley

https://ebookultra.com/download/python-cookbook-third-edition-beazley/

Python cookbook 3ed. Edition Beazley D.

https://ebookultra.com/download/python-cookbook-3ed-edition-beazley-d/

Python Cookbook 1st Edition Alex Martelli

https://ebookultra.com/download/python-cookbook-1st-edition-alex-
martelli/

Maya Programming with Python Cookbook 1st Edition Herbez

https://ebookultra.com/download/maya-programming-with-python-
cookbook-1st-edition-herbez/
Python Data Visualization Cookbook 2nd Edition Igor
Milovanovic

https://ebookultra.com/download/python-data-visualization-
cookbook-2nd-edition-igor-milovanovic/

PHP Cookbook Solutions Examples for PHP Programmers 3rd

Edition David Sklar

https://ebookultra.com/download/php-cookbook-solutions-examples-for-
php-programmers-3rd-edition-david-sklar/

Python data science cookbook over 60 practical recipes to

help you explore Python and its robust data science
capabilities Subramanian
https://ebookultra.com/download/python-data-science-cookbook-
over-60-practical-recipes-to-help-you-explore-python-and-its-robust-
data-science-capabilities-subramanian/

Functional programming in Python 1st Edition David Mertz

https://ebookultra.com/download/functional-programming-in-python-1st-
edition-david-mertz/

Puppet Cookbook 3rd Edition Thomas Uphill

https://ebookultra.com/download/puppet-cookbook-3rd-edition-thomas-
uphill/
Python Cookbook 3rd Edition David Beazley Digital
Instant Download
Author(s): David Beazley, Brian K. Jones
ISBN(s): 9781449340360, 1449340369
Edition: 3
File Details: PDF, 9.64 MB
Year: 2013
Language: english
 THIRD EDITION

Python Cookbook

David Beazley and Brian K. Jones

Python Cookbook, Third Edition
by David Beazley and Brian K. Jones
Copyright © 2013 David Beazley and Brian Jones. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://my.safaribooksonline.com). For more information, contact our corporate/
institutional sales department: 800-998-9938 or [email protected].
Editors: Meghan Blanchette and Rachel Roumeliotis Indexer: WordCo Indexing Services
Production Editor: Kristen Borg Cover Designer: Karen Montgomery
Copyeditor: Jasmine Kwityn Interior Designer: David Futato
Proofreader: BIM Proofreading Services Illustrator: Robert Romano

May 2013: Third Edition

Revision History for the Third Edition:

2013-05-08: First release

See http://oreilly.com/catalog/errata.csp?isbn=9781449340377 for release details.

Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly
Media, Inc. Python Cookbook, the image of a springhaas, and related trade dress are trademarks of O’Reilly
Media, Inc.
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as
trademarks. Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trade‐
mark claim, the designations have been printed in caps or initial caps.
While every precaution has been taken in the preparation of this book, the publisher and authors assume
no responsibility for errors or omissions, or for damages resulting from the use of the information contained
herein.

ISBN: 978-1-449-34037-7
[LSI]
 Table of Contents

Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

1. Data Structures and Algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1. Unpacking a Sequence into Separate Variables 1
1.2. Unpacking Elements from Iterables of Arbitrary Length 3
1.3. Keeping the Last N Items 5
1.4. Finding the Largest or Smallest N Items 7
1.5. Implementing a Priority Queue 8
1.6. Mapping Keys to Multiple Values in a Dictionary 11
1.7. Keeping Dictionaries in Order 12
1.8. Calculating with Dictionaries 13
1.9. Finding Commonalities in Two Dictionaries 15
1.10. Removing Duplicates from a Sequence while Maintaining Order 17
1.11. Naming a Slice 18
1.12. Determining the Most Frequently Occurring Items in a Sequence 20
1.13. Sorting a List of Dictionaries by a Common Key 21
1.14. Sorting Objects Without Native Comparison Support 23
1.15. Grouping Records Together Based on a Field 24
1.16. Filtering Sequence Elements 26
1.17. Extracting a Subset of a Dictionary 28
1.18. Mapping Names to Sequence Elements 29
1.19. Transforming and Reducing Data at the Same Time 32
1.20. Combining Multiple Mappings into a Single Mapping 33

2. Strings and Text. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

2.1. Splitting Strings on Any of Multiple Delimiters 37
2.2. Matching Text at the Start or End of a String 38
2.3. Matching Strings Using Shell Wildcard Patterns 40
2.4. Matching and Searching for Text Patterns 42

iii
 2.5. Searching and Replacing Text 45
2.6. Searching and Replacing Case-Insensitive Text 46
2.7. Specifying a Regular Expression for the Shortest Match 47
2.8. Writing a Regular Expression for Multiline Patterns 48
2.9. Normalizing Unicode Text to a Standard Representation 50
2.10. Working with Unicode Characters in Regular Expressions 52
2.11. Stripping Unwanted Characters from Strings 53
2.12. Sanitizing and Cleaning Up Text 54
2.13. Aligning Text Strings 57
2.14. Combining and Concatenating Strings 58
2.15. Interpolating Variables in Strings 61
2.16. Reformatting Text to a Fixed Number of Columns 64
2.17. Handling HTML and XML Entities in Text 65
2.18. Tokenizing Text 66
2.19. Writing a Simple Recursive Descent Parser 69
2.20. Performing Text Operations on Byte Strings 78

3. Numbers, Dates, and Times. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

3.1. Rounding Numerical Values 83
3.2. Performing Accurate Decimal Calculations 84
3.3. Formatting Numbers for Output 87
3.4. Working with Binary, Octal, and Hexadecimal Integers 89
3.5. Packing and Unpacking Large Integers from Bytes 90
3.6. Performing Complex-Valued Math 92
3.7. Working with Infinity and NaNs 94
3.8. Calculating with Fractions 96
3.9. Calculating with Large Numerical Arrays 97
3.10. Performing Matrix and Linear Algebra Calculations 100
3.11. Picking Things at Random 102
3.12. Converting Days to Seconds, and Other Basic Time Conversions 104
3.13. Determining Last Friday’s Date 106
3.14. Finding the Date Range for the Current Month 107
3.15. Converting Strings into Datetimes 109
3.16. Manipulating Dates Involving Time Zones 110

4. Iterators and Generators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

4.1. Manually Consuming an Iterator 113
4.2. Delegating Iteration 114
4.3. Creating New Iteration Patterns with Generators 115
4.4. Implementing the Iterator Protocol 117
4.5. Iterating in Reverse 119
4.6. Defining Generator Functions with Extra State 120

iv | Table of Contents
 4.7. Taking a Slice of an Iterator 122
4.8. Skipping the First Part of an Iterable 123
4.9. Iterating Over All Possible Combinations or Permutations 125
4.10. Iterating Over the Index-Value Pairs of a Sequence 127
4.11. Iterating Over Multiple Sequences Simultaneously 129
4.12. Iterating on Items in Separate Containers 131
4.13. Creating Data Processing Pipelines 132
4.14. Flattening a Nested Sequence 135
4.15. Iterating in Sorted Order Over Merged Sorted Iterables 136
4.16. Replacing Infinite while Loops with an Iterator 138

5. Files and I/O. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

5.1. Reading and Writing Text Data 141
5.2. Printing to a File 144
5.3. Printing with a Different Separator or Line Ending 144
5.4. Reading and Writing Binary Data 145
5.5. Writing to a File That Doesn’t Already Exist 147
5.6. Performing I/O Operations on a String 148
5.7. Reading and Writing Compressed Datafiles 149
5.8. Iterating Over Fixed-Sized Records 151
5.9. Reading Binary Data into a Mutable Buffer 152
5.10. Memory Mapping Binary Files 153
5.11. Manipulating Pathnames 156
5.12. Testing for the Existence of a File 157
5.13. Getting a Directory Listing 158
5.14. Bypassing Filename Encoding 160
5.15. Printing Bad Filenames 161
5.16. Adding or Changing the Encoding of an Already Open File 163
5.17. Writing Bytes to a Text File 165
5.18. Wrapping an Existing File Descriptor As a File Object 166
5.19. Making Temporary Files and Directories 167
5.20. Communicating with Serial Ports 170
5.21. Serializing Python Objects 171

6. Data Encoding and Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

6.1. Reading and Writing CSV Data 175
6.2. Reading and Writing JSON Data 179
6.3. Parsing Simple XML Data 183
6.4. Parsing Huge XML Files Incrementally 186
6.5. Turning a Dictionary into XML 189
6.6. Parsing, Modifying, and Rewriting XML 191
6.7. Parsing XML Documents with Namespaces 193

Table of Contents | v
 6.8. Interacting with a Relational Database 195
6.9. Decoding and Encoding Hexadecimal Digits 197
6.10. Decoding and Encoding Base64 199
6.11. Reading and Writing Binary Arrays of Structures 199
6.12. Reading Nested and Variable-Sized Binary Structures 203
6.13. Summarizing Data and Performing Statistics 214

7. Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
7.1. Writing Functions That Accept Any Number of Arguments 217
7.2. Writing Functions That Only Accept Keyword Arguments 219
7.3. Attaching Informational Metadata to Function Arguments 220
7.4. Returning Multiple Values from a Function 221
7.5. Defining Functions with Default Arguments 222
7.6. Defining Anonymous or Inline Functions 224
7.7. Capturing Variables in Anonymous Functions 225
7.8. Making an N-Argument Callable Work As a Callable with Fewer
Arguments 227
7.9. Replacing Single Method Classes with Functions 231
7.10. Carrying Extra State with Callback Functions 232
7.11. Inlining Callback Functions 235
7.12. Accessing Variables Defined Inside a Closure 238

8. Classes and Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

8.1. Changing the String Representation of Instances 243
8.2. Customizing String Formatting 245
8.3. Making Objects Support the Context-Management Protocol 246
8.4. Saving Memory When Creating a Large Number of Instances 248
8.5. Encapsulating Names in a Class 250
8.6. Creating Managed Attributes 251
8.7. Calling a Method on a Parent Class 256
8.8. Extending a Property in a Subclass 260
8.9. Creating a New Kind of Class or Instance Attribute 264
8.10. Using Lazily Computed Properties 267
8.11. Simplifying the Initialization of Data Structures 270
8.12. Defining an Interface or Abstract Base Class 274
8.13. Implementing a Data Model or Type System 277
8.14. Implementing Custom Containers 283
8.15. Delegating Attribute Access 287
8.16. Defining More Than One Constructor in a Class 291
8.17. Creating an Instance Without Invoking init 293
8.18. Extending Classes with Mixins 294
8.19. Implementing Stateful Objects or State Machines 299

vi | Table of Contents
 8.20. Calling a Method on an Object Given the Name As a String 305
8.21. Implementing the Visitor Pattern 306
8.22. Implementing the Visitor Pattern Without Recursion 311
8.23. Managing Memory in Cyclic Data Structures 317
8.24. Making Classes Support Comparison Operations 321
8.25. Creating Cached Instances 323

9. Metaprogramming. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
9.1. Putting a Wrapper Around a Function 329
9.2. Preserving Function Metadata When Writing Decorators 331
9.3. Unwrapping a Decorator 333
9.4. Defining a Decorator That Takes Arguments 334
9.5. Defining a Decorator with User Adjustable Attributes 336
9.6. Defining a Decorator That Takes an Optional Argument 339
9.7. Enforcing Type Checking on a Function Using a Decorator 341
9.8. Defining Decorators As Part of a Class 345
9.9. Defining Decorators As Classes 347
9.10. Applying Decorators to Class and Static Methods 350
9.11. Writing Decorators That Add Arguments to Wrapped Functions 352
9.12. Using Decorators to Patch Class Definitions 355
9.13. Using a Metaclass to Control Instance Creation 356
9.14. Capturing Class Attribute Definition Order 359
9.15. Defining a Metaclass That Takes Optional Arguments 362
9.16. Enforcing an Argument Signature on *args and **kwargs 364
9.17. Enforcing Coding Conventions in Classes 367
9.18. Defining Classes Programmatically 370
9.19. Initializing Class Members at Definition Time 374
9.20. Implementing Multiple Dispatch with Function Annotations 376
9.21. Avoiding Repetitive Property Methods 382
9.22. Defining Context Managers the Easy Way 384
9.23. Executing Code with Local Side Effects 386
9.24. Parsing and Analyzing Python Source 388
9.25. Disassembling Python Byte Code 392

10. Modules and Packages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397

10.1. Making a Hierarchical Package of Modules 397
10.2. Controlling the Import of Everything 398
10.3. Importing Package Submodules Using Relative Names 399
10.4. Splitting a Module into Multiple Files 401
10.5. Making Separate Directories of Code Import Under a Common
Namespace 404
10.6. Reloading Modules 406

Table of Contents | vii

 10.7. Making a Directory or Zip File Runnable As a Main Script 407
10.8. Reading Datafiles Within a Package 408
10.9. Adding Directories to sys.path 409
10.10. Importing Modules Using a Name Given in a String 411
10.11. Loading Modules from a Remote Machine Using Import Hooks 412
10.12. Patching Modules on Import 428
10.13. Installing Packages Just for Yourself 431
10.14. Creating a New Python Environment 432
10.15. Distributing Packages 433

11. Network and Web Programming. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437

11.1. Interacting with HTTP Services As a Client 437
11.2. Creating a TCP Server 441
11.3. Creating a UDP Server 445
11.4. Generating a Range of IP Addresses from a CIDR Address 447
11.5. Creating a Simple REST-Based Interface 449
11.6. Implementing a Simple Remote Procedure Call with XML-RPC 454
11.7. Communicating Simply Between Interpreters 456
11.8. Implementing Remote Procedure Calls 458
11.9. Authenticating Clients Simply 461
11.10. Adding SSL to Network Services 464
11.11. Passing a Socket File Descriptor Between Processes 470
11.12. Understanding Event-Driven I/O 475
11.13. Sending and Receiving Large Arrays 481

12. Concurrency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485

12.1. Starting and Stopping Threads 485
12.2. Determining If a Thread Has Started 488
12.3. Communicating Between Threads 491
12.4. Locking Critical Sections 497
12.5. Locking with Deadlock Avoidance 500
12.6. Storing Thread-Specific State 504
12.7. Creating a Thread Pool 505
12.8. Performing Simple Parallel Programming 509
12.9. Dealing with the GIL (and How to Stop Worrying About It) 513
12.10. Defining an Actor Task 516
12.11. Implementing Publish/Subscribe Messaging 520
12.12. Using Generators As an Alternative to Threads 524
12.13. Polling Multiple Thread Queues 531
12.14. Launching a Daemon Process on Unix 534

13. Utility Scripting and System Administration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539

viii | Table of Contents

 13.1. Accepting Script Input via Redirection, Pipes, or Input Files 539
13.2. Terminating a Program with an Error Message 540
13.3. Parsing Command-Line Options 541
13.4. Prompting for a Password at Runtime 544
13.5. Getting the Terminal Size 545
13.6. Executing an External Command and Getting Its Output 545
13.7. Copying or Moving Files and Directories 547
13.8. Creating and Unpacking Archives 549
13.9. Finding Files by Name 550
13.10. Reading Configuration Files 552
13.11. Adding Logging to Simple Scripts 555
13.12. Adding Logging to Libraries 558
13.13. Making a Stopwatch Timer 559
13.14. Putting Limits on Memory and CPU Usage 561
13.15. Launching a Web Browser 563

14. Testing, Debugging, and Exceptions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565

14.1. Testing Output Sent to stdout 565
14.2. Patching Objects in Unit Tests 567
14.3. Testing for Exceptional Conditions in Unit Tests 570
14.4. Logging Test Output to a File 572
14.5. Skipping or Anticipating Test Failures 573
14.6. Handling Multiple Exceptions 574
14.7. Catching All Exceptions 576
14.8. Creating Custom Exceptions 578
14.9. Raising an Exception in Response to Another Exception 580
14.10. Reraising the Last Exception 582
14.11. Issuing Warning Messages 583
14.12. Debugging Basic Program Crashes 585
14.13. Profiling and Timing Your Program 587
14.14. Making Your Programs Run Faster 590

15. C Extensions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 597

15.1. Accessing C Code Using ctypes 599
15.2. Writing a Simple C Extension Module 605
15.3. Writing an Extension Function That Operates on Arrays 609
15.4. Managing Opaque Pointers in C Extension Modules 612
15.5. Defining and Exporting C APIs from Extension Modules 614
15.6. Calling Python from C 619
15.7. Releasing the GIL in C Extensions 625
15.8. Mixing Threads from C and Python 625
15.9. Wrapping C Code with Swig 627

Table of Contents | ix
 15.10. Wrapping Existing C Code with Cython 632
15.11. Using Cython to Write High-Performance Array Operations 638
15.12. Turning a Function Pointer into a Callable 643
15.13. Passing NULL-Terminated Strings to C Libraries 644
15.14. Passing Unicode Strings to C Libraries 648
15.15. Converting C Strings to Python 653
15.16. Working with C Strings of Dubious Encoding 654
15.17. Passing Filenames to C Extensions 657
15.18. Passing Open Files to C Extensions 658
15.19. Reading File-Like Objects from C 659
15.20. Consuming an Iterable from C 662
15.21. Diagnosing Segmentation Faults 663

A. Further Reading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 665

Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 667

x | Table of Contents
 Preface

Since 2008, the Python world has been watching the slow evolution of Python 3. It was
always known that the adoption of Python 3 would likely take a long time. In fact, even
at the time of this writing (2013), most working Python programmers continue to use
Python 2 in production. A lot has been made about the fact that Python 3 is not backward
compatible with past versions. To be sure, backward compatibility is an issue for anyone
with an existing code base. However, if you shift your view toward the future, you’ll find
that Python 3 offers much more than meets the eye.
Just as Python 3 is about the future, this edition of the Python Cookbook represents a
major change over past editions. First and foremost, this is meant to be a very forward
looking book. All of the recipes have been written and tested with Python 3.3 without
regard to past Python versions or the “old way” of doing things. In fact, many of the
recipes will only work with Python 3.3 and above. Doing so may be a calculated risk,
but the ultimate goal is to write a book of recipes based on the most modern tools and
idioms possible. It is hoped that the recipes can serve as a guide for people writing new
code in Python 3 or those who hope to modernize existing code.
Needless to say, writing a book of recipes in this style presents a certain editorial chal‐
lenge. An online search for Python recipes returns literally thousands of useful recipes
on sites such as ActiveState’s Python recipes or Stack Overflow. However, most of these
recipes are steeped in history and the past. Besides being written almost exclusively for
Python 2, they often contain workarounds and hacks related to differences between old
versions of Python (e.g., version 2.3 versus 2.4). Moreover, they often use outdated
techniques that have simply become a built-in feature of Python 3.3. Finding recipes
exclusively focused on Python 3 can be a bit more difficult.
Rather than attempting to seek out Python 3-specific recipes, the topics of this book are
merely inspired by existing code and techniques. Using these ideas as a springboard,
the writing is an original work that has been deliberately written with the most modern
Python programming techniques possible. Thus, it can serve as a reference for anyone
who wants to write their code in a modern style.

xi
In choosing which recipes to include, there is a certain realization that it is simply
impossible to write a book that covers every possible thing that someone might do with
Python. Thus, a priority has been given to topics that focus on the core Python language
as well as tasks that are common to a wide variety of application domains. In addition,
many of the recipes aim to illustrate features that are new to Python 3 and more likely
to be unknown to even experienced programmers using older versions. There is also a
certain preference to recipes that illustrate a generally applicable programming tech‐
nique (i.e., programming patterns) as opposed to those that narrowly try to address a
very specific practical problem. Although certain third-party packages get coverage, a
majority of the recipes focus on the core language and standard library.

Who This Book Is For

This book is aimed at more experienced Python programmers who are looking to
deepen their understanding of the language and modern programming idioms. Much
of the material focuses on some of the more advanced techniques used by libraries,
frameworks, and applications. Throughout the book, the recipes generally assume that
the reader already has the necessary background to understand the topic at hand (e.g.,
general knowledge of computer science, data structures, complexity, systems program‐
ming, concurrency, C programming, etc.). Moreover, the recipes are often just skeletons
that aim to provide essential information for getting started, but which require the
reader to do more research to fill in the details. As such, it is assumed that the reader
knows how to use search engines and Python’s excellent online documentation.
Many of the more advanced recipes will reward the reader’s patience with a much greater
insight into how Python actually works under the covers. You will learn new tricks and
techniques that can be applied to your own code.

Who This Book Is Not For

This is not a book designed for beginners trying to learn Python for the first time. In
fact, it already assumes that you know the basics that might be taught in a Python tutorial
or more introductory book. This book is also not designed to serve as a quick reference
manual (e.g., quickly looking up the functions in a specific module). Instead, the book
aims to focus on specific programming topics, show possible solutions, and serve as a
springboard for jumping into more advanced material you might find online or in a
reference.

xii | Preface
Conventions Used in This Book
The following typographical conventions are used in this book:
Italic
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Constant width
Used for program listings, as well as within paragraphs to refer to program elements
such as variable or function names, databases, data types, environment variables,
statements, and keywords.
Constant width bold
Shows commands or other text that should be typed literally by the user.
Constant width italic
Shows text that should be replaced with user-supplied values or by values deter‐
mined by context.

This icon signifies a tip, suggestion, or general note.

This icon indicates a warning or caution.

Online Code Examples

Almost all of the code examples in this book are available online at http://github.com/
dabeaz/python-cookbook. The authors welcome bug fixes, improvements, and com‐
ments.

Using Code Examples

This book is here to help you get your job done. In general, if this book includes code
examples, you may use the code in this book in your programs and documentation. You
do not need to contact us for permission unless you’re reproducing a significant portion
of the code. For example, writing a program that uses several chunks of code from this
book does not require permission. Selling or distributing a CD-ROM of examples from
O’Reilly books does require permission. Answering a question by citing this book and
quoting example code does not require permission. Incorporating a significant amount

Preface | xiii
of example code from this book into your product’s documentation does require per‐
mission.
We appreciate, but do not require, attribution. An attribution usually includes the title,
author, publisher, and ISBN. For example: Python Cookbook, 3rd edition, by David
Beazley and Brian K. Jones (O’Reilly). Copyright 2013 David Beazley and Brian Jones,
978-1-449-34037-7.
If you feel your use of code examples falls outside fair use or the permission given here,
feel free to contact us at [email protected].

Safari® Books Online

Safari Books Online is an on-demand digital library that delivers ex‐
pert content in both book and video form from the world’s leading
authors in technology and business.
Technology professionals, software developers, web designers, and business and crea‐
tive professionals use Safari Books Online as their primary resource for research, prob‐
lem solving, learning, and certification training.
Safari Books Online offers a range of product mixes and pricing programs for organi‐
zations, government agencies, and individuals. Subscribers have access to thousands of
books, training videos, and prepublication manuscripts in one fully searchable database
from publishers like O’Reilly Media, Prentice Hall Professional, Addison-Wesley Pro‐
fessional, Microsoft Press, Sams, Que, Peachpit Press, Focal Press, Cisco Press, John
Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT
Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, Course Technol‐
ogy, and dozens more. For more information about Safari Books Online, please visit us
online.

How to Contact Us
Please address comments and questions concerning this book to the publisher:

O’Reilly Media, Inc.

1005 Gravenstein Highway North
Sebastopol, CA 95472
800-998-9938 (in the United States or Canada)
707-829-0515 (international or local)
707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional
information. You can access this page at http://oreil.ly/python_cookbook_3e.

xiv | Preface
To comment or ask technical questions about this book, send email to bookques
[email protected].
For more information about our books, courses, conferences, and news, see our website
at http://www.oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://www.youtube.com/oreillymedia

Acknowledgments
We would like to acknowledge the technical reviewers, Jake Vanderplas, Robert Kern,
and Andrea Crotti, for their very helpful comments, as well as the general Python com‐
munity for their support and encouragement. We would also like to thank the editors
of the prior edition, Alex Martelli, Anna Ravenscroft, and David Ascher. Although this
edition is newly written, the previous edition provided an initial framework for selecting
the topics and recipes of interest. Last, but not least, we would like to thank readers of
the early release editions for their comments and suggestions for improvement.

David Beazley’s Acknowledgments

Writing a book is no small task. As such, I would like to thank my wife Paula and my
two boys for their patience and support during this project. Much of the material in this
book was derived from content I developed teaching Python-related training classes
over the last six years. Thus, I’d like to thank all of the students who have taken my
courses and ultimately made this book possible. I’d also like to thank Ned Batchelder,
Travis Oliphant, Peter Wang, Brian Van de Ven, Hugo Shi, Raymond Hettinger, Michael
Foord, and Daniel Klein for traveling to the four corners of the world to teach these
courses while I stayed home in Chicago to work on this project. Meghan Blanchette and
Rachel Roumeliotis of O’Reilly were also instrumental in seeing this project through to
completion despite the drama of several false starts and unforeseen delays. Last, but not
least, I’d like to thank the Python community for their continued support and putting
up with my flights of diabolical fancy.
David M. Beazley
http://www.dabeaz.com
https://twitter.com/dabeaz

Preface | xv
Brian Jones’ Acknowledgments
I would like to thank both my coauthor, David Beazley, as well as Meghan Blanchette
and Rachel Roumeliotis of O’Reilly, for working with me on this project. I would also
like to thank my amazing wife, Natasha, for her patience and encouragement in this
project, and her support in all of my ambitions. Most of all, I’d like to thank the Python
community at large. Though I have contributed to the support of various open source
projects, languages, clubs, and the like, no work has been so gratifying and rewarding
as that which has been in the service of the Python community.
Brian K. Jones
http://www.protocolostomy.com
https://twitter.com/bkjones

xvi | Preface
 CHAPTER 1
Data Structures and Algorithms

Python provides a variety of useful built-in data structures, such as lists, sets, and dic‐
tionaries. For the most part, the use of these structures is straightforward. However,
common questions concerning searching, sorting, ordering, and filtering often arise.
Thus, the goal of this chapter is to discuss common data structures and algorithms
involving data. In addition, treatment is given to the various data structures contained
in the collections module.

1.1. Unpacking a Sequence into Separate Variables

Problem
You have an N-element tuple or sequence that you would like to unpack into a collection
of N variables.

Solution
Any sequence (or iterable) can be unpacked into variables using a simple assignment
operation. The only requirement is that the number of variables and structure match
the sequence. For example:
>>> p = (4, 5)
>>> x, y = p
>>> x
4
>>> y
5
>>>

>>> data = [ 'ACME', 50, 91.1, (2012, 12, 21) ]

>>> name, shares, price, date = data
>>> name

1
 'ACME'
>>> date
(2012, 12, 21)

>>> name, shares, price, (year, mon, day) = data

>>> name
'ACME'
>>> year
2012
>>> mon
12
>>> day
21
>>>
If there is a mismatch in the number of elements, you’ll get an error. For example:
>>> p = (4, 5)
>>> x, y, z = p
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: need more than 2 values to unpack
>>>

Discussion
Unpacking actually works with any object that happens to be iterable, not just tuples or
lists. This includes strings, files, iterators, and generators. For example:
>>> s = 'Hello'
>>> a, b, c, d, e = s
>>> a
'H'
>>> b
'e'
>>> e
'o'
>>>
When unpacking, you may sometimes want to discard certain values. Python has no
special syntax for this, but you can often just pick a throwaway variable name for it. For
example:
>>> data = [ 'ACME', 50, 91.1, (2012, 12, 21) ]
>>> _, shares, price, _ = data
>>> shares
50
>>> price
91.1
>>>
However, make sure that the variable name you pick isn’t being used for something else
already.

2 | Chapter 1: Data Structures and Algorithms

Random documents with unrelated
content Scribd suggests to you:
 K E N T ' S A LTA R- P I E C E

which, combined with the inscription engraved beneath, is a very

bitter satire on the painter; though it must be acknowledged that the
original, which has been for many years in the vestry-room of St.
Clement Danes, amply justifies the ridicule.
This picture produced a small tract, with the following title:—
"A letter from a parishioner of St. Clement Danes, to Edmund
(Gibson), Lord Bishop of London, occasioned by his Lordship's
causing the picture over the altar to be taken down, with some
observations on the use and abuse of church paintings in general,
and of that picture in particular."
In this tract, after some compliments to the prelate, the writer
works himself into a violent rage at the introduction of this piece of
popish foppery, and asks some questions which in a degree elucidate
part of the inscription under Hogarth's copy:—
"To what end or purpose was it put there, but to affront our most
gracious sovereign, by placing at our very altar the known
resemblance of a person who is the wife of his utter enemy, and
pensioner to the whore of Babylon?
"When I say the known resemblance, I speak not only according
to my own knowledge, but appeal to all mankind who have seen the
Princess Sobieski, or any picture or resemblance of her, if the picture
of that angel in the white garment and blue mantle, which is there
supposed to be beating time to the music, is not directly a great
likeness of that princess.
"Whether it was done by chance or on purpose, I shall not
determine; but be it which it will, it has given great offence, and
your Lordship has acted the part of a wise and good prelate to order
its removal."
It was probably during the time of Hogarth's apprenticeship that
he engraved the annexed print, entitled

T H E R A P E O F T H E LO C K .
 T H E R A P E O F T H E LO C K .

I by no means think, as Mr. Nichols asserts, that this is one of the

poorest of Hogarth's performances; for though slight, and not
intended to be impressed on paper, the air of the figures is easy, and
the faces, especially those of Sir Plume and the heroine of the story,
extremely characteristic. It is said to have been engraven on the lid
of a snuff-box for some gentleman characterized in Pope's admirable
mock-heroic poem, probably Lord Petre, who is here represented as
holding the lock of hair in his left hand. Sir Plume,—the round-faced
and insignificant Sir Plume,—
"Of amber snuff-box justly vain,
And the nice conduct of a clouded cane;"
for Sir George Brown, who was the only one of the party that took
the thing serious. He was angry that the poet should make him talk
nothing but nonsense; and, in truth (as Mr. Warburton adds), one
could not well blame him.
As this little story was intended to be viewed on gold, the figures
in the copy are not reversed, but left as they were originally
engraven on the box; from which I believe there are only three
impressions extant, one of which was sold by Greenwood at Mr.
Gulston's sale, on the 7th of February 1786, for £33.
The following account of the persons for whom Hogarth painted
several of his early pictures is copied from his own handwriting, and
may sometimes be useful in tracing the pedigree of a portrait.
By this list, it appears that the two pictures of "Before and After"
were painted for a Mr. Thomson; but as it is not probable that
Hogarth delineated this subject twice, I think that these two pictures
were the property of the late Lord Besborough. They were sold on
his Lordship's demise, in February 1801, at Christie's rooms.
"Account taken, January 1, 1731, of all the pictures that
remain unfinished.—Half payment received.
A family piece, consisting of four figures, for Mr. Rich, 1728.
An assembly of twenty-five figures, for Lord Castlemain,
August 28, 1729.
Family of four figures; Mr. Wood, 1728.
A conversation of six figures; Mr. Cock, Nov. 1728.
A family of five figures; Mr. Jones, March 1730.
The Committee of the House of Commons, for Sir Archibald
Grant, Nov. 5, 1729.
The Beggar's Opera; ditto.
Single figure; Mr. Kirkham, April 18, 1730.
Family of nine; Mr. Vernon, Feb. 27, 1730.
 Another of two; Mr. Cooper.
Another of five; Duke of Montague.
Two little pictures; ditto.
Single figure; Sir Robert Pye, Nov. 18, 1730.
Two little pictures, called "Before and After," for Mr. Thomson,
Dec. 7, 1730.
A head, for Mr. Sarmond, Jan. 12, 1730-31.
Pictures bespoke for the present year 1731."

With this his memorandum ends; and I regret that he has not
recorded the prices he received for the pictures. Mr. Nichols
conjectures that they were originally very low; he is most probably
right with respect to those that were painted in the early part of
Hogarth's life. But let it be recollected that for the portrait of Garrick
in Richard III. he received two hundred pounds, which, as the artist
himself remarks, was a more liberal remuneration than had been
paid to any contemporary painter. When my late friend Mr.
Gainsborough began to paint portraits at Bath (at a period when
much higher prices were paid), his general rule was five guineas for
a three-quarters portrait.
Below is inserted a copy from one of Hogarth's early engravings,
the arms of the Duchess of Kendal, mistress to George I., probably
done on a piece of plate at the time he was Gamble's apprentice.
The original, of the same size, is in the Editor's possession. It is
drawn in a correct and spirited style; and considering the age of the
artist, and the purpose for which it was engraven, not demanding
much attention or exertion, gave some promise of the excellence
which he afterwards attained.
In this point of view, to an admirer of Hogarth it becomes in some
degree interesting, which will, I hope, plead my apology for the
insertion of this solitary specimen of his boyish heraldry. On no other
ground should so insignificant a production as a coat of arms have
found a place in this volume.

T H E K E N D A L A R M S.
 CHAPTER II.

MARRIES. PAINTS SMALL CONVERSATIONS, WHICH SUBJECTS HE QUITS FOR

FAMILIAR PRINTS. ATTEMPTS HISTORY; BUT FINDING IT IS NOT
ENCOURAGED IN ENGLAND, RETURNS TO ENGRAVING FROM HIS OWN
DESIGNS. OCCASIONALLY TAKES PORTRAITS LARGE AS LIFE, FOR WHICH
HE INCURS MUCH ABUSE. TO PROVE HIS POWERS AND VINDICATE HIS
FAME, PAINTS THE ADMIRABLE PORTRAIT OF CAPTAIN CORAM, AND
PRESENTS IT TO THE FOUNDLING HOSPITAL.

then married, and commenced painter of small

conversation pieces, from twelve to fifteen inches high.
This having novelty, succeeded for a few years. But
though it gave somewhat more scope to the fancy, was still
but a less kind of drudgery; and as I could not bring myself to
act like some of my brethren, and make it a sort of a
manufactory to be carried on by the help of background and
drapery painters, it was not sufficiently profitable to pay the
expenses my family required. I therefore turned my thoughts
to a still more novel mode, viz. painting and engraving
modern moral subjects, a field not broken up in any country
or any age.
"The reasons which induced me to adopt this mode of
designing were, that I thought both writers and painters had,
in the historical style, totally overlooked that intermediate
species of subjects which may be placed between the sublime
and grotesque; I therefore wished to compose pictures on
canvas, similar to representations on the stage, and further
hope that they will be tried by the same test, and criticised by
the same criterion. Let it be observed, that I mean to speak
only of those scenes where the human species are actors,
and these I think have not often been delineated in a way of
which they are worthy and capable.
 "In these compositions, those subjects that will both
entertain and improve the mind bid fair to be of the greatest
public utility, and must therefore be entitled to rank in the
highest class. If the execution is difficult (though that is but a
secondary merit), the author has a claim to a higher degree
of praise. If this be admitted, comedy in painting as well as
writing ought to be allotted the first place, as most capable of
all these perfections, though the sublime, as it is called, has
been opposed to it. Ocular demonstration will carry more
conviction to the mind of a sensible man, than all he would
find in a thousand volumes; and this has been attempted in
the prints I have composed. Let the decision be left to every
unprejudiced eye; let the figures in either pictures or prints be
considered as players dressed either for the sublime,—for
genteel comedy,[15] or farce,—for high or low life. I have
endeavoured to treat my subjects as a dramatic writer: my
picture is my stage, and men and women my players, who by
means of certain actions and gestures are to exhibit a dumb
show.
"Before I had done anything of much consequence in this
walk, I entertained some hopes of succeeding in what the
puffers in books call the great style of history painting; so
that without having had a stroke of this grand business
before, I quitted small portraits and familiar conversations,
and with a smile at my own temerity, commenced history
painter, and on a great staircase at St. Bartholomew's
Hospital painted two Scripture stories (the 'Pool of Bethesda'
and the 'Good Samaritan'), with figures seven feet high.
These I presented to the charity,[16] and thought they might
serve as a specimen to show that were there an inclination in
England for encouraging historical pictures, such a first essay
might prove the painting them more easily attainable than is
generally imagined. But as religion, the great promoter of this
style in other countries, rejected it in England, I was unwilling
to sink into a portrait manufacturer; and, still ambitious of
being singular, dropped all expectations of advantage from
that source, and returned to the pursuit of my former
dealings with the public at large. This I found was most likely
to answer my purpose, provided I could strike the passions,
and by small sums from many, by the sale of prints which I
could engrave from my own pictures, thus secure my property
to myself.
"In pursuing my studies, I made all possible use of the
technical memory which I have before described, by
observing and endeavouring to retain in my mind lineally such
objects as best suited my purpose; so that be where I would,
while my eyes were open, I was at my studies, and acquiring
something useful to my profession. By this means, whatever I
saw, whether a remarkable incident or a trifling subject,
became more truly a picture than one that was drawn by a
camera-obscura. And thus the most striking objects, whether
of beauty or deformity, were by habit the most easily
impressed and retained in my imagination. A redundancy of
matter being by this means acquired, it is natural to suppose
I introduced it into my works on every occasion that I could.
"By this idle way of proceeding I grew so profane as to
admire nature beyond the first productions of art, and
acknowledged I saw, or fancied, delicacies in the life so far
surpassing the utmost efforts of imitation, that when I drew
the comparison in my mind, I could not help uttering
blasphemous expressions against the divinity even of Raphael
Urbino, Correggio, and Michael Angelo. For this, though my
brethren have most unmercifully abused me, I hope to be
forgiven. I confess to have frequently said, that I thought the
style of painting which I had adopted, admitting that my
powers were not equal to doing it justice, might one time or
other come into better hands, and be made more entertaining
and more useful than the eternal blazonry and tedious
repetition of hackneyed, beaten subjects, either from the
Scriptures or the old ridiculous stories of heathen gods; as
neither the religion of one or the other requires promoting
among Protestants, as it formerly did in Greece, and at a later
period in Rome.[17]
"For these and other heretical opinions, as I have before
observed, I was deemed vain, and accused of enviously
attempting what I was unable to execute.
"The chief things that have brought much obloquy on me
are: First, the attempting portrait-painting; Secondly, writing
the Analysis of Beauty; Thirdly, painting the picture of
'Sigismunda;' and, Fourthly, publishing the first print of 'The
Times.'
"In the ensuing pages it shall be my endeavour to vindicate
myself from these aspersions, and each of the subjects taken
in the order they occurred shall be occasionally interspersed
with some thoughts by the way on the state of the arts,
institution of a Royal Academy, Society of Arts, etc., as being
remotely, if not immediately, connected with my own pursuits.
"Though small whole lengths and prints of familiar
conversations were my principal pursuit, yet by those who
were partial to me I was sometimes employed to paint
portraits as large as life, and for this I was most barbarously
abused. My opponents acknowledged, that in the particular
branches to which I had devoted my attention I had some
little merit; but as neither history nor portrait were my
province, nothing but what they were pleased to term
extreme vanity could induce me to attempt either one or the
other; for it would be interfering in that branch of which I had
no knowledge, and in which I had therefore no concern.
"At this I was rather piqued, and, as well as I could,
defended my conduct and explained my motives. Some part
of this defence it will be necessary to repeat; and it will also
be proper to recollect, that after having had my plates pirated
in almost all sizes, I, in 1735, applied to Parliament for
redress, and obtained it in so liberal a manner as hath not
only answered my own purpose, but made prints a
considerable article in the commerce of this country; there
being now more business of this kind done here than in Paris,
or anywhere else, and as well.
"The dealers in pictures and prints found their craft in
danger by what they called a new-fangled innovation. Their
trade of living and getting fortunes by the ingenuity of the
industrious has, I know, suffered much by my interference;
and if the detection of this band of public cheats and
oppressors of the rising artists be a crime, I confess myself
most guilty.
"To put this matter in a fair point of view, it will be
necessary to state the situation of the arts and artists at this
period. In doing which, I shall probably differ from every
other author, as I think the books hitherto written on the
subject have had a tendency to confirm prejudice and error,
rather than diffuse information and truth. My notions of
painting differ not only from those who have formed their
opinions from books, but from those who have taken them
upon trust.
"I am therefore under the necessity of submitting to the
public what may possibly be deemed peculiar opinions, but
without the least hope of bringing over either men whose
interests are concerned, or who implicitly rely upon the
authority of a tribe of picture dealers and puny judges that
delight in the marvellous, and determine to admire what they
do not understand; but I have hope of succeeding a little with
such as dare to think for themselves, and can believe their
own eyes.
"As introductory to the subject, let us begin with
considering that branch of the art which is termed still life—a
species of painting which ought to be held in the lowest
estimation.
 "Whatever is or can be perfectly fixed, from the plainest to
the most complicated object, from a bottle and glass to a
statue of the human figure, may be denominated still life.
Ship and landscape painting ought unquestionably to come
into the same class; for if copied exactly as they chance to
appear, the painters have no occasion of judgment; yet with
those who do not consider the few talents necessary, even
this tribe sometimes pass for very capital artists.
"'Well painted, and finely pencilled!' are phrases perpetually
repeated by coach and sign painters. Merely well painted or
pencilled is chiefly the effect of much practice; and we
frequently see that those who are in these particulars very
excellent cannot advance a step further.
"As to portrait-painting, the chief branch of the art by which
a painter can procure himself a tolerable livelihood, and the
only one by which a lover of money can get a fortune; a man
of very moderate talents may have great success in it, as the
artifice and address of a mercer is infinitely more useful than
the abilities of a painter. By the manner in which the present
race of professors in England conduct it, that also becomes
still life as much as any of the preceding. Admitting that the
artist has no further view than merely copying the figure, this
must be admitted to its full extent; for the sitter ought to be
still as a statue, and no one will dispute a statue being as
much still life as fruit, flowers, a gallipot, or a broken earthen
pan. It must, indeed, be acknowledged they do not seem
ashamed of the title, for their figures are frequently so
executed as to be as still as a post. Posture and drapery, as it
is called, is usually supplied by a journeyman, who puts a
coat, etc. on a wooden figure like a jointed doll, which they
call a layman, and copies it in every fold as it chances to
come; and all this is done at so easy a rate, as enables the
principal to get more money in a week than a man of the first
professional talents can in three months. If they have a
sufficient quantity of silks, satins, and velvets to dress their
layman, they may thus carry on a very profitable manufactory
without a ray of genius. There is a living instance well known
to the connoisseurs in this town, of one of the best copiers of
pictures, particularly those by Rubens, who is almost an idiot.
[18] Mere correctness, therefore, if in still life, from an apple
or a rose, to the face,—nay, even the whole figure, if you take
it merely as it presents itself,—requires only an exact eye and
an adroit hand. Their pattern is before them, and much
practice with little study is usually sufficient to bring them into
high vogue. By perpetual attention to this branch only, one
should imagine they would attain a certain stroke—quite the
reverse; for though the whole business lies in an oval of four
inches long, which they have before them, they are obliged to
repeat and alter the eyes, mouth, and nose, three or four
times before they can make it what they think right. The little
praise due to their productions ought, in most cases, to be
given to the drapery-man, whose pay is only one part in ten,
while the other nine, as well as all the reputation, is
engrossed by the master phiz-monger for a proportion which
he may complete in five or six hours; and even this, little as it
is, gives him so much importance in his own eyes, that he
assumes a consequential air, sets his arms akimbo, and,
strutting among the historical artists, cries, 'How we apples
swim!'
"For men who drudge in this mechanical part merely for
gain, to commence dealers in pictures is natural. In this, also,
great advantage may accrue from the labour and ingenuity of
others. They stand in the catalogue of painters; and having
little to study in their own way, become great connoisseurs,
not in the points where real perfection lies, for there they
must be deficient, as their ideas have been confined to the
oval; but their great inquiry is, how the old masters stand in
the public estimation, that they may regulate their prices
accordingly, both in buying and selling. You may know these
painter-dealers by their constant attendance at auctions. They
collect under pretence of a love for the arts, but sell, knowing
the reputation they have stamped on the commodity they
have once purchased, in the opinion of the ignorant admirer
of pictures, drawings, and prints, which, thus warranted,
almost invariably produce them treble their original purchase
money, and treble their real worth. Unsanctioned by their
authority,[19] and unascertained by tradition, the best
preserved and highest finished picture (though it should have
been painted by Raphael) will not, at a public auction,
produce five shillings; while a despicable, damaged, and
repaired old canvas, sanctioned by their praise, shall be
purchased at any price, and find a place in the noblest
collections. All this is very well understood by the dealers,
who, on every occasion where their own interest is
concerned, are wondrously loquacious in adoring the
mysterious beauties! spirited touches! brilliant colours! and
the Lord knows what, of these ancient worn-out wonders! But
whoever should dare to hint that (admitting them to be
originally painted by Raphael) there is little left to admire in
them, would be instantly stigmatized as vilifying the great
masters, and, to invalidate his judgment, accused of envy and
self-conceit. By these misrepresentations, if he has an
independent fortune, he only suffers the odium; but if a
young man, without any other property than his talents,
presumes boldly to give an opinion, he may be undone by his
temerity; for the whole herd will unite and try to hunt him
down.
"Such is the situation of the arts and artists at this time.
Credulity,—an implicit confidence in the opinions of others,—
and not daring to think for themselves, leads the whole town
into error, and thus they become the prey of ignorant and
designing knaves.
"With respect to portrait-painting, whatever talents a
professor may have, if he is not in fashion, and cannot afford
to hire a drapery-man, he will not do; but if he is in vogue,
and can employ a journeyman and place a layman in the
garret of his manufactory, his fortune is made, and, as his
two coadjutors are kept in the background, his own fame is
established.
"If a painter comes from abroad, his being an exotic will be
much in his favour; and if he has address enough to persuade
the public that he had brought a new discovered mode of
colouring, and paints his faces all red, all blue, or all purple,
he has nothing to do but to hire one of these painted tailors
as an assistant, for without him the manufactory cannot go
on, and my life for his success.
"Vanloo,[20] a French portrait-painter, being told that the
English were to be cajoled by any one who had a sufficient
portion of assurance, came to this country, set his trumpeters
to work, and by the assistance of puffing monopolized all the
people of fashion in the kingdom. Down went at once *,—*,
—*,—*,—*,— etc. etc. etc.,[21] painters who before his arrival
were highly fashionable and eminent, but by this foreign
interloper were driven into the greatest distress and poverty.
"By this inundation of folly and fuss, I must confess I was
much disgusted, and determined to try if by any means I
could stem the torrent, and 'by opposing end it.' I laughed at
the pretensions of these quacks in colouring, ridiculed their
productions as feeble and contemptible, and asserted that it
required neither taste nor talents to excel their most popular
performances. This interference excited much enmity,
because, as my opponents told me, my studies were in
another way. You talk, added they, with ineffable contempt of
portrait-painting; if it is so easy a task, why do not you
convince the world by painting a portrait yourself? Provoked
at this language, I one day at the Academy in St. Martin's
Lane put the following question: Supposing any man at this
time were to paint a portrait as well as Vandyke, would it be
seen or acknowledged, and could the artist enjoy the benefit
or acquire the reputation due to his performance?
"They asked me in reply if I could paint one as well? and I
frankly answered, 'I believed I could.'[22] My query as to the
credit I should obtain if I did, was replied to by Mr. Ramsay,
and confirmed by the president and about twenty members
present: 'Our opinions must be consulted, and we will never
allow it.' Piqued at this cavalier treatment, I resolved to try
my own powers; and if I did what I attempted, determined to
affirm that I had done it. In this decided manner I had a habit
of speaking; and if I only did myself justice, to have adopted
half words would have been affectation. Vanity, as I
understand it, consists in affirming you have done that which
you have not done, not in frankly asserting what you are
convinced is truth.
"A watchmaker may say, 'The watch which I have made for
you is as good as Quare, or Tompion, or any other man could
have made.' If it really is so, he is neither called vain nor
branded with infamy, but deemed an honest and fair man for
being as good as his word. Why should not the same privilege
be allowed to a painter? The modern artist, though he will not
warrant his works as the watchmaker, has the impudence to
demand twice as much money for painting them as was
charged by those whom he acknowledges his superiors in the
art.
"Of the mighty talents said to be requisite for portrait-
painting I had not the most exalted opinion, and thought
that, if I chose to practise in this branch, I could at least
equal my contemporaries, for whose glittering productions I
really had not much reverence. In answer to this there are
who will say with Peachum in the play, 'All professions be-
rogue one another;' but let it be taken into the account that
men with the same pursuits are naturally rivals, and when put
in competition with each other must necessarily be so,—what
racer ever wished that his opponent might outrun him? what
boxer ever chose to be beat in pure complaisance to his
antagonist? The artist who pretends to be pleased and
gratified when he sees himself excelled by his competitor
must have lost all reverence for truth, or be totally dead to
that spirit which I believe to be one great source of
excellence in all human attempts; and if he is so polite and
civil as to confess superiority in one he knows to be his
inferior, he must be either a fool or an hypocrite, perhaps
both. If he has temper enough to be silent, it is surely
sufficient; but this I have seldom seen, even amongst the
most complaisant and liberal of the faculty.
"Those who will honestly speak their feelings must confess
that all this is natural to man. One of the highest
gratifications of superiority arises from the pleasure which
attends instructing men who do not know so much as
ourselves; but when they verge on being rivals, the pleasure
in a degree ceases. Hence the story of Rubens advising
Vandyke to paint horses and faces, to prevent, as it is said,
his being put in competition with himself in history-painting.
Had either of these great artists lived in England at this time,
they would have found men of very moderate parts—mere
face painters—who, if they chanced to be in vogue, might
with ease get a thousand a year, when they with all their
talents would scarcely have found employment.
"To return to my dispute with Mr. Ramsay on the abilities
necessary for portrait-painting: as I found the performances
of professors in this branch of the art were held in such
estimation, I determined to have a brush at it. I had
occasionally painted portraits; but as they required constant
practice to take a likeness with facility, and the life must not
be rigidly followed, my portraitures met with a fate somewhat
similar to those of Rembrandt. By some they were said to be
nature itself, by others declared most execrable; so that time
only can decide whether I was the best or the worst face
 painter of my day, for a medium was never so much as
suggested.
"The portrait which I painted with most pleasure, and in
which I particularly wished to excel, was that of Captain
Coram, for the Foundling Hospital; and if I am so wretched an
artist as my enemies assert, it is somewhat strange that this,
which was one of the first I painted the size of life, should
stand the test of twenty years' competition, and be generally
thought the best portrait in the place, notwithstanding the
first painters in the kingdom exerted all their talents to vie
with it.[23] To this I refer Mr. Rams-eye[24] and his quick-
sighted and impartial coadjutors."

C A P TA I N T H O M AS C O R A M

was born in the year 1668, bred to the sea, and passed the first part
of his life as master of a vessel trading to the colonies. While he
resided in the vicinity of Rotherhithe, his avocations obliging him to
go early into the city and return late, he frequently saw deserted
infants exposed to the inclemencies of the seasons, and through the
indigence or cruelty of their parents left to casual relief or untimely
death. This naturally excited his compassion, and led him to project
the establishment of an hospital for the reception of exposed and
deserted young children; in which humane design he laboured more
than seventeen years, and at last, by his unwearied application,
obtained the Royal Charter, bearing date the 17th of October 1739,
for its incorporation.
He was highly instrumental in promoting another good design, viz.
the procuring a bounty upon naval stores imported from the colonies
to Georgia and Nova Scotia. But the charitable plan which he lived to
make some progress in, though not to complete, was a scheme for
uniting the Indians in North America more closely with the British
Government, by an establishment for the education of Indian girls.
Indeed, he spent a great part of his life in serving the public, and
with so total a disregard to his private interest, that in his old age he
was himself supported by a pension of somewhat more than an
hundred pounds a year,[25] raised for him at the solicitation of Sir
Sampson Gideon and Dr. Brocklesby, by the voluntary subscriptions
of public-spirited persons, at the head of whom was the late
Frederick Prince of Wales. On application being made to this
venerable and good old man to know whether a subscription being
opened for his benefit would not offend him, he gave this noble
answer: "I have not wasted the little wealth of which I was formerly
possessed in self-indulgence or vain expenses, and am not ashamed
to confess that in this my old age I am poor."
This singularly humane, persevering, and memorable man died at
his lodgings near Leicester Square, March 29, 1751, and was
interred, pursuant to his own desire, in the vault under the chapel of
the Foundling Hospital, where an historic epitaph records his virtues,
as Hogarth's portrait has preserved his honest countenance.
Hogarth thus resumes his narrative:—
"For the portrait of Mr. Garrick in Richard III. I was paid two
hundred pounds[26] (which was more than any English artist
ever received for a single portrait), and that, too, by the
sanction of several painters who had been previously
consulted about the price, which was not given without
mature consideration.
"Notwithstanding all this, the current remark was, that
portraits were not my province, and I was tempted to
abandon the only lucrative branch of my art, for the practice
brought the whole nest of phiz-mongers on my back, where
they buzzed like so many hornets. All these people have their
friends, whom they incessantly teach to call my women
harlots, my 'Essay on Beauty' borrowed,[27] and my
composition and engraving contemptible.
"This so much disgusted me, that I sometimes declared I
would never paint another portrait, and frequently refused
 when applied to; for I found by mortifying experience, that
whoever would succeed in this branch must adopt the mode
recommended in one of Gay's fables, and make divinities of
all who sit to him.[28] Whether or not this childish affectation
will ever be done away, is a doubtful question: none of those
who have attempted to reform it have yet succeeded; nor,
unless portrait-painters in general become more honest, and
their customers less vain, is there much reason to expect they
ever will."

Though thus in a state of warfare with his brother artists, he was

occasionally gratified by the praise of men whose judgment was
universally acknowledged, and whose sanction became an higher
honour, from its being neither lightly nor indiscriminately given. The
following letter from the facetious Mr. George Faulkner notices the
estimation in which the author of The Battle of the Books held the
painter of "The Battle of the Pictures:"—
To Mr. William Hogarth, at his house in Leicester Fields, London.
"Sir,—I was favoured with a letter from Mr. Delany, who
tells me that you are going to publish three prints.[29] Your
reputation here is sufficiently known to recommend anything
of yours, and I shall be glad to serve you. The duty on prints
is ten per cent. in Ireland. You may send me fifty sets,
provided you will take back what I cannot sell. I desire no
other profit than what you allow in London to those who sell
them again. I have often the favour of drinking your health
with Doctor Swift, who is a great admirer of yours, and hath
made mention of you in his poems with great honour,[30] and
desired me to thank you for your kind present, and to accept
of his service.—I am, Sir, your most obedient and most
humble servant,
"George Faulkner.
"Dublin, Nov. 15, 1740."
 Hogarth about this time painted the portrait of Dr. Benjamin
Hoadley, Bishop of Winchester, which, though rather French, is in a
grand style. Concerning it, Dr. John Hoadley wrote the following
whimsical epistle to the artist:—
To Mr. Wm. Hogarth.
"Dear Billy.—You were so kind as to say you would touch
up the Doctor if I would send it to town. Lo! it is here. I am at
Alresford for a day or two, to shear my flock and to feed 'em
(money, you know, is the sinews of war); and having this
morning taken down all my pictures, in order to have my
room painted, I thought I might as well pack up Dr. Benjamin,
and send him packing to London. My love to him, and desire
him, when his wife says he looks charmingly, to drive
immediately to Leicester Fields (Square I mean, I beg your
pardon), and sit an hour or two, or three, in your painting-
room. Do not set it by and forget it now,—don't you. My
humble service waits upon Mrs. Hogarth, and all good wishes
upon your honour; and I am, dear Sir, your obliged and
affectionate
"J. Hoadley."
 CHAPTER III.

OF ACADEMIES. HOGARTH'S OPINION OF THAT NOW DENOMINATED ROYAL;

AND OF THE SOCIETY FOR THE ENCOURAGEMENT OF ARTS,
MANUFACTURES, AND COMMERCE, GIVING PREMIUMS FOR PICTURES AND
DRAWINGS.

mong Hogarth's loose papers I found the rough draft of a

letter (addressed but not directed) to a nobleman, declaring
his disapprobation of a scheme by which certain projectors
were endeavouring to establish a Royal Academy, and stating that he
had a plan which would be much more useful. I do not know that he
ever sent the epistle, or admitting he did, that it was honoured with
an answer. But as I think it probable he had given the subject some
consideration, with the hope of bringing his project to bear, and that
in the following pages relative to the Royal Academy he has stated
what he meant to have said to the Peer, I have inserted it:—
"My Lord,—Mr. Martin has informed me that when some of
my thoughts relative to the establishment of a public
academy are put into writing, you will peruse them. I have
made a rough sketch, but to fit it for inspection will require
much time, which it will be needless to take till I have your
Lordship's opinion on two or three leading points on which
the whole will turn, and which I cannot with propriety commit
to paper. A verbal statement of them will not take up more
than half an hour; and if, when known, they are not
concurred in, I will not take up your Lordship's time by
arguing on their propriety. But I am vain enough to think, that
though I must say strong, and perhaps startling things, with
regard to myself and others, I can prove every position which
I shall advance.
 "I have reason to believe that another project is in hand:
this the author will naturally defend in opposition to mine, but
it shall not create controversy; for being now upwards of sixty
years of age, and in a very poor state of health, I would
rather lose a favourite point than break a night's rest.
"Mr. Ramsey, if I judge right, is no stranger to the plan I
allude to, and I know his opinion differs from mine, and am
firmly persuaded his interest will induce him to support it.—I
am, my Lord, etc.,
"W. H."

"Much has been said about the immense benefit likely to

result from the establishment of an academy in this country;
but as I do not see it in the same light with many of my
contemporaries, I shall take the freedom of making my
objections to the plan on which they propose forming it; and
as a sort of preliminary to the subject, state some slight
particulars concerning the fate of former attempts at similar
establishments.
"The first place of this sort was in Queen Street, about sixty
years ago; it was begun by some gentlemen-painters of the
first rank, who in their general forms imitated the plan of that
in France, but conducted their business with far less fuss and
solemnity; yet the little that there was, in a very short time
became the object of ridicule. Jealousies arose, parties were
formed, and the president and all his adherents found
themselves comically represented as marching in ridiculous
procession round the walls of the room. The first proprietors
soon put a padlock on the door; the rest, by their right as
subscribers, did the same, and thus ended this academy.
"Sir James Thornhill, at the head of one of these parties,
then set up another in a room he built at the back of his own
house,—now next the playhouse,—and furnished tickets
gratis to all that required admission; but so few would lay
themselves under such an obligation, that this also soon sunk
into insignificance. Mr. Vanderbank headed the rebellious
party, and converted an old Presbyterian meeting-house into
an academy, with the addition of a woman figure, to make it
the more inviting to subscribers. This lasted a few years; but
the treasurer sinking the subscription money, the lamp, stove,
etc. were seized for rent, and that also dropped.
"Sir James dying, I became possessed of his neglected
apparatus; and thinking that an academy conducted on
proper and moderate principles had some use, proposed that
a number of artists should enter into a subscription for the
hire of a place large enough to admit thirty or forty people to
draw after a naked figure. This was soon agreed to, and a
room taken in St. Martin's Lane. To serve the society, I lent
them the furniture which had belonged to Sir James
Thornhill's academy; and as I attributed the failure of that
and Mr. Vanderbank's to the leading members assuming a
superiority which their fellow-students could not brook, I
proposed that every member should contribute an equal sum
to the establishment, and have an equal right to vote in every
question relative to the society. As to electing presidents,
directors, professors, etc., I considered it as a ridiculous
imitation of the foolish parade of the French Academy, by the
establishment of which Louis XIV. got a large portion of fame
and flattery on very easy terms. But I could never learn that
the arts were benefited, or that members acquired any other
advantages than what arose to a few leaders from their paltry
salaries, not more I am told than £50 a year; which, as must
always be the case, were engrossed by those who had most
influence, without any regard to their relative merit.[31] As a
proof of the little benefit the arts derived from this Royal
Academy, Voltaire asserts that, after its establishment, no one
work of genius appeared in the country: the whole band,
adds the same lively and sensible writer, became mannerists
and imitators.[32] It may be said in answer to this, that all
painting is but imitation. Granted; but if we go no further
than copying what has been done before, without entering
into the spirit, causes, and effects, what are we doing? If we
vary from our original, we fall off from it, and it ceases to be
a copy; and if we strictly adhere to it, we can have no hopes
of getting beyond it; for 'if two men ride on a horse, one of
them must be behind.'
"To return to our own academy. By the regulations I have
mentioned of a general equality, etc., it has now subsisted
near thirty years, and is, to every useful purpose, equal to
that in France, or any other; but this does not satisfy. The
members, finding his present Majesty's partiality to the arts,
met at the Turk's Head in Gerard Street, Soho, laid out the
public money in advertisements to call all sorts of artists
together, and have resolved to draw up and present a
ridiculous address to King, Lords, and Commons to do for
them what they have (as well as it can be) done for
themselves. Thus to pester the three great estates of the
empire, about twenty or thirty students, drawing after a man
or a horse, appears, as it must be acknowledged, foolish
enough; but the real motive is, that a few bustling characters,
who have access to people of rank, think they can thus get a
superiority over their brethren, be appointed to places, and
have salaries as in France, for telling a lad when an arm or a
leg is too long or too short.
"Not approving of this plan, I opposed it; and having
refused to assign to the society the property which I had
before lent them, I am accused of acrimony, ill-nature, and
spleen, and held forth as an enemy to the arts and artists.
How far their mighty project will succeed, I neither know nor
care; certain I am it deserves to be laughed at, and laughed
at it has been.[33] The business rests in the breast of Majesty,
and the simple question now is, whether he will do what Sir
James Thornhill did before him, i.e. establish an academy
with the little addition of a royal name, and salaries for those
professors who can make most interest and obtain the
greatest patronage. As his Majesty's beneficence to the arts
will unquestionably induce him to do that which he thinks
most likely to promote them, would it not be more useful if he
were to furnish his own gallery with one picture by each of
the most eminent painters among his own subjects? This
might possibly set an example to a few of the opulent
nobility; but even then it is to be feared that there never can
be a market in this country, for the great number of works
which, by encouraging parents to place their children in this
line, it would probably cause to be painted. The world is
already glutted with these commodities, which do not perish
fast enough to want such a supply.
"In answer to this and other objections which I have
sometimes made to those who display so much zeal for
increasing learners, and crowding the profession, I am asked
if I consider what the arts were in Greece, what immense
benefits accrued to the city of Rome from the possession of
their works, and what advantages the people of France derive
from the encouragement given by their Royal Academy? It is
added, why cannot we have one on the same principles? That
we may not be led away by sounds without meaning, let us
take a cursory view of these things separately, and in the
same order that they occurred.
"The height to which the arts were carried in Greece was
owing to a variety of causes, concerning some of which we
can now only form conjectures. They made a part of their
system of government, and were connected with their modes
of worship. Their temples were crowded with deities of their
own manufacture, and in places of public resort were
depicted such actions of their fellow-citizens as deserved
commemoration; which, being displayed in a language legible
to all, incited the spectator to emulate the virtues they
represented. The artists who could perform such wonders
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookultra.com