Contents

About ................................................................................................................................................................................... 1
Chapter 1: Getting started with Python Language ...................................................................................... 2
Section 1.1: Getting Started ........................................................................................................................................... 2
Section 1.2: Creating variables and assigning values ................................................................................................ 6
Section 1.3: Block Indentation ..................................................................................................................................... 10
Section 1.4: Datatypes ................................................................................................................................................. 11
Section 1.5: Collection Types ...................................................................................................................................... 15
Section 1.6: IDLE - Python GUI .................................................................................................................................... 19
Section 1.7: User Input ................................................................................................................................................. 21
Section 1.8: Built in Modules and Functions .............................................................................................................. 21
Section 1.9: Creating a module ................................................................................................................................... 25
Section 1.10: Installation of Python 2.7.x and 3.x ....................................................................................................... 26
Section 1.11: String function - str() and repr() ........................................................................................................... 28
Section 1.12: Installing external modules using pip ................................................................................................... 29
Section 1.13: Help Utility ............................................................................................................................................... 31
Chapter 2: Python Data Types ............................................................................................................................ 33
Section 2.1: String Data Type ..................................................................................................................................... 33
Section 2.2: Set Data Types ....................................................................................................................................... 33
Section 2.3: Numbers data type ................................................................................................................................ 33
Section 2.4: List Data Type ......................................................................................................................................... 34
Section 2.5: Dictionary Data Type ............................................................................................................................. 34
Section 2.6: Tuple Data Type ..................................................................................................................................... 34
Chapter 3: Indentation ............................................................................................................................................. 35
Section 3.1: Simple example ....................................................................................................................................... 35
Section 3.2: How Indentation is Parsed ..................................................................................................................... 35
Section 3.3: Indentation Errors ................................................................................................................................... 36
Chapter 4: Comments and Documentation .................................................................................................. 37
Section 4.1: Single line, inline and multiline comments ............................................................................................ 37
Section 4.2: Programmatically accessing docstrings .............................................................................................. 37
Section 4.3: Write documentation using docstrings ................................................................................................ 38
Chapter 5: Date and Time ...................................................................................................................................... 42
Section 5.1: Parsing a string into a timezone aware datetime object .................................................................... 42
Section 5.2: Constructing timezone-aware datetimes ............................................................................................ 42
Section 5.3: Computing time dierences .................................................................................................................. 44
Section 5.4: Basic datetime objects usage ............................................................................................................... 44
Section 5.5: Switching between time zones .............................................................................................................. 45
Section 5.6: Simple date arithmetic ........................................................................................................................... 45
Section 5.7: Converting timestamp to datetime ...................................................................................................... 46
Section 5.8: Subtracting months from a date accurately ....................................................................................... 46
Section 5.9: Parsing an arbitrary ISO 8601 timestamp with minimal libraries ...................................................... 46
Section 5.10: Get an ISO 8601 timestamp .................................................................................................................. 47
Section 5.11: Parsing a string with a short time zone name into a timezone aware datetime object ................ 47
Section 5.12: Fuzzy datetime parsing (extracting datetime out of a text) ............................................................ 48
Section 5.13: Iterate over dates .................................................................................................................................. 49
Chapter 6: Date Formatting .................................................................................................................................. 50
Section 6.1: Time between two date-times ............................................................................................................... 50
Section 6.2: Outputting datetime object to string .................................................................................................... 50
 Section 6.3: Parsing string to datetime object ......................................................................................................... 50
Chapter 7: Enum .......................................................................................................................................................... 51
Section 7.1: Creating an enum (Python 2.4 through 3.3) ......................................................................................... 51
Section 7.2: Iteration ................................................................................................................................................... 51
Chapter 8: Set ............................................................................................................................................................... 52
Section 8.1: Operations on sets .................................................................................................................................. 52
Section 8.2: Get the unique elements of a list .......................................................................................................... 53
Section 8.3: Set of Sets ................................................................................................................................................ 53
Section 8.4: Set Operations using Methods and Builtins ......................................................................................... 53
Section 8.5: Sets versus multisets .............................................................................................................................. 55
Chapter 9: Simple Mathematical Operators ................................................................................................. 57
Section 9.1: Division ..................................................................................................................................................... 57
Section 9.2: Addition .................................................................................................................................................... 58
Section 9.3: Exponentiation ........................................................................................................................................ 59
Section 9.4: Trigonometric Functions ........................................................................................................................ 60
Section 9.5: Inplace Operations ................................................................................................................................. 61
Section 9.6: Subtraction .............................................................................................................................................. 61
Section 9.7: Multiplication ........................................................................................................................................... 61
Section 9.8: Logarithms .............................................................................................................................................. 62
Section 9.9: Modulus ................................................................................................................................................... 62
Chapter 10: Bitwise Operators ............................................................................................................................. 65
Section 10.1: Bitwise NOT ............................................................................................................................................ 65
Section 10.2: Bitwise XOR (Exclusive OR) .................................................................................................................. 66
Section 10.3: Bitwise AND ............................................................................................................................................ 67
Section 10.4: Bitwise OR .............................................................................................................................................. 67
Section 10.5: Bitwise Left Shift .................................................................................................................................... 67
Section 10.6: Bitwise Right Shift .................................................................................................................................. 68
Section 10.7: Inplace Operations ................................................................................................................................ 68
Chapter 11: Boolean Operators ............................................................................................................................ 69
Section 11.1: `and` and `or` are not guaranteed to return a boolean ...................................................................... 69
Section 11.2: A simple example ................................................................................................................................... 69
Section 11.3: Short-circuit evaluation ......................................................................................................................... 69
Section 11.4: and ........................................................................................................................................................... 70
Section 11.5: or .............................................................................................................................................................. 70
Section 11.6: not ............................................................................................................................................................ 71
Chapter 12: Operator Precedence ...................................................................................................................... 72
Section 12.1: Simple Operator Precedence Examples in python ............................................................................. 72
Chapter 13: Variable Scope and Binding ......................................................................................................... 73
Section 13.1: Nonlocal Variables ................................................................................................................................. 73
Section 13.2: Global Variables .................................................................................................................................... 73
Section 13.3: Local Variables ...................................................................................................................................... 74
Section 13.4: The del command ................................................................................................................................. 75
Section 13.5: Functions skip class scope when looking up names ......................................................................... 76
Section 13.6: Local vs Global Scope ........................................................................................................................... 77
Section 13.7: Binding Occurrence ............................................................................................................................... 79
Chapter 14: Conditionals ......................................................................................................................................... 80
Section 14.1: Conditional Expression (or "The Ternary Operator") ......................................................................... 80
Section 14.2: if, elif, and else ....................................................................................................................................... 80
Section 14.3: Truth Values ........................................................................................................................................... 80
 Section 14.4: Boolean Logic Expressions ................................................................................................................... 81
Section 14.5: Using the cmp function to get the comparison result of two objects ............................................. 83
Section 14.6: Else statement ....................................................................................................................................... 83
Section 14.7: Testing if an object is None and assigning it ...................................................................................... 84
Section 14.8: If statement ............................................................................................................................................ 84
Chapter 15: Comparisons ........................................................................................................................................ 86
Section 15.1: Chain Comparisons ................................................................................................................................ 86
Section 15.2: Comparison by `is` vs `==` ...................................................................................................................... 87
Section 15.3: Greater than or less than ...................................................................................................................... 88
Section 15.4: Not equal to ........................................................................................................................................... 88
Section 15.5: Equal To ................................................................................................................................................. 89
Section 15.6: Comparing Objects ............................................................................................................................... 89
Chapter 16: Loops ....................................................................................................................................................... 91
Section 16.1: Break and Continue in Loops ................................................................................................................ 91
Section 16.2: For loops ................................................................................................................................................ 93
Section 16.3: Iterating over lists .................................................................................................................................. 93
Section 16.4: Loops with an "else" clause .................................................................................................................. 94
Section 16.5: The Pass Statement .............................................................................................................................. 96
Section 16.6: Iterating over dictionaries .................................................................................................................... 97
Section 16.7: The "half loop" do-while ........................................................................................................................ 98
Section 16.8: Looping and Unpacking ....................................................................................................................... 98
Section 16.9: Iterating dierent portion of a list with dierent step size ............................................................... 99
Section 16.10: While Loop .......................................................................................................................................... 100
Chapter 17: Arrays .................................................................................................................................................... 102
Section 17.1: Access individual elements through indexes ..................................................................................... 102
Section 17.2: Basic Introduction to Arrays .............................................................................................................. 102
Section 17.3: Append any value to the array using append() method ................................................................ 103
Section 17.4: Insert value in an array using insert() method ................................................................................ 103
Section 17.5: Extend python array using extend() method ................................................................................... 103
Section 17.6: Add items from list into array using fromlist() method .................................................................. 104
Section 17.7: Remove any array element using remove() method ..................................................................... 104
Section 17.8: Remove last array element using pop() method ............................................................................ 104
Section 17.9: Fetch any element through its index using index() method ........................................................... 104
Section 17.10: Reverse a python array using reverse() method ........................................................................... 104
Section 17.11: Get array buer information through buer_info() method ........................................................ 105
Section 17.12: Check for number of occurrences of an element using count() method .................................... 105
Section 17.13: Convert array to string using tostring() method ............................................................................ 105
Section 17.14: Convert array to a python list with same elements using tolist() method .................................. 105
Section 17.15: Append a string to char array using fromstring() method ........................................................... 105
Chapter 18: Multidimensional arrays .............................................................................................................. 106
Section 18.1: Lists in lists ............................................................................................................................................ 106
Section 18.2: Lists in lists in lists in.. .......................................................................................................................... 106
Chapter 19: Dictionary ............................................................................................................................................ 108
Section 19.1: Introduction to Dictionary ................................................................................................................... 108
Section 19.2: Avoiding KeyError Exceptions ........................................................................................................... 109
Section 19.3: Iterating Over a Dictionary ................................................................................................................. 109
Section 19.4: Dictionary with default values ........................................................................................................... 110
Section 19.5: Merging dictionaries ........................................................................................................................... 111
Section 19.6: Accessing keys and values ................................................................................................................ 111
Section 19.7: Accessing values of a dictionary ....................................................................................................... 112
 Section 19.8: Creating a dictionary .......................................................................................................................... 112
Section 19.9: Creating an ordered dictionary ......................................................................................................... 113
Section 19.10: Unpacking dictionaries using the ** operator ................................................................................. 113
Section 19.11: The trailing comma ............................................................................................................................ 114
Section 19.12: The dict() constructor ........................................................................................................................ 114
Section 19.13: Dictionaries Example ......................................................................................................................... 114
Section 19.14: All combinations of dictionary values .............................................................................................. 115
Chapter 20: List ......................................................................................................................................................... 117
Section 20.1: List methods and supported operators ............................................................................................ 117
Section 20.2: Accessing list values .......................................................................................................................... 122
Section 20.3: Checking if list is empty ..................................................................................................................... 123
Section 20.4: Iterating over a list ............................................................................................................................. 123
Section 20.5: Checking whether an item is in a list ................................................................................................ 124
Section 20.6: Any and All .......................................................................................................................................... 124
Section 20.7: Reversing list elements ...................................................................................................................... 125
Section 20.8: Concatenate and Merge lists ............................................................................................................ 125
Section 20.9: Length of a list .................................................................................................................................... 126
Section 20.10: Remove duplicate values in list ....................................................................................................... 126
Section 20.11: Comparison of lists ............................................................................................................................ 127
Section 20.12: Accessing values in nested list ........................................................................................................ 127
Section 20.13: Initializing a List to a Fixed Number of Elements ........................................................................... 128
Chapter 21: List comprehensions ...................................................................................................................... 130
Section 21.1: List Comprehensions ........................................................................................................................... 130
Section 21.2: Conditional List Comprehensions ...................................................................................................... 132
Section 21.3: Avoid repetitive and expensive operations using conditional clause ............................................ 134
Section 21.4: Dictionary Comprehensions ............................................................................................................... 135
Section 21.5: List Comprehensions with Nested Loops .......................................................................................... 136
Section 21.6: Generator Expressions ........................................................................................................................ 138
Section 21.7: Set Comprehensions ........................................................................................................................... 140
Section 21.8: Refactoring filter and map to list comprehensions ......................................................................... 140
Section 21.9: Comprehensions involving tuples ...................................................................................................... 141
Section 21.10: Counting Occurrences Using Comprehension ............................................................................... 142
Section 21.11: Changing Types in a List .................................................................................................................... 142
Section 21.12: Nested List Comprehensions ............................................................................................................ 142
Section 21.13: Iterate two or more list simultaneously within list comprehension .............................................. 143
Chapter 22: List slicing (selecting parts of lists) ....................................................................................... 144
Section 22.1: Using the third "step" argument ........................................................................................................ 144
Section 22.2: Selecting a sublist from a list ............................................................................................................ 144
Section 22.3: Reversing a list with slicing ................................................................................................................ 144
Section 22.4: Shifting a list using slicing .................................................................................................................. 144
Chapter 23: groupby() ............................................................................................................................................ 146
Section 23.1: Example 4 ............................................................................................................................................. 146
Section 23.2: Example 2 ............................................................................................................................................ 146
Section 23.3: Example 3 ............................................................................................................................................ 147
Chapter 24: Linked lists ......................................................................................................................................... 149
Section 24.1: Single linked list example ................................................................................................................... 149
Chapter 25: Linked List Node ............................................................................................................................. 154
Section 25.1: Write a simple Linked List Node in python ....................................................................................... 154
Chapter 26: Filter ...................................................................................................................................................... 155
 Section 26.1: Basic use of filter ................................................................................................................................. 155
Section 26.2: Filter without function ........................................................................................................................ 155
Section 26.3: Filter as short-circuit check ............................................................................................................... 156
Section 26.4: Complementary function: filterfalse, ifilterfalse .............................................................................. 156
Chapter 27: Heapq ................................................................................................................................................... 158
Section 27.1: Largest and smallest items in a collection ....................................................................................... 158
Section 27.2: Smallest item in a collection .............................................................................................................. 158
Chapter 28: Tuple ..................................................................................................................................................... 160
Section 28.1: Tuple ..................................................................................................................................................... 160
Section 28.2: Tuples are immutable ........................................................................................................................ 161
Section 28.3: Packing and Unpacking Tuples ........................................................................................................ 161
Section 28.4: Built-in Tuple Functions ..................................................................................................................... 162
Section 28.5: Tuple Are Element-wise Hashable and Equatable ......................................................................... 163
Section 28.6: Indexing Tuples ................................................................................................................................... 164
Section 28.7: Reversing Elements ............................................................................................................................ 164
Chapter 29: Basic Input and Output ............................................................................................................... 165
Section 29.1: Using the print function ...................................................................................................................... 165
Section 29.2: Input from a File ................................................................................................................................. 165
Section 29.3: Read from stdin .................................................................................................................................. 167
Section 29.4: Using input() and raw_input() .......................................................................................................... 167
Section 29.5: Function to prompt user for a number ............................................................................................ 167
Section 29.6: Printing a string without a newline at the end ................................................................................. 168
Chapter 30: Files & Folders I/O ......................................................................................................................... 171
Section 30.1: File modes ............................................................................................................................................ 171
Section 30.2: Reading a file line-by-line .................................................................................................................. 172
Section 30.3: Iterate files (recursively) .................................................................................................................... 173
Section 30.4: Getting the full contents of a file ...................................................................................................... 173
Section 30.5: Writing to a file ................................................................................................................................... 174
Section 30.6: Check whether a file or path exists .................................................................................................. 175
Section 30.7: Random File Access Using mmap .................................................................................................... 176
Section 30.8: Replacing text in a file ....................................................................................................................... 176
Section 30.9: Checking if a file is empty ................................................................................................................. 176
Section 30.10: Read a file between a range of lines .............................................................................................. 177
Section 30.11: Copy a directory tree ........................................................................................................................ 177
Section 30.12: Copying contents of one file to a dierent file .............................................................................. 177
Chapter 31: os.path .................................................................................................................................................. 178
Section 31.1: Join Paths ............................................................................................................................................. 178
Section 31.2: Path Component Manipulation .......................................................................................................... 178
Section 31.3: Get the parent directory ..................................................................................................................... 178
Section 31.4: If the given path exists ........................................................................................................................ 178
Section 31.5: check if the given path is a directory, file, symbolic link, mount point etc .................................... 179
Section 31.6: Absolute Path from Relative Path ..................................................................................................... 179
Chapter 32: Iterables and Iterators ................................................................................................................ 180
Section 32.1: Iterator vs Iterable vs Generator ....................................................................................................... 180
Section 32.2: Extract values one by one ................................................................................................................. 181
Section 32.3: Iterating over entire iterable ............................................................................................................. 181
Section 32.4: Verify only one element in iterable .................................................................................................. 181
Section 32.5: What can be iterable .......................................................................................................................... 182
Section 32.6: Iterator isn't reentrant! ....................................................................................................................... 182
Chapter 33: Functions ............................................................................................................................................. 183
Section 33.1: Defining and calling simple functions ............................................................................................... 183
Section 33.2: Defining a function with an arbitrary number of arguments ........................................................ 184
Section 33.3: Lambda (Inline/Anonymous) Functions .......................................................................................... 187
Section 33.4: Defining a function with optional arguments .................................................................................. 189
Section 33.5: Defining a function with optional mutable arguments ................................................................... 190
Section 33.6: Argument passing and mutability .................................................................................................... 191
Section 33.7: Returning values from functions ....................................................................................................... 192
Section 33.8: Closure ................................................................................................................................................. 192
Section 33.9: Forcing the use of named parameters ............................................................................................ 193
Section 33.10: Nested functions ............................................................................................................................... 194
Section 33.11: Recursion limit .................................................................................................................................... 194
Section 33.12: Recursive Lambda using assigned variable ................................................................................... 195
Section 33.13: Recursive functions ........................................................................................................................... 195
Section 33.14: Defining a function with arguments ................................................................................................ 196
Section 33.15: Iterable and dictionary unpacking .................................................................................................. 196
Section 33.16: Defining a function with multiple arguments ................................................................................. 198
Chapter 34: Defining functions with list arguments .............................................................................. 199
Section 34.1: Function and Call ................................................................................................................................. 199
Chapter 35: Functional Programming in Python ...................................................................................... 201
Section 35.1: Lambda Function ................................................................................................................................ 201
Section 35.2: Map Function ...................................................................................................................................... 201
Section 35.3: Reduce Function ................................................................................................................................. 201
Section 35.4: Filter Function ..................................................................................................................................... 201
Chapter 36: Partial functions .............................................................................................................................. 202
Section 36.1: Raise the power ................................................................................................................................... 202
Chapter 37: Decorators ......................................................................................................................................... 203
Section 37.1: Decorator function .............................................................................................................................. 203
Section 37.2: Decorator class ................................................................................................................................... 204
Section 37.3: Decorator with arguments (decorator factory) .............................................................................. 205
Section 37.4: Making a decorator look like the decorated function .................................................................... 207
Section 37.5: Using a decorator to time a function ............................................................................................... 207
Section 37.6: Create singleton class with a decorator .......................................................................................... 208
Chapter 38: Classes ................................................................................................................................................. 209
Section 38.1: Introduction to classes ........................................................................................................................ 209
Section 38.2: Bound, unbound, and static methods .............................................................................................. 210
Section 38.3: Basic inheritance ................................................................................................................................ 212
Section 38.4: Monkey Patching ................................................................................................................................ 214
Section 38.5: New-style vs. old-style classes .......................................................................................................... 214
Section 38.6: Class methods: alternate initializers ................................................................................................. 215
Section 38.7: Multiple Inheritance ............................................................................................................................ 217
Section 38.8: Properties ............................................................................................................................................ 219
Section 38.9: Default values for instance variables ............................................................................................... 220
Section 38.10: Class and instance variables ........................................................................................................... 221
Section 38.11: Class composition .............................................................................................................................. 222
Section 38.12: Listing All Class Members ................................................................................................................. 223
Section 38.13: Singleton class ................................................................................................................................... 224
Section 38.14: Descriptors and Dotted Lookups .................................................................................................... 225
Chapter 39: Metaclasses ....................................................................................................................................... 226
 Section 39.1: Basic Metaclasses ............................................................................................................................... 226
Section 39.2: Singletons using metaclasses ........................................................................................................... 227
Section 39.3: Using a metaclass .............................................................................................................................. 227
Section 39.4: Introduction to Metaclasses .............................................................................................................. 227
Section 39.5: Custom functionality with metaclasses ........................................................................................... 228
Section 39.6: The default metaclass ....................................................................................................................... 229
Chapter 40: String Formatting ......................................................................................................................... 232
Section 40.1: Basics of String Formatting ............................................................................................................... 232
Section 40.2: Alignment and padding ..................................................................................................................... 233
Section 40.3: Format literals (f-string) .................................................................................................................... 234
Section 40.4: Float formatting ................................................................................................................................. 234
Section 40.5: Named placeholders ......................................................................................................................... 235
Section 40.6: String formatting with datetime ....................................................................................................... 236
Section 40.7: Formatting Numerical Values ........................................................................................................... 236
Section 40.8: Nested formatting .............................................................................................................................. 237
Section 40.9: Format using Getitem and Getattr ................................................................................................... 237
Section 40.10: Padding and truncating strings, combined .................................................................................... 237
Section 40.11: Custom formatting for a class ......................................................................................................... 238
Chapter 41: String Methods ................................................................................................................................ 240
Section 41.1: Changing the capitalization of a string ............................................................................................. 240
Section 41.2: str.translate: Translating characters in a string ............................................................................... 241
Section 41.3: str.format and f-strings: Format values into a string ...................................................................... 242
Section 41.4: String module's useful constants ....................................................................................................... 243
Section 41.5: Stripping unwanted leading/trailing characters from a string ...................................................... 244
Section 41.6: Reversing a string ............................................................................................................................... 245
Section 41.7: Split a string based on a delimiter into a list of strings ................................................................... 245
Section 41.8: Replace all occurrences of one substring with another substring ................................................ 246
Section 41.9: Testing what a string is composed of ............................................................................................... 247
Section 41.10: String Contains ................................................................................................................................... 249
Section 41.11: Join a list of strings into one string ................................................................................................... 249
Section 41.12: Counting number of times a substring appears in a string .......................................................... 250
Section 41.13: Case insensitive string comparisons ................................................................................................ 250
Section 41.14: Justify strings ..................................................................................................................................... 251
Section 41.15: Test the starting and ending characters of a string ...................................................................... 252
Section 41.16: Conversion between str or bytes data and unicode characters .................................................. 253
Chapter 42: Using loops within functions .................................................................................................... 255
Section 42.1: Return statement inside loop in a function ...................................................................................... 255
Chapter 43: Importing modules ........................................................................................................................ 256
Section 43.1: Importing a module ............................................................................................................................ 256
Section 43.2: The __all__ special variable ............................................................................................................ 257
Section 43.3: Import modules from an arbitrary filesystem location .................................................................. 258
Section 43.4: Importing all names from a module ................................................................................................ 258
Section 43.5: Programmatic importing ................................................................................................................... 259
Section 43.6: PEP8 rules for Imports ....................................................................................................................... 259
Section 43.7: Importing specific names from a module ........................................................................................ 260
Section 43.8: Importing submodules ....................................................................................................................... 260
Section 43.9: Re-importing a module ...................................................................................................................... 260
Section 43.10: __import__() function ..................................................................................................................... 261
Chapter 44: Dierence between Module and Package ...................................................................... 262
Section 44.1: Modules ................................................................................................................................................ 262
 Section 44.2: Packages ............................................................................................................................................. 262
Chapter 45: Math Module .................................................................................................................................... 264
Section 45.1: Rounding: round, floor, ceil, trunc ...................................................................................................... 264
Section 45.2: Trigonometry ...................................................................................................................................... 265
Section 45.3: Pow for faster exponentiation ........................................................................................................... 266
Section 45.4: Infinity and NaN ("not a number") ................................................................................................... 266
Section 45.5: Logarithms .......................................................................................................................................... 269
Section 45.6: Constants ............................................................................................................................................ 269
Section 45.7: Imaginary Numbers ........................................................................................................................... 270
Section 45.8: Copying signs ..................................................................................................................................... 270
Section 45.9: Complex numbers and the cmath module ...................................................................................... 270
Chapter 46: Complex math ................................................................................................................................. 273
Section 46.1: Advanced complex arithmetic ........................................................................................................... 273
Section 46.2: Basic complex arithmetic .................................................................................................................. 274
Chapter 47: Collections module ....................................................................................................................... 275
Section 47.1: collections.Counter .............................................................................................................................. 275
Section 47.2: collections.OrderedDict ...................................................................................................................... 276
Section 47.3: collections.defaultdict ......................................................................................................................... 277
Section 47.4: collections.namedtuple ...................................................................................................................... 278
Section 47.5: collections.deque ................................................................................................................................ 279
Section 47.6: collections.ChainMap .......................................................................................................................... 280
Chapter 48: Operator module ........................................................................................................................... 282
Section 48.1: Itemgetter ............................................................................................................................................ 282
Section 48.2: Operators as alternative to an infix operator ................................................................................. 282
Section 48.3: Methodcaller ....................................................................................................................................... 282
Chapter 49: JSON Module .................................................................................................................................... 284
Section 49.1: Storing data in a file ............................................................................................................................ 284
Section 49.2: Retrieving data from a file ................................................................................................................ 284
Section 49.3: Formatting JSON output ................................................................................................................... 284
Section 49.4: `load` vs `loads`, `dump` vs `dumps` ................................................................................................... 285
Section 49.5: Calling `json.tool` from the command line to pretty-print JSON output ...................................... 286
Section 49.6: JSON encoding custom objects ........................................................................................................ 286
Section 49.7: Creating JSON from Python dict ...................................................................................................... 287
Section 49.8: Creating Python dict from JSON ...................................................................................................... 287
Chapter 50: Sqlite3 Module ................................................................................................................................. 289
Section 50.1: Sqlite3 - Not require separate server process ................................................................................. 289
Section 50.2: Getting the values from the database and Error handling ........................................................... 289
Chapter 51: The os Module ................................................................................................................................... 291
Section 51.1: makedirs - recursive directory creation ............................................................................................ 291
Section 51.2: Create a directory ............................................................................................................................... 292
Section 51.3: Get current directory .......................................................................................................................... 292
Section 51.4: Determine the name of the operating system ................................................................................ 292
Section 51.5: Remove a directory ............................................................................................................................ 292
Section 51.6: Follow a symlink (POSIX) .................................................................................................................... 292
Section 51.7: Change permissions on a file ............................................................................................................. 292
Chapter 52: The locale Module .......................................................................................................................... 293
Section 52.1: Currency Formatting US Dollars Using the locale Module ............................................................. 293
Chapter 53: Itertools Module .............................................................................................................................. 294
Section 53.1: Combinations method in Itertools Module ....................................................................................... 294
 Section 53.2: itertools.dropwhile .............................................................................................................................. 294
Section 53.3: Zipping two iterators until they are both exhausted ...................................................................... 295
Section 53.4: Take a slice of a generator ............................................................................................................... 295
Section 53.5: Grouping items from an iterable object using a function .............................................................. 296
Section 53.6: itertools.takewhile ............................................................................................................................... 297
Section 53.7: itertools.permutations ........................................................................................................................ 297
Section 53.8: itertools.repeat .................................................................................................................................... 298
Section 53.9: Get an accumulated sum of numbers in an iterable ...................................................................... 298
Section 53.10: Cycle through elements in an iterator ............................................................................................ 298
Section 53.11: itertools.product ................................................................................................................................. 298
Section 53.12: itertools.count .................................................................................................................................... 299
Section 53.13: Chaining multiple iterators together ............................................................................................... 300
Chapter 54: Asyncio Module ............................................................................................................................... 301
Section 54.1: Coroutine and Delegation Syntax ..................................................................................................... 301
Section 54.2: Asynchronous Executors ................................................................................................................... 302
Section 54.3: Using UVLoop ..................................................................................................................................... 303
Section 54.4: Synchronization Primitive: Event ....................................................................................................... 303
Section 54.5: A Simple Websocket .......................................................................................................................... 304
Section 54.6: Common Misconception about asyncio .......................................................................................... 304
Chapter 55: Random module ............................................................................................................................. 307
Section 55.1: Creating a random user password ................................................................................................... 307
Section 55.2: Create cryptographically secure random numbers ....................................................................... 307
Section 55.3: Random and sequences: shue, choice and sample .................................................................... 308
Section 55.4: Creating random integers and floats: randint, randrange, random, and uniform ...................... 309
Section 55.5: Reproducible random numbers: Seed and State ............................................................................ 310
Section 55.6: Random Binary Decision ................................................................................................................... 311
Chapter 56: Functools Module ........................................................................................................................... 312
Section 56.1: partial ................................................................................................................................................... 312
Section 56.2: cmp_to_key ....................................................................................................................................... 312
Section 56.3: lru_cache ............................................................................................................................................. 312
Section 56.4: total_ordering ..................................................................................................................................... 313
Section 56.5: reduce .................................................................................................................................................. 314
Chapter 57: The dis module ................................................................................................................................ 315
Section 57.1: What is Python bytecode? ................................................................................................................. 315
Section 57.2: Constants in the dis module .............................................................................................................. 315
Section 57.3: Disassembling modules ..................................................................................................................... 315
Chapter 58: The base64 Module ....................................................................................................................... 317
Section 58.1: Encoding and Decoding Base64 ....................................................................................................... 318
Section 58.2: Encoding and Decoding Base32 ....................................................................................................... 319
Section 58.3: Encoding and Decoding Base16 ........................................................................................................ 320
Section 58.4: Encoding and Decoding ASCII85 ...................................................................................................... 320
Section 58.5: Encoding and Decoding Base85 ....................................................................................................... 321
Chapter 59: Queue Module .................................................................................................................................. 322
Section 59.1: Simple example ................................................................................................................................... 322
Chapter 60: Deque Module .................................................................................................................................. 324
Section 60.1: Basic deque using ............................................................................................................................... 324
Section 60.2: Available methods in deque .............................................................................................................. 324
Section 60.3: limit deque size ................................................................................................................................... 325
Section 60.4: Breadth First Search .......................................................................................................................... 325
Chapter 61: Webbrowser Module ...................................................................................................................... 326
Section 61.1: Opening a URL with Default Browser ................................................................................................ 326
Section 61.2: Opening a URL with Dierent Browsers ........................................................................................... 327
Chapter 62: tkinter ................................................................................................................................................... 328
Section 62.1: Geometry Managers ........................................................................................................................... 328
Section 62.2: A minimal tkinter Application ............................................................................................................ 329
Chapter 63: pyautogui module .......................................................................................................................... 331
Section 63.1: Mouse Functions .................................................................................................................................. 331
Section 63.2: Keyboard Functions ........................................................................................................................... 331
Section 63.3: Screenshot And Image Recognition ................................................................................................. 331
Chapter 64: Indexing and Slicing ...................................................................................................................... 332
Section 64.1: Basic Slicing ......................................................................................................................................... 332
Section 64.2: Reversing an object ........................................................................................................................... 333
Section 64.3: Slice assignment ................................................................................................................................. 333
Section 64.4: Making a shallow copy of an array .................................................................................................. 333
Section 64.5: Indexing custom classes: __getitem__, __setitem__ and __delitem__ ................................... 334
Section 64.6: Basic Indexing ..................................................................................................................................... 335
Chapter 65: Plotting with Matplotlib .............................................................................................................. 337
Section 65.1: Plots with Common X-axis but dierent Y-axis : Using twinx() ....................................................... 337
Section 65.2: Plots with common Y-axis and dierent X-axis using twiny() ....................................................... 338
Section 65.3: A Simple Plot in Matplotlib ................................................................................................................. 340
Section 65.4: Adding more features to a simple plot : axis labels, title, axis ticks, grid, and legend ................ 341
Section 65.5: Making multiple plots in the same figure by superimposition similar to MATLAB ...................... 342
Section 65.6: Making multiple Plots in the same figure using plot superimposition with separate plot
commands ......................................................................................................................................................... 343
Chapter 66: graph-tool .......................................................................................................................................... 345
Section 66.1: PyDotPlus ............................................................................................................................................. 345
Section 66.2: PyGraphviz .......................................................................................................................................... 345
Chapter 67: Generators ......................................................................................................................................... 347
Section 67.1: Introduction .......................................................................................................................................... 347
Section 67.2: Infinite sequences ............................................................................................................................... 349
Section 67.3: Sending objects to a generator ........................................................................................................ 350
Section 67.4: Yielding all values from another iterable ......................................................................................... 351
Section 67.5: Iteration ............................................................................................................................................... 351
Section 67.6: The next() function ............................................................................................................................. 351
Section 67.7: Coroutines ........................................................................................................................................... 352
Section 67.8: Refactoring list-building code ........................................................................................................... 352
Section 67.9: Yield with recursion: recursively listing all files in a directory ........................................................ 353
Section 67.10: Generator expressions ...................................................................................................................... 354
Section 67.11: Using a generator to find Fibonacci Numbers ............................................................................... 354
Section 67.12: Searching ........................................................................................................................................... 354
Section 67.13: Iterating over generators in parallel ............................................................................................... 355
Chapter 68: Reduce ................................................................................................................................................. 356
Section 68.1: Overview .............................................................................................................................................. 356
Section 68.2: Using reduce ....................................................................................................................................... 356
Section 68.3: Cumulative product ............................................................................................................................ 357
Section 68.4: Non short-circuit variant of any/all ................................................................................................. 357
Chapter 69: Map Function .................................................................................................................................... 358
Section 69.1: Basic use of map, itertools.imap and future_builtins.map ............................................................. 358
 Section 69.2: Mapping each value in an iterable ................................................................................................... 358
Section 69.3: Mapping values of dierent iterables .............................................................................................. 359
Section 69.4: Transposing with Map: Using "None" as function argument (python 2.x only) .......................... 361
Section 69.5: Series and Parallel Mapping .............................................................................................................. 361
Chapter 70: Exponentiation ................................................................................................................................ 365
Section 70.1: Exponentiation using builtins: ** and pow() ...................................................................................... 365
Section 70.2: Square root: math.sqrt() and cmath.sqrt ......................................................................................... 365
Section 70.3: Modular exponentiation: pow() with 3 arguments .......................................................................... 366
Section 70.4: Computing large integer roots ......................................................................................................... 366
Section 70.5: Exponentiation using the math module: math.pow() ..................................................................... 367
Section 70.6: Exponential function: math.exp() and cmath.exp() ......................................................................... 368
Section 70.7: Exponential function minus 1: math.expm1() .................................................................................... 368
Section 70.8: Magic methods and exponentiation: builtin, math and cmath ...................................................... 369
Section 70.9: Roots: nth-root with fractional exponents ....................................................................................... 370
Chapter 71: Searching ............................................................................................................................................ 371
Section 71.1: Searching for an element .................................................................................................................... 371
Section 71.2: Searching in custom classes: __contains__ and __iter__ ........................................................... 371
Section 71.3: Getting the index for strings: str.index(), str.rindex() and str.find(), str.rfind() ............................... 372
Section 71.4: Getting the index list and tuples: list.index(), tuple.index() .............................................................. 373
Section 71.5: Searching key(s) for a value in dict ................................................................................................... 373
Section 71.6: Getting the index for sorted sequences: bisect.bisect_left() .......................................................... 374
Section 71.7: Searching nested sequences ............................................................................................................. 374
Chapter 72: Sorting, Minimum and Maximum ............................................................................................ 376
Section 72.1: Make custom classes orderable ........................................................................................................ 376
Section 72.2: Special case: dictionaries ................................................................................................................... 378
Section 72.3: Using the key argument .................................................................................................................... 379
Section 72.4: Default Argument to max, min ......................................................................................................... 379
Section 72.5: Getting a sorted sequence ................................................................................................................ 380
Section 72.6: Extracting N largest or N smallest items from an iterable ............................................................ 380
Section 72.7: Getting the minimum or maximum of several values .................................................................... 381
Section 72.8: Minimum and Maximum of a sequence ........................................................................................... 381
Chapter 73: Counting .............................................................................................................................................. 382
Section 73.1: Counting all occurrence of all items in an iterable: collections.Counter ........................................ 382
Section 73.2: Getting the most common value(-s): collections.Counter.most_common() ................................ 382
Section 73.3: Counting the occurrences of one item in a sequence: list.count() and tuple.count() .................. 382
Section 73.4: Counting the occurrences of a substring in a string: str.count() ................................................... 383
Section 73.5: Counting occurrences in numpy array ............................................................................................ 383
Chapter 74: The Print Function ......................................................................................................................... 384
Section 74.1: Print basics ........................................................................................................................................... 384
Section 74.2: Print parameters ................................................................................................................................ 385
Chapter 75: Regular Expressions (Regex) ................................................................................................... 388
Section 75.1: Matching the beginning of a string ................................................................................................... 388
Section 75.2: Searching ............................................................................................................................................ 389
Section 75.3: Precompiled patterns ......................................................................................................................... 389
Section 75.4: Flags .................................................................................................................................................... 390
Section 75.5: Replacing ............................................................................................................................................. 391
Section 75.6: Find All Non-Overlapping Matches ................................................................................................... 391
Section 75.7: Checking for allowed characters ...................................................................................................... 392
Section 75.8: Splitting a string using regular expressions ..................................................................................... 392
Section 75.9: Grouping .............................................................................................................................................. 392
 Section 75.10: Escaping Special Characters ........................................................................................................... 393
Section 75.11: Match an expression only in specific locations ............................................................................... 394
Section 75.12: Iterating over matches using `re.finditer` ........................................................................................ 395
Chapter 76: Copying data .................................................................................................................................... 396
Section 76.1: Copy a dictionary ................................................................................................................................ 396
Section 76.2: Performing a shallow copy ............................................................................................................... 396
Section 76.3: Performing a deep copy .................................................................................................................... 396
Section 76.4: Performing a shallow copy of a list .................................................................................................. 396
Section 76.5: Copy a set ........................................................................................................................................... 396
Chapter 77: Context Managers (“with” Statement) ............................................................................... 398
Section 77.1: Introduction to context managers and the with statement ............................................................ 398
Section 77.2: Writing your own context manager ................................................................................................. 398
Section 77.3: Writing your own contextmanager using generator syntax ......................................................... 399
Section 77.4: Multiple context managers ................................................................................................................ 400
Section 77.5: Assigning to a target .......................................................................................................................... 400
Section 77.6: Manage Resources ............................................................................................................................. 401
Chapter 78: The __name__ special variable ........................................................................................... 402
Section 78.1: __name__ == '__main__' ................................................................................................................. 402
Section 78.2: Use in logging ..................................................................................................................................... 402
Section 78.3: function_class_or_module.__name__ .......................................................................................... 402
Chapter 79: Checking Path Existence and Permissions ......................................................................... 404
Section 79.1: Perform checks using os.access ........................................................................................................ 404
Chapter 80: Creating Python packages ....................................................................................................... 406
Section 80.1: Introduction ......................................................................................................................................... 406
Section 80.2: Uploading to PyPI .............................................................................................................................. 406
Section 80.3: Making package executable ............................................................................................................. 408
Chapter 81: Usage of "pip" module: PyPI Package Manager ............................................................. 410
Section 81.1: Example use of commands ................................................................................................................ 410
Section 81.2: Handling ImportError Exception ........................................................................................................ 410
Section 81.3: Force install .......................................................................................................................................... 411
Chapter 82: pip: PyPI Package Manager ...................................................................................................... 412
Section 82.1: Install Packages .................................................................................................................................. 412
Section 82.2: To list all packages installed using `pip` ........................................................................................... 412
Section 82.3: Upgrade Packages ............................................................................................................................ 412
Section 82.4: Uninstall Packages ............................................................................................................................. 413
Section 82.5: Updating all outdated packages on Linux ...................................................................................... 413
Section 82.6: Updating all outdated packages on Windows ................................................................................ 413
Section 82.7: Create a requirements.txt file of all packages on the system ....................................................... 413
Section 82.8: Using a certain Python version with pip .......................................................................................... 414
Section 82.9: Create a requirements.txt file of packages only in the current virtualenv .................................. 414
Section 82.10: Installing packages not yet on pip as wheels ................................................................................ 415
Chapter 83: Parsing Command Line arguments ...................................................................................... 418
Section 83.1: Hello world in argparse ...................................................................................................................... 418
Section 83.2: Using command line arguments with argv ..................................................................................... 418
Section 83.3: Setting mutually exclusive arguments with argparse .................................................................... 419
Section 83.4: Basic example with docopt ............................................................................................................... 420
Section 83.5: Custom parser error message with argparse ................................................................................. 420
Section 83.6: Conceptual grouping of arguments with argparse.add_argument_group() ............................. 421
Section 83.7: Advanced example with docopt and docopt_dispatch ................................................................. 422
Chapter 84: Subprocess Library ...................................................................................................................... 424
Section 84.1: More flexibility with Popen ................................................................................................................. 424
Section 84.2: Calling External Commands .............................................................................................................. 425
Section 84.3: How to create the command list argument .................................................................................... 425
Chapter 85: setup.py .............................................................................................................................................. 427
Section 85.1: Purpose of setup.py ............................................................................................................................ 427
Section 85.2: Using source control metadata in setup.py .................................................................................... 427
Section 85.3: Adding command line scripts to your python package ................................................................. 428
Section 85.4: Adding installation options ................................................................................................................ 428
Chapter 86: Recursion ............................................................................................................................................ 430
Section 86.1: The What, How, and When of Recursion .......................................................................................... 430
Section 86.2: Tree exploration with recursion ........................................................................................................ 433
Section 86.3: Sum of numbers from 1 to n .............................................................................................................. 434
Section 86.4: Increasing the Maximum Recursion Depth ...................................................................................... 434
Section 86.5: Tail Recursion - Bad Practice ............................................................................................................ 435
Section 86.6: Tail Recursion Optimization Through Stack Introspection ............................................................ 435
Chapter 87: Type Hints .......................................................................................................................................... 437
Section 87.1: Adding types to a function ................................................................................................................. 437
Section 87.2: NamedTuple ....................................................................................................................................... 438
Section 87.3: Generic Types ..................................................................................................................................... 438
Section 87.4: Variables and Attributes .................................................................................................................... 438
Section 87.5: Class Members and Methods ............................................................................................................ 439
Section 87.6: Type hints for keyword arguments .................................................................................................. 439
Chapter 88: Exceptions .......................................................................................................................................... 440
Section 88.1: Catching Exceptions ............................................................................................................................ 440
Section 88.2: Do not catch everything! ................................................................................................................... 440
Section 88.3: Re-raising exceptions ......................................................................................................................... 441
Section 88.4: Catching multiple exceptions ............................................................................................................ 441
Section 88.5: Exception Hierarchy ........................................................................................................................... 442
Section 88.6: Else ....................................................................................................................................................... 444
Section 88.7: Raising Exceptions .............................................................................................................................. 444
Section 88.8: Creating custom exception types ..................................................................................................... 445
Section 88.9: Practical examples of exception handling ....................................................................................... 445
Section 88.10: Exceptions are Objects too .............................................................................................................. 446
Section 88.11: Running clean-up code with finally .................................................................................................. 446
Section 88.12: Chain exceptions with raise from .................................................................................................... 447
Chapter 89: Raise Custom Errors / Exceptions ......................................................................................... 448
Section 89.1: Custom Exception ............................................................................................................................... 448
Section 89.2: Catch custom Exception .................................................................................................................... 448
Chapter 90: Commonwealth Exceptions ....................................................................................................... 450
Section 90.1: Other Errors ......................................................................................................................................... 450
Section 90.2: NameError: name '???' is not defined .............................................................................................. 451
Section 90.3: TypeErrors .......................................................................................................................................... 452
Section 90.4: Syntax Error on good code ............................................................................................................... 453
Section 90.5: IndentationErrors (or indentation SyntaxErrors) ............................................................................ 454
Chapter 91: urllib ....................................................................................................................................................... 456
Section 91.1: HTTP GET .............................................................................................................................................. 456
Section 91.2: HTTP POST .......................................................................................................................................... 456
Section 91.3: Decode received bytes according to content type encoding ........................................................ 457
Chapter 92: Web scraping with Python ......................................................................................................... 458
Section 92.1: Scraping using the Scrapy framework ............................................................................................. 458
Section 92.2: Scraping using Selenium WebDriver ................................................................................................ 458
Section 92.3: Basic example of using requests and lxml to scrape some data ................................................. 459
Section 92.4: Maintaining web-scraping session with requests ........................................................................... 459
Section 92.5: Scraping using BeautifulSoup4 ......................................................................................................... 460
Section 92.6: Simple web content download with urllib.request .......................................................................... 460
Section 92.7: Modify Scrapy user agent ................................................................................................................. 460
Section 92.8: Scraping with curl ............................................................................................................................... 460
Chapter 93: HTML Parsing .................................................................................................................................... 462
Section 93.1: Using CSS selectors in BeautifulSoup ................................................................................................ 462
Section 93.2: PyQuery ............................................................................................................................................... 462
Section 93.3: Locate a text after an element in BeautifulSoup ............................................................................ 463
Chapter 94: Manipulating XML .......................................................................................................................... 464
Section 94.1: Opening and reading using an ElementTree ................................................................................... 464
Section 94.2: Create and Build XML Documents .................................................................................................... 464
Section 94.3: Modifying an XML File ........................................................................................................................ 465
Section 94.4: Searching the XML with XPath .......................................................................................................... 465
Section 94.5: Opening and reading large XML files using iterparse (incremental parsing) ............................. 466
Chapter 95: Python Requests Post .................................................................................................................. 468
Section 95.1: Simple Post .......................................................................................................................................... 468
Section 95.2: Form Encoded Data ........................................................................................................................... 469
Section 95.3: File Upload .......................................................................................................................................... 469
Section 95.4: Responses ........................................................................................................................................... 470
Section 95.5: Authentication ..................................................................................................................................... 470
Section 95.6: Proxies ................................................................................................................................................. 471
Chapter 96: Distribution ........................................................................................................................................ 473
Section 96.1: py2app ................................................................................................................................................. 473
Section 96.2: cx_Freeze ............................................................................................................................................ 474
Chapter 97: Property Objects ............................................................................................................................ 475
Section 97.1: Using the @property decorator for read-write properties ............................................................. 475
Section 97.2: Using the @property decorator ....................................................................................................... 475
Section 97.3: Overriding just a getter, setter or a deleter of a property object ................................................. 476
Section 97.4: Using properties without decorators ................................................................................................ 476
Chapter 98: Overloading ...................................................................................................................................... 479
Section 98.1: Operator overloading ......................................................................................................................... 479
Section 98.2: Magic/Dunder Methods .................................................................................................................... 480
Section 98.3: Container and sequence types ......................................................................................................... 481
Section 98.4: Callable types ..................................................................................................................................... 482
Section 98.5: Handling unimplemented behaviour ................................................................................................ 482
Chapter 99: Polymorphism .................................................................................................................................. 484
Section 99.1: Duck Typing ......................................................................................................................................... 484
Section 99.2: Basic Polymorphism .......................................................................................................................... 484
Chapter 100: Method Overriding ...................................................................................................................... 488
Section 100.1: Basic method overriding ................................................................................................................... 488
Chapter 101: User-Defined Methods ................................................................................................................ 489
Section 101.1: Creating user-defined method objects ............................................................................................ 489
Section 101.2: Turtle example ................................................................................................................................... 490
Chapter 102: String representations of class instances: __str__ and __repr__
methods ........................................................................................................................................................................ 491
Section 102.1: Motivation ........................................................................................................................................... 491
Section 102.2: Both methods implemented, eval-round-trip style __repr__() .................................................. 495
Chapter 103: Debugging ........................................................................................................................................ 496
Section 103.1: Via IPython and ipdb ......................................................................................................................... 496
Section 103.2: The Python Debugger: Step-through Debugging with _pdb_ .................................................... 496
Section 103.3: Remote debugger ............................................................................................................................. 498
Chapter 104: Reading and Writing CSV ........................................................................................................ 499
Section 104.1: Using pandas ..................................................................................................................................... 499
Section 104.2: Writing a TSV file .............................................................................................................................. 499
Chapter 105: Writing to CSV from String or List ...................................................................................... 501
Section 105.1: Basic Write Example .......................................................................................................................... 501
Section 105.2: Appending a String as a newline in a CSV file ............................................................................... 501
Chapter 106: Dynamic code execution with `exec` and `eval` ............................................................. 502
Section 106.1: Executing code provided by untrusted user using exec, eval, or ast.literal_eval ....................... 502
Section 106.2: Evaluating a string containing a Python literal with ast.literal_eval ........................................... 502
Section 106.3: Evaluating statements with exec ..................................................................................................... 502
Section 106.4: Evaluating an expression with eval ................................................................................................. 503
Section 106.5: Precompiling an expression to evaluate it multiple times ............................................................ 503
Section 106.6: Evaluating an expression with eval using custom globals ........................................................... 503
Chapter 107: PyInstaller - Distributing Python Code .............................................................................. 504
Section 107.1: Installation and Setup ........................................................................................................................ 504
Section 107.2: Using Pyinstaller ................................................................................................................................ 504
Section 107.3: Bundling to One Folder ..................................................................................................................... 505
Section 107.4: Bundling to a Single File ................................................................................................................... 505
Chapter 108: Data Visualization with Python ............................................................................................. 506
Section 108.1: Seaborn .............................................................................................................................................. 506
Section 108.2: Matplotlib ........................................................................................................................................... 508
Section 108.3: Plotly ................................................................................................................................................... 509
Section 108.4: MayaVI ............................................................................................................................................... 511
Chapter 109: The Interpreter (Command Line Console) ....................................................................... 513
Section 109.1: Getting general help .......................................................................................................................... 513
Section 109.2: Referring to the last expression ...................................................................................................... 513
Section 109.3: Opening the Python console ............................................................................................................ 514
Section 109.4: The PYTHONSTARTUP variable ...................................................................................................... 514
Section 109.5: Command line arguments ............................................................................................................... 514
Section 109.6: Getting help about an object ........................................................................................................... 515
Chapter 110: *args and **kwargs ....................................................................................................................... 518
Section 110.1: Using **kwargs when writing functions ............................................................................................ 518
Section 110.2: Using *args when writing functions .................................................................................................. 518
Section 110.3: Populating kwarg values with a dictionary ..................................................................................... 519
Section 110.4: Keyword-only and Keyword-required arguments ........................................................................ 519
Section 110.5: Using **kwargs when calling functions ............................................................................................ 519
Section 110.6: **kwargs and default values ............................................................................................................. 519
Section 110.7: Using *args when calling functions .................................................................................................. 520
Chapter 111: Garbage Collection ........................................................................................................................ 521
Section 111.1: Reuse of primitive objects .................................................................................................................. 521
Section 111.2: Eects of the del command .............................................................................................................. 521
Section 111.3: Reference Counting ............................................................................................................................ 522
 Section 111.4: Garbage Collector for Reference Cycles ......................................................................................... 522
Section 111.5: Forcefully deallocating objects ......................................................................................................... 523
Section 111.6: Viewing the refcount of an object ..................................................................................................... 524
Section 111.7: Do not wait for the garbage collection to clean up ........................................................................ 524
Section 111.8: Managing garbage collection ........................................................................................................... 524
Chapter 112: Pickle data serialisation ............................................................................................................. 526
Section 112.1: Using Pickle to serialize and deserialize an object .......................................................................... 526
Section 112.2: Customize Pickled Data .................................................................................................................... 526
Chapter 113: Binary Data ...................................................................................................................................... 528
Section 113.1: Format a list of values into a byte object ........................................................................................ 528
Section 113.2: Unpack a byte object according to a format string ...................................................................... 528
Section 113.3: Packing a structure ............................................................................................................................ 528
Chapter 114: Idioms .................................................................................................................................................. 530
Section 114.1: Dictionary key initializations .............................................................................................................. 530
Section 114.2: Switching variables ............................................................................................................................ 530
Section 114.3: Use truth value testing ...................................................................................................................... 530
Section 114.4: Test for "__main__" to avoid unexpected code execution .......................................................... 531
Chapter 115: Data Serialization .......................................................................................................................... 533
Section 115.1: Serialization using JSON .................................................................................................................... 533
Section 115.2: Serialization using Pickle ................................................................................................................... 533
Chapter 116: Multiprocessing ............................................................................................................................... 535
Section 116.1: Running Two Simple Processes ......................................................................................................... 535
Section 116.2: Using Pool and Map ........................................................................................................................... 535
Chapter 117: Multithreading ................................................................................................................................. 537
Section 117.1: Basics of multithreading .................................................................................................................... 537
Section 117.2: Communicating between threads .................................................................................................... 538
Section 117.3: Creating a worker pool ...................................................................................................................... 539
Section 117.4: Advanced use of multithreads .......................................................................................................... 539
Section 117.5: Stoppable Thread with a while Loop ............................................................................................... 541
Chapter 118: Processes and Threads .............................................................................................................. 542
Section 118.1: Global Interpreter Lock ...................................................................................................................... 542
Section 118.2: Running in Multiple Threads ............................................................................................................. 543
Section 118.3: Running in Multiple Processes .......................................................................................................... 544
Section 118.4: Sharing State Between Threads ....................................................................................................... 544
Section 118.5: Sharing State Between Processes .................................................................................................... 545
Chapter 119: Python concurrency ..................................................................................................................... 546
Section 119.1: The multiprocessing module ............................................................................................................. 546
Section 119.2: The threading module ....................................................................................................................... 547
Section 119.3: Passing data between multiprocessing processes ........................................................................ 547
Chapter 120: Parallel computation .................................................................................................................. 550
Section 120.1: Using the multiprocessing module to parallelise tasks ................................................................. 550
Section 120.2: Using a C-extension to parallelize tasks ........................................................................................ 550
Section 120.3: Using Parent and Children scripts to execute code in parallel .................................................... 550
Section 120.4: Using PyPar module to parallelize .................................................................................................. 551
Chapter 121: Sockets ................................................................................................................................................ 552
Section 121.1: Raw Sockets on Linux ......................................................................................................................... 552
Section 121.2: Sending data via UDP ....................................................................................................................... 552
Section 121.3: Receiving data via UDP ..................................................................................................................... 553
Section 121.4: Sending data via TCP ........................................................................................................................ 553
 Section 121.5: Multi-threaded TCP Socket Server ................................................................................................... 553
Chapter 122: Websockets ...................................................................................................................................... 556
Section 122.1: Simple Echo with aiohttp ................................................................................................................... 556
Section 122.2: Wrapper Class with aiohttp ............................................................................................................. 556
Section 122.3: Using Autobahn as a Websocket Factory ...................................................................................... 557
Chapter 123: Sockets And Message Encryption/Decryption Between Client and Server
............................................................................................................................................................................................ 559
Section 123.1: Server side Implementation .............................................................................................................. 559
Section 123.2: Client side Implementation .............................................................................................................. 561
Chapter 124: Python Networking ..................................................................................................................... 563
Section 124.1: Creating a Simple Http Server .......................................................................................................... 563
Section 124.2: Creating a TCP server ...................................................................................................................... 563
Section 124.3: Creating a UDP Server ..................................................................................................................... 564
Section 124.4: Start Simple HttpServer in a thread and open the browser ......................................................... 564
Section 124.5: The simplest Python socket client-server example ....................................................................... 565
Chapter 125: Python HTTP Server .................................................................................................................... 567
Section 125.1: Running a simple HTTP server ......................................................................................................... 567
Section 125.2: Serving files ........................................................................................................................................ 567
Section 125.3: Basic handling of GET, POST, PUT using BaseHTTPRequestHandler ......................................... 568
Section 125.4: Programmatic API of SimpleHTTPServer ....................................................................................... 569
Chapter 126: Flask .................................................................................................................................................... 571
Section 126.1: Files and Templates ........................................................................................................................... 571
Section 126.2: The basics .......................................................................................................................................... 571
Section 126.3: Routing URLs ..................................................................................................................................... 572
Section 126.4: HTTP Methods ................................................................................................................................... 573
Section 126.5: Jinja Templating ............................................................................................................................... 573
Section 126.6: The Request Object .......................................................................................................................... 574
Chapter 127: Introduction to RabbitMQ using AMQPStorm ................................................................ 576
Section 127.1: How to consume messages from RabbitMQ .................................................................................. 576
Section 127.2: How to publish messages to RabbitMQ ......................................................................................... 577
Section 127.3: How to create a delayed queue in RabbitMQ ................................................................................ 577
Chapter 128: Descriptor ......................................................................................................................................... 580
Section 128.1: Simple descriptor ............................................................................................................................... 580
Section 128.2: Two-way conversions ....................................................................................................................... 581
Chapter 129: tempfile NamedTemporaryFile ............................................................................................. 582
Section 129.1: Create (and write to a) known, persistent temporary file ............................................................. 582
Chapter 130: Input, Subset and Output External Data Files using Pandas ................................. 584
Section 130.1: Basic Code to Import, Subset and Write External Data Files Using Pandas ............................... 584
Chapter 131: Unzipping Files ................................................................................................................................ 586
Section 131.1: Using Python ZipFile.extractall() to decompress a ZIP file ............................................................ 586
Section 131.2: Using Python TarFile.extractall() to decompress a tarball ........................................................... 586
Chapter 132: Working with ZIP archives ........................................................................................................ 587
Section 132.1: Examining Zipfile Contents ............................................................................................................... 587
Section 132.2: Opening Zip Files ............................................................................................................................... 587
Section 132.3: Extracting zip file contents to a directory ....................................................................................... 588
Section 132.4: Creating new archives ...................................................................................................................... 588
Chapter 133: Getting start with GZip .............................................................................................................. 589
Section 133.1: Read and write GNU zip files ............................................................................................................ 589
Chapter 134: Stack ................................................................................................................................................... 590
Section 134.1: Creating a Stack class with a List Object ........................................................................................ 590
Section 134.2: Parsing Parentheses ......................................................................................................................... 591
Chapter 135: Working around the Global Interpreter Lock (GIL) ..................................................... 593
Section 135.1: Multiprocessing.Pool .......................................................................................................................... 593
Section 135.2: Cython nogil: ...................................................................................................................................... 594
Chapter 136: Deployment ..................................................................................................................................... 595
Section 136.1: Uploading a Conda Package ............................................................................................................ 595
Chapter 137: Logging .............................................................................................................................................. 597
Section 137.1: Introduction to Python Logging ........................................................................................................ 597
Section 137.2: Logging exceptions ........................................................................................................................... 598
Chapter 138: Web Server Gateway Interface (WSGI) ............................................................................. 601
Section 138.1: Server Object (Method) ..................................................................................................................... 601
Chapter 139: Python Server Sent Events ...................................................................................................... 602
Section 139.1: Flask SSE ............................................................................................................................................. 602
Section 139.2: Asyncio SSE ........................................................................................................................................ 602
Chapter 140: Alternatives to switch statement from other languages ....................................... 604
Section 140.1: Use what the language oers: the if/else construct ..................................................................... 604
Section 140.2: Use a dict of functions ...................................................................................................................... 604
Section 140.3: Use class introspection ..................................................................................................................... 605
Section 140.4: Using a context manager ................................................................................................................ 606
Chapter 141: List destructuring (aka packing and unpacking) ......................................................... 607
Section 141.1: Destructuring assignment ................................................................................................................. 607
Section 141.2: Packing function arguments ............................................................................................................ 608
Section 141.3: Unpacking function arguments ........................................................................................................ 610
Chapter 142: Accessing Python source code and bytecode .............................................................. 611
Section 142.1: Display the bytecode of a function .................................................................................................. 611
Section 142.2: Display the source code of an object ............................................................................................. 611
Section 142.3: Exploring the code object of a function .......................................................................................... 612
Chapter 143: Mixins .................................................................................................................................................. 613
Section 143.1: Mixin ..................................................................................................................................................... 613
Section 143.2: Overriding Methods in Mixins ........................................................................................................... 614
Chapter 144: Attribute Access ........................................................................................................................... 615
Section 144.1: Basic Attribute Access using the Dot Notation ............................................................................... 615
Section 144.2: Setters, Getters & Properties ............................................................................................................ 615
Chapter 145: ArcPy .................................................................................................................................................. 618
Section 145.1: createDissolvedGDB to create a file gdb on the workspace ........................................................ 618
Section 145.2: Printing one field's value for all rows of feature class in file geodatabase using Search
Cursor ................................................................................................................................................................. 618
Chapter 146: Abstract Base Classes (abc) ................................................................................................... 619
Section 146.1: Setting the ABCMeta metaclass ....................................................................................................... 619
Section 146.2: Why/How to use ABCMeta and @abstractmethod ...................................................................... 619
Chapter 147: Plugin and Extension Classes ................................................................................................. 621
Section 147.1: Mixins ................................................................................................................................................... 621
Section 147.2: Plugins with Customized Classes ..................................................................................................... 622
Chapter 148: Immutable datatypes(int, float, str, tuple and frozensets) .................................. 624
Section 148.1: Individual characters of strings are not assignable ....................................................................... 624
Section 148.2: Tuple's individual members aren't assignable ............................................................................... 624
 Section 148.3: Frozenset's are immutable and not assignable ............................................................................. 624
Chapter 149: Incompatibilities moving from Python 2 to Python 3 ................................................ 625
Section 149.1: Integer Division ................................................................................................................................... 625
Section 149.2: Unpacking Iterables .......................................................................................................................... 626
Section 149.3: Strings: Bytes versus Unicode .......................................................................................................... 628
Section 149.4: Print statement vs. Print function .................................................................................................... 630
Section 149.5: Dierences between range and xrange functions ........................................................................ 631
Section 149.6: Raising and handling Exceptions ..................................................................................................... 632
Section 149.7: Leaked variables in list comprehension .......................................................................................... 634
Section 149.8: True, False and None ........................................................................................................................ 635
Section 149.9: User Input ........................................................................................................................................... 635
Section 149.10: Comparison of dierent types ....................................................................................................... 636
Section 149.11: .next() method on iterators renamed ............................................................................................. 636
Section 149.12: filter(), map() and zip() return iterators instead of sequences ................................................... 637
Section 149.13: Renamed modules ........................................................................................................................... 637
Section 149.14: Removed operators <> and ``, synonymous with != and repr() ................................................... 638
Section 149.15: long vs. int ......................................................................................................................................... 638
Section 149.16: All classes are "new-style classes" in Python 3 ............................................................................ 639
Section 149.17: Reduce is no longer a built-in ......................................................................................................... 640
Section 149.18: Absolute/Relative Imports .............................................................................................................. 640
Section 149.19: map() ................................................................................................................................................ 642
Section 149.20: The round() function tie-breaking and return type .................................................................... 643
Section 149.21: File I/O .............................................................................................................................................. 644
Section 149.22: cmp function removed in Python 3 ............................................................................................... 644
Section 149.23: Octal Constants ............................................................................................................................... 645
Section 149.24: Return value when writing to a file object .................................................................................... 645
Section 149.25: exec statement is a function in Python 3 ...................................................................................... 645
Section 149.26: encode/decode to hex no longer available ................................................................................. 646
Section 149.27: Dictionary method changes .......................................................................................................... 647
Section 149.28: Class Boolean Value ....................................................................................................................... 647
Section 149.29: hasattr function bug in Python 2 ................................................................................................... 648
Chapter 150: 2to3 tool ............................................................................................................................................ 650
Section 150.1: Basic Usage ........................................................................................................................................ 650
Chapter 151: Non-ocial Python implementations ................................................................................ 652
Section 151.1: IronPython ........................................................................................................................................... 652
Section 151.2: Jython ................................................................................................................................................. 652
Section 151.3: Transcrypt .......................................................................................................................................... 653
Chapter 152: Abstract syntax tree ................................................................................................................... 656
Section 152.1: Analyze functions in a python script ................................................................................................ 656
Chapter 153: Unicode and bytes ....................................................................................................................... 658
Section 153.1: Encoding/decoding error handling .................................................................................................. 658
Section 153.2: File I/O ................................................................................................................................................ 658
Section 153.3: Basics .................................................................................................................................................. 659
Chapter 154: Python Serial Communication (pyserial) ......................................................................... 661
Section 154.1: Initialize serial device ......................................................................................................................... 661
Section 154.2: Read from serial port ....................................................................................................................... 661
Section 154.3: Check what serial ports are available on your machine .............................................................. 661
Chapter 155: Neo4j and Cypher using Py2Neo ......................................................................................... 664
Section 155.1: Adding Nodes to Neo4j Graph .......................................................................................................... 664
 Section 155.2: Importing and Authenticating .......................................................................................................... 664
Section 155.3: Adding Relationships to Neo4j Graph ............................................................................................. 664
Section 155.4: Query 1 : Autocomplete on News Titles .......................................................................................... 664
Section 155.5: Query 2 : Get News Articles by Location on a particular date ..................................................... 665
Section 155.6: Cypher Query Samples .................................................................................................................... 665
Chapter 156: Basic Curses with Python .......................................................................................................... 666
Section 156.1: The wrapper() helper function ......................................................................................................... 666
Section 156.2: Basic Invocation Example ................................................................................................................ 666
Chapter 157: Templates in python ................................................................................................................... 667
Section 157.1: Simple data output program using template ................................................................................. 667
Section 157.2: Changing delimiter ............................................................................................................................ 667
Chapter 158: Pillow ................................................................................................................................................... 668
Section 158.1: Read Image File ................................................................................................................................. 668
Section 158.2: Convert files to JPEG ........................................................................................................................ 668
Chapter 159: The pass statement .................................................................................................................... 669
Section 159.1: Ignore an exception ........................................................................................................................... 669
Section 159.2: Create a new Exception that can be caught .................................................................................. 669
Chapter 160: CLI subcommands with precise help output .................................................................. 671
Section 160.1: Native way (no libraries) ................................................................................................................... 671
Section 160.2: argparse (default help formatter) .................................................................................................. 671
Section 160.3: argparse (custom help formatter) .................................................................................................. 672
Chapter 161: Database Access ............................................................................................................................ 674
Section 161.1: SQLite ................................................................................................................................................... 674
Section 161.2: Accessing MySQL database using MySQLdb ................................................................................. 679
Section 161.3: Connection .......................................................................................................................................... 680
Section 161.4: PostgreSQL Database access using psycopg2 .............................................................................. 681
Section 161.5: Oracle database ................................................................................................................................ 682
Section 161.6: Using sqlalchemy ............................................................................................................................... 684
Chapter 162: Connecting Python to SQL Server ....................................................................................... 685
Section 162.1: Connect to Server, Create Table, Query Data ................................................................................ 685
Chapter 163: PostgreSQL ...................................................................................................................................... 686
Section 163.1: Getting Started ................................................................................................................................... 686
Chapter 164: Python and Excel .......................................................................................................................... 687
Section 164.1: Read the excel data using xlrd module ........................................................................................... 687
Section 164.2: Format Excel files with xlsxwriter ..................................................................................................... 687
Section 164.3: Put list data into a Excel's file ........................................................................................................... 688
Section 164.4: OpenPyXL .......................................................................................................................................... 689
Section 164.5: Create excel charts with xlsxwriter .................................................................................................. 689
Chapter 165: Turtle Graphics .............................................................................................................................. 693
Section 165.1: Ninja Twist (Turtle Graphics) ............................................................................................................ 693
Chapter 166: Python Persistence ...................................................................................................................... 694
Section 166.1: Python Persistence ............................................................................................................................ 694
Section 166.2: Function utility for save and load .................................................................................................... 695
Chapter 167: Design Patterns ............................................................................................................................. 696
Section 167.1: Introduction to design patterns and Singleton Pattern ................................................................. 696
Section 167.2: Strategy Pattern ................................................................................................................................ 698
Section 167.3: Proxy ................................................................................................................................................... 699
Chapter 168: hashlib ................................................................................................................................................ 701
 Section 168.1: MD5 hash of a string ......................................................................................................................... 701
Section 168.2: algorithm provided by OpenSSL ..................................................................................................... 702
Chapter 169: Creating a Windows service using Python ....................................................................... 703
Section 169.1: A Python script that can be run as a service .................................................................................. 703
Section 169.2: Running a Flask web application as a service ............................................................................... 704
Chapter 170: Mutable vs Immutable (and Hashable) in Python ....................................................... 706
Section 170.1: Mutable vs Immutable ....................................................................................................................... 706
Section 170.2: Mutable and Immutable as Arguments .......................................................................................... 708
Chapter 171: configparser .................................................................................................................................... 710
Section 171.1: Creating configuration file programmatically ................................................................................. 710
Section 171.2: Basic usage ......................................................................................................................................... 710
Chapter 172: Optical Character Recognition .............................................................................................. 711
Section 172.1: PyTesseract ........................................................................................................................................ 711
Section 172.2: PyOCR ................................................................................................................................................ 711
Chapter 173: Virtual environments .................................................................................................................. 713
Section 173.1: Creating and using a virtual environment ....................................................................................... 713
Section 173.2: Specifying specific python version to use in script on Unix/Linux ............................................... 715
Section 173.3: Creating a virtual environment for a dierent version of python ............................................... 715
Section 173.4: Making virtual environments using Anaconda .............................................................................. 715
Section 173.5: Managing multiple virtual environments with virtualenvwrapper ............................................... 716
Section 173.6: Installing packages in a virtual environment ................................................................................. 717
Section 173.7: Discovering which virtual environment you are using .................................................................. 718
Section 173.8: Checking if running inside a virtual environment .......................................................................... 719
Section 173.9: Using virtualenv with fish shell ......................................................................................................... 719
Chapter 174: Python Virtual Environment - virtualenv ......................................................................... 721
Section 174.1: Installation .......................................................................................................................................... 721
Section 174.2: Usage ................................................................................................................................................. 721
Section 174.3: Install a package in your Virtualenv ............................................................................................... 721
Section 174.4: Other useful virtualenv commands ................................................................................................. 722
Chapter 175: Virtual environment with virtualenvwrapper ................................................................ 724
Section 175.1: Create virtual environment with virtualenvwrapper ...................................................................... 724
Chapter 176: Create virtual environment with virtualenvwrapper in windows ........................ 726
Section 176.1: Virtual environment with virtualenvwrapper for windows ............................................................. 726
Chapter 177: sys ........................................................................................................................................................ 727
Section 177.1: Command line arguments ................................................................................................................ 727
Section 177.2: Script name ........................................................................................................................................ 727
Section 177.3: Standard error stream ...................................................................................................................... 727
Section 177.4: Ending the process prematurely and returning an exit code ....................................................... 727
Chapter 178: ChemPy - python package ...................................................................................................... 728
Section 178.1: Parsing formulae ............................................................................................................................... 728
Section 178.2: Balancing stoichiometry of a chemical reaction ........................................................................... 728
Section 178.3: Balancing reactions .......................................................................................................................... 728
Section 178.4: Chemical equilibria ............................................................................................................................ 729
Section 178.5: Ionic strength ..................................................................................................................................... 729
Section 178.6: Chemical kinetics (system of ordinary dierential equations) .................................................... 729
Chapter 179: pygame .............................................................................................................................................. 731
Section 179.1: Pygame's mixer module .................................................................................................................... 731
Section 179.2: Installing pygame .............................................................................................................................. 732
Chapter 180: Pyglet ................................................................................................................................................. 734
Section 180.1: Installation of Pyglet .......................................................................................................................... 734
Section 180.2: Hello World in Pyglet ........................................................................................................................ 734
Section 180.3: Playing Sound in Pyglet .................................................................................................................... 734
Section 180.4: Using Pyglet for OpenGL ................................................................................................................. 734
Section 180.5: Drawing Points Using Pyglet and OpenGL ..................................................................................... 734
Chapter 181: Audio .................................................................................................................................................... 736
Section 181.1: Working with WAV files ...................................................................................................................... 736
Section 181.2: Convert any soundfile with python and mpeg ............................................................................ 736
Section 181.3: Playing Windows' beeps .................................................................................................................... 736
Section 181.4: Audio With Pyglet .............................................................................................................................. 737
Chapter 182: pyaudio .............................................................................................................................................. 738
Section 182.1: Callback Mode Audio I/O .................................................................................................................. 738
Section 182.2: Blocking Mode Audio I/O ................................................................................................................. 739
Chapter 183: shelve .................................................................................................................................................. 741
Section 183.1: Creating a new Shelf .......................................................................................................................... 741
Section 183.2: Sample code for shelve .................................................................................................................... 742
Section 183.3: To summarize the interface (key is a string, data is an arbitrary object): .................................. 742
Section 183.4: Write-back ......................................................................................................................................... 742
Chapter 184: IoT Programming with Python and Raspberry PI ....................................................... 744
Section 184.1: Example - Temperature sensor ........................................................................................................ 744
Chapter 185: kivy - Cross-platform Python Framework for NUI Development ....................... 748
Section 185.1: First App .............................................................................................................................................. 748
Chapter 186: Pandas Transform: Preform operations on groups and concatenate the
results ............................................................................................................................................................................. 750
Section 186.1: Simple transform ............................................................................................................................... 750
Section 186.2: Multiple results per group ................................................................................................................ 751
Chapter 187: Similarities in syntax, Dierences in meaning: Python vs. JavaScript ............. 752
Section 187.1: `in` with lists ......................................................................................................................................... 752
Chapter 188: Call Python from C# ................................................................................................................... 753
Section 188.1: Python script to be called by C# application .................................................................................. 753
Section 188.2: C# code calling Python script .......................................................................................................... 753
Chapter 189: ctypes ................................................................................................................................................. 755
Section 189.1: ctypes arrays ..................................................................................................................................... 755
Section 189.2: Wrapping functions for ctypes ........................................................................................................ 755
Section 189.3: Basic usage ........................................................................................................................................ 756
Section 189.4: Common pitfalls ................................................................................................................................ 756
Section 189.5: Basic ctypes object ........................................................................................................................... 757
Section 189.6: Complex usage .................................................................................................................................. 758
Chapter 190: Writing extensions ........................................................................................................................ 760
Section 190.1: Hello World with C Extension ............................................................................................................ 760
Section 190.2: C Extension Using c++ and Boost .................................................................................................... 760
Section 190.3: Passing an open file to C Extensions .............................................................................................. 762
Chapter 191: Python Lex-Yacc ............................................................................................................................. 763
Section 191.1: Getting Started with PLY .................................................................................................................... 763
Section 191.2: The "Hello, World!" of PLY - A Simple Calculator ............................................................................ 763
Section 191.3: Part 1: Tokenizing Input with Lex ....................................................................................................... 765
Section 191.4: Part 2: Parsing Tokenized Input with Yacc ...................................................................................... 768
Chapter 192: Unit Testing ...................................................................................................................................... 772
Section 192.1: Test Setup and Teardown within a unittest.TestCase .................................................................... 772
Section 192.2: Asserting on Exceptions ................................................................................................................... 772
Section 192.3: Testing Exceptions ............................................................................................................................ 773
Section 192.4: Choosing Assertions Within Unittests ............................................................................................. 774
Section 192.5: Unit tests with pytest ........................................................................................................................ 775
Section 192.6: Mocking functions with unittest.mock.create_autospec ............................................................... 778
Chapter 193: py.test ................................................................................................................................................. 780
Section 193.1: Setting up py.test ............................................................................................................................... 780
Section 193.2: Intro to Test Fixtures ......................................................................................................................... 780
Section 193.3: Failing Tests ....................................................................................................................................... 783
Chapter 194: Profiling ............................................................................................................................................. 785
Section 194.1: %%timeit and %timeit in IPython ...................................................................................................... 785
Section 194.2: Using cProfile (Preferred Profiler) ................................................................................................... 785
Section 194.3: timeit() function ................................................................................................................................. 785
Section 194.4: timeit command line ......................................................................................................................... 786
Section 194.5: line_profiler in command line .......................................................................................................... 786
Chapter 195: Python speed of program ....................................................................................................... 788
Section 195.1: Deque operations .............................................................................................................................. 788
Section 195.2: Algorithmic Notations ....................................................................................................................... 788
Section 195.3: Notation .............................................................................................................................................. 789
Section 195.4: List operations ................................................................................................................................... 790
Section 195.5: Set operations ................................................................................................................................... 790
Chapter 196: Performance optimization ....................................................................................................... 792
Section 196.1: Code profiling ..................................................................................................................................... 792
Chapter 197: Security and Cryptography .................................................................................................... 794
Section 197.1: Secure Password Hashing ................................................................................................................. 794
Section 197.2: Calculating a Message Digest ......................................................................................................... 794
Section 197.3: Available Hashing Algorithms .......................................................................................................... 794
Section 197.4: File Hashing ....................................................................................................................................... 795
Section 197.5: Generating RSA signatures using pycrypto ................................................................................... 795
Section 197.6: Asymmetric RSA encryption using pycrypto ................................................................................. 796
Section 197.7: Symmetric encryption using pycrypto ............................................................................................ 797
Chapter 198: Secure Shell Connection in Python ...................................................................................... 798
Section 198.1: ssh connection ................................................................................................................................... 798
Chapter 199: Python Anti-Patterns .................................................................................................................. 799
Section 199.1: Overzealous except clause ............................................................................................................... 799
Section 199.2: Looking before you leap with processor-intensive function ........................................................ 799
Chapter 200: Common Pitfalls ........................................................................................................................... 802
Section 200.1: List multiplication and common references .................................................................................. 802
Section 200.2: Mutable default argument .............................................................................................................. 805
Section 200.3: Changing the sequence you are iterating over ............................................................................ 806
Section 200.4: Integer and String identity .............................................................................................................. 809
Section 200.5: Dictionaries are unordered ............................................................................................................. 810
Section 200.6: Variable leaking in list comprehensions and for loops ................................................................ 811
Section 200.7: Chaining of or operator ................................................................................................................... 811
Section 200.8: sys.argv[0] is the name of the file being executed ...................................................................... 812
Section 200.9: Accessing int literals' attributes ...................................................................................................... 812
Section 200.10: Global Interpreter Lock (GIL) and blocking threads ................................................................... 813
 Section 200.11: Multiple return .................................................................................................................................. 814
Section 200.12: Pythonic JSON keys ....................................................................................................................... 814
Chapter 201: Hidden Features ............................................................................................................................ 816
Section 201.1: Operator Overloading ....................................................................................................................... 816
Credits ............................................................................................................................................................................ 817
You may also like ...................................................................................................................................................... 831