An Informal Introduction to Python

In the following examples, input and output are distinguished by the presence or absence of prompts ( and ): to repeat the exampl
You can use the “Copy” button (it appears in the upper-right corner when hovering over or tapping a code example), which strips p
Many of the examples in this manual, even those entered at the interactive prompt, include comments. Comments in Python start
Some examples:
# this is the first comment
spam = 1 # and this is the second comment
# ... and now a third!
text = "# This is not a comment because it's inside quotes."
3.1. Using Python as a Calculator
Let’s try some simple Python commands. Start the interpreter and wait for the primary prompt, >>>. (It shouldn’t take long.)
3.1.1. Numbers
The interpreter acts as a simple calculator: you can type an expression at it and it will write the value. Expression syntax is straigh
2+2
4
50 - 5*6
20
(50 - 5*6) / 4
5.0
8 / 5 # division always returns a floating-point number
1.6
The integer numbers (e.g. 2, 4, 20) have type , the ones with a fractional part (e.g. 5.0, 1.6) have type . We will see more about nu
Division (/) always returns a float. To do and get an integer result you can use the // operator; to calculate the remainder you can u
17 / 3 # classic division returns a float
5.666666666666667
17 // 3 # floor division discards the fractional part
5
17 % 3 # the % operator returns the remainder of the division
2
5 * 3 + 2 # floored quotient * divisor + remainder
17
With Python, it is possible to use the ** operator to calculate powers :
5 ** 2 # 5 squared
25
2 ** 7 # 2 to the power of 7
128
The equal sign (=) is used to assign a value to a variable. Afterwards, no result is displayed before the next interactive prompt:
width = 20
height = 5 * 9
width * height
900
If a variable is not “defined” (assigned a value), trying to use it will give you an error:
n # try to access an undefined variable
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'n' is not defined
There is full support for floating point; operators with mixed type operands convert the integer operand to floating point:
4 * 3.75 - 1
14.0
In interactive mode, the last printed expression is assigned to the variable _. This means that when you are using Python as a des
tax = 12.5 / 100
price = 100.50
price * tax
12.5625
price + _
113.0625
round(_, 2)
113.06
This variable should be treated as read-only by the user. Don’t explicitly assign a value to it — you would create an independent lo
In addition to and , Python supports other types of numbers, such as and . Python also has built-in support for , and uses the j or
3.1.2. Text
Python can manipulate text (represented by type , so-called “strings”) as well as numbers. This includes characters “!”, words “rab
'spam eggs' # single quotes
'spam eggs'
"Paris rabbit got your back :)! Yay!" # double quotes
'Paris rabbit got your back :)! Yay!'
'1975' # digits and numerals enclosed in quotes are also strings
'1975'
To quote a quote, we need to “escape” it, by preceding it with \. Alternatively, we can use the other type of quotation marks:
'doesn\'t' # use \' to escape the single quote...
"doesn't"
"doesn't" # ...or use double quotes instead
"doesn't"
'"Yes," they said.'
'"Yes," they said.'
"\"Yes,\" they said."
'"Yes," they said.'
'"Isn\'t," they said.'
'"Isn\'t," they said.'
In the Python shell, the string definition and output string can look different. The function produces a more readable output, by om
s = 'First line.\nSecond line.' # \n means newline
s # without print(), special characters are included in the string
'First line.\nSecond line.'
print(s) # with print(), special characters are interpreted, so \n produces new line
First line.
Second line.
If you don’t want characters prefaced by \ to be interpreted as special characters, you can use raw strings by adding an r before th
print('C:\some\name') # here \n means newline!
C:\some
ame
print(r'C:\some\name') # note the r before the quote
C:\some\name
There is one subtle aspect to raw strings: a raw string may not end in an odd number of \ characters; see for more information and
String literals can span multiple lines. One way is using triple-quotes: """...""" or '''...'''. End-of-line characters are automatically inclu
print("""\
Usage: thingy [OPTIONS]
-h Display this usage message
-H hostname Hostname to connect to
""")
Usage: thingy [OPTIONS]
-h Display this usage message
-H hostname Hostname to connect to
Strings can be concatenated (glued together) with the + operator, and repeated with *:
# 3 times 'un', followed by 'ium'
3 * 'un' + 'ium'
'unununium'
Two or more string literals (i.e. the ones enclosed between quotes) next to each other are automatically concatenated.
'Py' 'thon'
'Python'
This feature is particularly useful when you want to break long strings:
text = ('Put several strings within parentheses '
'to have them joined together.')
text
'Put several strings within parentheses to have them joined together.'
This only works with two literals though, not with variables or expressions:
prefix = 'Py'
prefix 'thon' # can't concatenate a variable and a string literal
File "<stdin>", line 1
prefix 'thon'
^^^^^^
SyntaxError: invalid syntax
('un' * 3) 'ium'
File "<stdin>", line 1
('un' * 3) 'ium'
^^^^^
SyntaxError: invalid syntax
If you want to concatenate variables or a variable and a literal, use +:
prefix + 'thon'
'Python'
Strings can be indexed (subscripted), with the first character having index 0. There is no separate character type; a character is sim
word = 'Python'
word[0] # character in position 0
'P'
word[5] # character in position 5
'n'
Indices may also be negative numbers, to start counting from the right:
word[-1] # last character
'n'
word[-2] # second-last character
'o'
word[-6]
'P'
Note that since -0 is the same as 0, negative indices start from -1.
In addition to indexing, slicing is also supported. While indexing is used to obtain individual characters, slicing allows you to obtain
word[0:2] # characters from position 0 (included) to 2 (excluded)
'Py'
word[2:5] # characters from position 2 (included) to 5 (excluded)
'tho'
Slice indices have useful defaults; an omitted first index defaults to zero, an omitted second index defaults to the size of the string
word[:2] # character from the beginning to position 2 (excluded)
'Py'
word[4:] # characters from position 4 (included) to the end
'on'
word[-2:] # characters from the second-last (included) to the end
'on'
Note how the start is always included, and the end always excluded. This makes sure that s[:i] + s[i:] is always equal to s:
word[:2] + word[2:]
'Python'
word[:4] + word[4:]
'Python'
One way to remember how slices work is to think of the indices as pointing between characters, with the left edge of the first chara
+---+---+---+---+---+---+
|P|y|t|h|o|n|
+---+---+---+---+---+---+
0 1 2 3 4 5 6
-6 -5 -4 -3 -2 -1
The first row of numbers gives the position of the indices 0…6 in the string; the second row gives the corresponding negative indic
For non-negative indices, the length of a slice is the difference of the indices, if both are within bounds. For example, the length of
Attempting to use an index that is too large will result in an error:
word[42] # the word only has 6 characters
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: string index out of range
However, out of range slice indexes are handled gracefully when used for slicing:
word[4:42]
'on'
word[42:]
''
Python strings cannot be changed — they are . Therefore, assigning to an indexed position in the string results in an error:
word[0] = 'J'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'str' object does not support item assignment
word[2:] = 'py'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'str' object does not support item assignment
If you need a different string, you should create a new one:
'J' + word[1:]
'Jython'
word[:2] + 'py'
'Pypy'
The built-in function returns the length of a string:
s = 'supercalifragilisticexpialidocious'
len(s)
34
See also
Strings are examples of sequence types, and support the common operations supported by such types.
Strings support a large number of methods for basic transformations and searching.
String literals that have embedded expressions.
Information about string formatting with .
The old formatting operations invoked when strings are the left operand of the % operator are described in more detail here.
3.1.3. Lists
Python knows a number of compound data types, used to group together other values. The most versatile is the list, which can be
squares = [1, 4, 9, 16, 25]
squares
[1, 4, 9, 16, 25]
Like strings (and all other built-in types), lists can be indexed and sliced:
squares[0] # indexing returns the item
1
squares[-1]
25
squares[-3:] # slicing returns a new list
[9, 16, 25]
Lists also support operations like concatenation:
squares + [36, 49, 64, 81, 100]
[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
Unlike strings, which are , lists are a type, i.e. it is possible to change their content:
cubes = [1, 8, 27, 65, 125] # something's wrong here
4 ** 3 # the cube of 4 is 64, not 65!
64
cubes[3] = 64 # replace the wrong value
cubes
[1, 8, 27, 64, 125]
You can also add new items at the end of the list, by using the list.append() method (we will see more about methods later):
cubes.append(216) # add the cube of 6
cubes.append(7 ** 3) # and the cube of 7
cubes
[1, 8, 27, 64, 125, 216, 343]
Simple assignment in Python never copies data. When you assign a list to a variable, the variable refers to the existing list. Any ch
rgb = ["Red", "Green", "Blue"]
rgba = rgb
id(rgb) == id(rgba) # they reference the same object
True
rgba.append("Alph")
rgb
["Red", "Green", "Blue", "Alph"]
All slice operations return a new list containing the requested elements. This means that the following slice returns a of the list:
correct_rgba = rgba[:]
correct_rgba[-1] = "Alpha"
correct_rgba
["Red", "Green", "Blue", "Alpha"]
rgba
["Red", "Green", "Blue", "Alph"]
Assignment to slices is also possible, and this can even change the size of the list or clear it entirely:
letters = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
letters
['a', 'b', 'c', 'd', 'e', 'f', 'g']
# replace some values
letters[2:5] = ['C', 'D', 'E']
letters
['a', 'b', 'C', 'D', 'E', 'f', 'g']
# now remove them
letters[2:5] = []
letters
['a', 'b', 'f', 'g']
# clear the list by replacing all the elements with an empty list
letters[:] = []
letters
[]
The built-in function also applies to lists:
letters = ['a', 'b', 'c', 'd']
len(letters)
4
It is possible to nest lists (create lists containing other lists), for example:
a = ['a', 'b', 'c']
n = [1, 2, 3]
x = [a, n]
x
[['a', 'b', 'c'], [1, 2, 3]]
x[0]
['a', 'b', 'c']
x[0][1]
'b'
3.2. First Steps Towards Programming
Of course, we can use Python for more complicated tasks than adding two and two together. For instance, we can write an initial s
# Fibonacci series:
# the sum of two elements defines the next
a, b = 0, 1
while a < 10:
print(a)
a, b = b, a+b
0
1
1
2
3
5
8
This example introduces several new features.
The first line contains a multiple assignment: the variables a and b simultaneously get the new values 0 and 1. On the last line this
The loop executes as long as the condition (here: a < 10) remains true. In Python, like in C, any non-zero integer value is true; ze
The body of the loop is indented: indentation is Python’s way of grouping statements. At the interactive prompt, you have to type a
The function writes the value of the argument(s) it is given. It differs from just writing the expression you want to write (as we did e
i = 256*256
print('The value of i is', i)
The value of i is 65536
The keyword argument end can be used to avoid the newline after the output, or end the output with a different string:
a, b = 0, 1
while a < 1000:
 print(a, end=',')
a, b = b, a+b
0,1,1,2,3,5,8,13,21,34,55,89,144,233,377,610,987,
Footnotes
[]
Since ** has higher precedence than -, -3**2 will be interpreted as -(3**2) and thus result in -9. To avoid this and get 9, you can us
[]
Unlike other languages, special characters such as \n have the same meaning with both single ('...') and double ("...") quotes. The
dev (3.15)pre (3.14)3.13.53.123.113.103.93.83.73.63.53.43.33.23.13.02.72.6
Top of Form
Bottom of Form
Greek | Ελληνικ■EnglishSpanish | españolFrench | françaisItalian | italianoJapanese | ■■■Korean | ■■■Polish | polskiBrazilian
Theme Auto Light Dark
Previous topic
Next topic
This page
4. More Control Flow Tools
As well as the statement just introduced, Python uses a few more that we will encounter in this chapter.
4.1. if Statements
Perhaps the most well-known statement type is the statement. For example:
x = int(input("Please enter an integer: "))
Please enter an integer: 42
if x < 0:
x=0
print('Negative changed to zero')
elif x == 0:
print('Zero')
elif x == 1:
print('Single')
else:
print('More')
More
There can be zero or more parts, and the part is optional. The keyword ‘elif’ is short for ‘else if’, and is useful to avoid excessive i
If you’re comparing the same value to several constants, or checking for specific types or attributes, you may also find the match s
4.2. for Statements
The statement in Python differs a bit from what you may be used to in C or Pascal. Rather than always iterating over an arithmetic
# Measure some strings:
words = ['cat', 'window', 'defenestrate']
for w in words:
print(w, len(w))
cat 3
window 6
defenestrate 12
Code that modifies a collection while iterating over that same collection can be tricky to get right. Instead, it is usually more straigh
# Create a sample collection
users = {'Hans': 'active', 'Éléonore': 'inactive', '■■■': 'active'}
# Strategy: Iterate over a copy
for user, status in users.copy().items():
if status == 'inactive':
del users[user]
# Strategy: Create a new collection
active_users = {}
for user, status in users.items():
if status == 'active':
active_users[user] = status
4.3. The Function
If you do need to iterate over a sequence of numbers, the built-in function comes in handy. It generates arithmetic progressions:
for i in range(5):
print(i)
0
1
2
3
4
The given end point is never part of the generated sequence; range(10) generates 10 values, the legal indices for items of a seque
list(range(5, 10))
[5, 6, 7, 8, 9]
list(range(0, 10, 3))
[0, 3, 6, 9]
list(range(-10, -100, -30))
[-10, -40, -70]
To iterate over the indices of a sequence, you can combine and as follows:
a = ['Mary', 'had', 'a', 'little', 'lamb']
for i in range(len(a)):
print(i, a[i])
0 Mary
1 had
2a
3 little
4 lamb
In most such cases, however, it is convenient to use the function, see .
A strange thing happens if you just print a range:
range(10)
range(0, 10)
In many ways the object returned by behaves as if it is a list, but in fact it isn’t. It is an object which returns the successive items o
We say such an object is , that is, suitable as a target for functions and constructs that expect something from which they can obta
sum(range(4)) # 0 + 1 + 2 + 3
6
Later we will see more functions that return iterables and take iterables as arguments. In chapter , we will discuss in more detail ab
4.4. break and continue Statements
The statement breaks out of the innermost enclosing or loop:
for n in range(2, 10):
for x in range(2, n):
if n % x == 0:
print(f"{n} equals {x} * {n//x}")
break
4 equals 2 * 2
6 equals 2 * 3
8 equals 2 * 4
9 equals 3 * 3
The statement continues with the next iteration of the loop:
for num in range(2, 10):
if num % 2 == 0:
print(f"Found an even number {num}")
continue
print(f"Found an odd number {num}")
Found an even number 2
Found an odd number 3
Found an even number 4
Found an odd number 5
Found an even number 6
Found an odd number 7
Found an even number 8
Found an odd number 9
4.5. else Clauses on Loops
In a for or while loop the break statement may be paired with an else clause. If the loop finishes without executing the break, the e
In a loop, the else clause is executed after the loop finishes its final iteration, that is, if no break occurred.
In a loop, it’s executed after the loop’s condition becomes false.
In either kind of loop, the else clause is not executed if the loop was terminated by a . Of course, other ways of ending the loop ea
This is exemplified in the following for loop, which searches for prime numbers:
for n in range(2, 10):
for x in range(2, n):
if n % x == 0:
print(n, 'equals', x, '*', n//x)
break
else:
# loop fell through without finding a factor
print(n, 'is a prime number')
2 is a prime number
3 is a prime number
4 equals 2 * 2
5 is a prime number
6 equals 2 * 3
7 is a prime number
8 equals 2 * 4
9 equals 3 * 3
(Yes, this is the correct code. Look closely: the else clause belongs to the for loop, not the if statement.)
One way to think of the else clause is to imagine it paired with the if inside the loop. As the loop executes, it will run a sequence lik
When used with a loop, the else clause has more in common with the else clause of a statement than it does with that of if statem
4.6. pass Statements
The statement does nothing. It can be used when a statement is required syntactically but the program requires no action. For ex
while True:
pass # Busy-wait for keyboard interrupt (Ctrl+C)
This is commonly used for creating minimal classes:
class MyEmptyClass:
pass
Another place can be used is as a place-holder for a function or conditional body when you are working on new code, allowing yo
def initlog(*args):
pass # Remember to implement this!
4.7. match Statements
A statement takes an expression and compares its value to successive patterns given as one or more case blocks. This is superfi
The simplest form compares a subject value against one or more literals:
def http_error(status):
match status:
case 400:
return "Bad request"
case 404:
return "Not found"
case 418:
return "I'm a teapot"
case _:
return "Something's wrong with the internet"
Note the last block: the “variable name” _ acts as a wildcard and never fails to match. If no case matches, none of the branches is
You can combine several literals in a single pattern using | (“or”):
case 401 | 403 | 404:
return "Not allowed"
Patterns can look like unpacking assignments, and can be used to bind variables:
# point is an (x, y) tuple
match point:
case (0, 0):
print("Origin")
case (0, y):
print(f"Y={y}")
case (x, 0):
print(f"X={x}")
case (x, y):
print(f"X={x}, Y={y}")
case _:
raise ValueError("Not a point")
Study that one carefully! The first pattern has two literals, and can be thought of as an extension of the literal pattern shown above
If you are using classes to structure your data you can use the class name followed by an argument list resembling a constructor,
class Point:
def __init__(self, x, y):
self.x = x
self.y = y
def where_is(point):
match point:
case Point(x=0, y=0):
print("Origin")
case Point(x=0, y=y):
print(f"Y={y}")
case Point(x=x, y=0):
print(f"X={x}")
case Point():
print("Somewhere else")
case _:
print("Not a point")
You can use positional parameters with some builtin classes that provide an ordering for their attributes (e.g. dataclasses). You ca
Point(1, var)
Point(1, y=var)
Point(x=1, y=var)
Point(y=var, x=1)
A recommended way to read patterns is to look at them as an extended form of what you would put on the left of an assignment, t
Patterns can be arbitrarily nested. For example, if we have a short list of Points, with __match_args__ added, we could match it lik
class Point:
__match_args__ = ('x', 'y')
def __init__(self, x, y):
self.x = x
self.y = y
match points:
case []:
print("No points")
case [Point(0, 0)]:
print("The origin")
case [Point(x, y)]:
print(f"Single point {x}, {y}")
case [Point(0, y1), Point(0, y2)]:
print(f"Two on the Y axis at {y1}, {y2}")
case _:
print("Something else")
We can add an if clause to a pattern, known as a “guard”. If the guard is false, match goes on to try the next case block. Note that
match point:
case Point(x, y) if x == y:
print(f"Y=X at {x}")
case Point(x, y):
print(f"Not on the diagonal")
Several other key features of this statement:
Like unpacking assignments, tuple and list patterns have exactly the same meaning and actually match arbitrary sequences. An im
Sequence patterns support extended unpacking: [x, y, *rest] and (x, y, *rest) work similar to unpacking assignments. The name aft
Mapping patterns: {"bandwidth": b, "latency": l} captures the "bandwidth" and "latency" values from a dictionary. Unlike sequence p
Subpatterns may be captured using the as keyword:
case (Point(x1, y1), Point(x2, y2) as p2): ...
will capture the second element of the input as p2 (as long as the input is a sequence of two points)
Most literals are compared by equality, however the singletons True, False and None are compared by identity.
Patterns may use named constants. These must be dotted names to prevent them from being interpreted as capture variable:
from enum import Enum
class Color(Enum):
RED = 'red'
GREEN = 'green'
BLUE = 'blue'
color = Color(input("Enter your choice of 'red', 'blue' or 'green': "))
match color:
case Color.RED:
print("I see red!")
case Color.GREEN:
print("Grass is green")
case Color.BLUE:
print("I'm feeling the blues :(")
For a more detailed explanation and additional examples, you can look into which is written in a tutorial format.
4.8. Defining Functions
We can create a function that writes the Fibonacci series to an arbitrary boundary:
def fib(n): # write Fibonacci series less than n
"""Print a Fibonacci series less than n."""
a, b = 0, 1
while a < n:
print(a, end=' ')
a, b = b, a+b
print()
# Now call the function we just defined:
fib(2000)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597
The keyword introduces a function definition. It must be followed by the function name and the parenthesized list of formal parame
The first statement of the function body can optionally be a string literal; this string literal is the function’s documentation string, or
The execution of a function introduces a new symbol table used for the local variables of the function. More precisely, all variable a
The actual parameters (arguments) to a function call are introduced in the local symbol table of the called function when it is called
A function definition associates the function name with the function object in the current symbol table. The interpreter recognizes th
fib
<function fib at 10042ed0>
f = fib
f(100)
0 1 1 2 3 5 8 13 21 34 55 89
Coming from other languages, you might object that fib is not a function but a procedure since it doesn’t return a value. In fact, eve
fib(0)
print(fib(0))
None
It is simple to write a function that returns a list of the numbers of the Fibonacci series, instead of printing it:
def fib2(n): # return Fibonacci series up to n
"""Return a list containing the Fibonacci series up to n."""
result = []
a, b = 0, 1
while a < n:
result.append(a) # see below
a, b = b, a+b
return result
f100 = fib2(100) # call it
f100 # write the result
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
This example, as usual, demonstrates some new Python features:
The statement returns with a value from a function. return without an expression argument returns None. Falling off the end of a fu
The statement result.append(a) calls a method of the list object result. A method is a function that ‘belongs’ to an object and is nam
4.9. More on Defining Functions
It is also possible to define functions with a variable number of arguments. There are three forms, which can be combined.
4.9.1. Default Argument Values
The most useful form is to specify a default value for one or more arguments. This creates a function that can be called with fewer
def ask_ok(prompt, retries=4, reminder='Please try again!'):
while True:
reply = input(prompt)
if reply in {'y', 'ye', 'yes'}:
return True
if reply in {'n', 'no', 'nop', 'nope'}:
 return False
retries = retries - 1
if retries < 0:
raise ValueError('invalid user response')
print(reminder)
This function can be called in several ways:
giving only the mandatory argument: ask_ok('Do you really want to quit?')
giving one of the optional arguments: ask_ok('OK to overwrite the file?', 2)
or even giving all arguments: ask_ok('OK to overwrite the file?', 2, 'Come on, only yes or no!')
This example also introduces the keyword. This tests whether or not a sequence contains a certain value.
The default values are evaluated at the point of function definition in the defining scope, so that
i=5
def f(arg=i):
print(arg)
i=6
f()
will print 5.
Important warning: The default value is evaluated only once. This makes a difference when the default is a mutable object such as
def f(a, L=[]):
L.append(a)
return L
print(f(1))
print(f(2))
print(f(3))
This will print
[1]
[1, 2]
[1, 2, 3]
If you don’t want the default to be shared between subsequent calls, you can write the function like this instead:
def f(a, L=None):
if L is None:
L = []
L.append(a)
return L
4.9.2. Keyword Arguments
Functions can also be called using of the form kwarg=value. For instance, the following function:
def parrot(voltage, state='a stiff', action='voom', type='Norwegian Blue'):
print("-- This parrot wouldn't", action, end=' ')
print("if you put", voltage, "volts through it.")
print("-- Lovely plumage, the", type)
print("-- It's", state, "!")
accepts one required argument (voltage) and three optional arguments (state, action, and type). This function can be called in any
parrot(1000) # 1 positional argument
parrot(voltage=1000) # 1 keyword argument
parrot(voltage=1000000, action='VOOOOOM') # 2 keyword arguments
parrot(action='VOOOOOM', voltage=1000000) # 2 keyword arguments
parrot('a million', 'bereft of life', 'jump') # 3 positional arguments
parrot('a thousand', state='pushing up the daisies') # 1 positional, 1 keyword
but all the following calls would be invalid:
parrot() # required argument missing
parrot(voltage=5.0, 'dead') # non-keyword argument after a keyword argument
parrot(110, voltage=220) # duplicate value for the same argument
parrot(actor='John Cleese') # unknown keyword argument
In a function call, keyword arguments must follow positional arguments. All the keyword arguments passed must match one of the
def function(a):
pass
function(0, a=0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: function() got multiple values for argument 'a'
When a final formal parameter of the form **name is present, it receives a dictionary (see ) containing all keyword arguments exce
def cheeseshop(kind, *arguments, **keywords):
print("-- Do you have any", kind, "?")
print("-- I'm sorry, we're all out of", kind)
for arg in arguments:
print(arg)
print("-" * 40)
for kw in keywords:
print(kw, ":", keywords[kw])
It could be called like this:
cheeseshop("Limburger", "It's very runny, sir.",
"It's really very, VERY runny, sir.",
shopkeeper="Michael Palin",
client="John Cleese",
sketch="Cheese Shop Sketch")
and of course it would print:
-- Do you have any Limburger ?
-- I'm sorry, we're all out of Limburger
It's very runny, sir.
It's really very, VERY runny, sir.
----------------------------------------
shopkeeper : Michael Palin
client : John Cleese
sketch : Cheese Shop Sketch
Note that the order in which the keyword arguments are printed is guaranteed to match the order in which they were provided in th
4.9.3. Special parameters
By default, arguments may be passed to a Python function either by position or explicitly by keyword. For readability and performa
A function definition may look like:
def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):
----------- ---------- ----------
| | |
| Positional or keyword |
| - Keyword only
-- Positional only
where / and * are optional. If used, these symbols indicate the kind of parameter by how the arguments may be passed to the func
4.9.3.1. Positional-or-Keyword Arguments
If / and * are not present in the function definition, arguments may be passed to a function by position or by keyword.
4.9.3.2. Positional-Only Parameters
Looking at this in a bit more detail, it is possible to mark certain parameters as positional-only. If positional-only, the parameters’ o
Parameters following the / may be positional-or-keyword or keyword-only.
4.9.3.3. Keyword-Only Arguments
To mark parameters as keyword-only, indicating the parameters must be passed by keyword argument, place an * in the argumen
4.9.3.4. Function Examples
Consider the following example function definitions paying close attention to the markers / and *:
def standard_arg(arg):
print(arg)
def pos_only_arg(arg, /):
print(arg)
def kwd_only_arg(*, arg):
print(arg)
def combined_example(pos_only, /, standard, *, kwd_only):
print(pos_only, standard, kwd_only)
The first function definition, standard_arg, the most familiar form, places no restrictions on the calling convention and arguments m
standard_arg(2)
2
standard_arg(arg=2)
2
The second function pos_only_arg is restricted to only use positional parameters as there is a / in the function definition:
pos_only_arg(1)
1
pos_only_arg(arg=1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: pos_only_arg() got some positional-only arguments passed as keyword arguments: 'arg'
The third function kwd_only_arg only allows keyword arguments as indicated by a * in the function definition:
kwd_only_arg(3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: kwd_only_arg() takes 0 positional arguments but 1 was given
kwd_only_arg(arg=3)
3
And the last uses all three calling conventions in the same function definition:
combined_example(1, 2, 3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: combined_example() takes 2 positional arguments but 3 were given
combined_example(1, 2, kwd_only=3)
123
combined_example(1, standard=2, kwd_only=3)
123
combined_example(pos_only=1, standard=2, kwd_only=3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: combined_example() got some positional-only arguments passed as keyword arguments: 'pos_only'
Finally, consider this function definition which has a potential collision between the positional argument name and **kwds which ha
def foo(name, **kwds):
return 'name' in kwds
There is no possible call that will make it return True as the keyword 'name' will always bind to the first parameter. For example:
foo(1, **{'name': 2})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() got multiple values for argument 'name'
But using / (positional only arguments), it is possible since it allows name as a positional argument and 'name' as a key in the keyw
def foo(name, /, **kwds):
return 'name' in kwds
foo(1, **{'name': 2})
True
In other words, the names of positional-only parameters can be used in **kwds without ambiguity.
4.9.3.5. Recap
The use case will determine which parameters to use in the function definition:
def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):
As guidance:
Use positional-only if you want the name of the parameters to not be available to the user. This is useful when parameter names h
Use keyword-only when names have meaning and the function definition is more understandable by being explicit with names or y
For an API, use positional-only to prevent breaking API changes if the parameter’s name is modified in the future.
4.9.4. Arbitrary Argument Lists
Finally, the least frequently used option is to specify that a function can be called with an arbitrary number of arguments. These ar
def write_multiple_items(file, separator, *args):
file.write(separator.join(args))
Normally, these variadic arguments will be last in the list of formal parameters, because they scoop up all remaining input argumen
def concat(*args, sep="/"):
return sep.join(args)
concat("earth", "mars", "venus")
'earth/mars/venus'
concat("earth", "mars", "venus", sep=".")
'earth.mars.venus'
4.9.5. Unpacking Argument Lists
The reverse situation occurs when the arguments are already in a list or tuple but need to be unpacked for a function call requiring
list(range(3, 6)) # normal call with separate arguments
[3, 4, 5]
args = [3, 6]
list(range(*args)) # call with arguments unpacked from a list
[3, 4, 5]
In the same fashion, dictionaries can deliver keyword arguments with the **-operator:
def parrot(voltage, state='a stiff', action='voom'):
print("-- This parrot wouldn't", action, end=' ')
print("if you put", voltage, "volts through it.", end=' ')
print("E's", state, "!")
d = {"voltage": "four million", "state": "bleedin' demised", "action": "VOOM"}
parrot(**d)
-- This parrot wouldn't VOOM if you put four million volts through it. E's bleedin' demised !
4.9.6. Lambda Expressions
Small anonymous functions can be created with the keyword. This function returns the sum of its two arguments: lambda a, b: a+
def make_incrementor(n):
return lambda x: x + n
f = make_incrementor(42)
f(0)
42
f(1)
43
The above example uses a lambda expression to return a function. Another use is to pass a small function as an argument. For in
pairs = [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')]
pairs.sort(key=lambda pair: pair[1])
pairs
[(4, 'four'), (1, 'one'), (3, 'three'), (2, 'two')]
4.9.7. Documentation Strings
Here are some conventions about the content and formatting of documentation strings.
The first line should always be a short, concise summary of the object’s purpose. For brevity, it should not explicitly state the objec
If there are more lines in the documentation string, the second line should be blank, visually separating the summary from the rest
The Python parser does not strip indentation from multi-line string literals in Python, so tools that process documentation have to s
Here is an example of a multi-line docstring:
def my_function():
"""Do nothing, but document it.
No, really, it doesn't do anything.
"""
pass
print(my_function.__doc__)
Do nothing, but document it.
No, really, it doesn't do anything.
4.9.8. Function Annotations
are completely optional metadata information about the types used by user-defined functions (see and for more information).
are stored in the __annotations__ attribute of the function as a dictionary and have no effect on any other part of the function. Par
def f(ham: str, eggs: str = 'eggs') -> str:
print("Annotations:", f.__annotations__)
print("Arguments:", ham, eggs)
return ham + ' and ' + eggs
f('spam')
Annotations: {'ham': <class 'str'>, 'return': <class 'str'>, 'eggs': <class 'str'>}
Arguments: spam eggs
'spam and eggs'
4.10. Intermezzo: Coding Style
Now that you are about to write longer, more complex pieces of Python, it is a good time to talk about coding style. Most language
For Python, has emerged as the style guide that most projects adhere to; it promotes a very readable and eye-pleasing coding sty
Use 4-space indentation, and no tabs.
4 spaces are a good compromise between small indentation (allows greater nesting depth) and large indentation (easier to read).
Wrap lines so that they don’t exceed 79 characters.
This helps users with small displays and makes it possible to have several code files side-by-side on larger displays.
Use blank lines to separate functions and classes, and larger blocks of code inside functions.
When possible, put comments on a line of their own.
Use docstrings.
Use spaces around operators and after commas, but not directly inside bracketing constructs: a = f(1, 2) + g(3, 4).
Name your classes and functions consistently; the convention is to use UpperCamelCase for classes and lowercase_with_unders
Don’t use fancy encodings if your code is meant to be used in international environments. Python’s default, UTF-8, or even plain A
Likewise, don’t use non-ASCII characters in identifiers if there is only the slightest chance people speaking a different language wi
Footnotes
[]
Actually, call by object reference would be a better description, since if a mutable object is passed, the caller will see any changes
Data Structures
This chapter describes some things you’ve learned about already in more detail, and adds some new things as well.
5.1. More on Lists
The list data type has some more methods. Here are all of the methods of list objects:
list.append(x)
Add an item to the end of the list. Similar to a[len(a):] = [x].
list.extend(iterable)
Extend the list by appending all the items from the iterable. Similar to a[len(a):] = iterable.
list.insert(i, x)
Insert an item at a given position. The first argument is the index of the element before which to insert, so a.insert(0, x) inserts at th
list.remove(x)
Remove the first item from the list whose value is equal to x. It raises a if there is no such item.
list.pop([i])
Remove the item at the given position in the list, and return it. If no index is specified, a.pop() removes and returns the last item in
list.clear()
Remove all items from the list. Similar to del a[:].
list.index(x[, start[, end]])
Return zero-based index in the list of the first item whose value is equal to x. Raises a if there is no such item.
The optional arguments start and end are interpreted as in the slice notation and are used to limit the search to a particular subseq
list.count(x)
Return the number of times x appears in the list.
list.sort(*, key=None, reverse=False)
Sort the items of the list in place (the arguments can be used for sort customization, see for their explanation).
list.reverse()
Reverse the elements of the list in place.
list.copy()
Return a shallow copy of the list. Similar to a[:].
An example that uses most of the list methods:
fruits = ['orange', 'apple', 'pear', 'banana', 'kiwi', 'apple', 'banana']
fruits.count('apple')
2
fruits.count('tangerine')
0
fruits.index('banana')
3
fruits.index('banana', 4) # Find next banana starting at position 4
6
fruits.reverse()
fruits
['banana', 'apple', 'kiwi', 'banana', 'pear', 'apple', 'orange']
fruits.append('grape')
fruits
['banana', 'apple', 'kiwi', 'banana', 'pear', 'apple', 'orange', 'grape']
fruits.sort()
fruits
['apple', 'apple', 'banana', 'banana', 'grape', 'kiwi', 'orange', 'pear']
fruits.pop()
'pear'
You might have noticed that methods like insert, remove or sort that only modify the list have no return value printed – they return
Another thing you might notice is that not all data can be sorted or compared. For instance, [None, 'hello', 10] doesn’t sort because
5.1.1. Using Lists as Stacks
The list methods make it very easy to use a list as a stack, where the last element added is the first element retrieved (“last-in, firs
stack = [3, 4, 5]
stack.append(6)
stack.append(7)
stack
[3, 4, 5, 6, 7]
stack.pop()
7
stack
[3, 4, 5, 6]
stack.pop()
6
stack.pop()
5
stack
[3, 4]
5.1.2. Using Lists as Queues
It is also possible to use a list as a queue, where the first element added is the first element retrieved (“first-in, first-out”); however,
To implement a queue, use which was designed to have fast appends and pops from both ends. For example:
from collections import deque
queue = deque(["Eric", "John", "Michael"])
queue.append("Terry") # Terry arrives
queue.append("Graham") # Graham arrives
queue.popleft() # The first to arrive now leaves
'Eric'
queue.popleft() # The second to arrive now leaves
'John'
queue # Remaining queue in order of arrival
deque(['Michael', 'Terry', 'Graham'])
5.1.3. List Comprehensions
List comprehensions provide a concise way to create lists. Common applications are to make new lists where each element is the
For example, assume we want to create a list of squares, like:
squares = []
for x in range(10):
squares.append(x**2)
squares
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
Note that this creates (or overwrites) a variable named x that still exists after the loop completes. We can calculate the list of squar
squares = list(map(lambda x: x**2, range(10)))
or, equivalently:
squares = [x**2 for x in range(10)]
which is more concise and readable.
A list comprehension consists of brackets containing an expression followed by a for clause, then zero or more for or if clauses. Th
[(x, y) for x in [1,2,3] for y in [3,1,4] if x != y]
[(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
and it’s equivalent to:
combs = []
for x in [1,2,3]:
for y in [3,1,4]:
if x != y:
combs.append((x, y))
combs
[(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]
Note how the order of the and statements is the same in both these snippets.
If the expression is a tuple (e.g. the (x, y) in the previous example), it must be parenthesized.
vec = [-4, -2, 0, 2, 4]
# create a new list with the values doubled
[x*2 for x in vec]
[-8, -4, 0, 4, 8]
# filter the list to exclude negative numbers
[x for x in vec if x >= 0]
[0, 2, 4]
# apply a function to all the elements
[abs(x) for x in vec]
[4, 2, 0, 2, 4]
# call a method on each element
freshfruit = [' banana', ' loganberry ', 'passion fruit ']
[weapon.strip() for weapon in freshfruit]
['banana', 'loganberry', 'passion fruit']
# create a list of 2-tuples like (number, square)
[(x, x**2) for x in range(6)]
[(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25)]
# the tuple must be parenthesized, otherwise an error is raised
[x, x**2 for x in range(6)]
File "<stdin>", line 1
[x, x**2 for x in range(6)]
^^^^^^^
SyntaxError: did you forget parentheses around the comprehension target?
# flatten a list using a listcomp with two 'for'
vec = [[1,2,3], [4,5,6], [7,8,9]]
[num for elem in vec for num in elem]
[1, 2, 3, 4, 5, 6, 7, 8, 9]
List comprehensions can contain complex expressions and nested functions:
from math import pi
[str(round(pi, i)) for i in range(1, 6)]
['3.1', '3.14', '3.142', '3.1416', '3.14159']
5.1.4. Nested List Comprehensions
The initial expression in a list comprehension can be any arbitrary expression, including another list comprehension.
Consider the following example of a 3x4 matrix implemented as a list of 3 lists of length 4:
matrix = [
[1, 2, 3, 4],
[5, 6, 7, 8],
[9, 10, 11, 12],
]
The following list comprehension will transpose rows and columns:
[[row[i] for row in matrix] for i in range(4)]
[[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
As we saw in the previous section, the inner list comprehension is evaluated in the context of the that follows it, so this example is
transposed = []
for i in range(4):
transposed.append([row[i] for row in matrix])
transposed
[[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
which, in turn, is the same as:
transposed = []
for i in range(4):
# the following 3 lines implement the nested listcomp
transposed_row = []
for row in matrix:
transposed_row.append(row[i])
transposed.append(transposed_row)
transposed
[[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]
In the real world, you should prefer built-in functions to complex flow statements. The function would do a great job for this use ca
list(zip(*matrix))
[(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)]
See for details on the asterisk in this line.
5.2. The del statement
There is a way to remove an item from a list given its index instead of its value: the statement. This differs from the pop() method
a = [-1, 1, 66.25, 333, 333, 1234.5]
del a[0]
a
[1, 66.25, 333, 333, 1234.5]
del a[2:4]
a
[1, 66.25, 1234.5]
del a[:]
a
[]
can also be used to delete entire variables:
del a
Referencing the name a hereafter is an error (at least until another value is assigned to it). We’ll find other uses for later.
5.3. Tuples and Sequences
We saw that lists and strings have many common properties, such as indexing and slicing operations. They are two examples of s
A tuple consists of a number of values separated by commas, for instance:
t = 12345, 54321, 'hello!'
t[0]
12345
t
(12345, 54321, 'hello!')
# Tuples may be nested:
u = t, (1, 2, 3, 4, 5)
u
((12345, 54321, 'hello!'), (1, 2, 3, 4, 5))
# Tuples are immutable:
t[0] = 88888
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
# but they can contain mutable objects:
v = ([1, 2, 3], [3, 2, 1])
v
([1, 2, 3], [3, 2, 1])
As you see, on output tuples are always enclosed in parentheses, so that nested tuples are interpreted correctly; they may be inpu
Though tuples may seem similar to lists, they are often used in different situations and for different purposes. Tuples are , and usu
A special problem is the construction of tuples containing 0 or 1 items: the syntax has some extra quirks to accommodate these. E
empty = ()
singleton = 'hello', # <-- note trailing comma
len(empty)
0
len(singleton)
1
singleton
('hello',)
The statement t = 12345, 54321, 'hello!' is an example of tuple packing: the values 12345, 54321 and 'hello!' are packed together
x, y, z = t
This is called, appropriately enough, sequence unpacking and works for any sequence on the right-hand side. Sequence unpackin
5.4. Sets
Python also includes a data type for sets. A set is an unordered collection with no duplicate elements. Basic uses include member
Curly braces or the function can be used to create sets. Note: to create an empty set you have to use set(), not {}; the latter create
Here is a brief demonstration:
basket = {'apple', 'orange', 'apple', 'pear', 'orange', 'banana'}
print(basket) # show that duplicates have been removed
{'orange', 'banana', 'pear', 'apple'}
'orange' in basket # fast membership testing
True
'crabgrass' in basket
False
# Demonstrate set operations on unique letters from two words
a = set('abracadabra')
b = set('alacazam')
a # unique letters in a
{'a', 'r', 'b', 'c', 'd'}
a-b # letters in a but not in b
{'r', 'd', 'b'}
a|b # letters in a or b or both
{'a', 'c', 'r', 'd', 'b', 'm', 'z', 'l'}
a&b # letters in both a and b
{'a', 'c'}
a^b # letters in a or b but not both
{'r', 'd', 'b', 'm', 'z', 'l'}
Similarly to , set comprehensions are also supported:
a = {x for x in 'abracadabra' if x not in 'abc'}
a
{'r', 'd'}
5.5. Dictionaries
Another useful data type built into Python is the dictionary (see ). Dictionaries are sometimes found in other languages as “associa
It is best to think of a dictionary as a set of key: value pairs, with the requirement that the keys are unique (within one dictionary). A
The main operations on a dictionary are storing a value with some key and extracting the value given the key. It is also possible to
Performing list(d) on a dictionary returns a list of all the keys used in the dictionary, in insertion order (if you want it sorted, just use
Here is a small example using a dictionary:
tel = {'jack': 4098, 'sape': 4139}
tel['guido'] = 4127
tel
{'jack': 4098, 'sape': 4139, 'guido': 4127}
tel['jack']
4098
del tel['sape']
tel['irv'] = 4127
tel
{'jack': 4098, 'guido': 4127, 'irv': 4127}
list(tel)
['jack', 'guido', 'irv']
sorted(tel)
['guido', 'irv', 'jack']
'guido' in tel
True
'jack' not in tel
False
The constructor builds dictionaries directly from sequences of key-value pairs:
dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])
{'sape': 4139, 'guido': 4127, 'jack': 4098}
In addition, dict comprehensions can be used to create dictionaries from arbitrary key and value expressions:
{x: x**2 for x in (2, 4, 6)}
{2: 4, 4: 16, 6: 36}
When the keys are simple strings, it is sometimes easier to specify pairs using keyword arguments:
dict(sape=4139, guido=4127, jack=4098)
{'sape': 4139, 'guido': 4127, 'jack': 4098}
5.6. Looping Techniques
When looping through dictionaries, the key and corresponding value can be retrieved at the same time using the method.
knights = {'gallahad': 'the pure', 'robin': 'the brave'}
for k, v in knights.items():
print(k, v)
gallahad the pure
robin the brave
When looping through a sequence, the position index and corresponding value can be retrieved at the same time using the functio
for i, v in enumerate(['tic', 'tac', 'toe']):
print(i, v)
0 tic
1 tac
2 toe
To loop over two or more sequences at the same time, the entries can be paired with the function.
questions = ['name', 'quest', 'favorite color']
answers = ['lancelot', 'the holy grail', 'blue']
for q, a in zip(questions, answers):
print('What is your {0}? It is {1}.'.format(q, a))
What is your name? It is lancelot.
What is your quest? It is the holy grail.
What is your favorite color? It is blue.
To loop over a sequence in reverse, first specify the sequence in a forward direction and then call the function.
for i in reversed(range(1, 10, 2)):
print(i)
9
7
5
3
1
To loop over a sequence in sorted order, use the function which returns a new sorted list while leaving the source unaltered.
basket = ['apple', 'orange', 'apple', 'pear', 'orange', 'banana']
for i in sorted(basket):
print(i)
apple
apple
banana
orange
orange
pear
Using on a sequence eliminates duplicate elements. The use of in combination with over a sequence is an idiomatic way to loop
basket = ['apple', 'orange', 'apple', 'pear', 'orange', 'banana']
for f in sorted(set(basket)):
print(f)
apple
banana
orange
pear
It is sometimes tempting to change a list while you are looping over it; however, it is often simpler and safer to create a new list ins
import math
raw_data = [56.2, float('NaN'), 51.7, 55.3, 52.5, float('NaN'), 47.8]
filtered_data = []
for value in raw_data:
if not math.isnan(value):
filtered_data.append(value)
filtered_data
[56.2, 51.7, 55.3, 52.5, 47.8]
5.7. More on Conditions
The conditions used in while and if statements can contain any operators, not just comparisons.
The comparison operators in and not in are membership tests that determine whether a value is in (or not in) a container. The ope
Comparisons can be chained. For example, a < b == c tests whether a is less than b and moreover b equals c.
Comparisons may be combined using the Boolean operators and and or, and the outcome of a comparison (or of any other Boolea
The Boolean operators and and or are so-called short-circuit operators: their arguments are evaluated from left to right, and evalua
It is possible to assign the result of a comparison or other Boolean expression to a variable. For example,
string1, string2, string3 = '', 'Trondheim', 'Hammer Dance'
non_null = string1 or string2 or string3
non_null
'Trondheim'
Note that in Python, unlike C, assignment inside expressions must be done explicitly with the :=. This avoids a common class of p
5.8. Comparing Sequences and Other Types
Sequence objects typically may be compared to other objects with the same sequence type. The comparison uses lexicographical
(1, 2, 3) < (1, 2, 4)
[1, 2, 3] < [1, 2, 4]
'ABC' < 'C' < 'Pascal' < 'Python'
(1, 2, 3, 4) < (1, 2, 4)
(1, 2) < (1, 2, -1)
(1, 2, 3) == (1.0, 2.0, 3.0)
(1, 2, ('aa', 'ab')) < (1, 2, ('abc', 'a'), 4)
Note that comparing objects of different types with < or > is legal provided that the objects have appropriate comparison methods.
Footnotes
[]
Other languages may return the mutated object, which allows method chaining, such as d->insert("a")->remove("b")->sort();.
Modules
If you quit from the Python interpreter and enter it again, the definitions you have made (functions and variables) are lost. Therefor
To support this, Python has a way to put definitions in a file and use them in a script or in an interactive instance of the interpreter.
A module is a file containing Python definitions and statements. The file name is the module name with the suffix .py appended. W
# Fibonacci numbers module
def fib(n):
"""Write Fibonacci series up to n."""
a, b = 0, 1
while a < n:
print(a, end=' ')
a, b = b, a+b
print()
def fib2(n):
"""Return Fibonacci series up to n."""
result = []
a, b = 0, 1
while a < n:
result.append(a)
a, b = b, a+b
return result
Now enter the Python interpreter and import this module with the following command:
import fibo
This does not add the names of the functions defined in fibo directly to the current (see for more details); it only adds the module
fibo.fib(1000)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
fibo.fib2(100)
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
fibo.__name__
'fibo'
If you intend to use a function often you can assign it to a local name:
fib = fibo.fib
fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
6.1. More on Modules
A module can contain executable statements as well as function definitions. These statements are intended to initialize the module
Each module has its own private namespace, which is used as the global namespace by all functions defined in the module. Thus
Modules can import other modules. It is customary but not required to place all statements at the beginning of a module (or script,
There is a variant of the statement that imports names from a module directly into the importing module’s namespace. For examp
from fibo import fib, fib2
fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
This does not introduce the module name from which the imports are taken in the local namespace (so in the example, fibo is not
There is even a variant to import all names that a module defines:
from fibo import *
fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
This imports all names except those beginning with an underscore (_). In most cases Python programmers do not use this facility
Note that in general the practice of importing * from a module or package is frowned upon, since it often causes poorly readable co
If the module name is followed by as, then the name following as is bound directly to the imported module.
import fibo as fib
fib.fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
This is effectively importing the module in the same way that import fibo will do, with the only difference of it being available as fib.
It can also be used when utilising with similar effects:
from fibo import fib as fibonacci
fibonacci(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
Note
For efficiency reasons, each module is only imported once per interpreter session. Therefore, if you change your modules, you mu
6.1.1. Executing modules as scripts
When you run a Python module with
python fibo.py <arguments>
the code in the module will be executed, just as if you imported it, but with the __name__ set to "__main__". That means that by a
if __name__ == "__main__":
import sys
fib(int(sys.argv[1]))
you can make the file usable as a script as well as an importable module, because the code that parses the command line only run
python fibo.py 50
0 1 1 2 3 5 8 13 21 34
If the module is imported, the code is not run:
import fibo
This is often used either to provide a convenient user interface to a module, or for testing purposes (running the module as a scrip
6.1.2. The Module Search Path
When a module named spam is imported, the interpreter first searches for a built-in module with that name. These module names
The directory containing the input script (or the current directory when no file is specified).
(a list of directory names, with the same syntax as the shell variable PATH).
The installation-dependent default (by convention including a site-packages directory, handled by the module).
More details are at .
Note
On file systems which support symlinks, the directory containing the input script is calculated after the symlink is followed. In other
After initialization, Python programs can modify . The directory containing the script being run is placed at the beginning of the sea
6.1.3. “Compiled” Python files
To speed up loading modules, Python caches the compiled version of each module in the __pycache__ directory under the name
Python checks the modification date of the source against the compiled version to see if it’s out of date and needs to be recompile
Python does not check the cache in two circumstances. First, it always recompiles and does not store the result for the module tha
Some tips for experts:
You can use the or switches on the Python command to reduce the size of a compiled module. The -O switch removes assert sta
A program doesn’t run any faster when it is read from a .pyc file than when it is read from a .py file; the only thing that’s faster abou
The module can create .pyc files for all modules in a directory.
There is more detail on this process, including a flow chart of the decisions, in .
6.2. Standard Modules
Python comes with a library of standard modules, described in a separate document, the Python Library Reference (“Library Refer
import sys
sys.ps1
'>>> '
sys.ps2
'... '
sys.ps1 = 'C> '
C> print('Yuck!')
Yuck!
C>
These two variables are only defined if the interpreter is in interactive mode.
The variable sys.path is a list of strings that determines the interpreter’s search path for modules. It is initialized to a default path ta
import sys
sys.path.append('/ufs/guido/lib/python')
6.3. The Function
The built-in function is used to find out which names a module defines. It returns a sorted list of strings:
import fibo, sys
dir(fibo)
['__name__', 'fib', 'fib2']
dir(sys)
['__breakpointhook__', '__displayhook__', '__doc__', '__excepthook__',
'__interactivehook__', '__loader__', '__name__', '__package__', '__spec__',
 '__stderr__', '__stdin__', '__stdout__', '__unraisablehook__',
'_clear_type_cache', '_current_frames', '_debugmallocstats', '_framework',
'_getframe', '_git', '_home', '_xoptions', 'abiflags', 'addaudithook',
'api_version', 'argv', 'audit', 'base_exec_prefix', 'base_prefix',
'breakpointhook', 'builtin_module_names', 'byteorder', 'call_tracing',
'callstats', 'copyright', 'displayhook', 'dont_write_bytecode', 'exc_info',
'excepthook', 'exec_prefix', 'executable', 'exit', 'flags', 'float_info',
'float_repr_style', 'get_asyncgen_hooks', 'get_coroutine_origin_tracking_depth',
'getallocatedblocks', 'getdefaultencoding', 'getdlopenflags',
'getfilesystemencodeerrors', 'getfilesystemencoding', 'getprofile',
'getrecursionlimit', 'getrefcount', 'getsizeof', 'getswitchinterval',
'gettrace', 'hash_info', 'hexversion', 'implementation', 'int_info',
'intern', 'is_finalizing', 'last_traceback', 'last_type', 'last_value',
'maxsize', 'maxunicode', 'meta_path', 'modules', 'path', 'path_hooks',
'path_importer_cache', 'platform', 'prefix', 'ps1', 'ps2', 'pycache_prefix',
'set_asyncgen_hooks', 'set_coroutine_origin_tracking_depth', 'setdlopenflags',
'setprofile', 'setrecursionlimit', 'setswitchinterval', 'settrace', 'stderr',
'stdin', 'stdout', 'thread_info', 'unraisablehook', 'version', 'version_info',
'warnoptions']
Without arguments, lists the names you have defined currently:
a = [1, 2, 3, 4, 5]
import fibo
fib = fibo.fib
dir()
['__builtins__', '__name__', 'a', 'fib', 'fibo', 'sys']
Note that it lists all types of names: variables, modules, functions, etc.
does not list the names of built-in functions and variables. If you want a list of those, they are defined in the standard module :
import builtins
dir(builtins)
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException',
'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning',
'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError',
'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning',
'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False',
'FileExistsError', 'FileNotFoundError', 'FloatingPointError',
'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError',
'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError',
'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError',
'MemoryError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented',
'NotImplementedError', 'OSError', 'OverflowError',
'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError',
'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning',
'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError',
'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError',
'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError',
'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning',
'ValueError', 'Warning', 'ZeroDivisionError', '_', '__build_class__',
'__debug__', '__doc__', '__import__', '__name__', '__package__', 'abs',
'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable',
'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits',
'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit',
'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr',
'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass',
'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview',
'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property',
'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice',
'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars',
'zip']
6.4. Packages
Packages are a way of structuring Python’s module namespace by using “dotted module names”. For example, the module name
Suppose you want to design a collection of modules (a “package”) for the uniform handling of sound files and sound data. There a
sound/ Top-level package
__init__.py Initialize the sound package
formats/ Subpackage for file format conversions
__init__.py
wavread.py
wavwrite.py
aiffread.py
aiffwrite.py
auread.py
auwrite.py
...
effects/ Subpackage for sound effects
__init__.py
echo.py
surround.py
reverse.py
...
filters/ Subpackage for filters
__init__.py
equalizer.py
vocoder.py
karaoke.py
...
When importing the package, Python searches through the directories on sys.path looking for the package subdirectory.
The __init__.py files are required to make Python treat directories containing the file as packages (unless using a , a relatively adv
Users of the package can import individual modules from the package, for example:
import sound.effects.echo
This loads the submodule sound.effects.echo. It must be referenced with its full name.
sound.effects.echo.echofilter(input, output, delay=0.7, atten=4)
An alternative way of importing the submodule is:
from sound.effects import echo
This also loads the submodule echo, and makes it available without its package prefix, so it can be used as follows:
echo.echofilter(input, output, delay=0.7, atten=4)
Yet another variation is to import the desired function or variable directly:
from sound.effects.echo import echofilter
Again, this loads the submodule echo, but this makes its function echofilter() directly available:
echofilter(input, output, delay=0.7, atten=4)
Note that when using from package import item, the item can be either a submodule (or subpackage) of the package, or some oth
Contrarily, when using syntax like import item.subitem.subsubitem, each item except for the last must be a package; the last item
6.4.1. Importing * From a Package
Now what happens when the user writes from sound.effects import *? Ideally, one would hope that this somehow goes out to the f
The only solution is for the package author to provide an explicit index of the package. The statement uses the following conventio
__all__ = ["echo", "surround", "reverse"]
This would mean that from sound.effects import * would import the three named submodules of the sound.effects package.
Be aware that submodules might become shadowed by locally defined names. For example, if you added a reverse function to the
__all__ = [
"echo", # refers to the 'echo.py' file
"surround", # refers to the 'surround.py' file
"reverse", # !!! refers to the 'reverse' function now !!!
]
def reverse(msg: str): # <-- this name shadows the 'reverse.py' submodule
return msg[::-1] # in the case of a 'from sound.effects import *'
If __all__ is not defined, the statement from sound.effects import * does not import all submodules from the package sound.effects
import sound.effects.echo
import sound.effects.surround
from sound.effects import *
In this example, the echo and surround modules are imported in the current namespace because they are defined in the sound.eff
Although certain modules are designed to export only names that follow certain patterns when you use import *, it is still considere
Remember, there is nothing wrong with using from package import specific_submodule! In fact, this is the recommended notation
6.4.2. Intra-package References
When packages are structured into subpackages (as with the sound package in the example), you can use absolute imports to ref
You can also write relative imports, with the from module import name form of import statement. These imports use leading dots to
from . import echo
from .. import formats
from ..filters import equalizer
Note that relative imports are based on the name of the current module’s package. Since the main module does not have a packa
6.4.3. Packages in Multiple Directories
Packages support one more special attribute, . This is initialized to be a of strings containing the name of the directory holding the
While this feature is not often needed, it can be used to extend the set of modules found in a package.
Footnotes
[]
In fact function definitions are also ‘statements’ that are ‘executed’; the execution of a module-level function definition adds the fun
Input and Output
There are several ways to present the output of a program; data can be printed in a human-readable form, or written to a file for fu
7.1. Fancier Output Formatting
So far we’ve encountered two ways of writing values: expression statements and the function. (A third way is using the method o
Often you’ll want more control over the formatting of your output than simply printing space-separated values. There are several w
To use , begin a string with f or F before the opening quotation mark or triple quotation mark. Inside this string, you can write a Pyt
year = 2016
event = 'Referendum'
f'Results of the {year} {event}'
'Results of the 2016 Referendum'
The method of strings requires more manual effort. You’ll still use { and } to mark where a variable will be substituted and can pro
yes_votes = 42_572_654
total_votes = 85_705_149
percentage = yes_votes / total_votes
'{:-9} YES votes {:2.2%}'.format(yes_votes, percentage)
' 42572654 YES votes 49.67%'
Notice how the yes_votes are padded with spaces and a negative sign only for negative numbers. The example also prints percen
Finally, you can do all the string handling yourself by using string slicing and concatenation operations to create any layout you ca
When you don’t need fancy output but just want a quick display of some variables for debugging purposes, you can convert any va
The function is meant to return representations of values which are fairly human-readable, while is meant to generate representa
Some examples:
s = 'Hello, world.'
str(s)
'Hello, world.'
repr(s)
"'Hello, world.'"
str(1/7)
'0.14285714285714285'
x = 10 * 3.25
y = 200 * 200
s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...'
print(s)
The value of x is 32.5, and y is 40000...
# The repr() of a string adds string quotes and backslashes:
hello = 'hello, world\n'
hellos = repr(hello)
print(hellos)
'hello, world\n'
# The argument to repr() may be any Python object:
repr((x, y, ('spam', 'eggs')))
"(32.5, 40000, ('spam', 'eggs'))"
The module contains a class that offers yet another way to substitute values into strings, using placeholders like $x and replacing
7.1.1. Formatted String Literals
(also called f-strings for short) let you include the value of Python expressions inside a string by prefixing the string with f or F and
An optional format specifier can follow the expression. This allows greater control over how the value is formatted. The following e
import math
print(f'The value of pi is approximately {math.pi:.3f}.')
The value of pi is approximately 3.142.
Passing an integer after the ':' will cause that field to be a minimum number of characters wide. This is useful for making columns l
table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678}
for name, phone in table.items():
print(f'{name:10} ==> {phone:10d}')
Sjoerd ==> 4127
Jack ==> 4098
Dcab ==> 7678
Other modifiers can be used to convert the value before it is formatted. '!a' applies , '!s' applies , and '!r' applies :
animals = 'eels'
print(f'My hovercraft is full of {animals}.')
My hovercraft is full of eels.
print(f'My hovercraft is full of {animals!r}.')
My hovercraft is full of 'eels'.
The = specifier can be used to expand an expression to the text of the expression, an equal sign, then the representation of the ev
bugs = 'roaches'
count = 13
area = 'living room'
print(f'Debugging {bugs=} {count=} {area=}')
Debugging bugs='roaches' count=13 area='living room'
See for more information on the = specifier. For a reference on these format specifications, see the reference guide for the .
7.1.2. The String format() Method
Basic usage of the method looks like this:
print('We are the {} who say "{}!"'.format('knights', 'Ni'))
We are the knights who say "Ni!"
The brackets and characters within them (called format fields) are replaced with the objects passed into the method. A number in
print('{0} and {1}'.format('spam', 'eggs'))
spam and eggs
print('{1} and {0}'.format('spam', 'eggs'))
eggs and spam
If keyword arguments are used in the method, their values are referred to by using the name of the argument.
print('This {food} is {adjective}.'.format(
food='spam', adjective='absolutely horrible'))
This spam is absolutely horrible.
Positional and keyword arguments can be arbitrarily combined:
print('The story of {0}, {1}, and {other}.'.format('Bill', 'Manfred',
other='Georg'))
The story of Bill, Manfred, and Georg.
If you have a really long format string that you don’t want to split up, it would be nice if you could reference the variables to be form
table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
print('Jack: {0[Jack]:d}; Sjoerd: {0[Sjoerd]:d}; '
'Dcab: {0[Dcab]:d}'.format(table))
Jack: 4098; Sjoerd: 4127; Dcab: 8637678
This could also be done by passing the table dictionary as keyword arguments with the ** notation.
table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
print('Jack: {Jack:d}; Sjoerd: {Sjoerd:d}; Dcab: {Dcab:d}'.format(**table))
Jack: 4098; Sjoerd: 4127; Dcab: 8637678
This is particularly useful in combination with the built-in function , which returns a dictionary containing all local variables:
table = {k: str(v) for k, v in vars().items()}
message = " ".join([f'{k}: ' + '{' + k +'};' for k in table.keys()])
print(message.format(**table))
__name__: __main__; __doc__: None; __package__: None; __loader__: ...
As an example, the following lines produce a tidily aligned set of columns giving integers and their squares and cubes:
for x in range(1, 11):
print('{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x))
1 1 1
2 4 8
3 9 27
4 16 64
5 25 125
 6 36 216
7 49 343
8 64 512
9 81 729
10 100 1000
For a complete overview of string formatting with , see .
7.1.3. Manual String Formatting
Here’s the same table of squares and cubes, formatted manually:
for x in range(1, 11):
print(repr(x).rjust(2), repr(x*x).rjust(3), end=' ')
# Note use of 'end' on previous line
print(repr(x*x*x).rjust(4))
1 1 1
2 4 8
3 9 27
4 16 64
5 25 125
6 36 216
7 49 343
8 64 512
9 81 729
10 100 1000
(Note that the one space between each column was added by the way works: it always adds spaces between its arguments.)
The method of string objects right-justifies a string in a field of a given width by padding it with spaces on the left. There are simila
There is another method, , which pads a numeric string on the left with zeros. It understands about plus and minus signs:
'12'.zfill(5)
'00012'
'-3.14'.zfill(7)
'-003.14'
'3.14159265359'.zfill(5)
'3.14159265359'
7.1.4. Old string formatting
The % operator (modulo) can also be used for string formatting. Given format % values (where format is a string), % conversion sp
import math
print('The value of pi is approximately %5.3f.' % math.pi)
The value of pi is approximately 3.142.
More information can be found in the section.
7.2. Reading and Writing Files
returns a , and is most commonly used with two positional arguments and one keyword argument: open(filename, mode, encodin
f = open('workfile', 'w', encoding="utf-8")
The first argument is a string containing the filename. The second argument is another string containing a few characters describin
Normally, files are opened in text mode, that means, you read and write strings from and to the file, which are encoded in a specifi
In text mode, the default when reading is to convert platform-specific line endings (\n on Unix, \r\n on Windows) to just \n. When w
It is good practice to use the keyword when dealing with file objects. The advantage is that the file is properly closed after its suite
with open('workfile', encoding="utf-8") as f:
read_data = f.read()
# We can check that the file has been automatically closed.
f.closed
True
If you’re not using the keyword, then you should call f.close() to close the file and immediately free up any system resources used
Warning
Calling f.write() without using the with keyword or calling f.close() might result in the arguments of f.write() not being completely wr
After a file object is closed, either by a statement or by calling f.close(), attempts to use the file object will automatically fail.
f.close()
f.read()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: I/O operation on closed file.
7.2.1. Methods of File Objects
The rest of the examples in this section will assume that a file object called f has already been created.
To read a file’s contents, call f.read(size), which reads some quantity of data and returns it as a string (in text mode) or bytes objec
f.read()
'This is the entire file.\n'
f.read()
''
f.readline() reads a single line from the file; a newline character (\n) is left at the end of the string, and is only omitted on the last lin
f.readline()
'This is the first line of the file.\n'
f.readline()
'Second line of the file\n'
f.readline()
''
For reading lines from a file, you can loop over the file object. This is memory efficient, fast, and leads to simple code:
for line in f:
print(line, end='')
This is the first line of the file.
Second line of the file
If you want to read all the lines of a file in a list you can also use list(f) or f.readlines().
f.write(string) writes the contents of string to the file, returning the number of characters written.
f.write('This is a test\n')
15
Other types of objects need to be converted – either to a string (in text mode) or a bytes object (in binary mode) – before writing th
value = ('the answer', 42)
s = str(value) # convert the tuple to string
f.write(s)
18
f.tell() returns an integer giving the file object’s current position in the file represented as number of bytes from the beginning of the
To change the file object’s position, use f.seek(offset, whence). The position is computed from adding offset to a reference point; t
f = open('workfile', 'rb+')
f.write(b'0123456789abcdef')
16
f.seek(5) # Go to the 6th byte in the file
5
f.read(1)
b'5'
f.seek(-3, 2) # Go to the 3rd byte before the end
13
f.read(1)
b'd'
In text files (those opened without a b in the mode string), only seeks relative to the beginning of the file are allowed (the exception
File objects have some additional methods, such as and which are less frequently used; consult the Library Reference for a comp
7.2.2. Saving structured data with
Strings can easily be written to and read from a file. Numbers take a bit more effort, since the method only returns strings, which w
Rather than having users constantly writing and debugging code to save complicated data types to files, Python allows you to use
Note
The JSON format is commonly used by modern applications to allow for data exchange. Many programmers are already familiar w
If you have an object x, you can view its JSON string representation with a simple line of code:
import json
x = [1, 'simple', 'list']
json.dumps(x)
'[1, "simple", "list"]'
Another variant of the function, called , simply serializes the object to a . So if f is a object opened for writing, we can do this:
json.dump(x, f)
To decode the object again, if f is a or object which has been opened for reading:
x = json.load(f)
Note
JSON files must be encoded in UTF-8. Use encoding="utf-8" when opening JSON file as a for both of reading and writing.
This simple serialization technique can handle lists and dictionaries, but serializing arbitrary class instances in JSON requires a bit
See also
- the pickle module
Contrary to , pickle is a protocol which allows the serialization of arbitrarily complex Python objects. As such, it is specific to Python
Errors and Exceptions
Until now error messages haven’t been more than mentioned, but if you have tried out the examples you have probably seen som
8.1. Syntax Errors
Syntax errors, also known as parsing errors, are perhaps the most common kind of complaint you get while you are still learning P
while True print('Hello world')
File "<stdin>", line 1
while True print('Hello world')
^^^^^
SyntaxError: invalid syntax
The parser repeats the offending line and displays little arrows pointing at the place where the error was detected. Note that this is
The file name (<stdin> in our example) and line number are printed so you know where to look in case the input came from a file.
8.2. Exceptions
Even if a statement or expression is syntactically correct, it may cause an error when an attempt is made to execute it. Errors dete
10 * (1/0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
10 * (1/0)
~^~
ZeroDivisionError: division by zero
4 + spam*3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
4 + spam*3
^^^^
NameError: name 'spam' is not defined
'2' + 2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
'2' + 2
~~~~^~~
TypeError: can only concatenate str (not "int") to str
The last line of the error message indicates what happened. Exceptions come in different types, and the type is printed as part of t
The rest of the line provides detail based on the type of exception and what caused it.
The preceding part of the error message shows the context where the exception occurred, in the form of a stack traceback. In gen
lists the built-in exceptions and their meanings.
8.3. Handling Exceptions
It is possible to write programs that handle selected exceptions. Look at the following example, which asks the user for input until a
while True:
try:
x = int(input("Please enter a number: "))
break
except ValueError:
print("Oops! That was no valid number. Try again...")
The statement works as follows.
First, the try clause (the statement(s) between the and keywords) is executed.
If no exception occurs, the except clause is skipped and execution of the statement is finished.
If an exception occurs during execution of the clause, the rest of the clause is skipped. Then, if its type matches the exception nam
If an exception occurs which does not match the exception named in the except clause, it is passed on to outer statements; if no h
A statement may have more than one except clause, to specify handlers for different exceptions. At most one handler will be exec
... except (RuntimeError, TypeError, NameError):
... pass
A class in an clause matches exceptions which are instances of the class itself or one of its derived classes (but not the other way
class B(Exception):
pass
class C(B):
pass
class D(C):
pass
for cls in [B, C, D]:
 try:
raise cls()
except D:
print("D")
except C:
print("C")
except B:
print("B")
Note that if the except clauses were reversed (with except B first), it would have printed B, B, B — the first matching except clause
When an exception occurs, it may have associated values, also known as the exception’s arguments. The presence and types of t
The except clause may specify a variable after the exception name. The variable is bound to the exception instance which typically
try:
raise Exception('spam', 'eggs')
except Exception as inst:
print(type(inst)) # the exception type
print(inst.args) # arguments stored in .args
print(inst) # __str__ allows args to be printed directly,
# but may be overridden in exception subclasses
x, y = inst.args # unpack args
print('x =', x)
print('y =', y)
<class 'Exception'>
('spam', 'eggs')
('spam', 'eggs')
x = spam
y = eggs
The exception’s output is printed as the last part (‘detail’) of the message for unhandled exceptions.
is the common base class of all exceptions. One of its subclasses, , is the base class of all the non-fatal exceptions. Exceptions w
can be used as a wildcard that catches (almost) everything. However, it is good practice to be as specific as possible with the type
The most common pattern for handling is to print or log the exception and then re-raise it (allowing a caller to handle the exception
import sys
try:
f = open('myfile.txt')
s = f.readline()
i = int(s.strip())
except OSError as err:
print("OS error:", err)
except ValueError:
print("Could not convert data to an integer.")
except Exception as err:
print(f"Unexpected {err=}, {type(err)=}")
raise
The … statement has an optional else clause, which, when present, must follow all except clauses. It is useful for code that must
for arg in sys.argv[1:]:
try:
f = open(arg, 'r')
except OSError:
print('cannot open', arg)
else:
print(arg, 'has', len(f.readlines()), 'lines')
f.close()
The use of the else clause is better than adding additional code to the clause because it avoids accidentally catching an exception
Exception handlers do not handle only exceptions that occur immediately in the try clause, but also those that occur inside function
def this_fails():
x = 1/0
try:
this_fails()
except ZeroDivisionError as err:
print('Handling run-time error:', err)
Handling run-time error: division by zero
8.4. Raising Exceptions
The statement allows the programmer to force a specified exception to occur. For example:
raise NameError('HiThere')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
raise NameError('HiThere')
NameError: HiThere
The sole argument to indicates the exception to be raised. This must be either an exception instance or an exception class (a clas
raise ValueError # shorthand for 'raise ValueError()'
If you need to determine whether an exception was raised but don’t intend to handle it, a simpler form of the statement allows you
try:
raise NameError('HiThere')
except NameError:
print('An exception flew by!')
raise
An exception flew by!
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
raise NameError('HiThere')
NameError: HiThere
8.5. Exception Chaining
If an unhandled exception occurs inside an section, it will have the exception being handled attached to it and included in the erro
try:
open("database.sqlite")
except OSError:
raise RuntimeError("unable to handle error")
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
open("database.sqlite")
~~~~^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'database.sqlite'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
raise RuntimeError("unable to handle error")
RuntimeError: unable to handle error
To indicate that an exception is a direct consequence of another, the statement allows an optional clause:
# exc must be exception instance or None.
raise RuntimeError from exc
This can be useful when you are transforming exceptions. For example:
def func():
raise ConnectionError
try:
func()
except ConnectionError as exc:
raise RuntimeError('Failed to open database') from exc
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
func()
~~~~^^
File "<stdin>", line 2, in func
ConnectionError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
raise RuntimeError('Failed to open database') from exc
RuntimeError: Failed to open database
It also allows disabling automatic exception chaining using the from None idiom:
try:
open('database.sqlite')
except OSError:
raise RuntimeError from None
Traceback (most recent call last):
File "<stdin>", line 4, in <module>
raise RuntimeError from None
RuntimeError
For more information about chaining mechanics, see .
8.6. User-defined Exceptions
Programs may name their own exceptions by creating a new exception class (see for more about Python classes). Exceptions sh
Exception classes can be defined which do anything any other class can do, but are usually kept simple, often only offering a num
Most exceptions are defined with names that end in “Error”, similar to the naming of the standard exceptions.
Many standard modules define their own exceptions to report errors that may occur in functions they define.
8.7. Defining Clean-up Actions
The statement has another optional clause which is intended to define clean-up actions that must be executed under all circumsta
try:
raise KeyboardInterrupt
finally:
print('Goodbye, world!')
Goodbye, world!
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
raise KeyboardInterrupt
KeyboardInterrupt
If a clause is present, the finally clause will execute as the last task before the statement completes. The finally clause runs whet
If an exception occurs during execution of the try clause, the exception may be handled by an clause. If the exception is not hand
An exception could occur during execution of an except or else clause. Again, the exception is re-raised after the finally clause has
If the finally clause executes a , or statement, exceptions are not re-raised.
If the try statement reaches a , or statement, the finally clause will execute just prior to the break, continue or return statement’s e
If a finally clause includes a return statement, the returned value will be the one from the finally clause’s return statement, not the v
For example:
def bool_return():
try:
return True
finally:
return False
bool_return()
False
A more complicated example:
def divide(x, y):
try:
result = x / y
except ZeroDivisionError:
print("division by zero!")
else:
print("result is", result)
finally:
print("executing finally clause")
divide(2, 1)
result is 2.0
executing finally clause
divide(2, 0)
division by zero!
executing finally clause
divide("2", "1")
executing finally clause
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
divide("2", "1")
~~~~~~^^^^^^^^^^
File "<stdin>", line 3, in divide
 result = x / y
~~^~~
TypeError: unsupported operand type(s) for /: 'str' and 'str'
As you can see, the clause is executed in any event. The raised by dividing two strings is not handled by the clause and therefor
In real world applications, the clause is useful for releasing external resources (such as files or network connections), regardless o
8.8. Predefined Clean-up Actions
Some objects define standard clean-up actions to be undertaken when the object is no longer needed, regardless of whether or no
for line in open("myfile.txt"):
print(line, end="")
The problem with this code is that it leaves the file open for an indeterminate amount of time after this part of the code has finished
with open("myfile.txt") as f:
for line in f:
print(line, end="")
After the statement is executed, the file f is always closed, even if a problem was encountered while processing the lines. Objects
8.9. Raising and Handling Multiple Unrelated Exceptions
There are situations where it is necessary to report several exceptions that have occurred. This is often the case in concurrency fr
The builtin wraps a list of exception instances so that they can be raised together. It is an exception itself, so it can be caught like
def f():
excs = [OSError('error 1'), SystemError('error 2')]
raise ExceptionGroup('there were problems', excs)
f()
+ Exception Group Traceback (most recent call last):
| File "<stdin>", line 1, in <module>
| f()
| ~^^
| File "<stdin>", line 3, in f
| raise ExceptionGroup('there were problems', excs)
| ExceptionGroup: there were problems (2 sub-exceptions)
+-+---------------- 1 ----------------
| OSError: error 1
+---------------- 2 ----------------
| SystemError: error 2
+------------------------------------
try:
f()
except Exception as e:
print(f'caught {type(e)}: e')
caught <class 'ExceptionGroup'>: e
By using except* instead of except, we can selectively handle only the exceptions in the group that match a certain type. In the foll
def f():
raise ExceptionGroup(
"group1",
[
OSError(1),
SystemError(2),
ExceptionGroup(
"group2",
[
OSError(3),
RecursionError(4)
]
)
]
)
try:
f()
except* OSError as e:
print("There were OSErrors")
except* SystemError as e:
print("There were SystemErrors")
There were OSErrors
There were SystemErrors
+ Exception Group Traceback (most recent call last):
| File "<stdin>", line 2, in <module>
| f()
| ~^^
| File "<stdin>", line 2, in f
| raise ExceptionGroup(
| ...<12 lines>...
| )
| ExceptionGroup: group1 (1 sub-exception)
+-+---------------- 1 ----------------
| ExceptionGroup: group2 (1 sub-exception)
+-+---------------- 1 ----------------
| RecursionError: 4
+------------------------------------
Note that the exceptions nested in an exception group must be instances, not types. This is because in practice the exceptions wo
excs = []
for test in tests:
try:
test.run()
except Exception as e:
excs.append(e)
if excs:
raise ExceptionGroup("Test Failures", excs)
8.10. Enriching Exceptions with Notes
When an exception is created in order to be raised, it is usually initialized with information that describes the error that has occurre
try:
raise TypeError('bad type')
except Exception as e:
e.add_note('Add some information')
e.add_note('Add some more information')
raise
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
raise TypeError('bad type')
TypeError: bad type
Add some information
Add some more information
For example, when collecting exceptions into an exception group, we may want to add context information for the individual errors
def f():
raise OSError('operation failed')
excs = []
for i in range(3):
try:
f()
except Exception as e:
e.add_note(f'Happened in Iteration {i+1}')
excs.append(e)
raise ExceptionGroup('We have some problems', excs)
+ Exception Group Traceback (most recent call last):
| File "<stdin>", line 1, in <module>
| raise ExceptionGroup('We have some problems', excs)
| ExceptionGroup: We have some problems (3 sub-exceptions)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "<stdin>", line 3, in <module>
| f()
| ~^^
| File "<stdin>", line 2, in f
 | raise OSError('operation failed')
| OSError: operation failed
| Happened in Iteration 1
+---------------- 2 ----------------
| Traceback (most recent call last):
| File "<stdin>", line 3, in <module>
| f()
| ~^^
| File "<stdin>", line 2, in f
| raise OSError('operation failed')
| OSError: operation failed
| Happened in Iteration 2
+---------------- 3 ----------------
| Traceback (most recent call last):
| File "<stdin>", line 3, in <module>
| f()
| ~^^
| File "<stdin>", line 2, in f
| raise OSError('operation failed')
| OSError: operation failed
| Happened in Iteration 3
+------------------------------------
Classes
Classes provide a means of bundling data and functionality together. Creating a new class creates a new type of object, allowing n
Compared with other programming languages, Python’s class mechanism adds classes with a minimum of new syntax and seman
In C++ terminology, normally class members (including the data members) are public (except see below ), and all member functio
(Lacking universally accepted terminology to talk about classes, I will make occasional use of Smalltalk and C++ terms. I would us
9.1. A Word About Names and Objects
Objects have individuality, and multiple names (in multiple scopes) can be bound to the same object. This is known as aliasing in o
9.2. Python Scopes and Namespaces
Before introducing classes, I first have to tell you something about Python’s scope rules. Class definitions play some neat tricks wi
Let’s begin with some definitions.
A namespace is a mapping from names to objects. Most namespaces are currently implemented as Python dictionaries, but that’s
By the way, I use the word attribute for any name following a dot — for example, in the expression z.real, real is an attribute of the
Attributes may be read-only or writable. In the latter case, assignment to attributes is possible. Module attributes are writable: you
Namespaces are created at different moments and have different lifetimes. The namespace containing the built-in names is create
The local namespace for a function is created when the function is called, and deleted when the function returns or raises an exce
A scope is a textual region of a Python program where a namespace is directly accessible. “Directly accessible” here means that a
Although scopes are determined statically, they are used dynamically. At any time during execution, there are 3 or 4 nested scope
the innermost scope, which is searched first, contains the local names
the scopes of any enclosing functions, which are searched starting with the nearest enclosing scope, contain non-local, but also no
the next-to-last scope contains the current module’s global names
the outermost scope (searched last) is the namespace containing built-in names
If a name is declared global, then all references and assignments go directly to the next-to-last scope containing the module’s glob
Usually, the local scope references the local names of the (textually) current function. Outside functions, the local scope reference
It is important to realize that scopes are determined textually: the global scope of a function defined in a module is that module’s n
A special quirk of Python is that – if no or statement is in effect – assignments to names always go into the innermost scope. Ass
The statement can be used to indicate that particular variables live in the global scope and should be rebound there; the stateme
9.2.1. Scopes and Namespaces Example
This is an example demonstrating how to reference the different scopes and namespaces, and how and affect variable binding:
def scope_test():
def do_local():
spam = "local spam"
def do_nonlocal():
nonlocal spam
spam = "nonlocal spam"
def do_global():
global spam
spam = "global spam"
spam = "test spam"
 do_local()
print("After local assignment:", spam)
do_nonlocal()
print("After nonlocal assignment:", spam)
do_global()
print("After global assignment:", spam)
scope_test()
print("In global scope:", spam)
The output of the example code is:
After local assignment: test spam
After nonlocal assignment: nonlocal spam
After global assignment: nonlocal spam
In global scope: global spam
Note how the local assignment (which is default) didn’t change scope_test's binding of spam. The assignment changed scope_tes
You can also see that there was no previous binding for spam before the assignment.
9.3. A First Look at Classes
Classes introduce a little bit of new syntax, three new object types, and some new semantics.
9.3.1. Class Definition Syntax
The simplest form of class definition looks like this:
class ClassName:
<statement-1>
.
.
.
<statement-N>
Class definitions, like function definitions ( statements) must be executed before they have any effect. (You could conceivably plac
In practice, the statements inside a class definition will usually be function definitions, but other statements are allowed, and some
When a class definition is entered, a new namespace is created, and used as the local scope — thus, all assignments to local vari
When a class definition is left normally (via the end), a class object is created. This is basically a wrapper around the contents of th
9.3.2. Class Objects
Class objects support two kinds of operations: attribute references and instantiation.
Attribute references use the standard syntax used for all attribute references in Python: obj.name. Valid attribute names are all the
class MyClass:
"""A simple example class"""
i = 12345
def f(self):
return 'hello world'
then MyClass.i and MyClass.f are valid attribute references, returning an integer and a function object, respectively. Class attribute
Class instantiation uses function notation. Just pretend that the class object is a parameterless function that returns a new instanc
x = MyClass()
creates a new instance of the class and assigns this object to the local variable x.
The instantiation operation (“calling” a class object) creates an empty object. Many classes like to create objects with instances cu
def __init__(self):
self.data = []
When a class defines an method, class instantiation automatically invokes __init__() for the newly created class instance. So in th
x = MyClass()
Of course, the method may have arguments for greater flexibility. In that case, arguments given to the class instantiation operator
class Complex:
def __init__(self, realpart, imagpart):
self.r = realpart
self.i = imagpart
x = Complex(3.0, -4.5)
x.r, x.i
(3.0, -4.5)
9.3.3. Instance Objects
Now what can we do with instance objects? The only operations understood by instance objects are attribute references. There ar
data attributes correspond to “instance variables” in Smalltalk, and to “data members” in C++. Data attributes need not be declared
x.counter = 1
while x.counter < 10:
x.counter = x.counter * 2
print(x.counter)
del x.counter
The other kind of instance attribute reference is a method. A method is a function that “belongs to” an object.
Valid method names of an instance object depend on its class. By definition, all attributes of a class that are function objects define
9.3.4. Method Objects
Usually, a method is called right after it is bound:
x.f()
In the MyClass example, this will return the string 'hello world'. However, it is not necessary to call a method right away: x.f is a me
xf = x.f
while True:
print(xf())
will continue to print hello world until the end of time.
What exactly happens when a method is called? You may have noticed that x.f() was called without an argument above, even thou
Actually, you may have guessed the answer: the special thing about methods is that the instance object is passed as the first argu
In general, methods work as follows. When a non-data attribute of an instance is referenced, the instance’s class is searched. If th
9.3.5. Class and Instance Variables
Generally speaking, instance variables are for data unique to each instance and class variables are for attributes and methods sha
class Dog:
kind = 'canine' # class variable shared by all instances
def __init__(self, name):
self.name = name # instance variable unique to each instance
>>> d = Dog('Fido')
>>> e = Dog('Buddy')
>>> d.kind # shared by all dogs
'canine'
>>> e.kind # shared by all dogs
'canine'
>>> d.name # unique to d
'Fido'
>>> e.name # unique to e
'Buddy'
As discussed in , shared data can have possibly surprising effects with involving objects such as lists and dictionaries. For examp
class Dog:
tricks = [] # mistaken use of a class variable
def __init__(self, name):
self.name = name
def add_trick(self, trick):
self.tricks.append(trick)
>>> d = Dog('Fido')
>>> e = Dog('Buddy')
>>> d.add_trick('roll over')
>>> e.add_trick('play dead')
>>> d.tricks # unexpectedly shared by all dogs
['roll over', 'play dead']
Correct design of the class should use an instance variable instead:
class Dog:
def __init__(self, name):
self.name = name
self.tricks = [] # creates a new empty list for each dog
def add_trick(self, trick):
self.tricks.append(trick)
>>> d = Dog('Fido')
>>> e = Dog('Buddy')
>>> d.add_trick('roll over')
>>> e.add_trick('play dead')
>>> d.tricks
['roll over']
>>> e.tricks
['play dead']
9.4. Random Remarks
If the same attribute name occurs in both an instance and in a class, then attribute lookup prioritizes the instance:
class Warehouse:
purpose = 'storage'
region = 'west'
w1 = Warehouse()
print(w1.purpose, w1.region)
storage west
w2 = Warehouse()
w2.region = 'east'
print(w2.purpose, w2.region)
storage east
Data attributes may be referenced by methods as well as by ordinary users (“clients”) of an object. In other words, classes are not
Clients should use data attributes with care — clients may mess up invariants maintained by the methods by stamping on their dat
There is no shorthand for referencing data attributes (or other methods!) from within methods. I find that this actually increases the
Often, the first argument of a method is called self. This is nothing more than a convention: the name self has absolutely no specia
Any function object that is a class attribute defines a method for instances of that class. It is not necessary that the function definiti
# Function defined outside the class
def f1(self, x, y):
return min(x, x+y)
class C:
f = f1
def g(self):
return 'hello world'
h=g
Now f, g and h are all attributes of class C that refer to function objects, and consequently they are all methods of instances of C —
Methods may call other methods by using method attributes of the self argument:
class Bag:
def __init__(self):
self.data = []
def add(self, x):
self.data.append(x)
def addtwice(self, x):
self.add(x)
self.add(x)
Methods may reference global names in the same way as ordinary functions. The global scope associated with a method is the m
Each value is an object, and therefore has a class (also called its type). It is stored as object.__class__.
9.5. Inheritance
Of course, a language feature would not be worthy of the name “class” without supporting inheritance. The syntax for a derived cla
class DerivedClassName(BaseClassName):
<statement-1>
.
.
.
<statement-N>
The name BaseClassName must be defined in a namespace accessible from the scope containing the derived class definition. In
class DerivedClassName(modname.BaseClassName):
Execution of a derived class definition proceeds the same as for a base class. When the class object is constructed, the base clas
There’s nothing special about instantiation of derived classes: DerivedClassName() creates a new instance of the class. Method re
Derived classes may override methods of their base classes. Because methods have no special privileges when calling other meth
An overriding method in a derived class may in fact want to extend rather than simply replace the base class method of the same
Python has two built-in functions that work with inheritance:
Use to check an instance’s type: isinstance(obj, int) will be True only if obj.__class__ is or some class derived from .
Use to check class inheritance: issubclass(bool, int) is True since is a subclass of . However, issubclass(float, int) is False since
9.5.1. Multiple Inheritance
Python supports a form of multiple inheritance as well. A class definition with multiple base classes looks like this:
class DerivedClassName(Base1, Base2, Base3):
<statement-1>
.
.
.
 <statement-N>
For most purposes, in the simplest cases, you can think of the search for attributes inherited from a parent class as depth-first, left
In fact, it is slightly more complex than that; the method resolution order changes dynamically to support cooperative calls to . This
Dynamic ordering is necessary because all cases of multiple inheritance exhibit one or more diamond relationships (where at leas
9.6. Private Variables
“Private” instance variables that cannot be accessed except from inside an object don’t exist in Python. However, there is a conve
Since there is a valid use-case for class-private members (namely to avoid name clashes of names with names defined by subclas
See also
The for details and special cases.
Name mangling is helpful for letting subclasses override methods without breaking intraclass method calls. For example:
class Mapping:
def __init__(self, iterable):
self.items_list = []
self.__update(iterable)
def update(self, iterable):
for item in iterable:
self.items_list.append(item)
__update = update # private copy of original update() method
class MappingSubclass(Mapping):
def update(self, keys, values):
# provides new signature for update()
# but does not break __init__()
for item in zip(keys, values):
self.items_list.append(item)
The above example would work even if MappingSubclass were to introduce a __update identifier since it is replaced with _Mappin
Note that the mangling rules are designed mostly to avoid accidents; it still is possible to access or modify a variable that is consid
Notice that code passed to exec() or eval() does not consider the classname of the invoking class to be the current class; this is si
9.7. Odds and Ends
Sometimes it is useful to have a data type similar to the Pascal “record” or C “struct”, bundling together a few named data items. T
from dataclasses import dataclass
@dataclass
class Employee:
name: str
dept: str
salary: int
john = Employee('john', 'computer lab', 1000)
john.dept
'computer lab'
john.salary
1000
A piece of Python code that expects a particular abstract data type can often be passed a class that emulates the methods of that
have attributes, too: is the instance object with the method m(), and is the corresponding to the method.
9.8. Iterators
By now you have probably noticed that most container objects can be looped over using a statement:
for element in [1, 2, 3]:
print(element)
for element in (1, 2, 3):
print(element)
for key in {'one':1, 'two':2}:
print(key)
for char in "123":
print(char)
for line in open("myfile.txt"):
print(line, end='')
This style of access is clear, concise, and convenient. The use of iterators pervades and unifies Python. Behind the scenes, the s
s = 'abc'
it = iter(s)
it
<str_iterator object at 0x10c90e650>
next(it)
'a'
next(it)
'b'
next(it)
'c'
next(it)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
next(it)
StopIteration
Having seen the mechanics behind the iterator protocol, it is easy to add iterator behavior to your classes. Define an method whic
class Reverse:
"""Iterator for looping over a sequence backwards."""
def __init__(self, data):
self.data = data
self.index = len(data)
def __iter__(self):
return self
def __next__(self):
if self.index == 0:
raise StopIteration
self.index = self.index - 1
return self.data[self.index]
rev = Reverse('spam')
iter(rev)
<__main__.Reverse object at 0x00A1DB50>
for char in rev:
print(char)
m
a
p
s
9.9. Generators
are a simple and powerful tool for creating iterators. They are written like regular functions but use the statement whenever they w
def reverse(data):
for index in range(len(data)-1, -1, -1):
yield data[index]
for char in reverse('golf'):
print(char)
f
l
o
g
Anything that can be done with generators can also be done with class-based iterators as described in the previous section. What
Another key feature is that the local variables and execution state are automatically saved between calls. This made the function e
In addition to automatic method creation and saving program state, when generators terminate, they automatically raise . In combi
9.10. Generator Expressions
Some simple generators can be coded succinctly as expressions using a syntax similar to list comprehensions but with parenthese
Examples:
sum(i*i for i in range(10)) # sum of squares
285
xvec = [10, 20, 30]
yvec = [7, 5, 3]
sum(x*y for x,y in zip(xvec, yvec)) # dot product
260
unique_words = set(word for line in page for word in line.split())
valedictorian = max((student.gpa, student.name) for student in graduates)
data = 'golf'
list(data[i] for i in range(len(data)-1, -1, -1))
['f', 'l', 'o', 'g']
Footnotes
[]
Except for one thing. Module objects have a secret read-only attribute called which returns the dictionary used to implement the m
Brief Tour of the Standard Library
10.1. Operating System Interface
The module provides dozens of functions for interacting with the operating system:
import os
os.getcwd() # Return the current working directory
'C:\\Python313'
os.chdir('/server/accesslogs') # Change current working directory
os.system('mkdir today') # Run the command mkdir in the system shell
0
Be sure to use the import os style instead of from os import *. This will keep from shadowing the built-in function which operates m
The built-in and functions are useful as interactive aids for working with large modules like :
import os
dir(os)
<returns a list of all module functions>
help(os)
<returns an extensive manual page created from the module's docstrings>
For daily file and directory management tasks, the module provides a higher level interface that is easier to use:
import shutil
shutil.copyfile('data.db', 'archive.db')
'archive.db'
shutil.move('/build/executables', 'installdir')
'installdir'
10.2. File Wildcards
The module provides a function for making file lists from directory wildcard searches:
import glob
glob.glob('*.py')
['primes.py', 'random.py', 'quote.py']
10.3. Command Line Arguments
Common utility scripts often need to process command line arguments. These arguments are stored in the module’s argv attribute
# File demo.py
import sys
print(sys.argv)
Here is the output from running python demo.py one two three at the command line:
['demo.py', 'one', 'two', 'three']
The module provides a more sophisticated mechanism to process command line arguments. The following script extracts one or m
import argparse
parser = argparse.ArgumentParser(
prog='top',
description='Show top lines from each file')
parser.add_argument('filenames', nargs='+')
parser.add_argument('-l', '--lines', type=int, default=10)
args = parser.parse_args()
print(args)
When run at the command line with python top.py --lines=5 alpha.txt beta.txt, the script sets args.lines to 5 and args.filenames to [
10.4. Error Output Redirection and Program Termination
The module also has attributes for stdin, stdout, and stderr. The latter is useful for emitting warnings and error messages to make
sys.stderr.write('Warning, log file not found starting a new one\n')
Warning, log file not found starting a new one
The most direct way to terminate a script is to use sys.exit().
10.5. String Pattern Matching
The module provides regular expression tools for advanced string processing. For complex matching and manipulation, regular ex
import re
re.findall(r'\bf[a-z]*', 'which foot or hand fell fastest')
['foot', 'fell', 'fastest']
re.sub(r'(\b[a-z]+) \1', r'\1', 'cat in the the hat')
'cat in the hat'
When only simple capabilities are needed, string methods are preferred because they are easier to read and debug:
'tea for too'.replace('too', 'two')
'tea for two'
10.6. Mathematics
The module gives access to the underlying C library functions for floating-point math:
import math
math.cos(math.pi / 4)
0.70710678118654757
math.log(1024, 2)
10.0
The module provides tools for making random selections:
import random
random.choice(['apple', 'pear', 'banana'])
'apple'
random.sample(range(100), 10) # sampling without replacement
[30, 83, 16, 4, 8, 81, 41, 50, 18, 33]
random.random() # random float from the interval [0.0, 1.0)
0.17970987693706186
random.randrange(6) # random integer chosen from range(6)
4
The module calculates basic statistical properties (the mean, median, variance, etc.) of numeric data:
import statistics
data = [2.75, 1.75, 1.25, 0.25, 0.5, 1.25, 3.5]
statistics.mean(data)
1.6071428571428572
statistics.median(data)
1.25
statistics.variance(data)
1.3720238095238095
The SciPy project <> has many other modules for numerical computations.
10.7. Internet Access
There are a number of modules for accessing the internet and processing internet protocols. Two of the simplest are for retrieving
from urllib.request import urlopen
with urlopen('http://worldtimeapi.org/api/timezone/etc/UTC.txt') as response:
for line in response:
line = line.decode() # Convert bytes to a str
if line.startswith('datetime'):
print(line.rstrip()) # Remove trailing newline
datetime: 2022-01-01T01:36:47.689215+00:00
import smtplib
server = smtplib.SMTP('localhost')
server.sendmail('[email protected]', '[email protected]',
"""To: [email protected]
From: [email protected]
Beware the Ides of March.
""")
server.quit()
(Note that the second example needs a mailserver running on localhost.)
10.8. Dates and Times
The module supplies classes for manipulating dates and times in both simple and complex ways. While date and time arithmetic i
# dates are easily constructed and formatted
from datetime import date
now = date.today()
now
datetime.date(2003, 12, 2)
now.strftime("%m-%d-%y. %d %b %Y is a %A on the %d day of %B.")
'12-02-03. 02 Dec 2003 is a Tuesday on the 02 day of December.'
# dates support calendar arithmetic
birthday = date(1964, 7, 31)
age = now - birthday
age.days
14368
10.9. Data Compression
Common data archiving and compression formats are directly supported by modules including: , , , , and .
import zlib
s = b'witch which has which witches wrist watch'
len(s)
41
t = zlib.compress(s)
len(t)
37
zlib.decompress(t)
b'witch which has which witches wrist watch'
zlib.crc32(s)
226805979
10.10. Performance Measurement
Some Python users develop a deep interest in knowing the relative performance of different approaches to the same problem. Pyt
For example, it may be tempting to use the tuple packing and unpacking feature instead of the traditional approach to swapping ar
from timeit import Timer
Timer('t=a; a=b; b=t', 'a=1; b=2').timeit()
0.57535828626024577
Timer('a,b = b,a', 'a=1; b=2').timeit()
0.54962537085770791
In contrast to ’s fine level of granularity, the and modules provide tools for identifying time critical sections in larger blocks of code
10.11. Quality Control
One approach for developing high quality software is to write tests for each function as it is developed and to run those tests frequ
The module provides a tool for scanning a module and validating tests embedded in a program’s docstrings. Test construction is a
def average(values):
"""Computes the arithmetic mean of a list of numbers.
>>> print(average([20, 30, 70]))
40.0
"""
return sum(values) / len(values)
import doctest
doctest.testmod() # automatically validate the embedded tests
The module is not as effortless as the module, but it allows a more comprehensive set of tests to be maintained in a separate file
import unittest
class TestStatisticalFunctions(unittest.TestCase):
def test_average(self):
self.assertEqual(average([20, 30, 70]), 40.0)
self.assertEqual(round(average([1, 5, 7]), 1), 4.3)
with self.assertRaises(ZeroDivisionError):
average([])
with self.assertRaises(TypeError):
average(20, 30, 70)
unittest.main() # Calling from the command line invokes all tests
10.12. Batteries Included
Python has a “batteries included” philosophy. This is best seen through the sophisticated and robust capabilities of its larger packa
The and modules make implementing remote procedure calls into an almost trivial task. Despite the modules’ names, no direct k
The package is a library for managing email messages, including MIME and other -based message documents. Unlike and whic
The package provides robust support for parsing this popular data interchange format. The module supports direct reading and w
The module is a wrapper for the SQLite database library, providing a persistent database that can be updated and accessed usin
Internationalization is supported by a number of modules including , , and the package.
Brief Tour of the Standard Library — Part II
This second tour covers more advanced modules that support professional programming needs. These modules rarely occur in sm
11.1. Output Formatting
The module provides a version of customized for abbreviated displays of large or deeply nested containers:
import reprlib
reprlib.repr(set('supercalifragilisticexpialidocious'))
"{'a', 'c', 'd', 'e', 'f', 'g', ...}"
The module offers more sophisticated control over printing both built-in and user defined objects in a way that is readable by the i
import pprint
t = [[[['black', 'cyan'], 'white', ['green', 'red']], [['magenta',
'yellow'], 'blue']]]
pprint.pprint(t, width=30)
[[[['black', 'cyan'],
'white',
['green', 'red']],
[['magenta', 'yellow'],
'blue']]]
The module formats paragraphs of text to fit a given screen width:
import textwrap
doc = """The wrap() method is just like fill() except that it returns
a list of strings instead of one big string with newlines to separate
the wrapped lines."""
print(textwrap.fill(doc, width=40))
The wrap() method is just like fill()
except that it returns a list of strings
instead of one big string with newlines
to separate the wrapped lines.
The module accesses a database of culture specific data formats. The grouping attribute of locale’s format function provides a dir
import locale
locale.setlocale(locale.LC_ALL, 'English_United States.1252')
'English_United States.1252'
conv = locale.localeconv() # get a mapping of conventions
x = 1234567.8
locale.format_string("%d", x, grouping=True)
'1,234,567'
locale.format_string("%s%.*f", (conv['currency_symbol'],
conv['frac_digits'], x), grouping=True)
'$1,234,567.80'
11.2. Templating
The module includes a versatile class with a simplified syntax suitable for editing by end-users. This allows users to customize th
The format uses placeholder names formed by $ with valid Python identifiers (alphanumeric characters and underscores). Surroun
from string import Template
t = Template('${village}folk send $$10 to $cause.')
t.substitute(village='Nottingham', cause='the ditch fund')
'Nottinghamfolk send $10 to the ditch fund.'
The method raises a when a placeholder is not supplied in a dictionary or a keyword argument. For mail-merge style applications
t = Template('Return the $item to $owner.')
d = dict(item='unladen swallow')
t.substitute(d)
Traceback (most recent call last):
...
KeyError: 'owner'
t.safe_substitute(d)
'Return the unladen swallow to $owner.'
Template subclasses can specify a custom delimiter. For example, a batch renaming utility for a photo browser may elect to use p
import time, os.path
photofiles = ['img_1074.jpg', 'img_1076.jpg', 'img_1077.jpg']
class BatchRename(Template):
delimiter = '%'
fmt = input('Enter rename style (%d-date %n-seqnum %f-format): ')
Enter rename style (%d-date %n-seqnum %f-format): Ashley_%n%f
t = BatchRename(fmt)
date = time.strftime('%d%b%y')
for i, filename in enumerate(photofiles):
base, ext = os.path.splitext(filename)
newname = t.substitute(d=date, n=i, f=ext)
print('{0} --> {1}'.format(filename, newname))
img_1074.jpg --> Ashley_0.jpg
img_1076.jpg --> Ashley_1.jpg
img_1077.jpg --> Ashley_2.jpg
Another application for templating is separating program logic from the details of multiple output formats. This makes it possible to
11.3. Working with Binary Data Record Layouts
The module provides and functions for working with variable length binary record formats. The following example shows how to
import struct
with open('myfile.zip', 'rb') as f:
data = f.read()
start = 0
for i in range(3): # show the first 3 file headers
start += 14
fields = struct.unpack('<IIIHH', data[start:start+16])
crc32, comp_size, uncomp_size, filenamesize, extra_size = fields
start += 16
filename = data[start:start+filenamesize]
start += filenamesize
extra = data[start:start+extra_size]
print(filename, hex(crc32), comp_size, uncomp_size)
start += extra_size + comp_size # skip to the next header
11.4. Multi-threading
Threading is a technique for decoupling tasks which are not sequentially dependent. Threads can be used to improve the respons
The following code shows how the high level module can run tasks in background while the main program continues to run:
import threading, zipfile
class AsyncZip(threading.Thread):
def __init__(self, infile, outfile):
threading.Thread.__init__(self)
self.infile = infile
self.outfile = outfile
def run(self):
f = zipfile.ZipFile(self.outfile, 'w', zipfile.ZIP_DEFLATED)
f.write(self.infile)
f.close()
print('Finished background zip of:', self.infile)
background = AsyncZip('mydata.txt', 'myarchive.zip')
background.start()
print('The main program continues to run in foreground.')
background.join() # Wait for the background task to finish
print('Main program waited until background was done.')
The principal challenge of multi-threaded applications is coordinating threads that share data or other resources. To that end, the t
While those tools are powerful, minor design errors can result in problems that are difficult to reproduce. So, the preferred approac
11.5. Logging
The module offers a full featured and flexible logging system. At its simplest, log messages are sent to a file or to sys.stderr:
import logging
logging.debug('Debugging information')
logging.info('Informational message')
logging.warning('Warning:config file %s not found', 'server.conf')
logging.error('Error occurred')
logging.critical('Critical error -- shutting down')
This produces the following output:
WARNING:root:Warning:config file server.conf not found
ERROR:root:Error occurred
CRITICAL:root:Critical error -- shutting down
By default, informational and debugging messages are suppressed and the output is sent to standard error. Other output options i
The logging system can be configured directly from Python or can be loaded from a user editable configuration file for customized
11.6. Weak References
Python does automatic memory management (reference counting for most objects and to eliminate cycles). The memory is freed
This approach works fine for most applications but occasionally there is a need to track objects only as long as they are being use
import weakref, gc
class A:
def __init__(self, value):
 self.value = value
def __repr__(self):
return str(self.value)
a = A(10) # create a reference
d = weakref.WeakValueDictionary()
d['primary'] = a # does not create a reference
d['primary'] # fetch the object if it is still alive
10
del a # remove the one reference
gc.collect() # run garbage collection right away
0
d['primary'] # entry was automatically removed
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
d['primary'] # entry was automatically removed
File "C:/python313/lib/weakref.py", line 46, in __getitem__
o = self.data[key]()
KeyError: 'primary'
11.7. Tools for Working with Lists
Many data structure needs can be met with the built-in list type. However, sometimes there is a need for alternative implementatio
The module provides an object that is like a list that stores only homogeneous data and stores it more compactly. The following e
from array import array
a = array('H', [4000, 10, 700, 22222])
sum(a)
26932
a[1:3]
array('H', [10, 700])
The module provides a object that is like a list with faster appends and pops from the left side but slower lookups in the middle. T
from collections import deque
d = deque(["task1", "task2", "task3"])
d.append("task4")
print("Handling", d.popleft())
Handling task1
unsearched = deque([starting_node])
def breadth_first_search(unsearched):
node = unsearched.popleft()
for m in gen_moves(node):
if is_goal(m):
return m
unsearched.append(m)
In addition to alternative list implementations, the library also offers other tools such as the module with functions for manipulating
import bisect
scores = [(100, 'perl'), (200, 'tcl'), (400, 'lua'), (500, 'python')]
bisect.insort(scores, (300, 'ruby'))
scores
[(100, 'perl'), (200, 'tcl'), (300, 'ruby'), (400, 'lua'), (500, 'python')]
The module provides functions for implementing heaps based on regular lists. The lowest valued entry is always kept at position z
from heapq import heapify, heappop, heappush
data = [1, 3, 5, 7, 9, 2, 4, 6, 8, 0]
heapify(data) # rearrange the list into heap order
heappush(data, -5) # add a new entry
[heappop(data) for i in range(3)] # fetch the three smallest entries
[-5, 0, 1]
11.8. Decimal Floating-Point Arithmetic
The module offers a datatype for decimal floating-point arithmetic. Compared to the built-in implementation of binary floating poin
financial applications and other uses which require exact decimal representation,
control over precision,
control over rounding to meet legal or regulatory requirements,
tracking of significant decimal places, or
applications where the user expects the results to match calculations done by hand.
For example, calculating a 5% tax on a 70 cent phone charge gives different results in decimal floating point and binary floating po
from decimal import *
round(Decimal('0.70') * Decimal('1.05'), 2)
Decimal('0.74')
round(.70 * 1.05, 2)
0.73
The result keeps a trailing zero, automatically inferring four place significance from multiplicands with two place significance. Deci
Exact representation enables the class to perform modulo calculations and equality tests that are unsuitable for binary floating po
Decimal('1.00') % Decimal('.10')
Decimal('0.00')
1.00 % 0.10
0.09999999999999995
sum([Decimal('0.1')]*10) == Decimal('1.0')
True
0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 == 1.0
False
The module provides arithmetic with as much precision as needed:
getcontext().prec = 36
Decimal(1) / Decimal(7)
Decimal('0.142857142857142857142857142857142857')
Virtual Environments and Packages
12.1. Introduction
Python applications will often use packages and modules that don’t come as part of the standard library. Applications will sometim
This means it may not be possible for one Python installation to meet the requirements of every application. If application A needs
The solution for this problem is to create a , a self-contained directory tree that contains a Python installation for a particular versio
Different applications can then use different virtual environments. To resolve the earlier example of conflicting requirements, applic
12.2. Creating Virtual Environments
The module used to create and manage virtual environments is called . will install the Python version from which the command wa
To create a virtual environment, decide upon a directory where you want to place it, and run the module as a script with the direct
python -m venv tutorial-env
This will create the tutorial-env directory if it doesn’t exist, and also create directories inside it containing a copy of the Python inter
A common directory location for a virtual environment is .venv. This name keeps the directory typically hidden in your shell and thu
Once you’ve created a virtual environment, you may activate it.
On Windows, run:
tutorial-env\Scripts\activate
On Unix or MacOS, run:
source tutorial-env/bin/activate
(This script is written for the bash shell. If you use the csh or fish shells, there are alternate activate.csh and activate.fish scripts yo
Activating the virtual environment will change your shell’s prompt to show what virtual environment you’re using, and modify the en
$ source ~/envs/tutorial-env/bin/activate
(tutorial-env) $ python
Python 3.5.1 (default, May 6 2016, 10:59:36)
...
>>> import sys
>>> sys.path
['', '/usr/local/lib/python35.zip', ...,
'~/envs/tutorial-env/lib/python3.5/site-packages']
>>>
To deactivate a virtual environment, type:
deactivate
into the terminal.
12.3. Managing Packages with pip
You can install, upgrade, and remove packages using a program called pip. By default pip will install packages from the . You can
pip has a number of subcommands: “install”, “uninstall”, “freeze”, etc. (Consult the guide for complete documentation for pip.)
You can install the latest version of a package by specifying a package’s name:
(tutorial-env) $ python -m pip install novas
Collecting novas
Downloading novas-3.1.1.3.tar.gz (136kB)
Installing collected packages: novas
Running setup.py install for novas
Successfully installed novas-3.1.1.3
You can also install a specific version of a package by giving the package name followed by == and the version number:
(tutorial-env) $ python -m pip install requests==2.6.0
Collecting requests==2.6.0
Using cached requests-2.6.0-py2.py3-none-any.whl
Installing collected packages: requests
Successfully installed requests-2.6.0
If you re-run this command, pip will notice that the requested version is already installed and do nothing. You can supply a differen
(tutorial-env) $ python -m pip install --upgrade requests
Collecting requests
Installing collected packages: requests
Found existing installation: requests 2.6.0
Uninstalling requests-2.6.0:
Successfully uninstalled requests-2.6.0
Successfully installed requests-2.7.0
python -m pip uninstall followed by one or more package names will remove the packages from the virtual environment.
python -m pip show will display information about a particular package:
(tutorial-env) $ python -m pip show requests
---
Metadata-Version: 2.0
Name: requests
Version: 2.7.0
Summary: Python HTTP for Humans.
Home-page: http://python-requests.org
Author: Kenneth Reitz
Author-email: [email protected]
License: Apache 2.0
Location: /Users/akuchling/envs/tutorial-env/lib/python3.4/site-packages
Requires:
python -m pip list will display all of the packages installed in the virtual environment:
(tutorial-env) $ python -m pip list
novas (3.1.1.3)
numpy (1.9.2)
pip (7.0.3)
requests (2.7.0)
setuptools (16.0)
python -m pip freeze will produce a similar list of the installed packages, but the output uses the format that python -m pip install e
(tutorial-env) $ python -m pip freeze > requirements.txt
(tutorial-env) $ cat requirements.txt
novas==3.1.1.3
numpy==1.9.2
requests==2.7.0
The requirements.txt can then be committed to version control and shipped as part of an application. Users can then install all the
(tutorial-env) $ python -m pip install -r requirements.txt
Collecting novas==3.1.1.3 (from -r requirements.txt (line 1))
...
Collecting numpy==1.9.2 (from -r requirements.txt (line 2))
...
Collecting requests==2.7.0 (from -r requirements.txt (line 3))
...
Installing collected packages: novas, numpy, requests
Running setup.py install for novas
Successfully installed novas-3.1.1.3 numpy-1.9.2 requests-2.7.0
pip has many more options. Consult the guide for complete documentation for pip. When you’ve written a package and want to m
What Now?
Reading this tutorial has probably reinforced your interest in using Python — you should be eager to apply Python to solving your
This tutorial is part of Python’s documentation set. Some other documents in the set are:
:
You should browse through this manual, which gives complete (though terse) reference material about types, functions, and the m
explains how to install additional modules written by other Python users.
: A detailed explanation of Python’s syntax and semantics. It’s heavy reading, but is useful as a complete guide to the language its
More Python resources:
: The major Python web site. It contains code, documentation, and pointers to Python-related pages around the web.
: Fast access to Python’s documentation.
: The Python Package Index, previously also nicknamed the Cheese Shop , is an index of user-created Python modules that are a
: The Python Cookbook is a sizable collection of code examples, larger modules, and useful scripts. Particularly notable contributio
collects links to Python-related videos from conferences and user-group meetings.
: The Scientific Python project includes modules for fast array computations and manipulations plus a host of packages for such th
For Python-related questions and problem reports, you can post to the newsgroup comp.lang.python, or send them to the mailing
Before posting, be sure to check the list of (also called the FAQ). The FAQ answers many of the questions that come up again an
Footnotes
[]
“Cheese Shop” is a Monty Python’s sketch: a customer enters a cheese shop, but whatever cheese he asks for, the clerk says it’s
Interactive Input Editing and History Substitution
Some versions of the Python interpreter support editing of the current input line and history substitution, similar to facilities found in
14.1. Tab Completion and History Editing
Completion of variable and module names is at interpreter startup so that the Tab key invokes the completion function; it looks at
14.2. Alternatives to the Interactive Interpreter
This facility is an enormous step forward compared to earlier versions of the interpreter; however, some wishes are left: It would be
One alternative enhanced interactive interpreter that has been around for quite some time is , which features tab completion, objec
. Floating-Point Arithmetic: Issues and Limitations
Floating-point numbers are represented in computer hardware as base 2 (binary) fractions. For example, the decimal fraction 0.62
Unfortunately, most decimal fractions cannot be represented exactly as binary fractions. A consequence is that, in general, the dec
The problem is easier to understand at first in base 10. Consider the fraction 1/3. You can approximate that as a base 10 fraction:
0.3
or, better,
0.33
or, better,
0.333
and so on. No matter how many digits you’re willing to write down, the result will never be exactly 1/3, but will be an increasingly b
In the same way, no matter how many base 2 digits you’re willing to use, the decimal value 0.1 cannot be represented exactly as a
0.0001100110011001100110011001100110011001100110011...
Stop at any finite number of bits, and you get an approximation. On most machines today, floats are approximated using a binary f
Many users are not aware of the approximation because of the way values are displayed. Python only prints a decimal approximat
0.1
0.1000000000000000055511151231257827021181583404541015625
That is more digits than most people find useful, so Python keeps the number of digits manageable by displaying a rounded value
1 / 10
0.1
Just remember, even though the printed result looks like the exact value of 1/10, the actual stored value is the nearest representab
Interestingly, there are many different decimal numbers that share the same nearest approximate binary fraction. For example, the
Historically, the Python prompt and built-in function would choose the one with 17 significant digits, 0.10000000000000001. Starti
Note that this is in the very nature of binary floating point: this is not a bug in Python, and it is not a bug in your code either. You’ll s
For more pleasant output, you may wish to use string formatting to produce a limited number of significant digits:
format(math.pi, '.12g') # give 12 significant digits
'3.14159265359'
format(math.pi, '.2f') # give 2 digits after the point
'3.14'
repr(math.pi)
'3.141592653589793'
It’s important to realize that this is, in a real sense, an illusion: you’re simply rounding the display of the true machine value.
One illusion may beget another. For example, since 0.1 is not exactly 1/10, summing three values of 0.1 may not yield exactly 0.3,
0.1 + 0.1 + 0.1 == 0.3
False
Also, since the 0.1 cannot get any closer to the exact value of 1/10 and 0.3 cannot get any closer to the exact value of 3/10, then p
round(0.1, 1) + round(0.1, 1) + round(0.1, 1) == round(0.3, 1)
False
Though the numbers cannot be made closer to their intended exact values, the function can be useful for comparing inexact value
math.isclose(0.1 + 0.1 + 0.1, 0.3)
True
Alternatively, the function can be used to compare rough approximations:
round(math.pi, ndigits=2) == round(22 / 7, ndigits=2)
True
Binary floating-point arithmetic holds many surprises like this. The problem with “0.1” is explained in precise detail below, in the “R
As that says near the end, “there are no easy answers.” Still, don’t be unduly wary of floating point! The errors in Python float oper
While pathological cases do exist, for most casual use of floating-point arithmetic you’ll see the result you expect in the end if you
For use cases which require exact decimal representation, try using the module which implements decimal arithmetic suitable for
Another form of exact arithmetic is supported by the module which implements arithmetic based on rational numbers (so the num
If you are a heavy user of floating-point operations you should take a look at the NumPy package and many other packages for ma
Python provides tools that may help on those rare occasions when you really do want to know the exact value of a float. The meth
x = 3.14159
x.as_integer_ratio()
(3537115888337719, 1125899906842624)
Since the ratio is exact, it can be used to losslessly recreate the original value:
x == 3537115888337719 / 1125899906842624
True
The method expresses a float in hexadecimal (base 16), again giving the exact value stored by your computer:
x.hex()
'0x1.921f9f01b866ep+1'
This precise hexadecimal representation can be used to reconstruct the float value exactly:
x == float.fromhex('0x1.921f9f01b866ep+1')
True
Since the representation is exact, it is useful for reliably porting values across different versions of Python (platform independence
Another helpful tool is the function which helps mitigate loss-of-precision during summation. It uses extended precision for interme
0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 == 1.0
False
sum([0.1] * 10) == 1.0
True
The goes further and tracks all of the “lost digits” as values are added onto a running total so that the result has only a single roun
arr = [-0.10430216751806065, -266310978.67179024, 143401161448607.16,
-143401161400469.7, 266262841.31058735, -0.003244936839808227]
float(sum(map(Fraction, arr))) # Exact summation with single rounding
8.042173697819788e-13
math.fsum(arr) # Single rounding
8.042173697819788e-13
sum(arr) # Multiple roundings in extended precision
8.042178034628478e-13
total = 0.0
for x in arr:
total += x # Multiple roundings in standard precision
total # Straight addition has no correct digits!
-0.0051575902860057365
15.1. Representation Error
This section explains the “0.1” example in detail, and shows how you can perform an exact analysis of cases like this yourself. Bas
Representation error refers to the fact that some (most, actually) decimal fractions cannot be represented exactly as binary (base
Why is that? 1/10 is not exactly representable as a binary fraction. Since at least 2000, almost all machines use IEEE 754 binary f
1 / 10 ~= J / (2**N)
as
J ~= 2**N / 10
and recalling that J has exactly 53 bits (is >= 2**52 but < 2**53), the best value for N is 56:
2**52 <= 2**56 // 10 < 2**53
True
That is, 56 is the only value for N that leaves J with exactly 53 bits. The best possible value for J is then that quotient rounded:
q, r = divmod(2**56, 10)
r
6
Since the remainder is more than half of 10, the best approximation is obtained by rounding up:
q+1
7205759403792794
Therefore the best possible approximation to 1/10 in IEEE 754 double precision is:
7205759403792794 / 2 ** 56
Dividing both the numerator and denominator by two reduces the fraction to:
3602879701896397 / 2 ** 55
Note that since we rounded up, this is actually a little bit larger than 1/10; if we had not rounded up, the quotient would have been
So the computer never “sees” 1/10: what it sees is the exact fraction given above, the best IEEE 754 double approximation it can g
0.1 * 2 ** 55
3602879701896397.0
If we multiply that fraction by 10**55, we can see the value out to 55 decimal digits:
3602879701896397 * 10 ** 55 // 2 ** 55
1000000000000000055511151231257827021181583404541015625
meaning that the exact number stored in the computer is equal to the decimal value 0.10000000000000000555111512312578270
format(0.1, '.17f')
'0.10000000000000001'
The and modules make these calculations easy:
from decimal import Decimal
from fractions import Fraction
Fraction.from_float(0.1)
Fraction(3602879701896397, 36028797018963968)
(0.1).as_integer_ratio()
(3602879701896397, 36028797018963968)
Decimal.from_float(0.1)
Decimal('0.1000000000000000055511151231257827021181583404541015625')
format(Decimal.from_float(0.1), '.17')
'0.10000000000000001'
Appendix
16.1. Interactive Mode
There are two variants of the interactive . The classic basic interpreter is supported on all platforms with minimal line control capab
On Windows, or Unix-like systems with support, a new interactive shell is used by default. This one supports color, multiline editin
When using the new interactive shell, exit the shell by typing exit or quit. Adding call parentheses after those commands is not req
If the new interactive shell is not desired, it can be disabled via the environment variable.
16.1.1. Error Handling
When an error occurs, the interpreter prints an error message and a stack trace. In interactive mode, it then returns to the primary
Typing the interrupt character (usually Control-C or Delete) to the primary or secondary prompt cancels the input and returns to the
16.1.2. Executable Python Scripts
On BSD’ish Unix systems, Python scripts can be made directly executable, like shell scripts, by putting the line
#!/usr/bin/env python3
(assuming that the interpreter is on the user’s PATH) at the beginning of the script and giving the file an executable mode. The #! m
The script can be given an executable mode, or permission, using the chmod command.
chmod +x myscript.py
On Windows systems, there is no notion of an “executable mode”. The Python installer automatically associates .py files with pyth
16.1.3. The Interactive Startup File
When you use Python interactively, it is frequently handy to have some standard commands executed every time the interpreter is
This file is only read in interactive sessions, not when Python reads commands from a script, and not when /dev/tty is given as the
If you want to read an additional start-up file from the current directory, you can program this in the global start-up file using code l
import os
filename = os.environ.get('PYTHONSTARTUP')
if filename and os.path.isfile(filename):
with open(filename) as fobj:
startup_file = fobj.read()
exec(startup_file)
16.1.4. The Customization Modules
Python provides two hooks to let you customize it: sitecustomize and usercustomize. To see how it works, you need first to find the
import site
site.getusersitepackages()
'/home/user/.local/lib/python3.x/site-packages'
Now you can create a file named usercustomize.py in that directory and put anything you want in it. It will affect every invocation o
sitecustomize works in the same way, but is typically created by an administrator of the computer in the global site-packages direc
Footnotes
[]
A problem with the GNU Readline package may prevent this