Python Fundamentals
Comprehensions, Iterables and Generators
Robert Smallshire Austin Bingham
@robsmallshire @austin_bingham
[email protected] [email protected] Presenter
Monday, 12 August, 13
Comprehensions
Types of comprehensions
• list c om p reh en s io n s
• s et c om preh en s io n s
Style!
ic t ion a ry c o m p rehensions
• d • declarative
• functional
• readable
• expressive
• effective
Monday, 12 August, 13
[ expr(item) for item in iterable ]
Monday, 12 August, 13
{ expr(item) for item in iterable }
Monday, 12 August, 13
{ key_expr:value_expr for item in iterable }
Monday, 12 August, 13
Duplicates: later keys overwrite earlier keys
Monday, 12 August, 13
Don't cram too much
complexity into
comprehensions!
>>> import os
>>> import glob
>>> file_sizes = {os.path.realpath(p): os.stat(p).st_size
... for p in glob.glob('*.py')}
>>> pp(file_sizes)
{'/Users/pyfund/examples/exceptional.py': 400,
'/Users/pyfund/examples/keypress.py': 778,
'/Users/pyfund/examples/scopes.py': 133,
'/Users/pyfund/examples/words.py': 1185}
Monday, 12 August, 13
Filtering works wit
h:
• list comprehensions
• set comprehensions
• dictionary comprehensions
optional filtering clause
[ expr(item) for item in iterable if predicate(item) ]
Monday, 12 August, 13
Moment of Zen
Simple is better
than complex
Code is written once
But read over and over
Fewer is clearer
Monday, 12 August, 13
Iteration protocols
Monday, 12 August, 13
Iteration protocols
Iterable protocol Iterator protocol
Iterable objects can be Iterator objects can be
passed to the built-in passed to the built-in
iter() function to get next() function to
an iterator. fetch the next item.
iterator = iter(iterable) item = next(iterator)
Monday, 12 August, 13
Stateful generators
Generators resume
execution b l e ) :
( c o un t , i t e ra
de f ta k e t e l e m en t s "
e f i r s t c o un
"Tak
Can maintain state in counter = 0 terable:
local variables for i t e m i n i
r = = c o u n t:
if counte
return
Complex control flow c o u n te r + = 1
yield item
Lazy evaluation
Monday, 12 August, 13
Laziness and
the Infinite
• Just in Time Computation
• Infinite (or large) sequences
• sensor readings
• mathematical series
• massive files
Monday, 12 August, 13
Generator comprehensions
Similar syntax to list
comprehensions
Create a generator
object
Concise
Lazy evaluation
Monday, 12 August, 13
parentheses
(expr(item) for item in iterable)
Monday, 12 August, 13
"Batteries Included"
Iteration Tools
d
de
lu
nc
sI
rie
tte
Monday, 12 August, 13 Ba
Comprehensions, Generators
& Iterables Summary
Comprehensions
Comprehensions are a concise syntax for describing lists, sets and dictionaries.
Comprehensions operate on an iterable source object and apply an optional predicate
filter and a mandatory expression, both of which are usually in terms of the current item.
Iterables are objects over which we can iterate item by item.
We retrieve an iterator from an iterable object using the built-in iter() function.
Iterators produce items one-by-one-from the underlying iterable series each time they
are passed to the built-in next() function
Monday, 12 August, 13
Comprehensions, Generators
& Iterables Summary
Generators
Generator functions allow us to describe series using imperative code.
Generator functions contain at least one use of the yield keyword.
Generators are iterators. When advanced with next() the generator starts or resumes
execution up to and including the next yield.
Each call to a generator function creates a new generator object.
Generators can maintain explicit state in local variables between iterations.
Generators are lazy, and so can model infinite series of data.
Generator expressions have a similar syntactic form to list comprehensions and allow
for a more declarative and concise way of creating generator objects.
Monday, 12 August, 13
Getting Started – Summary
Iteration tools
Built-ins such as
sum()
any()
zip()
all()
min()
max()
enumerate()
Standard library itertools module
chain()
islice()
count()
many more!
Monday, 12 August, 13
