Iterators
Iterators
By now you have probably noticed that most container objects can be looped over us-
ing a for statement:
This style of access is clear, concise, and convenient. The use of iterators pervades
and unifies Python. Behind the scenes, the for statement calls iter() on the container
object. The function returns an iterator object that defines the
method __next__() which accesses elements in the container one at a time. When
there are no more elements, __next__() raises a StopIteration exception which tells
the for loop to terminate. You can call the __next__() method using the next() built-in
function; this example shows how it all works:
>>>
>>> s = 'abc'
>>> it = iter(s)
>>> it
<str_iterator object at 0x10c90e650>
>>> next(it)
'a'
>>> next(it)
'b'
>>> next(it)
'c'
>>> next(it)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
next(it)
StopIteration
Having seen the mechanics behind the iterator protocol, it is easy to add iterator be-
havior to your classes. Define an __iter__() method which returns an object with
a __next__() method. If the class defines __next__(), then __iter__() can just re-
turn self:
class Reverse:
"""Iterator for looping over a sequence backwards."""
def __init__(self, data):
self.data = data
self.index = len(data)
def __iter__(self):
return self
def __next__(self):
if self.index == 0:
raise StopIteration
self.index = self.index - 1
return self.data[self.index]
>>>
9.9. Generators
Generators are a simple and powerful tool for creating iterators. They are written like
regular functions but use the yield statement whenever they want to return data. Each
time next() is called on it, the generator resumes where it left off (it remembers all the
data values and which statement was last executed). An example shows that genera-
tors can be trivially easy to create:
def reverse(data):
for index in range(len(data)-1, -1, -1):
yield data[index]
>>>
Anything that can be done with generators can also be done with class-based iterators
as described in the previous section. What makes generators so compact is that
the __iter__() and __next__() methods are created automatically.
Another key feature is that the local variables and execution state are automatically
saved between calls. This made the function easier to write and much more clear than
an approach using instance variables like self.index and self.data.
In addition to automatic method creation and saving program state, when generators
terminate, they automatically raise StopIteration. In combination, these features make
it easy to create iterators with no more effort than writing a regular function.
9.10. Generator Expressions
Some simple generators can be coded succinctly as expressions using a syntax similar
to list comprehensions but with parentheses instead of square brackets. These expres-
sions are designed for situations where the generator is used right away by an enclos-
ing function. Generator expressions are more compact but less versatile than full gen-
erator definitions and tend to be more memory friendly than equivalent list compre-
hensions.
Examples:
>>>