Asynchronous programming with
Coroutines
in Python
Asynchronous programming
with Coroutines
in Python
Ewoud Van Craeynest
January 31, 2017
2
Table of Contents
introduction
asyncio in Python 3.4 (briefly)
asyncio in Python 3.5
summary
(extra slides)
3
Introduction
What is async programming for us today?
4
Introduction
What is async programming?
I writing concurrent applications 1
I without the use of threads or multiprocessing
I in a cooperative multitasking fashion
5 1
not to be confused with parallelism
Introduction
Not unique to Python
I many other languages also provide similar functionality
I .NET got credited by Guido
I Go, Dart, JavaScript, Scala, Clojure, Erlang, . . .
I but also Perl and Ruby
6
Introduction
Not unique to Python
I C++17 was rumoured to include it
I some talk about it on conferences
I but targetted for a new TS
I and hopefully C++20
7
Introduction
Blocking
A style of writing code
I that doesn’t use blocking calls
I but rather an event loop
I (that mustn’t be blocked)
8
Introduction
Blocking
What’s the issue with blocking api’s?
Why do we now dislike blocking?
9
Introduction
Blocking
What’s the issue with blocking api’s?
I your thread is ”busy”
I but not doing anything useful
I waiting for disk
I waiting for network/database
I waiting for serial port or other io
Why do we now dislike blocking?
I because now we have an alternative
10
Introduction
Threading
Isn’t that why threading exists?
Why do we now frown upon threads?
11
Introduction
Threading
Isn’t that why threading exists?
I yes, threads were designed for multitasking
I at operating system level
12
Introduction
Threading
Why do we now frown upon threads?
I context switches are expensive
I don’t scale well
I think large numbers of sockets (C10K)
I synchronisation is hard to get right
I unpredictable scheduling
I race conditions
I deadlock
I starvation
13
Introduction
Threading
Threads: the goto of our generation 2
I at least for concurrency
14 2
doesn’t mean they can’t be useful if used correctly, like goto’s
Introduction
Threading
In the multicore age
I for parallelism however
I threads (or multiprocessing) are still a must
15
Introduction
Threading
Threads no more?
I No!
I just less of them
I one thread for all connections
I i.s.o. one thread per connection
I one for all video stuff
I one for all screen io
I one for all . . .
16
Introduction
Non blocking calls
Circling back
I we want code not to block a thread
I because we want to do things concurrently
17
Introduction
Non blocking calls
Wait a minute . . .
Isn’t all code blocking in a way?
18
Introduction
Non blocking calls
Isn’t all code blocking in a way?
I indeed, but let’s make a distinction
I executing code, crunching data
I waiting for I/O operations to finish
I can’t do much about the first
I except parallelize
I but the second is the subject of our attention
19
Introduction
Non blocking calls
Non blocking I/O
I we want I/O code not to block a thread
I to do things concurrently
I we need new api’s
20
Introduction
Not so new
I turns out, those api’s, aren’t all that new
I Python has a long history of async programming
I though not in the way we know it now
21
Introduction
History
History of async programming api’s
I There where a few predecessors to what we got in Python3.5
I gevent (greenlets, c stack hack)
I tulip (now asyncio)
I twisted (event driven)
I tornado
I ...
I all a bit hacked on top of Python2
I asyncio provisional package in 3.4
22
Introduction
History
Predecessors vs 3.5
I all rely on some form of select/poll loops
I so does Python asyncio
I but with nicer syntax
I supported from the language itself
I using new keywords
23
Introduction
Let’s take a look
24
asyncio
python 3.4
asyncio
I python 3.4 added asyncio 3
I Asynchronous I/O, event loop, coroutines and tasks
I this is where our story really starts
25 3
get it from PyPI for Python 3.3
asyncio
provisional
provisional in 3.4
I It was, however, a work in progress
Note
The asyncio package has been included in the standard library on
a provisional basis. Backwards incompatible changes (up to and
including removal of the module) may occur if deemed necessary
by the core developers.
26
asyncio
coroutines
An example:
I note the @asyncio.coroutine decorator
I and yield from statement
first coroutine
@asyncio.coroutine
def print_hello():
while True:
print("{} - Hello world!".format(int(time())))
yield from asyncio.sleep(3)
27
asyncio
coroutines
coroutines
I are ”special” functions
I which can be suspended and resumed
first coroutine
@asyncio.coroutine
def print_hello():
while True:
print("{} - Hello world!".format(int(time())))
yield from asyncio.sleep(3)
28
asyncio
coroutines
new style
I using coroutines is considered ”new style” async
I though we now consider this example code ”old syntax”
I see Python 3.5 coroutines later on
new style, not newst syntax
@asyncio.coroutine
def print_hello():
while True:
print("{} - Hello world!".format(int(time())))
yield from asyncio.sleep(3)
29
asyncio
callbacks
coroutine api
I async code that does use coroutines
I needs a coroutine api
I like asyncio.open_connection and its return objects
coroutine api
@asyncio.coroutine
def tcp_echo_client(message, loop):
reader, writer = yield from asyncio.open_connection(’127.0.0.1’, 888
loop=loop)
print(’Send: %r’ % message)
writer.write(message.encode())
data = yield from reader.read(100)
print(’Received: %r’ % data.decode())
30 writer.close()
asyncio in Python 3.5
asyncio in Python 3.5
31
asyncio in Python 3.5
new keywords
coroutines
I python 3.5 added new coroutine keywords
I async def and await
I removing the need for the @asyncio.coroutine decorator
and yield from
32
asyncio in Python 3.5
provisional
still provisional in 3.5 4
I The documentation has still the same note
Note
The asyncio package has been included in the standard library on
a provisional basis. Backwards incompatible changes (up to and
including removal of the module) may occur if deemed necessary
by the core developers.
33 4
note is gone in Python3.6 docs
asyncio in Python 3.5
coroutines
same examples
I using the new syntax async/await
new syntax
async def print_hello():
while True:
print("{} - Hello world!".format(int(time())))
await asyncio.sleep(3)
34
asyncio
coroutines
coroutines reiterated
I are ”special” functions
I which can be suspended and resumed
first coroutine
async def print_hello():
while True:
print("{} - Hello world!".format(int(time())))
await asyncio.sleep(3)
35
asyncio
event loop
event loop
I an event loop will take care of starting and resuming tasks
I but in turn, it claims the thread you’re on
running the event loop
loop = asyncio.get_event_loop()
loop.run_until_complete(print_hello()) # blocking!
36
asyncio
old style async
old style async
I not all async code uses coroutines
I in fact, many of the predecessors used callbacks
I triggered by certain events
async using callback
def process_input():
text = sys.stdin.readline()
n = int(text.strip())
print(’fib({}) = {}’.format(n, timed_fib(n)))
loop.add_reader(sys.stdin, process_input)
37
asyncio
callbacks
callback style async
I though used intensively in the past
I it escalates quickly in a cascade of callbacks and state
machines
I becoming a bit of a design anti-pattern in itself
I callback hell . . .
I but we didn’t really have another option
I and it did get us out of threading!
38
asyncio
callbacks
callback style async
I asyncio’s event loop supports scheduling regular callbacks
I using a fifo queue of registered callbacks
I in this case as soon as possible
async using callback
def hello_world(loop):
print(’Hello World’)
loop.stop()
loop = asyncio.get_event_loop()
loop.call_soon(hello_world, loop) # <--
loop.run_forever()
39
loop.close()
asyncio
callbacks
callback style async
I delayed callbacks are also possible
I call_later
I call_at
I event loop has own internal clock for computing timeouts
delayed async using callback
loop.call_later(0.5, hello_world, loop)
40
asyncio in Python 3.5
coroutines
same examples
I using the new syntax async/await with streams
new syntax
async def tcp_echo_client(message, loop):
reader, writer = await asyncio.open_connection(’127.0.0.1’, 8888,
loop=loop)
print(’Send: %r’ % message)
writer.write(message.encode())
data = await reader.read(100)
print(’Received: %r’ % data.decode())
print(’Close the socket’)
41 writer.close()
asyncio in Python 3.5
coroutines
suspend on yield from
I coroutine will be suspended
I until open_connection has finished
coroutine api
reader, writer = await asyncio.open_connection(’127.0.0.1’, 8888,
loop=loop)
42
asyncio
coroutine api in Python 3.5
coroutine api
I also the objects returned by open_connection have
coroutines
I though only for what blocks
I write is documented not to block
I but we do want to suspend until read finishes
I without blocking the thread
coroutine api
writer.write(message.encode())
data = await reader.read(100)
43
asyncio
callbacks in Python 3.5
coroutine api
I written as if it where synchronous code
I no callbacks and keeping state
I but nonblocking with suspend and resume behaviour
coroutine api
async def tcp_echo_client(message, loop):
reader, writer = await asyncio.open_connection(’127.0.0.1’, 8888,
loop=loop)
print(’Send: %r’ % message)
writer.write(message.encode())
data = await reader.read(100)
print(’Received: %r’ % data.decode())
44 writer.close()
asyncio in Python 3.5
coroutines api
coroutine api
I as with Python 3.4
I we need alternatives for all blocking api’s we might want to
use
I as usual Python comes with (some) batteries included
I additional batteries on PyPI
I though it must be mentioned that Twisted currently offers
more
45
asyncio in Python 3.5
coroutines api
batteries include:
I low level socket operations
I streams & connections
I sleep
I subprocess
I synchronisation primitives
I nonblocking, with coroutines
46
asyncio in Python 3.5
asyncio.org
asyncio.org
I lists PyPI libraries to use with asyncio
I things like:
I HTTP, ZMQ, DNS, Redis, memcache, Mongo, SQL, . . .
I REST, WebSockets, IRC, wsgi, Docker, . . .
I SIP, SSH, XMPP, SMTP, . . .
I files 5 , queue, read-write lock
I pyserial, cron, blender
47 5
workaround using threads
asyncio in Python 3.5
asyncserial
asyncserial
I coroutine api for serial port
asyncserial example
async def foo():
port = asyncserial.AsyncSerial(’/dev/ttyUSB1’)
await port.write(somedata)
response = await port.read(5)
await some_handle_response(response)
48
asyncio in Python 3.5
aiozmq.rpc
aiozmq.rpc
I RPC mechanism on top of ZeroMQ using coroutines
RPC with coroutines
client = await aiozmq.rpc.connect_rpc(connect=’tcp://127.0.0.1:5555’)
ret = await client.call.get_some_value()
await client.call.start_some_remote_action(some_calc(ret))
await asyncio.sleep(42)
await client.call.stop_some_remote_action()
49
asyncio in Python 3.5
aiohttp
aiohttp
I HTTP using coroutines
I even with and for can use coroutines
aiohttp
async def fetch(session, url):
with aiohttp.Timeout(10):
async with session.get(url) as response:
return await response.text()
if __name__ == ’__main__’:
loop = asyncio.get_event_loop()
with aiohttp.ClientSession(loop=loop) as session:
html = loop.run_until_complete(
fetch(session, ’http://python.org’))
50
print(html)
asyncio in Python 3.5
coroutines
AsyncSSH
aiohttp
async def run_client():
async with asyncssh.connect(’localhost’) as conn:
stdin, stdout, stderr = await conn.open_session(’echo "Hello!"’)
output = await stdout.read()
print(output, end=’’)
await stdout.channel.wait_closed()
status = stdout.channel.get_exit_status()
if status:
print(’Program exited with status %d’ % status, file=sys.std
else:
print(’Program exited successfully’)
51
asyncio.get_event_loop().run_until_complete(run_client())
asyncio in Python 3.5
blocking
blocking stuff
I blocking functions should not be called directly
I it will block the loop and all other tasks
I if no high level async API available
I run in executor, like ThreadPoolExecutor
blocking stuff in executor
await loop.run_in_executor(None, my_blocking_func, arg1, arg2)
52
asyncio in Python 3.5
coroutines
aiofiles
I file IO is blocking
I not easily made asynchronous
I aiofiles delegates to thread pool
I unblocking your event loop
I using the future mechanism
I discussion with Guido on GitHub about asynchronous files
blocking stuff behind the scenes
async with aiofiles.open(’filename’, mode=’r’) as f:
contents = await f.read()
53
asyncio in Python 3.5
testing
asynctest
I you’ll want to test coroutines
I but that requires a loop running
I loop aware test
I asynctest module
asynctest
import asynctest
import aiozmq.rpc
class MyRpcTest(asynctest.TestCase):
async def setUp(self ):
self.client = await aiozmq.rpc.connect_rpc(
connect=’tcp://127.0.0.1:5555’)
54
asyncio in Python 3.5
testing
don’t write your unit tests this way!
asynctest testcase
async def test_some_remote_action(self):
cc = self.client.call
r = await cc.get_some_value()
self.assertGreater(someValue, r)
await cc.start_some_remote_action(some_calc(ret))
for _ in range(5):
await time.sleep(0.5)
newr = await cc.get_some_value()
self.assertGreater(newr, r)
r = newr
await cc.stop_some_remote_action()
55
asyncio in Python 3.5
testing
asynctest
I loop aware test
I ideally run unrelated tests concurrently on the same loop
I realistic?
I perhaps not
asynctest
import asynctest
56
asyncio in Python 3.5
testing
asynctest: other features
I ClockedTestCase
I allows to control the loop clock
I run timed events
I without waiting for the wall clock
I accelerated tests anyone?
asynctest
import asynctest
57
asyncio in Python 3.5
testing
asynctest: other features
I CoroutineMock
I FileMock
I SocketMock
asynctest
import asynctest.selector
58
asyncio in Python 3.5
testing
pytest-asyncio
I for those on pytest iso unittest
I haven’t tried it . . .
I claim custom event loop support
I monkey patching coroutines allowed
pytest-asyncio
@pytest.mark.asyncio
async def test_some_asyncio_code():
res = await library.do_something()
assert b’expected result’ == res
59
asyncio in Python 3.5
stopping loop
stopping the loop
I some applications might require stopping the loop
I basically any await statement is an opportunity for the loop
to stop
I it will warn you about unfinished scheduled tasks on the loop
stopping the loop
loop.stop()
60
asyncio in Python 3.5
stopping loop
cancelling a task
I sometimes not required to stop whole loop
I a single task might suffice
stopping the loop
sometask = loop.create_task(my_coroutine())
. . .
sometask.cancel()
61
asyncio in Python 3.5
stopping loop
threadsafety
I the whole thing isn’t threadsafe
I why would it ?
I so take precautions from other threads
stopping threadsafe
loop.call_soon_threadsafe(loop.stop)
loop.call_soon_threadsafe(sometask.cancel)
62
asyncio in Python 3.5
exceptions
exceptions
I raised exceptions from a coroutine
I get set on the internal future object
I and reraised when awaited on
exceptions
async def foo():
raise Exception()
async def bar():
await foo() # Exception time
63
asyncio in Python 3.5
exceptions
exceptions
I but if never awaited
I aka exception never consumed
I it’s logged with traceback 6
exceptions
async def foo():
raise Exception()
asyncio.ensure_future(foo()) # will log unconsumed exception
64 6
Get more logging by enabling asyncio debug mode
asyncio in Python 3.5
logging
logging
I asyncio logs information on the logging module in logger
’asyncio’
I useful to redirect this away from frameworks that steal stdin
and stdout
I like robotframework
65
asyncio in Python 3.5
alternatives
alternatives to asyncio
I as is to be expected . . .
I not everyone completely agrees on Python’s implementation
I and offer partial or complete improvement over asyncio
66
asyncio in Python 3.5
alternatives to asyncio
other loops
I we can use loops other than the standard one
I like uvloop 7
I a fast, drop-in replacement of asyncio event loop
I implements asyncio.AbstractEventLoop
I promises Go-like performance
I expect others . . .
uvloop
import asyncio
import uvloop
loop = uvloop.new_event_loop()
asyncio.set_event_loop(loop)
67 7
https://github.com/MagicStack/uvloop
asyncio in Python 3.5
alternatives
curio: an alternative to asyncio
I by David Beazly
I based on task queueing
I not callback based event loop
I not just the loop
I complete async I/O library
I sockets, files, sleep, signals, synchronization, processes, ssl, ipc
I interactive monitoring
I claims 75 to 150% faster than asyncio
I claims 5 to 40% faster than uvloop
I and about the same speed as gevent
68
asyncio in Python 3.5
alternatives
alternatives to asyncio
I I like standard stuff
I but benefits promised by others make them enticing . . .
69
summary
summary
70
summary
asynchronous programming
I concurrency without threading
I write suspendable functions
I as if it was synchronous code
71
summary
asynchronous programming
I with callbacks in any version
I with @asyncio.coroutine in 3.4
I with async def coroutines in 3.5
72
summary
asynchronous programming
I needs nonblocking api’s
I expect to see many of them
I even replacing blocking ones
I as they can also be used blocking
73
summary
Python 3.6
what about Python 3.6 ?
I a christmas present
I minor asyncio improvements
I run_coroutine_threadsafe
I submit coroutines to event loops in other threads
I timeout() context manager
I simplifying timeouts handling code
I all changes backported to 3.5.x
74
summary
Python 3.6
Python 3.6
I deserves a presentation of its own
I but do checkout formatted string literals
formatted string literal
>>> name = "Fred"
>>> f"He said his name is {name}."
’He said his name is Fred.’
75
summary
Thank you for joining!
76
summary
77
asyncio in Python 3.5
behind the screens
extra slides
78
asyncio in Python 3.5
behind the screens
How to make your library coroutine enabled?
I it’s about operations that happen asynchronously
I often in hardware or network
I that finish somewhere in the future
79
asyncio in Python 3.5
behind the screens
usecase: asyncify pyserial
I use (part of) existing api
I use ”everything is a file” to get async behaviour
I use future objects
80
asyncio in Python 3.5
behind the screens
usecase: asyncify pyserial
I use (part of) existing api
reuse existing api
class AsyncSerialBase:
def __init__(self, port=None, loop=None, timeout=None, write_timeout
**kwargs):
if (timeout is not None
or write_timeout is not None
or inter_byte_timeout is not None):
raise NotImplementedError("Use asyncio timeout features")
self.ser = serial.serial_for_url(port, **kwargs)
if loop is None:
loop = asyncio.get_event_loop()
self._loop = loop
81
asyncio in Python 3.5
behind the screens
usecase: asyncify pyserial
I use ”everything is a file” to get async behaviour
I async by callback that is
going async
self._loop.add_reader(self.fileno(),
self._read_ready, n)
82
asyncio in Python 3.5
behind the screens
usecase: asyncify pyserial
I use future objects
I to replace callback api by coroutines
going async
def read(self, n):
assert self.read_future is None or self.read_future.cancelled()
future = asyncio.Future(loop=self._loop)
. . . # add_reader . . .
return future
83
asyncio in Python 3.5
behind the screens
usecase: asyncify pyserial
I use future objects
I to replace callback api by coroutines
future objects
def _read_ready(self, n):
self._loop.remove_reader(self.fileno())
if not self.read_future.cancelled():
try:
res = os.read(self.fileno(), n)
except Exception as exc:
self.read_future.set_exception(exc)
else:
self.read_future.set_result(res)
self.read_future = None
84
asyncio in Python 3.5
behind the screens
usecase: asyncify pyserial
I use future objects
I because a future returned by a regular function
I can be awaited on
I as if it was a coroutine
future objects
return future
85
asyncio in Python 3.5
behind the screens
usecase: asyncify pyserial
I cleanup
reuse existing api
def close(self):
if self.read_future is not None:
self._loop.remove_reader(self.fileno())
if self.write_future is not None:
self._loop.remove_writer(self.fileno())
self.ser.close()
86
