Column Pythonista’s Razor

Handle multiple tasks

Your Raspberry Pi project may need to deal with more than one
thing at a time. Learn how to handle multiple tasks in Python
everal articles in this whether or not a given thread is done one thread may overwrite the data

S column have covered

different techniques to
accomplish specific
tasks. What we haven’t covered, so
by using code like that below:

if my_thread.is_alive():
print('This thread is
from another. In these cases, you will
need to use lock objects to manage
access to these global variables. A
basic example would look like:
far, is how best to deal with the case still running')
when your Raspberry Pi project mylock = threading.Lock()
needs to manage several different At some point in the main body of counter = 0
Joey Bernard tasks concurrently. This month, your program, you are going to want def func1():
is a true Renaissance we will look at how to use the to use the results from the functions mylock.acquire()
man. He splits his
time between building
multitasking capabilities within running in these threads. When this counter = counter + 1
furniture, helping Python to manage multiple tasks. happens you can use the join() mylock.release()
researchers with In the standard library, there are method of the thread object. This
scientific computing
three main modules available. They halts the main core of your program As you can see, you create the lock
problems and writing
Android apps. are threading, multiprocessing and and forces it to wait until the thread object in the main body of your
concurrent. Each has its own exits. The thread exits by default program. Then, within the function
strengths and weaknesses. Since when the running function exits. code, you try to acquire the lock. If it
these are all part of the standard But, how do you write code that is free, you get access to it and it is
library, there should not be anything uses threads well? The first item locked. If the lock object has already
extra that you will need to install. to consider is whether you will be been locked by another thread, then
this call to acquire blocks and waits
until the lock has been released. This
Why "How do you write code that is why you need to be really careful
to always have a release statement
Python? uses threads well?" for every acquire statement.
Otherwise, you’ll have a bug that
It’s the official
First, we will look at the threading using data that is globally available will be almost impossible to find
language of the module. There are two ways that you or whether you are using data that after the fact. This also introduces a
Raspberry Pi. can use this module. The first is to should only be visible within the bottleneck to your program, so you
Read the docs at
use it to create new thread objects current thread. If you do need local want to make sure that whatever
www.python.org/doc
that can be told to run some target only data, you can create a local code exists between the acquire and
function within your program. The object that can store these values. lock is the bare minimum required
following is a simple example: The following code stores a string to do the necessary work. This is the
with your author’s name in it: simplest form of locking mechanism
import threading available in Python. If your needs
def my_func(): mydata = threading.local() are greater, you can look at some of
print("Hello World") mydata.myname = 'Joey the other options to see if they might
my_thread = threading. Bernard' offer better control access.
Thread(target=my_func) Along with controlling access
my_thread.start() This would be code used within the to global data, you may need to
function being run by a thread. If you communicate directly between
Assuming that your tasks can be need to use global data, you need threads. This can be handled through
partitioned into separate functions, to consider how different threads an event object, which can be used
you can create a thread for each may try to use this global data. If to set a flag to true or false and
of these functions. One thing to be everyone is reading from a given make that visible to other threads. As
aware of is that these new threads variable, you won’t run into any an example, the code below shows
will not start executing the function issues. The problem arises when how to set and use such a flag:
code until you call the start method. you have multiple threads that may
At that point, the target function will try to write a given variable. In this event1 = threading.Event()
start running asynchronously in the case you’ll end up with a situation def func1():
background. You can check to see known as a race condition, where ....

78
 Python column

Other ways to do
parallelisation
event1.set() use the multiprocessing module. This We’ve looked at how to handle parallel tasks strictly
.... module uses subprocesses to bypass within a Python program. But sometimes you need
def func2(): the GIL completely in order to get true to run other pieces of code asynchronously. In
.... parallel operation. In its most basic these cases, you can use the subprocess module
if event1.set(): use case, you could use something like to execute external code and interact with it. As an
print('I got a lag the code below to get behaviour similar example, we will look at how you could run the ls
from func1') to what you get with threads: program and use its output.
....
import multiprocessing import subprocess
Sometimes, the only communication def f(name): subprocess.run(["ls", "-l"],
you need is to know when all of the print('hello’, name) stdout=subprocess.PIPE)
threads have completed some stage of
their work. Say, you multiple threads p = multiprocessing. The run method accepts as input the external
loading data files and you need to wait Process(target=f, args=('bob’,)) program to be run, along with any parameters.
until everyone is done before moving p.start() By default, run doesn’t send the output from
on to the next stage. In this case, you p.join() the external program back in to the main Python
can do so with barrier objects. Below, code. In this example, we set the input parameter
you can see how you could add a This appears to be the same on the stdout to be the PIPE value, so the output from the
barrier to the two threads (above): surface, but what is happening in the external program is sent back to the calling Python
back-end is radically different. The code. Sometimes, you may want to run this external
barrier1 = threading. process object starts up a new Python program through a shell. In order to do this, you will
Barrier(2) engine in one of a number of ways. need to use the input parameter shell=True.
def func1(): The default on UNIX systems, like the The run method is a simplified interface for
.... Pi, is to fork a new process. The fork running external programs. When you need more
barrier1.wait() method essentially makes a complete control over how the external programs execute,
.... copy of the current Python engine and you can use the Popen method.
def func2(): executes the given function. Another
.... method is to spawn a new Python proc1 = subprocess.Popen(['/bin/ls', '-l'])
barrier1.wait() engine. In the spawn method, only the
.... parts of the current Python engine that To communicate with this external process, you can
is needed for the new Python engine. If use the communicate method. You can get both the
In the above code, you need to set you do need to change it, you can use stdout and stderr streams with the following code:
how many threads will take part in the the following code:
barrier object when you create it. Then, outstream, errstream = proc1.communicate()
when threads use it and call the wait multiprocessing.set_start_
method, they will block until all of the method('spawn') If you want to also send input to the external
threads call the wait method. process, you can include a parameter named input
The threading module is a light, fast If you need to start many with the data. This blocks until the external process
and easy method to add the ability subprocesses, this may help speed finishes and exits. If you need to read from these
divide up the processing within your your code up. The set_start_method streams without waiting for the external program
code, but it does suffer from one should only ever be called once in a to finish, you can get access to pipes for stdout and
major issue. Within the Python core given program. stderr streams. For example, the following code
engine, there is a structure called the Hopefully, this article has given reads from the standard output stream:
GIL (global interpreter lock). The GIL is you some ideas on how to include the
used to control access to certain core ability to manage multiple tasks in proc2 = subprocess.Popen(['ls', '-l'],
functions and data within the Python parallel. This can be a powerful tool stdout=subprocess.PIPE)
interpreter. This means that at certain to make the software design of your proc2_output = proc2.stdout
points, your threads will run only one project more flexible and capable. Be print(proc2_output.read())
at a time. This can introduce a serious aware that we have only been able to
bottleneck in some situations. If you cover the most basic topics in such a If you need to, you can explicitly stop the external
are in this boat, then you may need to short article. process using terminate() or kill() methods.

www.linuxuser.co.uk 79