Threading Programming Using Python
Threading Programming Using Python
USING PYTHON
CUAUHTEMOC CARBAJAL
ITESM CEM
APRIL 06, 2013
1
PROCESS
• Background
• A running program is called a
"process"
• Each process has memory, list of
open files, stack, program counter,
etc...
• Normally, a process executes
statements in a single sequence of
control flow.
• Process creation with
fork(),system(), popen(), etc...
• These commands create an entirely
new process.
• Child process runs independently of
the parent.
• Has own set of resources.
• There is minimal sharing of
information between parent and
child.
• Think about using the Unix shell.
2
ROBOT OPERATING SYSTEM (ROS)
3D visualization tool
3
THREAD BASICS
• Threads
• A thread is a light-weight process (it’s a
sequence of control flow).
• Except that it exists entirely inside a
process and shares resources.
• A single process may have multiple
threads of execution.
• Useful when an application wants to
perform many concurrent tasks on shared
data.
• Think about a browser (loading pages,
animations, etc.)
4
MULTITHREADED WEB SERVER
5
MULTITASKING EMBEDDED SYSTEM
6
ADVANTAGES OF THREADING
• Scheduling
• To execute a threaded program, must
rapidly switch between threads.
• This can be done by the user process (user-
level threads).
• Can be done by the kernel (kernel-level
threads).
• Resource Sharing
• Since threads share memory and other
resources, must be very careful.
• Operation performed in one thread could
cause problems in another.
• Synchronization
• Threads often need to coordinate
actions.
• Can get "race conditions" (outcome
dependent on order of thread execution)
• Often need to use locking primitives
(mutual exclusion locks, semaphores,
etc...)
8
PYTHON THREADS
• Comments
• Python threads are somewhat more restrictive than in C.
• Effectiveness may be limited on multiple CPUs (due to
interpreter lock).
• Threads can interact strangely with other Python modules
(especially signal handling).
• Not all extension modules are thread-safe.
• Thread-safe describes a program portion or routine that can be
called from multiple programming threads without unwanted
interaction between the threads.
10
´PYTHON THREAD MODULE
11
THREAD MODULE
import thread
import time
while 1:
pass
/threading/ex1.py C3PO
13
THREAD MODULE (CONT)
• Thread termination
• Thread silently exits when the function returns.
• Thread can explicitly exit by calling thread.exit() or sys.exit().
• Uncaught exception causes thread termination (and prints error message).
• However, other threads continue to run even if one had an error.
• Simple locks
• allocate_lock(). Creates a lock object, initially unlocked.
import thread
lk = thread.allocate_lock()
def foo():
lk.acquire() # Acquire the lock
# critical section
lk.release() # Release the lock
• Only one thread can acquire the lock at once.
• Threads block indefinitely until lock becomes available.
• You might use this if two or more threads were allowed to update a shared
data structure.
14
#!/usr/bin/env python An asterisk (*) is placed before the
variable name that will hold the values
import time of all nonkeyword variable arguments.
import thread
def myfunction(string,sleeptime,lock,*args):
while 1:
#entering critical section
lock.acquire()
print string," Now Sleeping after Lock acquired for ",sleeptime
time.sleep(sleeptime)
print string," Now releasing lock and then sleeping again "
lock.release()
# exiting critical section
time.sleep(sleeptime) # why?
if __name__ == "__main__":
lock = thread.allocate_lock()
thread.start_new_thread(myfunction,("Thread No:1",2,lock))
thread.start_new_thread(myfunction,("Thread No:2",2,lock))
while 1:
pass
/threading/ex2.py C3PO
15
THREAD MODULE (CONT)
17
PYTHON THREADING
MODULE
18
THREADING — HIGHER-LEVEL
THREADING INTERFACE
• Python manages to get a lot done using so little.
• The Threading module uses the built in thread package to
provide some very interesting features that would make
your programming a whole lot easier.
• There are in built mechanisms which provide critical section
locks, wait/notify locks etc.
• Major Components of the Threading module are:
• Thread Object
• Lock object
• RLock object
• Semaphore Object
• Condition Object
• Event Object
19
THREADING — HIGHER-LEVEL
THREADING INTERFACE (CONT)
• It is a high-level threads module
• Implements threads as classes (similar to Java)
• Provides an assortment of synchronization and locking primitives.
• Built using the low-level thread module.
• Creating a new thread (as a class)
• Idea: Inherit from the "Thread" class and provide a few methods
import threading, time
class PrintTime(threading.Thread):
def __init__(self,interval):
threading.Thread.__init__(self) # Required
self.interval = interval
def run(self):
while 1:
time.sleep(self.interval)
print time.ctime(time.time())
t = PrintTime(5) # Create a thread object
t.start() # Start it
# Do something else
while(1): pass /threading/ex5.py C3PO 20
THREAD CLASS
21
1 CREATE THREAD INSTANCE, PASSING
IN FUNCTION
#!/usr/bin/env
#!/usr/bin/envpythonpython
import
importthreading
threading
from
fromtime
timeimport
importsleep,
sleep,time,
time,ctime
ctime
loops
loops= =[ [4,4,2 2] ]
def
defloop(nloop,
loop(nloop,nsec):nsec):
print
print'start
'startloop',
loop',nloop,
nloop,'at:',
'at:',ctime(time())
ctime(time())
sleep(nsec)
sleep(nsec)
print
print'loop',
'loop',nloop,
nloop,'done
'doneat:',
at:',ctime(time())
ctime(time())
def
defmain():
main():
print
print'starting
'startingthreads...'
threads...'
threads
threads= =[][]
nloops
nloops= =range(len(loops))
range(len(loops))
forfor i in
i in nloops:
nloops:
t t= =threading.Thread(target=loop,
threading.Thread(target=loop,
args=(i,
args=(i,loops[i]))
loops[i]))
threads.append(t)
threads.append(t)
for
fori iininnloops:
nloops: # #start
startthreads
threads
threads[i].start()
threads[i].start()
for
fori iininnloops:
nloops: # #wait
waitfor
forall
all
threads[i].join()
threads[i].join() # #threads
threadstotofinish
finish
print
print'all
'allDONE
DONEat:',
at:',ctime(time())
ctime(time())
ifif__name__
__name__===='__main__':
'__main__':
main()
main() /threading/mtsleep3.py C3PO 22
2 CREATE THREAD INSTANCE, PASSING
IN CALLABLE CLASS INSTANCE
#!/usr/bin/env python def main():
import threading print 'starting threads...'
from time import sleep, time, ctime threads = [] empty list
loops = [ 4, 2 ] nloops = range(len(loops))
for i in nloops:
class ThreadFunc: t = threading.Thread( \
target=ThreadFunc(loop, (i, loops[i]),
def __init__(self, func, args, name=''): loop.__name__))
self.name = name threads.append(t)
self.func = func for i in nloops:
self.args = args threads[i].start()
for i in nloops:
def __call__(self): threads[i].join()
apply(self.func, self.args) print 'all DONE at:', ctime(time())
aclass.py
class Aclass: x = Aclass()
def __call__(self): Hi I am __init__ed
print 'Hi I am __call__ed'; x()
def __init__(self, *args, **keyargs): Hi I am __call__ed
print "Hi I am __init__ed";
Executing x = Aclass() will call __init__() and just x() will call __call__().
test.py (1, 2, 3)
class Test(object):
def __call__(self, *args, **kwargs): {}
print args ---...
print kwargs ()
print '-'*80 {'a': 1, 'c': 3, 'b': 2}
t = Test() ---...
t(1, 2, 3) -
t(a=1, b=2, c=3) (4, 5, 6)
t(4, 5, 6, d=4, e=5, f=6) {'e': 5, 'd': 4, 'f': 6}
callable(t) ---...
24
3 SUBCLASS THREAD AND CREATE
SUBCLASS INSTANCE
#!/usr/bin/env python def main():
import threading print 'starting at:', ctime()
from time import sleep, ctime threads = []
loops = [ 4, 2 ] nloops = range(len(loops))
for i in nloops:
class MyThread(threading.Thread): t = MyThread(loop, (i, loops[i]),
loop.__name__)
def __init__(self, func, args, name=''): threads.append(t)
threading.Thread.__init__(self) for i in nloops:
self.name = name threads[i].start()
self.func = func for i in nloops:
self.args = args threads[i].join()
• start()
• Start the thread’s activity.
• It must be called at most once per thread object.
• It arranges for the object’s run() method to be invoked in
a separate thread of control.
• This method will raise a RuntimeError if called more than
once on the same thread object.
• run()
• Method representing the thread’s activity.
• You may override this method in a subclass. The standard
run() method invokes the callable object passed to the
object’s constructor as the target argument, if any, with
sequential and keyword arguments taken from the args
and kwargs arguments, respectively. 27
CLASS THREADING.THREAD METHODS
(CONT)
• join([timeout])
• Wait until the thread terminates. This blocks the calling
thread until the thread whose join() method is called
terminates – either normally or through an unhandled
exception – or until the optional timeout occurs.
• When the timeout argument is present and not None, it
should be a floating point number specifying a timeout for
the operation in seconds (or fractions thereof).
• As join() always returns None, you must call isAlive()
after join() to decide whether a timeout happened – if
the thread is still alive, the join() call timed out.
• When the timeout argument is not present or None, the
operation will block until the thread terminates.
28
THREAD STATES
also called
ready state
29
THREAD STATES (CONT)
logging.basicConfig(level=logging.DEBUG,
format='(%(threadName)-10s) %(message)s',
)
def daemon():
logging.debug('Starting')
time.sleep(2)
logging.debug('Exiting')
d = threading.Thread(name='daemon', target=daemon)
d.setDaemon(True)
def non_daemon():
logging.debug('Starting') /threading/daemon.py C3PO
logging.debug('Exiting')
t = threading.Thread(name='non-daemon', target=non_daemon)
(daemon ) Starting
d.start() (non-daemon) Starting
32
t.start() (non-daemon) Exiting
SYNCHRONIZATION PRIMITIVES
35
LOCK OBJECTS
39
CONDITION VARIABLES
• Creates a condition variable.
• Synchronization primitive typically used when a thread is interested in an event or
state change.
• Classic problem: producer-consumer problem.
# Create data queue and a condition variable
data = [] releases the lock, and then blocks
cv = threading.Condition() until it is awakened by notify()
# Consumer thread
def consume_item():
cv.acquire() # Acquire the lock
while not len(data):
cv.wait() # Wait for data to show up;
r = data.pop()
cv.release() # Release the lock
return r wakes up one of the threads
waiting for the condition variable,
# Producer thread if any are waiting; does not
def produce_item(obj): release the lock
cv.acquire() # Acquire the lock
data.append("object")
cv.notify() # Notify a consumer
40
cv.release() # Release the lock
SEMAPHORE OBJECTS
• Semaphores
• A locking primitive based on a counter.
• Each acquire() method decrements the counter.
• Each release() method increments the counter.
• If the counter reaches zero, future acquire() methods block.
• Common use: limiting the number of threads allowed to execute code
41
EVENT
42
EVENT OBJECTS
• Events
• A communication primitive for coordinating threads.
• One thread signals an "event"
• Other threads wait for it to happen.
# Create an event object
e = Event()
# Signal the event
def signal_event():
e.set()
# Wait for event
def wait_for_event():
e.wait()
# Clear event
def clear_event():
e.clear()
44
PYTHON QUEUE
45
QUEUE
46
THE QUEUE MODULE
47
BASIC FIFO QUEUE
q = Queue.Queue()
for i in range(5):
q.put(i)
basic_fifo_queue.py C3PO 48
LIFO QUEUE
q = Queue.LifoQueue()
for i in range(5):
q.put(i)
lifo_queue.py C3PO 49
PRIORITY QUEUE
50
PRIORITY QUEUE (2)
import Queue
class Job(object):
def __init__(self, priority, description):
self.priority = priority
self.description = description
print 'New job:', description
return
def __cmp__(self, other):
return cmp(self.priority, other.priority)
q = Queue.PriorityQueue()
53
THREAD EXAMPLE USING THE RPI
(PUSH-BUTTON CIRCUIT)
import threading
import time
import RPi.GPIO as GPIO
class Button(threading.Thread):
"""A Thread that monitors a GPIO button"""
# A program will exit when only daemon threads are left alive
self.daemon = True
# start thread running
self.start()
54
CAS Raspberry Pi Education Manual
def pressed(self):
if self._pressed:
# clear the pressed flag now we have detected it
self._pressed = False
return True
else:
return False
def run(self):
previous = None
while 1:
# read gpio channel
current = GPIO.input(self.channel)
time.sleep(0.01) # wait 10 ms
55
THREAD EXAMPLE USING THE RPI
(CONT)
def onButtonPress():
print('Button has been pressed!')
try:
while True:
# ask for a name and say hello
name = input('Enter a name (or Q to quit): ')
if name.upper() == ('Q'):
break
print('Hello', name)
# check if button has been pressed
if button.pressed():
onButtonPress()
• Required Hardware
• Raspberry Pi with Debian Wheezy
installed
• GPSd compatible GPS Receiver
• Getting the Software
• Enter the following command to install
Python, GPSd, and the Python modules to
bring them together: GlobalSat BU-353 USB GPS
• sudo apt-get install python gpsd Navigation Receiver
gpsd-clients
• Plugging in the USB receiver should
start GPSd automatically.
• To make sure that GPSd is playing
nice, you can open cgps to see what
data it is receiving.
• cgps
57
http://www.danmandle.com/blog/getting-gpsd-to-work-with-python/
import os
from gps import *
from time import *
import time
import threading
class GpsPoller(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
global gpsd #bring it in scope
gpsd = gps(mode=WATCH_ENABLE) #starting the stream of info
self.current_value = None
self.running = True #setting the thread running to true
def run(self):
global gpsd
while gpsp.running:
gpsd.next() #this will continue to loop and grab EACH set of gpsd info to clear the buffer
58
if __name__ == '__main__':
gpsp = GpsPoller() # create the thread
try:
gpsp.start() # start it up
while True:
#It may take a second or two to get good data
os.system('clear')
print
print ' GPS reading'
print '----------------------------------------'
print 'latitude ' , gpsd.fix.latitude
print 'longitude ' , gpsd.fix.longitude
print 'time utc ' , gpsd.utc,' + ', gpsd.fix.time
print 'altitude (m)' , gpsd.fix.altitude
print 'eps ' , gpsd.fix.eps
print 'epx ' , gpsd.fix.epx
print 'epv ' , gpsd.fix.epv
print 'ept ' , gpsd.fix.ept
print 'speed (m/s) ' , gpsd.fix.speed
print 'climb ' , gpsd.fix.climb
print 'track ' , gpsd.fix.track
print 'mode ' , gpsd.fix.mode
print
print 'sats ' , gpsd.satellites
time.sleep(5) #set to whatever
except (KeyboardInterrupt, SystemExit): #when you press ctrl+c
print "\nKilling Thread..."
gpsp.running = False
gpsp.join() # wait for the thread to finish what it's doing
print "Done.\nExiting." 59