SuperFastPython.

com Cheat Sheet for Python multiprocessing

Why multiprocessing? Locks and Events Condition Variables and Barriers
Use processes for tasks that are CPU-bound, such Locks protect critical section, events are safe flags. Conditions for wait/notify, barriers for syncing.
as compute tasks and math operations.
Mutex lock Condition variable
Create, Config, Use Process Objects lock = Lock() condition = Condition()
lock.acquire() condition.acquire()
Import # ... # ...
from multiprocessing import * lock.release() condition.release()

Create, run target function Mutex lock, context manager Wait on condition to be notified (blocking)
process = Process(target=task) lock = Lock() with condition:
with lock: condition.wait()
Config process name # ...
process = Process(name=’MyProcess’) Wait on condition for expression (blocking)
Reentrant mutex lock, protect critical section with condition:
Config daemon process (background process) lock = RLock() condition.wait_for(check)
process = Process(daemon=True) with lock:
with lock: Notify any single thread waiting on condition
Extend process # ... with condition:
class CustomProcess(Process): condition.notify(n=1)
def run(): Semaphore, set num positions
# ... semaphore = Semaphore(10) Notify all threads waiting on condition
semaphore.acquire() with condition:
Start process (non-blocking) # ... condition.notify_all()
process.start() semaphore.release()
Barrier, set number of parties
Join process, wait to finish (blocking) Semaphore, context manager barrier = Barrier(5)
process.join() semaphore = Semaphore(10)
with semaphore: Arrive and wait at barrier (blocking)
Join process with timeout # ... barrier.wait()
process.join(timeout=5)
Create event, then set event Arrive and wait at barrier with timeout
Check if process is running (not finished) event = Event() barrier.wait(timeout=0.5)
if process.is_alive(): event.set()
# ...
Check if event is set
Terminate process (SIGTERM) if event.is_set():
process.terminate() # ...

Kill process (SIGKILL) Wait for event to be set (blocking)

process.kill() event.wait()

Access process native PID Wait for event with timeout

process.pid if event.wait(timeout=0.5):
# ...
Module Functions Shared ctypes Queues
Utilities for working with processes. Share data between processes safely. Via Queue, SimpleQueue, JoinableQueue

List of all active child processes Create a shared integer value Create queue
children = active_children() var = Value(‘i’, 100) queue = Queue()

Number of logical CPU cores in system Create a shared floating point value Create queue with limited capacity
num = cpu_count() var = Value(‘f’, 2.2) queue = Queue(100)

Process instance for current process Access a shared value Add item to queue (blocking, if limited)
process = current_process() data = var.value queue.put(item)

Process instance for parent process Change a shared value Add item, with timeout (blocking if limited)
process = parent_process() var.value = 200 try:
queue.put(timeout=0.5)
Add multiprocessing support, if frozen (win32) Create a shared integer array except queue.Full as e:
freeze_support() array = Array(‘i’, (1, 2, 3, 4, 5)) # ...

Create context with start method Access all values in an array Retrieve item from queue (blocking)
ctx = get_context(‘spawn’) for item in array: item = queue.get()
# ...
Set the default method for child processes Retrieve item from queue, with timeout
set_start_method(‘spawn’) Change one value in an array try:
array[0] = 22 item = queue.get(timeout=0.5)
Get a list of all supported start methods except queue.Empty as e:
methods = get_all_start_methods() Pipes # ...

Get the current default start method Create unidirectional pipe Check if queue is empty
method = get_start_method() receiver, sender = Pipe() if queue.empty():
# ...
Protect the entry point Create bidirectional pipe (duplex)
if __name__ == '__main__': conn1, conn2 = Pipe(duplex=True) Check if queue is full
# ... if queue.full():
Send object via pipe # ...
Managers sender.send(‘Hello there’)
Get current capacity of queue
Create and use manager Receive object via pipe (blocking) capacity = queue.qsize()
manager = Manager() data = receiver.recv()
manager.start()
# ... Block until there is data to receive from pipe
manager.shutdown() receiver.poll()

Manager, via context manager, shared lock Block until data to receive, with timeout
with Manager() as manager: if receiver.poll(timeout=0.5):
lock = manager.Lock() # ...