Concurrency

Rob Miller
Fall 2007

© Robert Miller 2007

Concurrency
Multiple computations running at the same time
¾ Concurrency is everywhere, whether we like it or not

Memory

Network

Multiple computers in a network Multiple processors in a computer

(or multiple cores in a single chip)

¾ Concurrency is useful, too

• Splitting up a computation into concurrent pieces is often faster
• Many apps must handle multiple simultaneous users (e.g. web sites)
• Even single-user applications are better with concurrency (e.g. Eclipse
compiling your Java code in the background while you’re
you re editing it)

© Robert Miller 2007

Models for Concurrent Programming
Shared Memory
¾ Analogy: two processors in a computer, sharing the same physical memory

Concurrent modules A and B

interact by reading & writing shared
A B state in memory

Shared memory

Message Passing
¾ Analogy: two computers in a network, communicating only by network
connections

A and B interact by sending

A B messages to each other through
a communication channel

© Robert Miller 2007

Shared Memory Example
Four customers using cash machines simultaneously
¾ Shared memory model – each cash machine reads and writes the account
balance directlyy

Cash
machines A B C D
deposit $256 withdraw $65 deposit $127 get balance
to account 1 from account 2 to account 1 of account 1

Bank $964 $115 $14 Sh d memory

Shared

account 1 account 2 account 3

© Robert Miller 2007

Race Condition
Suppose A and C run at the same time
A get balance $964 C get balance $964
add deposit + $256 add deposit + $127
write back
b k totall $1220 write back
b k totall $1081
¾ Neither answer is right!
Thiss iss a
an example
e a peo of a race
ace condition
co d t o
¾ A race condition means that the correctness of the program depends on
the relative timing of events in concurrent computations
• “AA is in a race with C
C” A get balance $964
¾ Some interleavings of events may be OK, e.g.: add deposit + $256
but other interleavings produce wrong answers write back total $1220
C get balance $1220
Correctness off a concurrent
C add deposit + $127
program should not depend on write back total $1347
accidents of timingg
¾ Race conditions are very bad! They may be rarely observed, hard to
reproduce, hard to debug, but©usually have very serious effects
Robert Miller 2007
Synchronization
A and C need to synchronize with each other
¾ Locks are a common synchronization mechanism
¾ Holding a lock means “I’m
I m changing this; don
don’tt touch it right now
now”
¾ Suppose C acquires the lock first; then A must wait to read and write the
balance until C finishes and releases the lock
¾ Ensures that A and C are synchronized
synchronized, but B can run independently on a
different account (with a different lock)

Cash
machines A B C D

waiting
iti ffor llockk
waiting for lock

Bank $964 $115 $14 Sh d memory

Shared
lock holder C B (free)
© Robert Miller 2007
account 1 account 2 account 3
Deadlocks
Suppose A and B are making simultaneous transfers
¾ A transfer between accounts needs to lock both accounts, so that money
can’t disappear
pp from the system
y
¾ A and B each acquire the lock on the “from” account
¾ Now each must wait for the other to give up the lock on the “to” account
¾ Stalemate! A and B are frozen,
frozen
and the accounts are locked up. A B
“Deadly embrace” transfer $100 transfer $200
¾ Deadlock occurs when concurrent from account 1 from account 2
modules are stuck waiting for each to account 2 to account 1
other to do something
¾ A deadlock may involve more than
two modules (e.g., a cycle of Shared
transfers among N accounts) memory
¾ You can have deadlock without A B
using locks – example later account 1 account 2
© Robert Miller 2007
Lock Granularity
Preventing the deadlock
¾ One solution is to change the locking granularity – e.g. use one lock on
the entire bank,, instead of a lock on each account

$964 $115 $14 $964 $115 $14

one lock per account one lock for the whole bank

Choosing lock granularity is hard

¾ If locking is too coarse,
coarse then you lose concurrency (e.g.
(e g only one cash
machine can run at a time)
¾ If locking is too fine, then you get race conditions and/or deadlocks
¾ Easy to get this wrong

© Robert Miller 2007

Message Passing Example
Modules interact by sending messages to each other
¾ Incoming requests are placed in a queue to be handled one at a time
¾ Sender doesn’t
doesn t stop working while waiting for an answer to its request; it
handles more requests from its own queue
¾ Reply eventually comes back as another message

A B C D
deposit
p $256 withdraw $65 deposit
p $127 g balance
get
to account 1 from account 2 to account 1 of account 1
get bal
queue for
dep$127
A
Account 1
dep$256 wdrw $65

Account 1 Account 2 Account 3

Accounts are bal: $964 bal: $115 bal: $14
now modules,
© Robert Miller 2007
not just memory locations
Message Passing Has the Same Risks
Message passing doesn’t eliminate race conditions
¾ Suppose the account state machine supports get-balance and withdraw
operations
p (with
( corresponding
p g messages)
g )
¾ Can Alice and Bob always stay out of the OVERDRAWN state?

Al
Alice B b
Bob
withdraw
withdraw
get-balance get-balance
if balance > $75,
$75 if balance > $50,
$50
withdraw $75 withdraw $50
OK OVERDRAWN
Account
bal: $100 get-
balance

¾ Lesson: need to carefully choose the atomic (indivisible) operations of the

state machine – withdraw-if-sufficient-funds would be better
¾ Message-passing can have deadlocks too2007
© Robert Miller – example later
Concurrency Is Hard to Test
Poor coverage
¾ Recall our notions of coverage from last lecture: all states, all transitions, or
all ppaths through
g a state machine
¾ Given two concurrent state machines (with N states and M states), the
combined system has N x M states (and many more transitions and paths)
¾ As concurrency increases
increases, the state space explodes
explodes, and achieving sufficient
coverage becomes infeasible
Poor reproducibility
¾ Transitions
T are nondeterministic,
d i i i depending
d d on relative
l timing off events
that are strongly influenced by the environment
• Delays can be caused by other running programs, other network traffic,
operating
i system scheduling
h d li d decisions,
ii variations
i i iin processor clock
l k
speed, etc.
¾ Test driver can’t possibly control all these factors
¾ So even if state coverage were feasible, the test driver can’t reliably
reproduce particular paths through the combined state machine
© Robert Miller 2007
We’ll Use Message Passing In 6.005
We’ll focus on message passing, not shared memory
¾ Locking strategy for shared-memory paradigm is hard to get right
¾ Message
Message-passing
passing paradigm often aligns directly with the real
real-world
world
workflow of a problem
¾ But message passing is less suited to some problems, e.g. a big shared data
structure

© Robert Miller 2007

Mechanisms for Message-Passing Concurrency
Between computers in a network: sockets & streams
¾ A socket represents one end of a bidirectional network connection
• Suppose your laptop opens a connection to web
web.mit.edu
mit edu
• Then your laptop has a socket representing its end of the connection,
and web.mit.edu has a socket on its end
¾ A byte stream is a sequence of bytes (possibly finite
finite, but not necessarily)
• Each direction of the network connection is a byte stream
• Byte streams are also used for reading and writing files

write(“hello”) read() → “hello”

streams
socket socket

© Robert Miller 2007

Mechanisms for Message-Passing Concurrency
Between processes in a computer
¾ A pipe is a stream between two processes
• Every process has at least two such streams: standard input & standard
output (in Java, System.in and System.out)
• Unix shell commands construct pipes between concurrent processes:
ls | grep java
¾ Sockets are sometimes used, too ls grep

Between threads in a process

¾ A queue is a data structure that stores Eclipse process
objects and returns them in FIFO order
• In Java: java.util.concurrent.BlockingQueue
editor compiler
• Graphical user interface toolkits have thread thread
event queues

© Robert Miller 2007

Processes vs.Threads
Process
¾ A process is an instance of a running program that is isolated from other
processes on the same machine (p
p (particularlyy for resources like memory)
y)
¾ Tries to make the program feel like it has the whole machine to itself – like
a fresh computer has been created, with fresh memory
¾ By default,
default processes have no shared memory (needs special effort)
¾ Automatically ready for message passing (standard input & output streams)
Thread
¾ A thread is a locus of control inside a running program (i.e. position in the
code + stack, representing the current point in the computation)
¾ Simulates making a fresh processor inside the computer, running the
same program and sharing the same memory as other threads in process
¾ Automatically ready for shared memory, because threads share all the
memory in the process (needs special effort to get “thread-local” memory
that’s private to the thread)
¾ Must set up message passing explicitly (e.g. by creating queues)
© Robert Miller 2007
Process Example: Quote Generator
One quote lookup at a time was a bottleneck
¾ The lookups were serialized – one had to finish before next could start
But Yahoo can handle multiple concurrent requests
¾ Overlapping the work for each lookup will speed up the overall program
Q
Quote Quote Yahoo
Generator Yahoo Generator
AAPL? AAPL?
JAVA?
time IBM?
open,ask,chg open,ask,chg

JAVA? open,ask,chg
open,ask,chg

open,ask,chg

original synchronous quote generator concurrent quote generator

© Robert Miller 2007
 Concurrent Quote Generator
QuoteApp process
QuoteApp

AsyncQuoter QuoteDisplayer RTFGenerator

start

AAPL open,ask,chg classes in white are unchanged

from QuoteGenerator
QuoterProc

QuoterProc process Network

Design rules Yahoo server process

¾ Every
E object
bj bbelongs
l to exactly
l one process (actually
( ll JJava enforces
f this)
hi )
¾ Modules in different processes communicate using streams
© Robert Miller 2007
 Error Stream
Processes actually have three streams by default
¾ Standard input, standard output, and standard error (System.err in Java)
QuoteApp
Q A start
Eclipse
QuoteApp process Console process

std output
Black text
AsyncQuoter
start StreamGobbler
Red text
open,
open std error
AAPL debugging msgs,
ask,
chg error msgs
QuoterProc

QuoterProc process

¾ QuoteApp
Q A redirects
di subprocess’s
b ’ standardd d error to its
i own std
d error
¾ This is useful, because it lets you see error messages from the subprocess
(like uncaught exceptions) in the Eclipse console view
© Robert Miller 2007
Deadlock
Not just helpful for debugging
¾ You must consume the standard error stream from a subprocess, even if
yyou don’t care what it says
y
¾ Failing to do this can lead to deadlock
• Streams have finite buffers – when the buffer is full, the writer has to
wait (“block”)
( block ) until a reader starts to empty it
• So if subprocess fills up its error stream, and main process doesn’t
empty it, then the subprocess will eventually block
• Meanwhile the main process eventually has to wait for replies back
from the subprocess – so both processes are waiting for each other
• Deadlock!

Can you deadlock on the input and output streams too?

© Robert Miller 2007

Summary
Concurrency
¾ Multiple computations running simultaneously
Shared memory & message-passing
Shared-memory message passing paradigms
¾ Shared memory needs a synchronization mechanism, like locks
¾ Message passing synchronizes on communication channels, like streams or
queues
Race conditions
¾ When correctness of result depends
p on relative timingg of events
Deadlock
¾ When concurrent modules get stuck waiting for each other
Processes & threads
¾ Process is like a virtual computer; thread is like a virtual processor

© Robert Miller 2007