CMPS 270: Software

Construction
Fall 2022

SAFE
CONCURRENCY
Adapted from: 6.031 Fall 2019 at MIT

Rida Assaf
Department of Computer
1
Science
 THREAD SAFETY
STRATEGIES
There are basically four ways to make variable access safe in shared-memory
concurrency:

• Confinement. Don’t share variables or data between threads.

• Immutability. Make the shared variables un-reassignable or the shared

data immutable. We’ve talked a lot about immutability already, but there
are some additional requirements for concurrent programming.

• Threadsafe data type. Encapsulate the shared data in an existing

threadsafe data type that does the coordination for you.

• Synchronization. Use synchronization to keep the threads from accessing

shared variables or data at the same time. Synchronization is what you
2
 WHAT THREAD-SAFE
• MEANS
A data type or static method is threadsafe if it behaves correctly
when used from multiple threads, regardless of how those threads
are executed, and without demanding additional coordination from
the calling code.

• “Behaves correctly” means satisfying its specification and

preserving its rep invariant.

• “Regardless of how threads are executed” means threads

might be on multiple processors or time-sliced on the same
processor.

• And “without additional coordination”

3 means that the data
1.
CONFINEMENT
 1. CONFINEMENT
• Our first way of achieving thread safety is confinement.Thread confinement is a simple
idea: you avoid races on reassignable references and mutable data by keeping the
references or data confined to a single thread. Don’t give any other threads the
ability to read or write them directly.
• Since shared mutable state is the root cause of a race condition, confinement solves it by
not sharing the mutable state.
• Local variables are always thread confined. A local variable is stored in the
stack, and each thread has its own stack.
• Avoid Global Variables:
• Unlike local variables, static variables are not automatically thread confined.
• If you have static variables in your program, then you have to make an argument that
only one thread will ever use them, and you
5 have to document that fact clearly.
 1. CONFINEMENT -
EXAMPLE
• Be careful – the variable is
thread confined, but if it’s an
object reference, you also
need to check the object it
points to. If the object is
mutable, then we want to
check that the object is
confined as well – there can’t
be references to it that are
reachable from any other
thread.
• Confinement is what makes 6
1. CONFINEMENT - EXAMPLE
This code starts the thread for computeFact(99) with an
anonymous Runnable.

1- When we start the program, we start with one thread

running main.

7
 1. CONFINEMENT -
EXAMPLE
2- Main creates a second thread using the anonymous Runnable
idiom, and starts that thread.

8
 1. CONFINEMENT -
EXAMPLE
3- At this point, we
have two concurrent
threads of execution.
Their interleaving is
unknown! But one
possibility for the next
thing that happens is
that thread 1 enters
computeFact.
9
 1. CONFINEMENT -
EXAMPLE
4- Then, the next thing that might
happen is that thread 2 also
enters computeFact. At this point,
we see how confinement helps
with thread safety. Each
execution of computeFact has
its own n, i, and result
variables, confined to that
thread. The data they point to
are also confined, and
immutable. If the BigInteger
objects were not confined – if
they were aliased from multiple 1
 1. CONFINEMENT -
EXAMPLE

5- The computeFact
computations proceed
independently, updating
their respective variables.

1
II.
IMMUTABILITY
 II. IMMUTABILITY
• Our second way of achieving thread safety is by
using un-reassignable references and immutable data
types. Immutability tackles the shared- mutable-state
cause of a race condition and solves it simply by
making the shared state not mutable.
• A variable declared final is un-reassignable, and
is safe to access from multiple threads. You can
only read the variable, not write it. Be careful, because
this safety applies only to the variable itself, and we
1
III. THREAD-SAFE
DATATYPES
 III. THREAD-SAFE
• DATATYPES
Our third major strategy for achieving thread safety is to store shared mutable data
in existing threadsafe data types.
• When a data type in the Java library is threadsafe, its documentation will
explicitly state that fact. For example, here’s what StringBuffer says:

[S tringBuffer is] A thread-safe, mutable sequence of characters. A string buffer

is like a String, but can be modified. At any point in time it contains some
particular sequence of characters, but the length and content of the sequence can
be changed through certain method calls.
• String buffers are safe for use by multiple threads.The methods are synchronized
where necessary so that all the operations on any particular instance behave as if
they occur in some serial order that is consistent with the order of the method calls
made by each of the individual threads involved.
1
 III. THREAD-SAFE
DATATYPES
• This is in contrast to StringBuilder:

[S tringBuilder is] A mutable sequence of characters. This

class provides an API compatible with StringBuffer, but with no
guarantee of synchronization. This class is designed for
use as a drop-in replacement for StringBuffer in places
where the string buffer was being used by a single
thread (as is generally the case). Where possible, it is
recommended that this class be used in preference to
StringBuffer as it will be faster under most implementations.
1
 III. THREAD-SAFE
DATATYPES
Threadsafe Collections:

• The collection interfaces in Java – List, Set, Map – have basic

implementations that are not threadsafe. The implementations of these
that you’ve been used to using, namely ArrayList, HashMap, and
HashSet, cannot be used safely from more than one thread.

• Fortunately, just like the Collections API provides wrapper methods that make
collections immutable, it provides another set of wrapper methods to make
collections threadsafe, while still mutable.

• These wrappers effectively make each method of the collection atomic with
respect to the other methods. An atomic action effectively happens all at
once – it doesn’t interleave its internal operations with those of other
actions, and none of the effects of 1 the action are visible to other threads
 SUMMARY SO FAR

• Confinement: not sharing the variables or

data.

• Immutability: sharing, but keeping

the data immutable and variables
un-reassignable.

• Threadsafe data types: storing the shared

mutable data in a single threadsafe datatype.
1
 THE BIG THREE
These ideas connect to our three key
properties of good software as follows:

• Safe from bugs. We’re trying to eliminate a major class

of concurrency bugs, race conditions, and eliminate
them by design, not just by accident
of timing.

• Easy to understand. Applying these general, simple design

patterns is far more understandable than a complex argument
about which thread inter-leavings are possible and which are
not.
1
 IV. LOCKS AND
SYNCHRONIZATI
ON
 OBJECTIVES

• Understand how a lock is used to

protect shared mutable data.
• Be able to recognize deadlock and know
strategies to prevent it.

2
 SYNCHRONIZATION
DEFINITION

• Prevent threads from accessing the shared data

at the same time. This is what we use to
implement a threadsafe type, but we didn’t
discuss it at the time.
• Basically we need a way for concurrent
modules that share memory to synchronize with
each other. 2
 LOCKS

Locks are one synchronization technique. A lock is

an abstraction that allows at most one thread to own
it at a time. Holding a lock is how one thread
tells other threads: “I’m working with this
thing, don’t touch it right now.”

2
 LOCKS CONT’D
Locks have two operations:

• Acquire allows a thread to take ownership of a lock. If a

thread tries to acquire a lock currently owned by
another thread, it blocks until the other thread
releases the lock. At that point, it will contend with
any other threads that are trying to acquire the lock. At
most one thread can own the lock at a time.

• Release relinquishes ownership of the lock, allowing

another thread to take ownership
2 of it.
 LOCKS CONT’D

Using a lock also tells the compiler and processor

that you’re using shared memory concurrently, so
that registers and caches will be flushed
out to shared storage. This avoids the
problem of reordering, ensuring that the owner of
a lock is always looking at up-to-date data.
2
 BLOCKING OPERATIONS
• Blocking means, in general, that a thread waits (without doing
further work) until an event occurs.

• An acquire(L) on thread 1 will block if another thread (say

thread 2) is holding lock L. The event it waits for is thread 2
performing release(L). At that point, if thread 1 can acquire
L, it continues running its code, with ownership of the lock. It is
possible that another thread (say thread 3) was also blocked
on acquire(L). If so, either thread 1 or 3 (the winner is
nondeterministic) will take the2 lock L and continue.The other
 REVISITING THE BANK
ACCOUNT
• Our first example of EXAMPLE
shared
memory concurrency was a bank
with cash machines.
• The bank has several cash
machines, all of which can read
and write the same account
objects in memory.
• Of course, without any
coordination between concurrent
2
 REVISITING THE BANK
ACCOUNT
EXAMPLE
• To solve this problem using locks, we can
add a lock that protects each bank
account. Now, before they can access or
update an account balance, cash
machines must first acquire the lock on
that account.
• In the diagram to the right, both A and B
are trying to access account 1. Suppose B
acquires the lock first. Then A must wait to
read or write the balance until B finishes
and releases the lock.This ensures that A
and B are synchronized, but another 2
 DEADLOCK

When proper and carefully, can rac

used
conditions. ly But then
locks another problem
prevent e
rears its ugly
Because the use of locks requires threads to wait hea
(acquire blocks when another thread is holdingd. the
lock), it’s possible to get into a situation where two
threads are waiting for each other — and hence
neither can make progress.
2
 DEADLOCK EXAMPLE
• In the figure to the right, suppose A
and B are making simultaneous
transfers between two accounts in
our bank.
• A transfer between accounts needs to
lock both accounts, so that money
can’t disappear from the system. A
and B each acquire the lock on
their respective “from” account first: A
acquires the lock on account 1, and B
acquires the lock on account
2. Now, each must acquire the lock on
their “to”
account: so A is waiting for B to
release the account 2 lock, and B3 is
 DEADLOCK CONT’D

Deadlock occurs when concurrent modules are

stuck waiting for each other to do something. A
deadlock may involve more than two modules:
the signal feature of deadlock is a cycle of
dependencies, e.g. A is waiting for B which is
waiting for C which is waiting for A. None of them
can make progress. 3
 DEADLOCKS WITHOUT
LOCKS
You can also have deadlock without using any
locks. For example, a message-passing system can
experience deadlock when message buffers fill up. If a
client fills up the server’s buffer with requests, and then
blocks waiting to add another request, the server may
then fill up the client’s buffer with results and then
block itself. So the client is waiting for the server, and
the server waiting for the client, and neither can
3
 WIZARD
• Suppose we’re modeling the
social network of a series of books.

• Like Facebook, this social network

is bidirectional: if x is friends with y,
is friends
then y with x. The friend()
defriend( ) methods and
enforce
invariant by modifying the that
reps of both
objects, which because they use the
monitor pattern means acquiring the
How synchronized
locks to Works: When a thread
both objects enters a synchronized
as well.
block or method, it acquires the lock. Other threads attempting to
enter a synchronized block or method on the same object are
blocked until the lock is released. The lock is released when the
thread exits the synchronized block or method, either normally or
through an exception.
the lock applies to this (the current object), not the that object
argument.
•For instance, if wizard1.friend(wizard2) is called, the thread will
acquire the lock on wizard1 (the this object) before executing the
 WIZARD EXAMPLE

Let’s create a couple of

wizards:

And then think about what happens So A is holding Harry and waiting
for Snape, and B is holding Snape
when two independent threads are and waiting for Harry. Both
repeatedly running: threads are stuck in friend(), so
neither one will ever manage to
exit the synchronized region and
release the lock to the other.This
3
is a classic deadly embrace.The
 WIZARD EXAMPLE
• We will deadlock ver y rapidly. Here’s why. abo to
Suppose
harr thread Aand is
y.friend(snape), ut
thread B is about to execute snape.friend(harrexecute
y).

• Thread A acquires the lock on harry (because the friend method is synchronized).
• Then thread B acquires the lock on snape (for the same reason).
• They both update their individual reps independently, and then try to call
friend() on the other object
— which requires them to acquire the lock on the other object.
• So A is holding Harry and waiting for Snape, and B is holding Snape and waiting
for Harry. Both threads are stuck in friend(), so neither one will ever manage to
exit the synchronized region and release the lock to the other.This is a classic
deadly embrace.The program simply stops.
• The essence of the problem is acquiring
3 multiple locks, and holding some of the
 DEADLOCK
SOLUTION
#1LOCK ORDERING
• One way to prevent deadlock is to put an ordering on the locks
that need to be acquired simultaneously, and ensuring that all
code acquires the locks in that order.
• In our social network example, we might always acquire the
locks on the Wizard objects in alphabetical order by the wizard’s
name. Since thread A and thread B are both going to need the
locks for Harry and Snape, they would both acquire them in that
order: Harry’s lock first, then Snape’s. If thread A gets Harry’s lock
before B does, it will also get Snape’s lock before B does, because
B can’t proceed until A releases 3Harry’s lock again. The ordering
 DEADLOCK
SOLUTION
#1 LOCK ORDERING

Here’s what the

code might
look like:

3
 DEADLOCK
SOLUTION
#1 isDRAWBACKS
Although lock ordering useful (particularly in code like operating
system kernels), it has a number of drawbacks in practice:
• First, it’s not modular — the code has to know about all the locks in
the system, or at least in its subsystem.
• Second, it may be difficult or impossible for the code to know
exactly which of those locks it will need before it even acquires
the first one. It may need to do some computation to figure it
out. Think about doing a depth-first search on the social
network graph, for example — how would you know which nodes
need to be locked, before you’ve3 even started looking for them?
 DEADLOCK SOLUTION
#2 COARSE-GRAINED LOCKING
A more common approach than lock ordering, particularly
for application programming (as opposed to operating
system or device driver programming), is to use coarser
locking — use a single lock to guard many object instances,
or even a whole subsystem of a program.

For example, we might have a single lock for an entire social

network, and have all the operations on any of its
constituent parts synchronize on that lock. In the code on
the right, all Wizards belong to a Castle, and we just use
that Castle object’s lock to synchronize:
• Coarse-grained locks can have a significant performance
penalty. If you guard a large pile of mutable data with a
single lock, then you’re giving up the ability to access any
of that data concurrently. In the worst case, having a
3
single lock protecting everything, your program
 DEADLOCK
EXERCISE
• In the code below three threads 1, 2, and 3 are trying to acquire
locks on objects alpha, beta, and gamma:

• This system is susceptible to deadlock. For each of the scenarios

below, determine whether the system is in deadlock if the
threads are currently on the indicated
4
 DEADLOCK
EXERCISE
• Scenario A:
• Thread 1 inside using alpha
• Thread 2 blocked on
synchronized (alpha)
• Thread 3 finished 41
 DEADLOCK
• Scenario B:
•
EXERCISE
Thread 1 finished
• Thread 2 blocked on synchronized
(beta)
• Thread 3 blocked on 2nd synchronized
(gamma)

4
 DEADLOCK
EXERCISE
• Scenario C:
•Thread 1 running synchronized (beta)
• Thread 2 blocked on synchronized
(gamma)
• Thread 3 blocked on 1st synchronized
(gamma)

4
 DEADLOCK
EXERCISE
• Scenario D:
• Thread 1 blocked on synchronized
(beta)
• Thread 2 finished
• Thread 3 blocked on 2nd synchronized
(gamma)

4
 DEADLOCK EXERCISE
• In the previous problem,
we saw deadlocks involving
beta and gamma. What about
alpha? Which of the below is
correct :
a)There is a possible
deadlock where thread 1 owns
the lock on alpha.
b)There is a possible
deadlock where thread 2 owns
the lock on alpha.
c)There is a possible
deadlock where thread 3 owns
the lock on alpha. 4
 DEADLOCK EXERCISE
• There are no deadlocks involving
alpha.
• We can reason about it this way:
in order to encounter deadlock,
threads must try to acquire locks
in different orders, creating a cycle in
the graph of who-is-waiting-for-who.
• So we look at alpha vs. beta: are
there two threads that try to
acquire these locks in the opposite
order? No. Only thread 2 acquires
them both at the same time.
• Next we look at alpha vs. gamma:
are there two threads that try to
acquire these locks in the opposite
4
order? No. Both thread 2 and
QUEUES AND
MESSAGE
PASSING
SHARED MEMORY VS MESSAGE
PASSING
• In the shared memory model, concurrent modules interact
by reading and writing shared mutable objects in memory.
Creating multiple threads inside a single Java process
is our primary example of shared-memory
concurrency.
• In the message passing model, concurrent modules
interact by sending immutable messages to one another
over a communication channel. That communication
channel might connect different computers over a network,
4
 SHARED MEMORY VS MESSAGE
PASSING
• Rather than synchronize with locks, message
passing systems synchronize on a shared
communication channel, e.g. a stream or a queue.
• Threads communicating with blocking queues is a
useful pattern for message passing within a single
process.
4
 ADVANTAGES OF MESSAGE
PASSING
• The message passing model has several advantages
over the shared memory model, which boil down to
greater safety from bugs. In message- passing, concurrent
modules interact explicitly, by passing messages through
the communication channel, rather than implicitly
through mutation of shared data.
• Message passing also immuta objects (the
shares only ble messages) requires
objects, which
between we have
modules, already memor
whereas seen can be a
sharing mutable
source of
shared bugs. y
5
 MESSAGE PASSING USING
QUEUES
We can use a queue with blocking operations for message passing between threads.
• In an ordinary Queue:
• add(e) adds element e to the end of the queue.
• remove() removes and returns the element at the head of the queue, or throws
an exception if the queue is empty.

• A BlockingQueue extends this interface:

• additionally supports operations that wait for the
queue to become non-empty when retrieving an element, and
wait for space to become available in the queue when storing an element.
• put(e) blocks until it can add element e to the end of the queue (if the queue
does not have a size bound, put will not block).
• take() blocks until it can remove and return the element at the head of the
queue, waiting until the queue is non-empty.
5
MESSAGE PASSING RACE
CONDITION
• In a previous lecture, we saw in the bank account
example that message-passing doesn’t eliminate the
possibility of race conditions. It’s still possible for
concurrent message-passing processes to interleave
their work in bad ways.
• This particularly happens when a client must send
multiple messages to the module to do what it needs,
because those messages (and the client’s processing
of their responses) may interleave with messages
5
 MESSAGE PASSING
DEADLOCK
• The blocking behavior of blocking queues is very convenient for
programming, but blocking also introduces the possibility of
deadlock. In a deadlock, two (or more) concurrent modules are
both blocked waiting for each other to do something. Since
they’re blocked, no module will be able to make anything
happen, and none of them will break the deadlock.
• Deadlock is much more common with locks than with
message-passing — but when the message-passing queues
have limited capacity, and become filled up to that capacity
with messages, then even a5 message-passing system can
 MORE DEADLOCK
SOLUTIONS
• One solution to deadlock is to design the system so
that there is no possibility of a cycle — so that if A is
waiting for B, it cannot be that B was already (or will
start) waiting for A.
• Another approach to deadlock is timeouts. If a
module has been blocked for too long (maybe 100
milliseconds? or 10 seconds? how to decide?), then
you stop blocking and throw an exception. Now the
5