Synchronization:

Distributed Mutual Exclusion

 2
Distributed Mutual
Exclusion: Introduction
 In the problem of mutual exclusion, concurrent access to a
shared resource by several uncoordinated user-requests is
serialized to secure the integrity of the shared resource
 It requires that the actions performed by a user on a shared
resource must be atomic.
 For correctness, it is necessary that the shared resource be
accessed by a single site (or process) at a time.
 Mutual exclusion is a fundamental issue in the design of
distributed systems and an efficient and robust technique for
mutual exclusion is essential to the viable design of distributed
systems.
 3
Single-Computer vs. Distributed
System

 In single-computer systems, the status of a shared resource and

the status of users is readily available in the shared memory,
and solutions to the mutual exclusion problem can be easily
implemented using shared variables (e.g., semaphores)
 However, in distributed systems, both the shared resources
and the users may be distributed and shared memory does not
exist
 Consequently, approaches based on shared variables are not
applicable to distributed systems and approaches based on
message passing must be used.
 4
DME Algorithms: Classification

 Mutual exclusion algorithms can be grouped into two classes.

 The algorithms in the first class are nontoken-based
 These algorithms require two or more successive rounds of message
exchanges among the sites
 These algorithms are assertion based because a site can enter its critical
section (CS) when an assertion defined on its local variables becomes true
 Mutual exclusion is enforced because the assertion becomes true only at
one site at any given time.
 The algorithms in the second class are token-based
 In these algorithms, a unique token (also known as the PRIVILEGE
message) is shared among the sites
 A site is allowed to enter its CS if it possesses the token and it continues to
hold the token until the execution of the CS is over
 These algorithms essentially differ in the way a site carries out the search
for the token.
 5
System Model

 At any instant, a site may have several requests for CS

 A site queues up these requests and serves them one at a time
 A site can be in one of the following three states:
 requesting CS
 In the requesting CS state, the site is blocked and cannot make further requests for CS
 executing CS
 or neither requesting nor executing CS (i.e., idle)
 In the idle state, the site is executing outside its CS
 In the token-based algorithms, a site can also be in a state where a site holding the
token is executing outside the CS
 Such a state is referred to as an idle token state.
 6
DME: Requirements

 The primary objective of a mutual exclusion algorithm is to maintain mutual

exclusion; that is, to guarantee that only one request accesses the CS at a time.
In addition, the following characteristics are considered important in a mutual
exclusion algorithm:
 Freedom from Deadlocks. Two or more sites should not endlessly wait for
messages that will never arrive.
 Freedom from starvation. A site should not be forced to wait indefinitely to
execute CS while other sites are repeatedly executing CS. That is, every
requesting site should get an opportunity to execute CS in a finite time.
 Fairness. Fairness dictates that requests must be executed in the order they are
made (or the order in which they arrive in the system). since a physical global
clock does not exist, time is determined by logical clocks. Note that fairness
implies freedom from starvation, but not vice-versa.
 Fault Tolerance. A mutual exclusion algorithm is fault-tolerant if in the wake
of a failure, it can reorganize itself so that it continues to function without any
(prolonged) disruptions
 7
DME: Performance

 The performance of mutual exclusion algorithms is generally

measured by the following four metrics:
 The number of messages necessary per CS invocation
 The synchronization delay, which is the time required after a site
leaves the CS and before the next site enters the CS
 The response time, which is the time interval a request waits for its CS
execution to be over after its request messages have been sent out
 The system throughput, which is the rate at which the system executes
requests for the CS.
system throughput = 1/ (sd + E)
where sd is the synchronization delay and E is the average critical
section execution time
 8

CS Request Its Request The site enters The site exits

Arrives message sent out the CS the CS

CS execution
time
Response Time
 9
DME: A Simple Solution

 In a simple solution to distributed mutual exclusion, a site, called the

control site, is assigned the task of granting permission for the CS
execution
 To request the CS, a site sends a REQUEST message to the control
site
 The control site queues up the requests for the CS and grants them
permission, one by one
 This method to achieve mutual exclusion in distributed systems
requires only three messages per CS execution
 10
DME: A Simple Solution

 This naive, centralized solution has several drawbacks.

 First, there is a single point of failure, the control site.
 Second, the control site is likely to be swamped with extra work.
Also, the communication links near the control site are likely to
be congested and become a bottleneck.
 Third, the synchronization delay of this algorithm is 2T because a
site should first release permission to the control site and then the
control site should grant permission to the next site to execute the
CS
Throughput = 1 / 2T + E
 11
Lamport’s Algorithm

 Lamport proposed DME algorithm which was based on his clock

synchronization scheme
 In Lamport’s algorithm
 Every site Si keeps a queue, request_queue, which contains
mutual exclusion requests ordered by their timestamps
 Algorithm requires messages to be delivered in the FIFO order
between every pair of sites
 12
DME: The Lamport’s Algorithm

 Requesting the CS
 When a site Si wants to enter the CS, it sends a REQUEST(tsi,i) message to all the
sites in its request set Ri and places the request on request_queuei, ((tsi, i) is the
timestamp of the request.)
 When a site Sj receives the REQUEST (tsi,i) message from site Si, it returns a
timestamped REPLY message to Si and places site Si’s request on request_queuej
 Executing the CS. Site Si enters the CS when the two following
conditions hold:
 L1: Si has received a message with timestamp larger than (tsi, i) from all other sites.
 L2: Si’s request is at the top of request_queue
 Releasing the CS
 Site Si, upon exiting the CS, removes its request from the top of its request queue and
sends a timestamped RELEASE message to all the sites in its request set
 When a site Sj receives a RELEASE message from site Si, it removes Si’s request
from its request queue
 13
DME: Lamport’s Algorithm

 Correctness
 Performance
 Requires 3(N-1) messages per CS invocation:
 (N-1) REQUEST, (N-1) REPLY, and (N-1) RELEASE messages,
Synchronization delay is T
 Optimization
 Can be optimized to require between 3(N-1) and 2(N-1) messages per
CS execution by suppressing REPLY messages in certain cases
 E.g. suppose site Sj receives a REQUEST message from site Si after it has
sent its own REQUEST messages with timestamp higher than the
timestamp of site Si’s request
 In this case, site Sj need not send a REPLY message to site Si.
 14
DME: The Ricart-Agrawala
Algorithm

 Its an optimization of Lamport’s algorithm

 RELEASE messages are merged with REPLY messages
 When a site Sj, receives a REQUEST message from site Si, it
sends a REPLY message to site Si if Sj is neither requesting nor
executing the CS or if site Sj is requesting and Si’s request’s
timestamp is smaller than site Sj’s own request’s timestamp. The
request is deferred otherwise
 Correctness
 Performance
 Optimization
 15
DME: A generalized Non-
token based Algorithm
 A generalized non-token-based mutual exclusion algorithm for distributed systems
includes all the existing non-token-based mutual exclusion algorithms as special
cases.
 The concept of information structure forms the basis for unifying different non-token-
based mutual exclusion algorithms.
 The information structure of a mutual exclusion algorithm defines the data structure
needed at a site to record the status of other sites.
 The information kept in the information structure is used by a site in making
decisions (i.e., from which sites to request permission) when invoking mutual
exclusion.
 The information structures at a site Si consists of the following three sets:
 a request set Ri,
 an inform set Ii, and
 a status set Sti.
 A site also maintains a variable CSSTAT, which indicates the site's knowledge of the
status of the CS.
 16

 In the generalised algorithm, each request for a CS is assigned a

timestamp that is maintained according to Lamport's scheme.
 Timestamps are used to prioritise requests in case of conflict.
 Requesting the critical section.
 1. To execute CS, a site sends timestamped REQUEST messages to all the
sites in its request set.
 2. On the receipt of a REQUEST message, a site Si takes the following
actions:
 It places the request in its queue (which is ordered by timestamps).
 If CSSTAT indicates that the CS is free, then it sends a GRANT message to the
site at the top of the queue and removes its entry from the queue.
 If the recipient of the GRANT message is in Sti, then CSSTAT is set to indicate
that the site is in CS
 17

 Executing the critical section.

 A site executes the CS only after it has received a GRANT message
from all the sites in its request set.
 Releasing the critical section.
 On exiting the CS, the site sends a RELEASE message to every site
in its inform set. On receiving a RELEASE message, a site Si takes
the following actions:
 CSSTAT is set to free.
 If its queue is nonempty, then it sends a GRANT message to the site at
the top of the queue and removes its entry from the queue. If the recipient
of the GRANT message is in Sti, then CSSTAT is set to indicate that the
site is in the CS.
 The previous action is repeated until CSSTAT indicates that a site is in
the CS or its queue becomes empty
 18
DME: Token based Algorithms

 In token-based algorithms, a unique token is shared among all sites. A site

is allowed to enter its CS if it possesses the token.
 Depending upon the way a site carries out its search for the token, there are
numerous token-based algorithms.
 Suzuki-Kasami’s broadcast algorithm
 Singhal’s heuristic algorithm
 Raymond’s tree-based algorithm
 Token-based algorithms use sequence numbers instead of timestamps.
 Every request for the token contains a sequence number and the sequence
numbers of sites advance independently.
 A site increments its sequence number counter every time it makes a
request for the token.
 A primary function of the sequence numbers is to distinguish between old
and current requests.
 19
DME: Suzuki-Kasami’s
Algorithm
 The Suzuki-Kasami's broadcast algorithm is a token-based exclusion
algorithm
 A unique token is shared among all processes.
 If a process possesses the token, it is allowed to enter its critical section
 Therefore it broadcasts a request message to all other processes.
 The process that possesses the token sends it to the requesting process if
the token-owner is not in the critical section.
 In that case, the token will be sent after leaving the critical section.
 In order to know actual requests, every process keeps an array of integers
RNi[1..n], where n is the number of processes, i the ID of process pi and
RNi[j] is the largest sequence number of requests received so far in a
request message from process pj.
 20

 The token itself has got an internal FIFO-queue in order to save the
requests and an array of integers LN[1..n],
 where n is the number of processes and LN[j] is the sequence number
of the request that process pj executed most recently.
 Thereby it is possible to check, if a process has already been executed
 So this array includes the most actual known requests of every process.
 21


Requesting the critical section
 If the requesting process does not have the token, it increments its sequence
number RNi[i], and sends a REQUEST(I, sn) message to all other sites
 When a site Sj receives this message, it sets RNj[i] to max(RNj[i], sn). If Sj has
the idle token, then it sends the token to Si if RNj[i]=LN[i]+1
 Executing the CS
 Site Si executes the CS when it has received the token
 Releasing the CS. Having finished the execution of the CS, site Si takes the
following actions:
 It sets LN[i] element of the token array equal to RNi[i]
 For everyt site Sj whose ID is not in the token queue, it appends its ID to the
token queue if RNi[j]=LN[j]+1
 If token queue is nonempty after the above update, then it deletes the top site ID
from the queue and sends the token to the site indicated by the ID
 22
DME: Raymond’s Tree-Based
Algorithm
 Developed by Kerry Raymond in 1989
 In progression:
 Ricart/Agrawala : 2*(N-1) messages
 Suzuki/Kasami : N messages
 Maekawa : sqrt(N) messages
 Raymond : log(N) messages
 23
Basic Ideas

 Nodes are arranged as a directed tree, such that the edges of the tree
are assigned directions toward the site (root of the tree) that has the
token
 Every site has a local variable holder that points to an immediate
neighbor node on a directed path to the root node
 Holder variable at sites, define logical tree structure among the sites
 If we follow holder variables at sites, every site has a directed path
leading to the site holding the token
 At root site, holder points to iself
 Every site keeps a FIFO queue, called request_q, which stores the
request fo those neighboring sites that have sent a request to this site,
but have not yet been sent the token
 24
Basic Topology
 25
Algorithm

 Requesting the CS
 When a site wants to enter the CS, it sends a REQUEST message to the node
along the directed path to the roots, provided it does not hold the token and its
request_q is empty. It then adds its request to its request_q
 When a site on the path recieves this message, it places the REQUEST in its
request_q and sends a REQUEST message along the directed path to the root
provided it has not sent out a REQUEST message on its outgoing edge
 When the root site receives a REQUEST message, it sends the token to the site
from which it received the REQUEST message and sets its holder variable to
the point at that site
 When a site receives the token, it deletes the top entry from its request_q, sends
the token to the site indicated in this entry, and sets its holder variable to point at
that site. If the request_q is nonempty at this point, then the site sends a
REQUEST message to the site which is pointed at by holder vairable
 26

 Executing the CS
 A site enters the CS when it receives the token and its own entry is at
the top of its request_q.
 In this case, the site deletes the top entry from its request_q and enters
the CS
 Releasing the CS. After a site has finished execution of the CS, it
takes the following antions:
 It its request_q is nonempty, then it deletes the top entry from its
request_q, sends the token to that site, and sets its holder variable to
point at that site
 If the request_q is nonempty at this point then site sends a REQUEST
message to the site which is pointed at by the holder vairable
 27
DME: Comparative
Performance Analysis
 Response Time. At low loads, there is hardly any contention among requests for
the CS. Therefore, the response time under low load conditions for many algorithms
is simply a round trip message delay (= 2T) to acquire permission or token plus the
time to execute the CS (= E).
 As the load is increased, response time increases in all mutual exclusion algorithms
because contention for access to the CS increases.
 Synchronization Delay is the delay due to the sequential message exchanges
required after a site leaves the CS and before the next site enters the CS. In many of
algorithms, a site exiting the CS directly sends a REPLY message or the token to
the next site to enter the CS, resulting in a synchronization delay of T.
 Message Traffic
 When the rate of CS requests is nonuniformly distributed over sites, Singhal's
heuristic algorithm has the potential to substantially reduce the message traffic by
adapting to the prevailing system state
 28
DME: Summary

 Token-based mutual exclusion algorithms are in general more message-

efficient than non-token-based mutual exclusion algorithms—a single token
message is returned instead of individual reply messages from a number of
sites. Thus, the token can be viewed as all the reply messages lumped together.
 While a general theory exists to unify all non-token-based mutual exclusion
algorithms, no such unifying theme exists for token-based mutual exclusion
algorithms. It has been found that, in general, algorithms that make decisions
based on the current state of the system are more efficient.
 In general, there is a tradeoff between speed and message complexity in mutual
exclusion algorithms. There is no single algorithm that can optimize both speed
and message complexity. Future research on the design of efficient mutual
exclusion algorithms should focus on hybrid mutual exclusion algorithms that
combine the advantages of two mutual exclusion algorithms to provide
improved performance in both speed and message complexity.