DISTRIBUTED COMPUTING

MODULE-2

# Introduction
Usually causality is tracked using physical time. However, in distributed systems, it is not possible to
have global physical time; it is possible to realize only an approximation of it. The knowledge of the causal
precedence relation among the events of processes helps solve a variety of problems in distributed systems.
Examples of some of these problems is as follows:
Distributed algorithms design The knowledge of the causal precedence relation among events helps ensure
liveness and fairness in mutual exclusion algorithms, helps maintain consistency in replicated databases, and
helps design correct deadlock detection algorithms to avoid phantom and undetected deadlocks.
Tracking of dependent events In distributed debugging, the knowledge of the causal dependency among
events helps construct a consistent state for resuming reexecution; in failure recovery, it helps build a
checkpoint; in replicated databases, it aids in the detection of file inconsistencies in case of a network
partitioning.
Knowledge about the progress The knowledge of the causal dependency among events helps measure the
progress of processes in the distributed computation. This is useful in discarding obsolete information,
garbage collection, and termination detection.
Concurrency measure The knowledge of how many events are causally dependent is useful in measuring
the amount of concurrency in a computation. All events that are not causally related can be executed
concurrently. Thus, an analysis of the causality in a computation gives an idea of the concurrency in the
program.
In a system of logical clocks, every process has a logical clock that is advanced using a set of rules. Every
event is assigned a timestamp and the causality relation between events can be generally inferred from their
timestamps. The timestamps assigned to events obey the fundamental monotonicity property; that is, if an event
a causally affects an event b, then the timestamp of a is smaller than the timestamp of b.

# A framework for a system of logical clocks

A system of logical clocks consists of a time domain T and a logical clock C. Elements of T form a partially
ordered set over a relation <. This relation is usually called the happened before or causal precedence.
Intuitively, this relation is analogous to the earlier than relation provided by the physical time. The logical
clock C is a function that maps an event e in a distributed system to an element in the time domain T, denoted
as C(e) and called the timestamp of e, and is defined as follows:
C : H → T,
such that the following property is satisfied:
for two events ei and ej , ei →ej =⇒ C(ei) < C(ej ).
This monotonicity property is called the clock consistency condition. When T and C satisfy the following
condition,
for two events ei and ej , ei →ej ⇔ C(ei) < C(ej ), the system of clocks is said to be strongly consistent.

# Implementation of logical clocks

Implementation of logical clocks requires addressing two issues : data structures local to every process to
represent logical time and a protocol (set of rules) to update the data structures to ensure the consistency
condition. Each process pi maintains data structures that allow it the following two capabilities:
• A local logical clock, denoted by lci, that helps process pi measure its own progress.
• A logical global clock, denoted by gci, that is a representation of process pi’s local view of the logical
global time. It allows this process to assign consistent timestamps to its local events. Typically, lc i is a
part of gci
The protocol ensures that a process’s logical clock, and thus its view of the global time, is managed
consistently. The protocol consists of the following two rules:
• R1 This rule governs how the local logical clock is updated by a process when it executes an event
(send, receive, or internal).
 • R2 This rule governs how a process updates its global logical clock to update its view of the global
time and global progress. It dictates what information about the logical time is piggybacked in a
message and how this information is used by the receiving process to update its view of the global
time.

# Scalar Time
• Proposed by Lamport in 1978 as an attempt to totally order events in a distributed system.
• Time domain is the set of non-negative integers.
• The logical local clock of a process pi and its local view of the global time are squashed into one
integer variable Ci .
• Rules R1 and R2 to update the clocks are as follows:R1: Before executing an event (send, receive, or
internal), process pi executes the following: Ci := Ci + d (d > 0)
• In general, every time R1 is executed, d can have a different value; however, typically d is kept at 1.
• R2: Each message piggybacks the clock value of its sender at sending time. When a process pi
receives a message with timestamp Cmsg , it executes the following actions:
◮ Ci := max(Ci, Cmsg )
◮ Execute R1.
◮ Deliver the message.
Figure shows evolution of scalar time

# Basic properties of scalar time

Consistency Property
• Scalar clocks satisfy the monotonicity and hence the consistency property:
for two events ei and ej , ei → ej =⇒ C(ei) < C(ej).
Total Ordering
• Scalar clocks can be used to totally order events in a distributed system.
• The main problem in totally ordering events is that two or more events at different processes may
have identical timestamp.
• For example in Figure, the third event of process P1 and the second event of process P2 have
identical scalar timestamp
• A tie-breaking mechanism is needed to order such events. A tie is broken as follows:
• Process identifiers are linearly ordered and tie among events with identical scalar timestamp is
broken on the basis of their process identifiers.
• The lower the process identifier in the ranking, the higher the priority.
• The timestamp of an event is denoted by a tuple (t, i) where t is its time of occurrence and i is the
identity of the process where it occurred.
• The total order relation ≺ on two events x and y with timestamps (h,i) and (k,j), respectively, is
defined as follows: x ≺ y ⇔ (h < k or (h = k and i < j))
Event counting
• If the increment value d is always 1, the scalar time has the following interesting property: if event e
has a timestamp h, then h-1 represents the minimum logical duration, counted in units of events,
required before producing the event e;
• We call it the height of the event e.
• In other words, h-1 events have been produced sequentially before the event e regardless of the
processes that produced these events.
• For example, in Figure, five events precede event b on the longest causal path ending at b.
No Strong Consistency
• The system of scalar clocks is not strongly consistent; that is, for two events ei and ej , C(ei) < C(ej)
=⇒ ei → ej .
• For example, in Figure, the third event of process P1 has smaller scalar timestamp than the third
event of process P2. However, the former did not happen before the latter.
• The reason that scalar clocks are not strongly consistent is that the logical local clock and logical
global clock of a process are squashed into one, resulting in the loss causal dependency information
among events at different processes.
• For example, in Figure, when process P2 receives the first message from process P1, it updates its
clock to 3, forgetting that the timestamp of the latest event at P1 on which it depends is 2.

# Vector Time
• The system of vector clocks was developed independently by Fidge, Mattern and Schmuck.
• In the system of vector clocks, the time domain is represented by a set of n-dimensional non-negative
integer vectors.
• Each process pi maintains a vector vti [1..n], where vti[i] is the local logical clock of pi and describes
the logical time progress at process pi . vti[j] represents process pi ’s latest knowledge of process pj
local time.
• If vti[j]=x, then process pi knows that local time at process pj has progressed till x.
• The entire vector vti constitutes pi ’s view of the global logical time and is used to timestamp events.
• Process pi uses the following two rules R1 and R2 to update its clock: R1: Before executing an event,
process pi updates its local logical time as follows: vti [i] := vti [i] + d ; (d > 0)
• R2: Each message m is piggybacked with the vector clock vt of the sender process at sending time. On
the receipt of such a message (m,vt), process pi executes the following sequence of actions:
1. Update its global logical time as follows: 1 ≤ k ≤ n : vti[k] := max(vti[k], vt[k])
2. Execute R1.
3. Deliver the message m

# Basic properties if vector time

Isomorphism
• If events in a distributed system are timestamped using a system of vector clocks, we have the
following property.
• If two events x and y have timestamps vh and vk, respectively,then
x → y ⇔ vh < vk
x || y ⇔ vh || vk.
• Thus, there is an isomorphism between the set of partially ordered events produced by a distributed
computation and their vector timestamps
Strong Consistency
• The system of vector clocks is strongly consistent; thus, by examining the vector timestamp of two
events, we can determine if the events are causally related.
• However, Charron-Bost showed that the dimension of vector clocks cannot be less than n, the total
number of processes in the distributed computation, for this property to hold.
Event Counting
• If d=1 (in rule R1), then the i th component of vector clock at process pi , vt i [i], denotes the number
of events that have occurred at pi until that instant.
• So, if an event e has timestamp vh, vh[j] denotes the number of events executed by process pj that
causally precede e. Clearly, ∑vh[j] − 1 represents the total number of events that causally precede e
in the distributed computation

# Election Algorithm
• In order to perform coordination, distributed systems employ the concept of coordinators.
• An algorithm for choosing a unique process to play a particular role(coordinator) is called an election
algorithm.
 • Election algorithm assumes that every active process in the system has a unique priority number.
• The process with highest priority will be chosen as the coordinator.
• When a coordinator fails, the algorithm elects that active process which has highest priority number.
• Then this number is send to every active process in the distributed system.

Bully Algorithm
There are 3 types of messages in bully algorithm
Election message – announces an election
Ok message – response to an election message
Coordinator message – announce the identity of the elected process
Steps:-
1. A process can begin an election by sending an election message to processes with high priority
number and waiting for ok messages in response.
2. If none arrives within time T, the process considers itself as the coordinator and sends a coordinator
message to all processes with lower identifiers announcing this
3. Otherwise the other process start election for a coordinator.
4. If coordinator does not respond to it within a time interval T, then it is assumed that coordinator has
failed.
5. Now process P sends election message to every process with high priority number.
6. It waits for responses, if no one responds within time interval T, then process P elects itself as a
coordinator.
7. Then it sends a message to all lower priority number processes that it is elected as their new
coordinator.
8. If a process that was previously down/failed comes back it take over the coordinator job.
9. Biggest guy always wins hence the name bully algorithm.

Ring based election algorithm

• This algorithm applies to systems organized as a logical ring.
 • In this algorithm we assume that the link between the processes are unidirectional.
• Every process can message to other process in clockwise direction only.
• Initially every process is marked as a non participant in an election.
• Any process can begin an election.
• It proceeds by making itself as a participant, placing its identifier in an election message and sending
it to its clockwise neighbour.
• When a process receives an election message it compares the identifier in the message with its own.
• If the arrived identifier is greater, then it forwards the message to its neighbour.
• If the arrived identifier is smaller, then it substitutes its own identifier in the message and forwards
it.
• If the received identifier is that of the receiver itself, then this process’s identifier must be the
greatest, and it becomes the coordinator.
• The coordinator marks itself as a coordinator and sends an elected message to its neighbour.

3 17

4
24

15 24
28

➢ The election wasstarted by process 17.

➢ Process forward to neighbour with greatest identifier
➢ The election message currently contains 24, and forwards
➢ The process 28 will replace 24 with its identifier when the message reaches it
➢ The election message currently contains 28, and
➢ Forwards until the received identifier is that of the receiver itself,
➢ It becomes the coordinator and sends a coordinator message to its neighbours

# Global state and snapshot recording algorithms

• A distributed computing system consists of spatially separated processes that do not share a
common memory and communicate asynchronously with each other by message passing over
communication channels.
• Each component of a distributed system has a local state.
• The state of a process is characterized by the state of its local memory and a history of its activity.
• The state of a channel is characterized by the set of messages sent along the channel less the
messages received along the channel.
• The global state of a distributed system is a collection of the local states of its components.
System model
• The system consists of a collection of n processes p1, p2, ..., pn that are connected by channels.
• There are no globally shared memory and physical global clock and processes communicate by
passing messages through communication channels.
• Cij denotes the channel from process pi to process pj and its state is denoted by SCij .
• The actions performed by a process are modeled as three types of events: Internal events, the
message send event and the message receive event.
• For a message mij that is sent by process pi to process pj , let send(mij) and rec(mij) denote its send
and receive events.
• At any instant, the state of process pi , denoted by LSi , is a result of the sequence of all the events
executed by pi till that instant.
• For an event e and a process state LSi , e∈LSi iff e belongs to the sequence of events that have taken
process pi to state LSi .
• For an event e and a process state LSi , e∉LSi iff e does not belong to the sequence of events that have
taken process pi to state LSi .
• For a channel Cij , the following set of messages can be defined based on the local states of the
processes pi and pj
Transit: transit(LSi , LSj) = {mij |send(mij) ∈ LSi ꓥ rec(mij) ∉ LSj }
• Thus, if a snapshot recording algorithm records the state of processes p i and pj as LSi and LSj ,
respectively, then it must record the state of channel Cij as transit (LSi_LSj).
• There are several models of communication among processes and different snapshot algorithms
have assumed different models of communication.
• In FIFO model, each channel acts as a first-in first-out message queue and thus, message ordering is
preserved by a channel.
• In non-FIFO model, a channel acts like a set in which the sender process adds messages and the
receiver process removes messages from it in a random order.
• A system that supports causal delivery of messages satisfies the following property: “For any two
messages mij and mkj , if send(mij) −→ send(mkj), then rec(mij) −→ rec(mkj)”.

Consistent global state

The global state of a distributed system is a collection of the local states of the processes and the channels.
Notationally, global state GS is defined as, GS = { ՍiLSi , Սi,j SCij }
A global state GS is a consistent global state iff it satisfies the following two conditions :
C1: send(mij)∈LSi ⇒ mij∈SCij ⊕ rec(mij)∈LSj . (⊕ is Ex-OR operator.)
C2: send(mij)∉LSi ⇒ mij ∉SCij ∧ rec(mij)∉LSj .
Condition C1 states the law of conservation of messages. Every message m ij that is recorded as sent in the
local state of a process pi must be captured in the state of the channel Cij or in the collected local state of the
receiver process pj .
Condition C2 states that in the collected global state, for every effect, its cause must be present. If a message
mij is not recorded as sent in the local state of process pi, then it must neither be present in the state of the
channel Cij nor in the collected local state of the receiver process pj . In a consistent global state, every
message that is recorded as received is also recorded as sent.

Interpretation in terms of cuts

• A cut in a space-time diagram is a line joining an arbitrary point on each process line that slices the
space-time diagram into a PAST and a FUTURE.
• A cut is a line joining an arbitrary point on each process line that slices the space–time diagram into a
PAST and a FUTURE.
• A consistent global state corresponds to a cut in which every message received in the PAST of the cut
was sent in the PAST of that cut. Such a cut is known as a consistent cut.
• For example, consider the space-time diagram for the computation illustrated in Figure. Cut C 1 is
inconsistent because message m1 is flowing from the FUTURE to the PAST. Cut C 2 is consistent and
message m4 must be captured in the state of channel C21.
 • Note that in a consistent snapshot, all the recorded local states of processes are concurrent; that is,
the recorded local state of no process casually affects the recorded local state of any other process.

Issues in recording a global state

• If a global physical clock were available, the following simple procedure could be used to record a
consistent global snapshot of a distributed system.
• In this, the initiator of the snapshot collection decides a future time at which the snapshot is to be
taken and broadcasts this time to every process.
• All processes take their local snapshots at that instant in the global time.
• The snapshot of channel Cij includes all the messages that process pj receives after taking the
snapshot and whose timestamp is smaller than the time of the snapshot. (All messages are
timestamped with the sender’s clock.) Clearly, if channels are not FIFO, a termination detection
scheme will be needed to determine when to stop waiting for messages on channels.
• However, a global physical clock is not available in a distributed system and the following two issues
need to be addressed in recording of a consistent global snapshot of a distributed system
• I1: How to distinguish between the messages to be recorded in the snapshot from those not to be
recorded.
Any message that is sent by a process before recording its snapshot, must be recorded in the
global snapshot (from C1).
Any message that is sent by a process after recording its snapshot, must not be recorded in
the global snapshot (from C2).
• I2: How to determine the instant when a process takes its snapshot.
A process pj must record its snapshot before processing a message mij that was sent by process
pi after recording its snapshot.