CST 402 DISTRIBUTED COMPUTING

MODULE 1

Prepared by: Ms. JENI MONI

Assistant Professor-CSE
 SYLLABUS
Distributed System – Definition, Relation to computer
system components, Motivation, Primitives for
distributed communication, Design issues, Challenges
and applications.
A model of distributed computations – Distributed
program, Model of distributed executions, Models of
communication networks, Global state of a distributed
system, Cuts of a distributed computation, Past and
future cones of an event, Models of process
communications.
 DISTRIBUTED SYSTEMS
• A distributed system is a collection of independent computers
that cooperate to solve a problem that cannot be individually
solved.
• A distributed system can be characterized in one of several
ways:
1. Crash of a single machine never prevents from doing
work.
2. A collection of computers that do not share common
memory or a common physical clock, that communicate
by a messages passing over a communication network,
and where each computer has its own memory and runs
its own operating system.
 CONT..
3. A collection of independent computers that appears to the
users of the system as a single coherent computer.
4. A term that describes a wide range of computers, from
weakly coupled systems such as wide-area networks, to
strongly coupled systems such as local area networks, to very
strongly coupled systems such as multiprocessor systems.
 FEATURES OF DISTRIBUTED COMPUTING
1. No common physical clock: It introduces the element of
“distribution” in the system and gives rise to the inherent
asynchrony amongst the processors.
2. No shared memory: Requires message-passing for
communication.
3. Geographical separation: The geographically wider apart
that the processors are, the more representative is the
system of a distributed system.
4. Autonomy and heterogeneity: The processors are “loosely
coupled” in that they have different speeds and each can be
running a different operating system, cooperate with one
another by offering services or solving a problem jointly.
 Relation to Computer System
Components

• Each computer has a memory-processing unit and the

computers are connected by a communication network.
CONT..
 CONT..
• Figure 1.2 shows the relationships of the software
components that run on each of the computers and use the
local operating system and network protocol stack for
functioning.
• The distributed software is termed as middleware.
• A distributed execution is the execution of processes across
the distributed system to collaboratively achieve a common
goal.
• The distributed system uses a layered architecture to break
down the complexity of system design.
 CONT..
• The middleware is the distributed software that drives the
distributed system, while providing transparency of
heterogeneity at the platform level
• The middleware layer does not contain the traditional
application layer functions of the network protocol stack, such
as http, mail, ftp, and telnet.
• Various primitives and calls to functions defined in various
libraries of the middleware layer are embedded in the user
program code.
• There exist several libraries to choose from to invoke
primitives for the more common functions – such as reliable
and ordered multicasting – of the middleware layer.
 MOTIVATION

1. Inherently distributed computations: In many applications

such as Money transfer in banking, or reaching consensus
among parties that are geographically distant in which
computations is inherently distributed.
2. Resource sharing: Resources such as peripherals, complete
data sets in databases, special libraries as well as data can’t
be fully replicated at all the sites. Also they can’t be placed
at a single site. So such resources are distributed across the
system.
3. Access to geographically remote data and resources: data
can be stored in the central server and all the associated
clients can access it from anywhere in the world.
 CONT..
4. Enhanced reliability: Reliability has many aspects
 Availability: The resources should be accessible at all time
 Integrity: The value/state of the resource should be
correct
 Fault-tolerance: The ability to recover from system
failures
5. Increased performance/cost ratio: By resource sharing
and accessing geographically remote data and resources,
the performance/ cost ratio is increased.
• Any task can be partitioned across the various computers in
the distributed system.
 CONT..
6. Scalability: Adding more processors does not pose a direct
bottleneck for the communication network.
7. Modularity and incremental expandability: Heterogeneous
processors may be easily added into the system without
affecting the performance, as long as those processors are
running the same middleware algorithms.
• Similarly, existing processors may be easily replaced by other
processors.
Primitives for Distributed Communication

• Blocking/non-blocking
synchronous/asynchronous primitives
 Blocking/Non-blocking,
Synchronous/Asynchronous Primitives
Message send and message receive communication primitives
are denoted as Send() and Receive().
Send() : has at least two parameters, the destination, and the
buffer in the user space
Receive() : at least two parameters, the source from which the
data is to be received and the user buffer into which the data is
to be received.
There are two ways of sending data when the Send primitive is
invoked
1) Buffered option
◦ The buffered option which is the standard option copies the
data from the user buffer to the kernel buffer. The data later gets
copied from the kernel buffer onto the network.
 CONT..
2) Unbuffered option
◦ In the unbuffered option, the data gets copied directly from the
user buffer onto the network.
◦ For the Receive primitive, the buffered option is usually
required because the data may already have arrived when the
primitive is invoked, and needs a storage place in the kernel.
 Synchronous and Asynchronous Primitives

 Synchronous Send or Receive

 Sender and Receiver handshake with each other.
 Send primitive completes when receive primitive
completes
 Receive primitive completes when the data to be received
is copied into the receiver’s user buffer.
Asynchronous Send
 Control returns back to the invoking process after the data
item to be sent has been copied out of the user-specified
buffer.
CONT..
 Blocking and Non blocking Primitives
 Blocking (send or receive)
• Control returns to the invoking process after the processing
for the primitive (whether in synchronous or asynchronous
mode) completes.
 Non-blocking (send or receive)
• Control returns back to the invoking process immediately
after invocation, even though the operation has not
completed.
• For a non-blocking Send, control returns to the process even
before the data is copied out of the user buffer.
• For a non-blocking Receive, control returns to the process
even before the data may have arrived from the sender.
 CONT..
Four Versions of the Send Primitive

1. Asynchronous blocking send

• Does not return until the required message has been copied
out of the buffer in the sender process.
• Does not wait for acknowledgement from the receiver
process.

2. Synchronous blocking send

• Does not return until acknowledgement arrives back to the
sender process that the required message has been placed
into the buffer of the receiver process.
 CONT..
3. Asynchronous non-blocking send
• Can return before the required message has been copied out
of the buffer from the sender process.
• Does not involve acknowledgement from the receiver
process.

4, Synchronous non-blocking send

• Can also return before acknowledgement arrives at the
sender process that the required message has been place into
the buffer of the receiver process.
 CONT..
Two Versions for the Receive primitive

1. Blocking receive
Does not return until the message has arrived and been copied
into the buffer of the receiver process.

2. Non-blocking receive
Can return before the message has arrived and been copied into
the buffer of the receiver process.
 PROCESSOR SYNCHRONY

• Processor synchrony indicates that all the processors execute

in a lock-step with their clocks synchronized.
• For a large granularity of code, usually termed as a step, the
processors are synchronized.
• This abstraction is Implemented using some of form of barrier
synchronization to ensure that no processor begins executing
the next step of code until all the processors have completed
executing the previous steps of code assigned to each of the
processors.
 Design Issues and Challenges

Design issues and challenges after categorizing them as

1. having a greater component related to systems design and
operating systems design (system perspective)
2. having a greater component related to algorithm design
3. emerging from recent technology advances and/or driven by
new applications
 Distributed Systems Challenges from a
System Perspective
1. Communication
2. Processes
3. Naming
4. Synchronization
5. Data storage and access
6. Consistency and replication
7. Fault tolerance
8. Security
9. Applications Programming Interface (API) and transparency
10. Scalability and modularity
1. Communication

• This task involves designing appropriate mechanisms for

communication among the processes in the network.
• Some example mechanisms are: remote procedure call (RPC),
remote object invocation (ROI), message-oriented
communication versus stream-oriented communication.
2. Processes
Some of the issues involved are:
– management of processes and threads at clients/servers;
– code migration;
– and the design of software and mobile agents.
 3. Naming

• Devising easy to use and robust schemes for names,

identifiers, and addresses is essential for locating resources
and processes in a transparent and scalable manner.
• Naming in mobile systems also provides additional challenges.

4. Synchronization
• Mechanisms for synchronization or coordination among the
processes are essential.
• Mutual exclusion is the classical example of synchronization,
but many other forms of synchronization, such as leader
election are also needed.
5. Data Storage and Access

• Schemes for data storage, and implicitly for accessing the data
in a fast and scalable manner across the network are
important for efficiency.
• Traditional issues such as file system design have to be
reconsidered in the setting of a distributed system.
6. Consistency and Replication
• To avoid bottlenecks, to provide fast access to data, and to
provide scalability, replication of data objects is highly
desirable.
• This leads to issues of managing the replicas, and dealing with
consistency among the replicas/caches in a distributed
setting.
• A simple example issue is deciding the level of granularity
(i.e., size) of data access.
7. Fault Tolerance

• Fault tolerance requires maintaining correct and efficient

operation in spite of any failures of links, nodes, and
processes.
• Process resilience, reliable communication, distributed
commit, checkpointing and recovery, agreement and
consensus, failure detection, and self-stabilization are some of
the mechanisms to provide fault-tolerance.
8. Security

• Distributed systems security involves

– various aspects of cryptography
– secure channels
– access control
– key management – generation and distribution
– authorization
– secure group management
9. Applications Programming Interface (API) and Transparency

• Transparency deals with hiding the implementation policies

from the user, and can be classified as follows
1. Access transparency hides differences in data
representation on different systems and provides uniform
operations to access system resources.
2. Location transparency makes the locations of resources
transparent to the users.
3. Migration transparency allows relocating resources without
changing names.
 CONT..
4. Relocation transparency: The ability to relocate the
resources as they are being accessed is.
5. Replication transparency does not let the user become
aware of any replication.
6. Concurrency transparency deals with masking the
concurrent use of shared resources for the user.
7. Failure transparency refers to the system being reliable and
fault-tolerant.
10. Scalability and Modularity

• The algorithms, data (objects), and services must be as

distributed as possible.
• Various techniques such as replication, caching and cache
management, and asynchronous processing help to achieve
scalability.
Algorithmic Challenges in Distributed
Computing
1. Designing useful execution models and frameworks
2. Dynamic distributed graph algorithms and distributed
routing algorithms
3. Time and global state in a distributed system
4. Synchronization/coordination mechanisms
5. Group communication, multicast, and ordered message
delivery
6. Monitoring distributed events and predicates
7. Distributed program design and verification tools
8. Debugging distributed programs
9. Data replication, consistency models, and caching
1. Designing Useful Execution Models and
Frameworks
• The interleaving model and partial order model are two
widely adopted models of distributed system executions.
• These models are particularly useful for operational reasoning
and the design of distributed algorithms.
• The input/output automata model and the TLA (temporal
logic of actions) are two other examples of models.
• Provides different degrees of infrastructure for reasoning
more formally with and proving the correctness of distributed
programs .
 2. Dynamic Distributed Graph Algorithms
and Distributed Routing Algorithms
• The distributed system is modeled as a distributed graph.
• Graph algorithms form the building blocks for a large number
of higher level communication, data dissemination, object
location, and object search functions.
• The algorithms need to deal with dynamically changing graph
characteristics, such as to model varying link loads in a routing
algorithm.
• The efficiency of these algorithms affects not only the user-
perceived latency but also the traffic and hence the load or
congestion in the network.
• Hence, the design of efficient distributed graph algorithms is
very important.
 3. Time and Global State in a Distributed
System
• The challenges pertain to providing accurate physical time,
and to providing a variant of time, called logical time
• Logical time is relative time, and eliminates the overheads of
providing physical time for applications where physical time is
not required.
• It is not possible for any one process to directly observe a
meaningful global state across all the processes, without using
extra state gathering effort which needs to be done in a
coordinated manner.
 4. Synchronization/Coordination
Mechanisms
• Synchronization is essential for the distributed processes to
overcome the limited observation of the system state from
the viewpoint of any one process.
• Overcoming this limited observation is necessary for taking
any actions that would impact other processes.
• The synchronization mechanisms can also be viewed as
resource management and concurrency management
mechanisms to streamline the behavior of the processes that
would otherwise act independently.
 Problems Requiring Synchronization
• Physical clock synchronization
• Leader election
• Mutual exclusion
• Deadlock detection and resolution
• Termination detection
• Garbage collection
 i. Physical Clock Synchronization

• Physical clocks usually diverge in their values due to hardware

limitations.
• Keeping them synchronized is a fundamental challenge to
maintain common time.
 ii. Leader Election

• All the processes need to agree on which process will play the
role of a distinguished process – called a leader process.
• A leader is necessary even for many distributed algorithms
because there is often some asymmetry as in initiating some
action like a broadcast or collecting the state of the system, or
in “regenerating” a token that gets “lost” in the system
 5. Group Communication, Multicast, and
Ordered Message Delivery
• A group is a collection of processes that share a common
context and collaborate on a common task within an
application domain.
• Specific algorithms need to be designed to enable efficient
group communication and group management wherein
processes can join and leave groups dynamically, or even
fail.
• When multiple processes send messages concurrently,
different recipients may receive the messages in different
orders, possibly violating the semantics of the distributed
program.
• Hence, formal specifications of the semantics of ordered
delivery need to be formulated, and then implemented.
 6. Monitoring distributed events and
predicates
• Predicates are used for specifying conditions on the global
system state, and are useful for applications such as
debugging, sensing the environment, and in industrial process
control.
• An important paradigm for monitoring distributed events is
that of event streaming, wherein streams of relevant events
reported from different processes are examined collectively to
detect predicates.
 7. Distributed Program Design and
Verification Tools
• Methodically designed and verifiably correct programs can
greatly reduce the overhead of software design, debugging,
and engineering.
• Designing mechanisms to achieve these design and
verification goals is a challenge.
 8. Debugging Distributed Programs

• Debugging distributed programs is harder because of the

concurrency in actions and the ensuing uncertainty due to the
large number of possible executions defined by the
interleaved concurrent actions.
• Adequate debugging mechanisms and tools need to be
designed to meet this challenge.
 9. Data Replication, Consistency Models,
and Caching
• Fast access to data and other resources requires them to be
replicated in the distributed system.
• Managing such replicas in the face of updates introduces the
problems of ensuring consistency among the replicas and
cached copies.
• Additionally, placement of the replicas in the systems is also a
challenge because resources usually cannot be freely
replicated.
Applications of Distributed Computing
and Newer Challenges
1. Mobile systems
2. Sensor networks
3. Ubiquitous or pervasive computing
4. Peer-to-peer computing
5. Publish-subscribe, content distribution, and multimedia
6. Distributed agents
7. Distributed data mining
8. Grid computing
9. Security in distributed systems
 1. Mobile Systems

• Mobile systems typically use wireless communication which is

based on electromagnetic waves and utilizes a shared
broadcast medium.
• The characteristics of communication are different; set of
problems such as
i. routing,
ii. location management,
iii. channel allocation,
iv. localization and position estimation, and
v. the overall management of mobility
 2. Sensor Networks

• A sensor is a processor with an electro-mechanical interface

that is capable of sensing physical parameters, such as
temperature, velocity, pressure, humidity, and chemicals.
• Sensors may be mobile or static; sensors may communicate
wirelessly, although they may also communicate across a wire
when they are statically installed.
• Sensors may introduces a whole new set of challenges, such
as position estimation and time estimation.
 3. Ubiquitous or Pervasive Computing
• Ubiquitous systems are essentially distributed systems
• Such systems are typically characterized as having many small
processors operating collectively in a dynamic ambient
network.
• The intelligent home, and the smart workplace are some
example of ubiquitous environments .
• The processors may be connected to more powerful
networks and processing resources in the background for
processing and collating data.
– For example, it may be convenient for users to control
their washing machine or their entertainment system from
their phone or a ‘universal remote control’ device in the
home. Equally, the washing machine could notify the user
via a smart badge or phone when the washing is done.
 4. Peer-to-peer Computing

• Peer-to-peer (P2P) computing represents computing over an

application layer network wherein all interactions among the
processors are at a “peer” level, without any hierarchy among
the processors.
• P2P computing arose as a paradigm shift from client–server
computing where the roles among the processors are
essentially asymmetrical.
• P2P networks are typically self-organizing, and may or may
not have a regular structure to the network.
• No central directories for name resolution and object lookup
are allowed.
5. Publish-Subscribe, Content Distribution,
and Multimedia
• In a dynamic environment where the information constantly fluctuates ,
there needs to be:
 an efficient mechanism for distributing this information (publish)
 an efficient mechanism to allow end users to indicate interest in
receiving specific kinds of information (subscribe)
 an efficient mechanism for aggregating large volumes of published
information and filtering it as per the user’s subscription filter.
• Content distribution refers to a class of mechanisms in which specific
information which can be broadly characterized by a set of parameters is
to be distributed to interested processes.
• Multimedia data is usually very large and information-intensive, requires
compression, and often requires special synchronization during storage
and playback .
 6. Distributed Agents

• Agents are software processes or robots that can move

around the system to do specific tasks for which they are
specially programmed.
• Agents collect and process information, and can exchange
such information with other agents.
• Challenges in distributed agent systems include coordination
mechanisms among the agents, controlling the mobility of the
agents, and their software design and interfaces.
 7. Distributed Data Mining

• The data is necessarily distributed and cannot be collected in

a single repository.
• Applications such as banking where the data is private and
sensitive, or in atmospheric weather prediction where the
data sets are far too massive to collect and process at a single
repository in real-time.
• In such cases, efficient distributed data mining algorithms are
required.
 8. Grid Computing

• Grid Computing is a subset of distributed computing

• Distributed computing uses a centralized resource manager
and all nodes cooperatively work together as a single unified
resource or a system.
• Grid computing utilizes a structure where each node has its
own resource manager and the system does not act as a
single unit.
• Many challenges in making grid computing a reality include:
– scheduling jobs in such a distributed environment,
– a framework for implementing quality of service and real-
time guarantees,
– Security of individual machines as well as of jobs being
executed in this setting.
 A Distributed Program
• A distributed program is composed of a set of n asynchronous
processes P1, p2,…,pi,…, pn that communicate by message
passing over the communication network.
• We assume that each process is running on a different
processor.
• The processes do not share a global memory and
communicate solely by passing messages.
• Let Cij denote the channel from process pi to process pj and
let mij denote a message sent from pi to pj .
• The communication delay is finite and unpredictable.
• Process execution and message transfer are asynchronous-a
process may execute an action spontaneously and a process
sending a message does not wait for the delivery of the
message to be complete.
 A Model of Distributed Executions
• The execution of a process consists of a sequential execution
of its actions.
• The actions are atomic and the actions of a process are
modeled as three types of events, namely,
 Internal events
 Message send events
 Message receive events
• For a message m, let send(m) and rec(m) denote its send and
receive events, respectively.
• The occurrence of events changes the states of respective
processes and channels, thus causing transitions in the global
system state.
 CONT..
• An internal event changes the state of the process at which it
occurs.
• A send event changes the state of the process that sends the
message and the state of the channel on which the message is
sent.
• A receive event changes the state of the process that receives
the message and the state of the channel on which the
message is received.
 CONT..
• Let 𝑒𝑖𝑥 denote the xth event at process pi.
• The events at a process are linearly ordered by their order of
occurrence.
• The execution of process 𝑃𝑖 produces a sequence of events
𝑒𝑖1 , 𝑒𝑖2 , , , 𝑒𝑖𝑥 , 𝑒𝑖𝑥+1 and is denoted by 𝐻𝑖 :
𝑯𝒊 =(𝒉𝒊 →𝒊 )
where hi is the set of events produced by 𝑝𝑖 and binary relation
→𝑖 defines a linear order on these events.
• Relation →𝑖 expresses causal dependencies among the events
of 𝑃𝑖 .
 CONT..
• For every message m that is exchanged between two
processes, we have
send(m)→𝑚𝑠𝑔 rec(m)
• Relation →𝑚𝑠𝑔 defines causal dependencies between the
pairs of corresponding send and receive events.
Space–Time Diagram of a Distributed Execution
Involving Three Processes

• A horizontal line represents the progress of the process; a dot

indicates an event; a slant arrow indicates a message transfer.
• In this figure, for process p1, the second event is a message
send event, the third event is an internal event, and the fourth
event is a message receive event.
 CASUAL PRECEDENCE RELATION
• The execution of a distributed application results in a set of
distributed events produced by the processes.
• Let H =𝑈𝑖 ℎ𝑖 denote the set of events executed in a distributed
computation.
• Next, we define a binary relation on the set H, denoted as →,
that expresses causal dependencies between events in the
distributed execution.
• Note that the relation → is Lamport’s “happens before”
relation.
• For any two events 𝑒𝑖 and 𝑒𝑗 , if 𝑒𝑖 → 𝑒𝑗 , then event 𝑒𝑗 is
directly or transitively dependent on event 𝑒𝑖 .
 CONT..
• Graphically, it means that there exists a path in the space–
time diagram that starts at 𝑒𝑖 and ends at 𝑒𝑗 .
• The relation → denotes flow of information in a distributed
computation and 𝑒𝑖 → 𝑒𝑗 dictates that all the information
available at 𝑒𝑖 is potentially accessible at 𝑒𝑗 .
 Logical vs. physical concurrency

• In a distributed computation, two events are logically

concurrent if and only if they do not causally affect each
other.
• Physical concurrency, on the other hand, has a connotation
that the events occur at the same instant in physical time.
• Note that two or more events may be logically concurrent
even though they do not occur at the same instant in physical
time.
• However, if processor speed and message delays had been
different, the execution of these events could have very well
coincided in physical time.
 Models of Communication Networks

• There are several models of the service provided by

communication networks, namely, FIFO (first-in, first-out),
non-FIFO, and causal ordering.
• FIFO model: each channel acts as a first-in first-out message
queue and thus, message ordering is preserved by a channel.
• Non-FIFO model: a channel acts like a set in which the sender
process adds messages and the receiver process removes
messages from it in a random order.
• Causal ordering model: based on Lamport’s “happens
before” relation.
 CONT..
• A system that supports the causal ordering model satisfies the
following property:
CO: For any two messages 𝒎𝒊𝒋 and 𝒎𝒌𝒋 ,
if send(𝒎𝒊𝒋 )send(𝒎𝒌𝒋 ) then rec(𝒎𝒊𝒋 )→ rec(𝒎𝒌𝒋 ).
• That is, this property ensures that all the messages are
processed in order that they were created
• Causally ordered delivery of messages implies FIFO message
delivery.
• Furthermore, note that CO ⊂ FIFO ⊂ Non-FIFO.
• Causal ordering model is useful in developing distributed
algorithms since it considerably simplifies the design of
distributed algorithms because it provides a built-in
synchronization.
 Global State of a Distributed System

• The global state of a distributed system is a collection of the

local states of its components, namely, the processes and the
communication channels.
• Notationally, global state GS can be defined as:

• The state of a process at any time is defined by the contents

of processor registers, stacks, local memory, etc. and depends
on the local context of the distributed application.
• The state of a channel is given by the set of messages carrying
in the channel. Also depends upon the states of the processes
it connects.
 CONT..
• The occurrence of events changes the states of respective
processes and channels, thus causing transitions in global
system state.
• An internal event changes the state of the process at which it
occurs.
• A send event changes the state of the process that sends the
message and the state of the channel on which the message is
sent.
• A receive event changes the state of the process that receives
the message and the state of the channel on which the
message is received.
 Cuts of a Distributed Computation

• In the space–time diagram of a distributed computation, a

zigzag line joining one arbitrary point on each process line is
termed a cut in the computation.
• Such a line slices the space–time diagram, and thus the set of
events in the distributed computation, into a PAST and a
FUTURE.
• The PAST contains all the events to the left of the cut and the
FUTURE contains all the events to the right of the cut.
• For a cut C, let PAST(C) and FUTURE(C) denote the set of
events in the PAST and FUTURE of C, respectively.
 CONT..
• Every cut corresponds to a global state and every global state
can be graphically represented as a cut in the computation’s
space–time diagram.
• A consistent global state corresponds to a cut in which every
message received in the PAST of the cut was sent in the PAST
of that cut. Such a cut is known as a consistent cut (In Figure
2.3, cut C2 is a consistent cut.).
• All messages that cross the cut from the PAST to the FUTURE
are in transit in the corresponding consistent global state.
• A cut is inconsistent if a message crosses the cut from the
FUTURE to the PAST (In Figure 2.3, cut C1 is a inconsistent
cut).
CONT..
 Past And Future Cones of an Event

Past cone of an event

• An event 𝑒𝑗 could have been affected only by all events 𝑒𝑖
such that 𝒆𝒊 →𝒆𝒋 and all the information available at 𝑒𝑖 could
be made accessible at 𝑒𝑗 . All such events 𝑒𝑖 belong to the past
of 𝑒𝑗 .
• Let Past(𝑒𝑗 ) denote all events in the past of 𝑒𝑗 on process 𝑝𝑖 ,
which can be defines as:

• max(Past(𝑒𝑗 ) is the latest event at process 𝑝𝑖 that affected

event 𝑒𝑗
• Note that max(𝑝𝑎𝑠𝑡𝑖 (𝑒𝑗 )) is always a message send event.
CONT..
 CONT..
Future cone of an event
• The future of an event 𝑒𝑗 , denoted by Future(𝑒𝑗 ), contains all
events 𝑒𝑖 that are causally affected by 𝑒𝑗 (see Figure 2.4).
• Future(𝑒𝑗 ) is defined as

• Likewise, we can define Future(𝑒𝑗 ) as the set of those events

of Future(𝑒𝑗 ) that are on process pi.
• min(Future(𝑒𝑗 ) as the first event on process 𝑝𝑖 that is affected
by 𝑒𝑗 .
• Note that min(Future(𝑒𝑗 )) is always a message receive event.
 Models of Process Communications

• There are two basic models of process communications –

synchronous and asynchronous.

• The synchronous communication model is a blocking type

where on a message send, the sender process blocks until the
message has been received by the receiver process.
• The sender process resumes execution only after it learns
that the receiver process has accepted the message.
• Thus, the sender and the receiver processes must
synchronize to exchange a message.
 CONT..
• Asynchronous communication model is a non-blocking type
where the sender and the receiver do not synchronize to
exchange a message.
• After having sent a message, the sender process does not wait
for the message to be delivered to the receiver process.
• The message is buffered by the system and is delivered to the
receiver process when it is ready to accept the message.
• A buffer overflow may occur if a process sends a large number
of messages in a burst to another process.
 Comparison Between Synchronous and
Asynchronous Communication
• Asynchronous communication provides higher parallelism
because the sender process can execute while the message is
in transit to the receiver.
• However, an implementation of asynchronous
communication requires more complex buffer management.
• In addition, due to higher degree of parallelism and non-
determinism, it is much more difficult to design, verify, and
implement distributed algorithms.
• The state space of such algorithms are likely to be much
larger.
• Synchronous communication is simpler to handle and
implement. However, due to frequent blocking, it is likely to
have poor performance and is likely to be more prone to
deadlocks.