Distributed Systems and Cloud Computing

Distributed Systems and Cloud Computing

(Generated in ChatGPT, compiled in Obsidian.md)

UNIT 1

1. Characteristics of Distributed Systems - Introduction

2. Examples of Distributed Systems
3. Advantages of Distributed Systems
4. System Models - Introduction
5. Networking and Internetworking
6. Inter Process Communication(IPC) - Message Passing and Shared Memory
7. Distributed Objects and Remote Method Invocation (RMI)
8. Remote Procedure Call (RPC)
9. Events and Notifications
10. Case Study - Java Remote Method Invocation (RMI)

UNIT 2

1. Time and Global States - Introduction

2. Logical Clocks
3. Synchronizing Events and Processes
4. Synchronizing Physical Clocks
5. Logical Time and Logical Clocks
6. Global States
7. Distributed Debugging
8. Coordination and Agreement : Distributed Mutual Exclusion
9. Elections in Distributed Systems
10. Multicast Communication in Distributed Systems
11. Consensus and Related Problems in Distributed Systems

UNIT 1 (Introduction to Distributed Systems)

Characteristics of Distributed Systems - Introduction
1. Definition of Distributed Systems
A distributed system is a model in which components located on networked computers communicate
and coordinate their actions by passing messages. The components interact with each other in order
to achieve a common goal.
1 / 123
 Distributed Systems and Cloud Computing
2. Key Characteristics of Distributed Systems

Resource Sharing: Distributed systems allow multiple users to share resources such as
hardware (e.g., printers, storage) and software (e.g., applications).
Concurrency: Multiple processes may execute simultaneously, enabling users to perform
various tasks at the same time without interference.
Scalability: The ability to expand or reduce the resources of the system according to the
demand. This includes both horizontal scaling (adding more machines) and vertical scaling
(adding resources to existing machines).
Fault Tolerance: The system's ability to continue operating properly in the event of a failure of
one or more components. Techniques such as replication and redundancy are often used to
achieve this.
Heterogeneity: Components in a distributed system can run on different platforms, which may
include different operating systems, hardware, or network protocols.
Transparency: Users should be unaware of the distribution of resources. This can include:
Location Transparency: Users do not need to know the physical location of resources.
Migration Transparency: Resources can move from one location to another without user
awareness.
Replication Transparency: Users are unaware of the replication of resources for fault
tolerance.
Openness: The system should be open for integration with other systems and allow for the
addition of new components. This involves adherence to standardized protocols.

3. Examples of Distributed Systems

The Internet: A vast network of computers that communicate with each other using various
protocols.
Cloud Computing: Services such as Amazon Web Services (AWS) and Microsoft Azure that
provide distributed computing resources over the internet.
Peer-to-Peer Networks: Systems like BitTorrent where each participant can act as both a client
and a server.
Distributed Databases: Databases that are spread across multiple sites, ensuring data is
available and fault-tolerant.

4. Applications of Distributed Systems

E-commerce: Handling transactions across different geographical locations.

Social Media: Platforms like Facebook or Twitter that distribute user data and interactions across
global servers.
Scientific Computing: Utilizing distributed resources for simulations and data processing.

2 / 123
 Distributed Systems and Cloud Computing
5. Conclusion
Understanding the characteristics of distributed systems is crucial as it lays the foundation for
designing and implementing scalable, fault-tolerant, and efficient systems. As technology continues to
evolve, the importance of distributed systems will only increase, particularly with the rise of cloud
computing and IoT (Internet of Things).

Additional Points
Challenges: Distributed systems can face challenges such as network latency, security issues,
and complexities in coordination and communication.
Future Trends: Increasing use of microservices, serverless architectures, and containerization
(e.g., Docker, Kubernetes) are shaping the future of distributed systems.

Examples of Distributed Systems

1. Client-Server Architecture
The Client-server model is a distributed application structure that partitions tasks or workloads
between the providers of a resource or service, called servers, and service requesters called clients.
In the client-server architecture, when the client computer sends a request for data to the server
through the internet, the server accepts the requested process and delivers the data packets
requested back to the client. Clients do not share any of their resources. Examples of the Client-
Server Model are Email, World Wide Web, etc.
In a client-server model, multiple clients request and receive services from a centralized server. This
architecture is widely used in many applications.

How Does the Client-Server Model Work?

In this article, we are going to take a dive into the **Client-Server** model and have a look at how
the **Internet** works via, web browsers. This article will help us have a solid WEB foundation and
help us easily work with WEB technologies.

**Client:** When we say the word **Client**, it means to talk of a person or an organization
using a particular service. Similarly in the digital world, a **Client** is a computer (**Host**) i.e.
capable of receiving information or using a particular service from the service providers
(**Servers**).
**Servers:** Similarly, when we talk about the word **Servers**, It means a person or medium
that serves something. Similarly in this digital world, a **Server** is a remote computer that
provides information (data) or access to particular services.

3 / 123
 Distributed Systems and Cloud Computing
So, it is the **Client** requesting something and the **Server** serving it as long as it is in the
database.

Characteristics:
Separation of concerns: Clients handle the user interface and user interaction, while
servers manage data and business logic.
Centralized control: The server can manage resources, data consistency, and security.

How the Browser Interacts With the Servers?

There are a few steps to follow to interacts with the servers of a client.

User enters the URL(Uniform Resource Locator) of the website or file. The Browser then
requests the DNS (DOMAIN NAME SYSTEM) Server.
DNS Server lookup for the address of the WEB Server.
The DNS Server* responds with the IP address of the WEB Server.
The Browser sends over an HTTP/HTTPS request to the WEB Server’s IP (provided by the DNS
server).
The Server sends over the necessary files for the website.
The Browser then renders the files and the website is displayed. This rendering is done with the
help of DOM (Document Object Model) interpreter, CSS interpreter, and JS Engine collectively
known as the JIT or (Just in Time) Compilers.

4 / 123
 Distributed Systems and Cloud Computing

Advantages of Client-Server Model

Centralized system with all data in a single place.

Cost efficient requires less maintenance cost and Data recovery is possible.
The capacity of the Client and Servers can be changed separately.

Disadvantages of Client-Server Model

Clients are prone to viruses, Trojans, and worms if present in the Server or uploaded into the
Server.
Servers are prone to Denial of Service (DOS) attacks.
Data packets may be spoofed or modified during transmission.
Phishing or capturing login credentials or other useful information of the user are common
and MITM(Man in the Middle) attacks are common.
Examples:
Web Applications: Browsers (clients) request web pages from a web server.
Database Systems: Applications connect to a database server to retrieve and manipulate
data.
Use Cases:
E-commerce websites, online banking, and enterprise applications.

2. Peer-to-Peer (P2P) Systems

5 / 123
 Distributed Systems and Cloud Computing
A peer-to-peer network is a simple network of computers. It first came into existence in the late 1970s.
Here each computer acts as a node for file sharing within the formed network. Here each node acts as
a server and thus there is no central server in the network. This allows the sharing of a huge amount
of data. The tasks are equally divided amongst the nodes. Each node connected in the network
shares an equal workload. For the network to stop working, all the nodes need to individually stop
working. This is because each node works independently.
In P2P systems, each participant (peer) can act both as a client and a server, sharing resources
directly with one another without a central authority.

History of P2P Networks

Before the development of P2P, USENET came into existence in 1979. The network enabled the
users to read and post messages. Unlike the forums we use today, it did not have a central server. It is
used to copy the new messages to all the servers of the node.

In the 1980s the first use of P2P networks occurred after personal computers were introduced.
In August 1988, the internet relay chat was the first P2P network built to share text and chat.
In June 1999, Napster was developed which was a file-sharing P2P software. It could be used to
share audio files as well. This software was shut down due to the illegal sharing of files. But the
concept of network sharing i.e P2P became popular.
In June 2000, Gnutella was the first decentralized P2P file sharing network. This allowed users to
access files on other users’ computers via a designated folder.

Types of P2P networks

1. Unstructured P2P networks: In this type of P2P network, each device is able to make an equal
contribution. This network is easy to build as devices can be connected randomly in the network.
But being unstructured, it becomes difficult to find content. For example, Napster, Gnutella, etc.
2. Structured P2P networks: It is designed using software that creates a virtual layer in order to
put the nodes in a specific structure. These are not easy to set up but can give easy access to
users to the content. For example, P-Grid, Kademlia, etc.
3. Hybrid P2P networks: It combines the features of both P2P networks and client-server
architecture. An example of such a network is to find a node using the central server.

Features of P2P network

These networks do not involve a large number of nodes, usually less than 12. All the computers
in the network store their own data but this data is accessible by the group.
Unlike client-server networks, P2P uses resources and also provides them. This results in
additional resources if the number of nodes increases. It requires specialized software. It allows
resource sharing among the network.

6 / 123
 Distributed Systems and Cloud Computing
Since the nodes act as clients and servers, there is a constant threat of attack.
Almost all OS today support P2P networks.

P2P Network Architecture

In the P2P network architecture, the computers connect with each other in a workgroup to share files,
and access to internet and printers.

Each computer in the network has the same set of responsibilities and capabilities.
Each device in the network serves as both a client and server.
The architecture is useful in residential areas, small offices, or small companies where each
computer act as an independent workstation and stores the data on its hard drive.
Each computer in the network has the ability to share data with other computers in the network.
The architecture is usually composed of workgroups of 12 or more computers.

Characteristics:

Decentralization: No single point of failure; peers can join and leave the network freely.
Resource Sharing: Each peer contributes resources (e.g., storage, processing power).

7 / 123
 Distributed Systems and Cloud Computing
How Does P2P Network Work?
Let’s understand the working of the Peer-to-Peer network through an example. Suppose, the user
wants to download a file through the peer-to-peer network then the download will be handled in this
way:

If the peer-to-peer software is not already installed, then the user first has to install the peer-to-
peer software on his computer.
This creates a virtual network of peer-to-peer application users.
The user then downloads the file, which is received in bits that come from multiple computers in
the network that have already that file.
The data is also sent from the user’s computer to other computers in the network that ask for the
data that exist on the user’s computer.
Thus, it can be said that in the peer-to-peer network the file transfer load is distributed among the
peer computers.

How to Use a P2P Network Efficiently?

Firstly secure your network via privacy solutions. Below are some of the measures to keep the P2P
network secure:

Share and download legal files: Double-check the files that are being downloaded before
sharing them with other employees. It is very important to make sure that only legal files are
downloaded.
Design strategy for sharing: Design a strategy that suits the underlying architecture in order to
manage applications and underlying data.
Keep security practices up-to-date: Keep a check on the cyber security threats which might
prevail in the network. Invest in good quality software that can sustain attacks and prevent the
network from being exploited. Update your software regularly.
Scan all downloads: This is used to constantly check and scan all the files for viruses before
downloading them. This helps to ensure that safe files are being downloaded and in case, any
file with potential threat is detected then report to the IT Staff.
Proper shutdown of P2P networking after use: It is very important to correctly shut down the
software to avoid unnecessary access to third persons to the files in the network. Even if the
windows are closed after file sharing but the software is still active then the unauthorized user
can still gain access to the network which can be a major security breach in the network.

P2P networks can be basically categorized into three levels.

The first level is the basic level which uses a USB to create a P2P network between two
systems.

8 / 123
 Distributed Systems and Cloud Computing
The second is the intermediate level which involves the usage of copper wires in order to
connect more than two systems.
The third is the advanced level which uses software to establish protocols in order to manage
numerous devices across the internet.

Applications of P2P Network

Below are some of the common uses of P2P network:

File sharing: P2P network is the most convenient, cost-efficient method for file sharing for
businesses. Using this type of network there is no need for intermediate servers to transfer the
file.
Blockchain: The P2P architecture is based on the concept of decentralization. When a peer-to-
peer network is enabled on the blockchain it helps in the maintenance of a complete replica of
the records ensuring the accuracy of the data at the same time. At the same time, peer-to-peer
networks ensure security also.
Direct messaging: P2P network provides a secure, quick, and efficient way to communicate.
This is possible due to the use of encryption at both the peers and access to easy messaging
tools.
Collaboration: The easy file sharing also helps to build collaboration among other peers in the
network.
File sharing networks: Many P2P file sharing networks like G2, and eDonkey have popularized
peer-to-peer technologies.
Content distribution: In a P2P network, unline the client-server system so the clients can both
provide and use resources. Thus, the content serving capacity of the P2P networks can actually
increase as more users begin to access the content.
IP Telephony: Skype is one good example of a P2P application in VoIP.

Advantages of P2P Network

Easy to maintain: The network is easy to maintain because each node is independent of the
other.
Less costly: Since each node acts as a server, therefore the cost of the central server is saved.
Thus, there is no need to buy an expensive server.
No network manager: In a P2P network since each node manages his or her own computer,
thus there is no need for a network manager.
Adding nodes is easy: Adding, deleting, and repairing nodes in this network is easy.
Less network traffic: In a P2P network, there is less network traffic than in a client/ server
network.

9 / 123
 Distributed Systems and Cloud Computing

Disadvantages of P2P Network

Data is vulnerable: Because of no central server, data is always vulnerable to getting lost
because of no backup.
Less secure: It becomes difficult to secure the complete network because each node is
independent.
Slow performance: In a P2P network, each computer is accessed by other computers in the
network which slows down the performance of the user.
Files hard to locate: In a P2P network, the files are not centrally stored, rather they are stored
on individual computers which makes it difficult to locate the files.
Examples:
File Sharing: BitTorrent allows users to download and share files directly from each other.
Cryptocurrencies: Bitcoin operates on a P2P network where transactions are verified by
users (nodes) rather than a central bank.
Some of the popular P2P networks are Gnutella, BitTorrent, eDonkey, Kazaa, Napster, and
Skype.
Use Cases:
Content distribution networks, collaborative applications, and decentralized applications
(DApps).

3. Grid Computing
Grid computing is a distributed computing model that involves a network of computers working
together to perform tasks by sharing their processing power and resources. It breaks down tasks into
smaller subtasks, allowing concurrent processing. All machines on that network work under the same
protocol to act as a virtual supercomputer. The tasks that they work on may include analyzing huge
datasets or simulating situations that require high computing power. Computers on the network
contribute resources like processing power and storage capacity to the network.
Grid Computing is a subset of distributed computing, where a virtual supercomputer comprises
machines on a network connected by some bus, mostly Ethernet or sometimes the Internet. It can
also be seen as a form of Parallel Computing where instead of many CPU cores on a single machine,
it contains multiple cores spread across various locations. The concept of grid computing isn’t new,
but it is not yet perfected as there are no standard rules and protocols established and accepted by
people.

Characteristics:

Resource pooling: Combines resources from multiple locations for computation.

Task scheduling: Efficiently allocates tasks among the available resources.

Why is Grid Computing Important?

10 / 123
 Distributed Systems and Cloud Computing
Scalability: It allows organizations to scale their computational resources dynamically. As
workloads increase, additional machines can be added to the grid, ensuring efficient processing.
Resource Utilization: By pooling resources from multiple computers, grid computing maximizes
resource utilization. Idle or underutilized machines contribute to tasks, reducing wastage.
Complex Problem Solving: Grids handle large-scale problems that require significant
computational power. Examples include climate modeling, drug discovery, and genome analysis.
Collaboration: Grids facilitate collaboration across geographical boundaries. Researchers,
scientists, and engineers can work together on shared projects.
Cost Savings: Organizations can reuse existing hardware, saving costs while accessing excess
computational resources. Additionally, cloud resources can be cost-effectively.

Working of Grid Computing

A Grid computing network mainly consists of these three types of machines

Control Node: A computer, usually a server or a group of servers which administrates the whole
network and keeps the account of the resources in the network pool.
Provider: The computer contributes its resources to the network resource pool.
User: The computer that uses the resources on the network.

Grid computing enables computers to request and share resources through a control node, allowing
machines to alternate between being users and providers based on demand. Nodes can be
homogeneous (same OS) or heterogeneous (different OS). Middleware manages the network,
ensuring that resources are utilized efficiently without overloading any provider. The concept, initially
described in Ian Foster and Carl Kesselman's 1999 book, likened computing power consumption to
electricity from a power grid. Today, grid computing functions as a collaborative distributed network,

11 / 123
 Distributed Systems and Cloud Computing
widely used in institutions for solving complex mathematical and analytical problems.

What are the Types of Grid Computing?

Computational grid: A computational grid is a collection of high-performance processors. It

enables researchers to utilize the combined computing capacity of the machines. Researchers
employ computational grid computing to complete resource-intensive activities like mathematical
calculations.
Scavenging grid: Similar to computational grids, CPU scavenging grids have a large number of
conventional computers. Scavenging refers to the process of searching for available computing
resources in a network of normal computers.
Data grid: A data grid is a grid computing network that connects multiple computers together to
enable huge amounts of data storage. You can access the stored data as if it were on your local
system, without worrying about where it is physically located on the grid.

Use Cases of Grid Computing

12 / 123
 Distributed Systems and Cloud Computing
Genomic Research
Drug Discovery
Cancer Research
Weather Forecasting
Risk Analysis
Computer-Aided Design (CAD)
Animation and Visual Effects
Collaborative Projects

Advantages of Grid Computing

Grid Computing provide high resources utilization.

Grid Computing allow parallel processing of task.
Grid Computing is designed to be scalable.

Disadvantages of Grid Computing

The software of the grid is still in the evolution stage.

Grid computing introduce Complexity.
Limited Flexibility
Security Risks
Examples:
SETI@home: A project that uses idle computer resources worldwide to analyze radio
signals for signs of extraterrestrial life.
Large-scale simulations: Weather forecasting, molecular modeling, and complex scientific
calculations.

4. Cloud Computing
Cloud Computing means storing and accessing the data and programs on remote servers that are
hosted on the internet instead of the computer’s hard drive or local server. Cloud computing is also
referred to as Internet-based computing, it is a technology where the resource is provided as a service
through the Internet to the user. The data that is stored can be files, images, documents, or any other
storable document.
Cloud computing delivers on-demand computing resources and services over the internet, allowing
users to access and utilize resources without managing physical hardware.

The following are some of the Operations that can be performed with Cloud Computing

Storage, backup, and recovery of data

13 / 123
 Distributed Systems and Cloud Computing
Delivery of software on demand
Development of new applications and services
Streaming videos and audio
Characteristics:
On-demand self-service: Users can provision resources as needed without human
intervention.
Scalability: Resources can be easily scaled up or down based on demand.
Pay-per-use model: Users pay only for the resources they consume.

Understanding How Cloud Computing Works?

Cloud computing helps users in easily accessing computing resources like storage, and processing
over internet rather than local hardwares. Here we discussing how it works in nutshell:

Infrastructure: Cloud computing depends on remote network servers hosted on internet for
store, manage, and process the data.
On-Demand Acess: Users can access cloud services and resources based on-demand they can
scale up or down the without having to invest for physical hardware.
Types of Services: Cloud computing offers various benefits such as cost saving, scalability,
reliability and acessibility it reduces capital expenditures, improves efficiency.

Origins Of Cloud Computing

Mainframe computing in the 1950s and the internet explosion in the 1990s came together to give rise t
o cloud computing. Since businesses like Amazon, Google, and Salesforce started providing web-
based services in the early 2000s. The term “cloud computing” has gained popularity. Scalability,
adaptability, and cost-effectiveness are to be facilitated by the concept’s on-demand internet-based
access to computational resources.

What is Virtualization In Cloud Computing?

Virtualization is the software technology that helps in providing the logical isolation of physical
resources. Creating logical isolation of physical resources such as RAM, CPU, and Storage.. over the
cloud is known as Virtualization in Cloud Computing. In simple we can say creating types of Virtual
Instances of computing resources over the cloud. It provides better management and utilization of
hardware resources with logical isolation making the applications independent of others. It facilitates
streamlining the resource allocation and enhancing scalability for multiple virtual computers within a
single physical source offering cost-effectiveness and better optimization of resources.

Architecture Of Cloud Computing

14 / 123
 Distributed Systems and Cloud Computing
Cloud computing architecture refers to the components and sub-components required for cloud
computing. These components typically refer to:

1. Front end ( Fat client, Thin client)

2. Back-end platforms ( Servers, Storage )
3. Cloud-based delivery and a network ( Internet, Intranet, Intercloud )

1. Front End ( User Interaction Enhancement )

The User Interface of Cloud Computing consists of 2 sections of clients. The Thin clients are the ones
that use web browsers facilitating portable and lightweight accessibilities and others are known as Fat
Clients that use many functionalities for offering a strong user experience.

2. Back-end Platforms ( Cloud Computing Engine )

The core of cloud computing is made at back-end platforms with several servers for storage and
processing computing. Management of Applications logic is managed through servers and effective
data handling is provided by storage. The combination of these platforms at the backend offers the
processing power, and capacity to manage and store data behind the cloud.

3. Cloud-Based Delivery and Network

15 / 123
 Distributed Systems and Cloud Computing
On-demand access to the computer and resources is provided over the Internet, Intranet, and
Intercloud. The Internet comes with global accessibility, the Intranet helps in internal communications
of the services within the organization and the Intercloud enables interoperability across various cloud
services. This dynamic network connectivity ensures an essential component of cloud computing
architecture on guaranteeing easy access and data transfer.

What Are The Types of Cloud Computing Services?

The following are the types of Cloud Computing:

1. Infrastructure as a Service (IaaS)

2. Platform as a Service (PaaS)
3. Software as a Service (SaaS)
4. Function as as Service (FaaS)

We'll study about all of this and more, in detail, in Unit 4.

Advantages of Cloud Computing

1. Cost Efficiency: Cloud Computing provides flexible pricing to the users with the principal pay-
as-you-go model. It helps in lessening capital expenditures of Infrastructure, particularly for small
and medium-sized businesses companies.
2. Flexibility and Scalability: Cloud services facilitate the scaling of resources based on demand.
It ensures the efficiency of businesses in handling various workloads without the need for large
amounts of investments in hardware during the periods of low demand.
3. Collaboration and Accessibility: Cloud computing provides easy access to data and
applications from anywhere over the internet. This encourages collaborative team participation

16 / 123
 Distributed Systems and Cloud Computing
from different locations through shared documents and projects in real-time resulting in quality
and productive outputs.
4. Automatic Maintenance and Updates: AWS Cloud takes care of the infrastructure
management and keeping with the latest software automatically making updates they is new
versions. Through this, AWS guarantee the companies always having access to the newest
technologies to focus completely on business operations and innovations.

Disadvantages Of Cloud Computing

1. Security Concerns: Storing of sensitive data on external servers raised more security concerns
which is one of the main drawbacks of cloud computing.
2. Downtime and Reliability: Even though cloud services are usually dependable, they may also
have unexpected interruptions and downtimes. These might be raised because of server
problems, Network issues or maintenance disruptions in Cloud providers which negative effect
on business operations, creating issues for users accessing their apps.
3. Dependency on Internet Connectivity: Cloud computing services heavily rely on Internet
connectivity. For accessing the cloud resources the users should have a stable and high-speed
internet connection for accessing and using cloud resources. In regions with limited internet
connectivity, users may face challenges in accessing their data and applications.
4. Cost Management Complexity: The main benefit of cloud services is their pricing model that
coming with *Pay as you go but it also leads to cost management complexities. On without
proper careful monitoring and utilization of resources optimization, Organizations may end up
with unexpected costs as per their use scale. Understanding and Controlled usage of cloud
services requires ongoing attention.

Examples:
Infrastructure as a Service (IaaS): Amazon EC2 provides virtual servers and storage.
Platform as a Service (PaaS): Google App Engine allows developers to build and deploy
applications without managing infrastructure.
Software as a Service (SaaS): Applications like Google Workspace and Microsoft 365
provide software via the cloud.
Use Cases:
Web hosting, data storage, application development, and enterprise solutions.

Understanding the various examples of distributed systems is essential for grasping the broad
applications and architectures that facilitate modern computing. Each model offers unique advantages
and serves different needs, from centralized resource management in client-server systems to
decentralized operations in peer-to-peer systems.

17 / 123
 Distributed Systems and Cloud Computing

Advantages of Distributed Systems

1. Resource Sharing

Maximized Utilization:
Distributed systems enable sharing of diverse resources—CPU cycles, memory, storage—
across different nodes. This leads to a higher overall resource utilization rate compared to
traditional systems where resources might sit idle.
Example: In a corporate environment, employees across different departments can access
shared databases and files, rather than maintaining duplicate copies on local machines.
Cost Efficiency:
Organizations can leverage existing hardware across different locations instead of investing
in centralized data centers. This helps in reducing capital expenditure.
Example: A small business can utilize cloud services for its IT needs rather than setting up
a full-fledged server room.

2. Scalability

Horizontal Scalability:
As demand grows, new nodes (servers) can be added to the system. This is often more
cost-effective than upgrading existing machines (vertical scaling).
Example: E-commerce platforms can handle seasonal spikes in traffic by adding additional
servers temporarily during peak times.
Load Balancing:
Load balancers can distribute requests across multiple servers, ensuring that no single
server bears too much load, which can lead to slowdowns or crashes.
Example: A content delivery network (CDN) uses load balancing to serve web pages
quickly by distributing the load among geographically dispersed servers.

3. Fault Tolerance and Reliability

Redundancy:
By replicating data and services across multiple nodes, distributed systems can recover
from hardware failures or outages without significant downtime.
Example: Cloud providers like AWS use data replication across different availability zones,
ensuring that data remains accessible even if one zone fails.
Graceful Degradation:
Instead of complete failure, parts of the system may fail while others continue functioning.
This ensures continuous service availability.
Example: If a specific service in a microservices architecture fails, other services may still
operate independently, allowing the application to function at reduced capacity.
18 / 123
 Distributed Systems and Cloud Computing
4. Enhanced Performance

Parallel Processing:
Tasks can be split into smaller sub-tasks and executed simultaneously on different nodes,
significantly speeding up processing times.
Example: In scientific simulations, vast calculations can be performed concurrently on
multiple machines, reducing the overall time to complete simulations.
Reduced Latency:
By deploying resources closer to users (edge computing), distributed systems can minimize
the time it takes for data to travel across the network.
Example: Streaming services often cache content on edge servers located near users,
improving playback speed and reducing buffering.

5. Flexibility and Adaptability

Heterogeneity:
Distributed systems can incorporate different hardware and software platforms, allowing
organizations to utilize the best technologies available.
Example: A company can use a mix of cloud services, local servers, and edge devices to
optimize its operations based on specific requirements.
Dynamic Resource Allocation:
Resources can be dynamically allocated based on demand, allowing for efficient use of
computing power and storage.
Example: In cloud environments, resources can be spun up or down based on application
load, optimizing costs.

6. Geographic Distribution

Global Reach:
Distributed systems can operate across various geographical locations, providing better
service to users by reducing the distance data must travel.
Example: Global companies can offer localized services, ensuring compliance with regional
regulations and enhancing user experience.
Disaster Recovery:
Geographic distribution enhances disaster recovery strategies, allowing businesses to back
up data and services in multiple locations to safeguard against data loss.
Example: Companies can implement backup solutions that replicate data across different
regions, ensuring data is preserved even in the event of natural disasters.

7. Improved Collaboration
19 / 123
 Distributed Systems and Cloud Computing
Multi-user Environments:
Distributed systems support collaborative applications where multiple users can work on
shared projects in real-time.
Example: Tools like Google Docs allow multiple users to edit documents simultaneously,
improving productivity.
Real-time Communication:
Many distributed systems facilitate real-time communication and data sharing, which is
critical for applications such as video conferencing and collaborative platforms.
Example: Applications like Slack or Microsoft Teams allow teams to communicate and
share information in real-time, enhancing collaboration.

8. Security and Isolation

Distributed Security Mechanisms:

Security features can be implemented at various nodes, reducing the risk of a single point of
failure in the security model.
Example: In a distributed database, data encryption can be applied at each node, ensuring
that even if one node is compromised, the data remains secure.
Data Isolation:
Sensitive information can be isolated within specific nodes or regions, improving compliance
with data protection regulations.
Example: Healthcare organizations may choose to keep patient data within specific
geographic boundaries to comply with regulations like HIPAA.

The advantages of distributed systems underscore their critical role in modern computing. By
capitalizing on resource sharing, scalability, fault tolerance, and flexibility, organizations can design
robust systems that meet the demands of a rapidly evolving technological landscape.

System Models - Introduction

1. Definition of System Models
System models provide a conceptual framework for understanding and designing distributed systems.
They abstract the complexities of the system's architecture and interactions, allowing for clearer
analysis, communication, and implementation of distributed applications.

2. Importance of System Models

Framework for Analysis: Helps in evaluating system performance, scalability, reliability, and
other critical attributes.

20 / 123
 Distributed Systems and Cloud Computing
Design Guidance: Offers guidelines for developers and engineers in structuring systems to
meet specific requirements.
Communication Tool: Provides a common language for stakeholders (e.g., developers,
managers, clients) to discuss system characteristics and functionalities.

3. Types of System Models

The 2 main types of system models mentioned in our syllabus used in distributed systems are:

1. Architectural Model

Architectural model in distributed computing system is the overall design and structure of the system,
and how its different components are organised to interact with each other and provide the desired
functionalities. It is an overview of the system, on how will the development, deployment and
operations take place. Construction of a good architectural model is required for efficient cost usage,
and highly improved scalability of the applications.

The key aspects of architectural model are:

1. Client-Server model

It is a centralised approach in which the clients initiate requests for services and severs respond by
providing those services. It mainly works on the request-response model where the client sends a
request to the server and the server processes it, and responds to the client accordingly.

It can be achieved by using TCP/IP, HTTP protocols on the transport layer.

21 / 123
 Distributed Systems and Cloud Computing
This is mainly used in web services, cloud computing, database management systems etc.

2. Peer-to-peer model

It is a decentralised approach in which all the distributed computing nodes, known as peers, are all the
same in terms of computing capabilities and can both request as well as provide services to other
peers. It is a highly scalable model because the peers can join and leave the system dynamically,
which makes it an ad-hoc form of network.

The resources are distributed and the peers need to look out for the required resources as and
when required.
The communication is directly done amongst the peers without any intermediaries according to
some set rules and procedures defined in the P2P networks.

22 / 123
 Distributed Systems and Cloud Computing
The best example of this type of computing is BitTorrent.

3. Layered model

It involves organising the system into multiple layers, where each layer will provision a specific
service. Each layer communicated with the adjacent layers using certain well-defined protocols
without affecting the integrity of the system. A hierarchical structure is obtained where each layer

23 / 123
 Distributed Systems and Cloud Computing
abstracts the underlying complexity of lower layers.

4. Micro-services model

In this system, a complex application or task, is decomposed into multiple independent tasks and
these services running on different servers. Each service performs only a single function and is
focussed on a specific business-capability. This makes the overall system more maintainable, scalable
and easier to understand. Services can be independently developed, deployed and scaled without

24 / 123
 Distributed Systems and Cloud Computing
affecting the ongoing services.

2. Fundamental Model
The fundamental model in a distributed computing system is a broad conceptual framework that helps
in understanding the key aspects of the distributed systems. These are concerned with more formal
description of properties that are generally common in all architectural models. It represents the
essential components that are required to understand a distributed system’s behaviour. Four
fundamental models are as follows:

1. Interaction Model

Distributed computing systems are full of many processes interacting with each other in highly
complex ways. Interaction model provides a framework to understand the mechanisms and patterns
that are used for communication and coordination among various processes. Different components
that are important in this model are –

Message Passing – It deals with passing messages that may contain, data, instructions, a
service request, or process synchronisation between different computing nodes. It may be
synchronous or asynchronous depending on the types of tasks and processes.

25 / 123
 Distributed Systems and Cloud Computing
Publish/Subscribe Systems – Also known as pub/sub system. In this the publishing process
can publish a message over a topic and the processes that are subscribed to that topic can take
it up and execute the process for themselves. It is more important in an event-driven architecture.

2. Remote Procedure Call (RPC)

It is a communication paradigm that has an ability to invoke a new process or a method on a remote
process as if it were a local procedure call. The client process makes a procedure call using RPC and
then the message is passed to the required server process using communication protocols. These
message passing protocols are abstracted and the result once obtained from the server process, is
sent back to the client process to continue execution.

3. Failure Model

This model addresses the faults and failures that occur in the distributed computing system. It
provides a framework to identify and rectify the faults that occur or may occur in the system. Fault
tolerance mechanisms are implemented so as to handle failures by replication and error detection and
recovery methods. Different failures that may occur are:

Crash failures – A process or node unexpectedly stops functioning.

26 / 123
 Distributed Systems and Cloud Computing
Omission failures – It involves a loss of message, resulting in absence of required
communication.
Timing failures – The process deviates from its expected time quantum and may lead to delays
or unsynchronised response times.
Byzantine failures – The process may send malicious or unexpected messages that conflict
with the set protocols.

2. Security Model

Distributed computing systems may suffer malicious attacks, unauthorised access and data breaches.
Security model provides a framework for understanding the security requirements, threats,
vulnerabilities, and mechanisms to safeguard the system and its resources.

Various aspects that are vital in the security model are:

Authentication: It verifies the identity of the users accessing the system. It ensures that only the
authorised and trusted entities get access. It involves –
Password-based authentication: Users provide a unique password to prove their identity.
Public-key cryptography: Entities possess a private key and a corresponding public key,
allowing verification of their authenticity.
Multi-factor authentication: Multiple factors, such as passwords, biometrics, or security
tokens, are used to validate identity.
27 / 123
 Distributed Systems and Cloud Computing
Encryption:
It is the process of transforming data into a format that is unreadable without a decryption
key. It protects sensitive information from unauthorized access or disclosure.
Data Integrity:
- Data integrity mechanisms protect against unauthorised modifications or tampering of data.
They ensure that data remains unchanged during storage, transmission, or processing. Data
integrity mechanisms include:
- Hash functions – Generating a hash value or checksum from data to verify its integrity.
- Digital signatures – Using cryptographic techniques to sign data and verify its authenticity and
integrity.

4. Characteristics of Effective System Models

Abstraction: Simplifies complex systems by focusing on essential components and interactions

while ignoring irrelevant details.
Consistency: Provides a coherent view of system operations, ensuring all components work
harmoniously.
Flexibility: Adaptable to changes in technology, requirements, or usage patterns.
Scalability: Capable of accommodating growth in users, data, and transactions without
significant redesign.

28 / 123
 Distributed Systems and Cloud Computing
Understanding system models is essential for effectively designing, implementing, and managing
distributed systems. By leveraging various models, stakeholders can better analyze system
requirements, ensure efficient communication, and ultimately create robust and scalable solutions.

Networking and Internetworking

1. Definition of Networking
Networking refers to the practice of connecting computers and other devices to share resources and
information. It involves both hardware (routers, switches, cables) and software (protocols,
applications) that enable communication between devices.

2. Key Components of Networking

Nodes: Devices connected to the network, such as computers, servers, printers, and
smartphones.
Links: The physical or wireless connections that facilitate communication between nodes.
Switches: Devices that connect multiple nodes on the same network, forwarding data packets
based on MAC addresses.
Routers: Devices that connect different networks, directing data packets between them based
on IP addresses.

3. Types of Networks

Local Area Network (LAN):

Covers a small geographical area (e.g., a single building or campus).
High data transfer rates and low latency.
Example: Office networks.
Wide Area Network (WAN):
Covers a broad geographical area, often using leased telecommunication lines.
Lower data transfer rates compared to LANs.
Example: The internet, connecting multiple LANs.
Metropolitan Area Network (MAN):
Spans a city or large campus.
Often used by municipalities or large organizations.
Example: City-wide Wi-Fi networks.
Personal Area Network (PAN):
Covers a very short range, typically for personal devices.
Example: Bluetooth connections between a smartphone and a headset.

29 / 123
 Distributed Systems and Cloud Computing
4. Networking Protocols
Protocols are standardized rules that govern how data is transmitted and received over a network.
Key protocols include:

Transmission Control Protocol (TCP):

Ensures reliable, ordered, and error-checked delivery of data between applications.
Works in conjunction with the Internet Protocol (IP).
Internet Protocol (IP):
Responsible for addressing and routing packets of data between devices across networks.
Versions include IPv4 (most commonly used) and IPv6 (to accommodate more devices).
Hypertext Transfer Protocol (HTTP):
Governs the transfer of hypertext documents on the web.
Secured variant: HTTPS, which uses encryption for secure communication.
File Transfer Protocol (FTP):
Used for transferring files between computers over a network.

5. Internetworking

Internetworking refers to the interconnection of multiple networks to form a larger, cohesive network
(the internet). This process involves various components and technologies:

Routers: Facilitate communication between different networks by forwarding packets based on

destination IP addresses.
Gateways: Act as a bridge between different network architectures or protocols, enabling data
transfer across incompatible systems.
Bridges: Connect two or more local area networks (LANs) to make them function as a single
network.

Internetworking is combined of 2 words, inter and networking which implies an association between
totally different nodes or segments. This connection area unit is established through intercessor
devices akin to routers or gateway. The first term for associate degree internetwork was catenet. This
interconnection is often among or between public, private, commercial, industrial, or governmental
networks. Thus, associate degree internetwork could be an assortment of individual networks,
connected by intermediate networking devices, that function as one giant network. Internetworking
refers to the trade, products, and procedures that meet the challenge of making and administering
internet works.
To enable communication, every individual network node or phase is designed with a similar protocol
or communication logic, that is Transfer Control Protocol (TCP) or Internet Protocol (IP). Once a
network communicates with another network having constant communication procedures, it’s called
Internetworking. Internetworking was designed to resolve the matter of delivering a packet of
information through many links.

30 / 123
 Distributed Systems and Cloud Computing
There is a minute difference between extending the network and Internetworking. Merely exploitation
of either a switch or a hub to attach 2 local area networks is an extension of LAN whereas connecting
them via the router is an associate degree example of Internetworking. Internetworking is enforced in
Layer three (Network Layer) of the OSI-ISO model. The foremost notable example of internetworking
is the Internet.

There is chiefly 3 units of Internetworking:

1. Extranet
2. Intranet
3. Internet

Intranets and extranets might or might not have connections to the net. If there is a connection to the
net, the computer network or extranet area unit is usually shielded from being accessed from the net if
it is not authorized. The net isn’t thought-about to be a section of the computer network or extranet,
though it should function as a portal for access to parts of the associate degree extranet.

1. Extranet – It’s a network of the internetwork that’s restricted in scope to one organization or
entity however that additionally has restricted connections to the networks of one or a lot of
different sometimes, however not essential. It’s the very lowest level of Internetworking, usually
enforced in an exceedingly personal area. Associate degree extranet may additionally be
classified as a Man, WAN, or different form of network however it cannot encompass one local
area network i.e. it should have a minimum of one reference to associate degree external
network.
2. Intranet – This associate degree computer network could be a set of interconnected networks,
which exploits the Internet Protocol and uses IP-based tools akin to web browsers and FTP tools,
that are underneath the management of one body entity. That body entity closes the computer
network to the remainder of the planet and permits solely specific users. Most typically, this
network is the internal network of a corporation or different enterprise. An outsized computer
network can usually have its own internet server to supply users with browsable data.
3. Internet – A selected Internetworking, consisting of a worldwide interconnection of governmental,
academic, public, and personal networks based mostly upon the Advanced analysis comes
Agency Network (ARPANET) developed by ARPA of the U.S. Department of Defence additionally
home to the World Wide Web (WWW) and cited as the ‘Internet’ to differentiate from all different
generic Internetworks. Participants within the web, or their service suppliers, use IP Addresses
obtained from address registries that manage assignments.

Internetworking has evolved as an answer to a few key problems: isolated LANs, duplication of
resources, and an absence of network management. Isolated LANs created transmission problems
between totally different offices or departments. Duplication of resources meant that constant
hardware and code had to be provided to every workplace or department, as did a separate support
employee. This lack of network management meant that no centralized methodology of managing and
troubleshooting networks existed.
One more form of the interconnection of networks usually happens among enterprises at the Link

31 / 123
 Distributed Systems and Cloud Computing
Layer of the networking model, i.e. at the hardware-centric layer below the amount of the TCP/IP
logical interfaces. Such interconnection is accomplished through network bridges and network
switches. This can be typically incorrectly termed internetworking, however, the ensuing system is just
a bigger, single subnetwork, and no internetworking protocol, akin to web Protocol, is needed to
traverse these devices.
However, one electronic network is also reborn into associate degree internetwork by dividing the
network into phases and logically dividing the segment traffic with routers. The Internet Protocol is
meant to supply an associate degree unreliable packet service across the network. The design avoids
intermediate network components maintaining any state of the network. Instead, this task is allotted to
the endpoints of every communication session. To transfer information correctly, applications should
utilize associate degree applicable Transport Layer protocol, akin to Transmission management
Protocol (TCP), that provides a reliable stream. Some applications use a less complicated,
connection-less transport protocol, User Datagram Protocol (UDP), for tasks that don’t need reliable
delivery of information or that need period of time service, akin to video streaming or voice chat.

6. Key Concepts in Internetworking

Addressing: Each device on a network is assigned a unique IP address to facilitate

communication.
Subnets: Dividing a larger network into smaller, manageable segments to improve performance
and security.
Network Address Translation (NAT): A method used to translate private IP addresses to a
public IP address, allowing multiple devices on a local network to access the internet.

7. Security Considerations

Firewalls: Security devices that monitor and control incoming and outgoing network traffic based
on predetermined security rules.
Virtual Private Networks (VPNs): Secure connections that allow users to access a private
network over the internet as if they were physically present in that network.
Encryption: Protects data transmitted over networks, ensuring confidentiality and integrity.

Networking and internetworking are foundational elements of distributed systems. Understanding the
principles of networking, the types of networks, protocols, and security measures is crucial for
designing and managing robust distributed applications. As the landscape of networking continues to
evolve with advancements like 5G and IoT, the importance of these concepts will only grow.

Inter Process Communication(IPC) - Message Passing and

Shared Memory
32 / 123
 Distributed Systems and Cloud Computing
Processes can coordinate and interact with one another using a method called inter-process
communication (IPC). Interprocess Communication (IPC) is a mechanism that allows processes to
communicate and synchronize their actions when executing in a distributed system. Through
facilitating process collaboration, it significantly contributes to improving the effectiveness, modularity,
and ease of software systems.

Types of Process
Independent process
Co-operating process

An independent process is not affected by the execution of other processes while a co-operating
process can be affected by other executing processes. Though one can think that those processes,
which are running independently, will execute very efficiently, in reality, there are many situations when
cooperative nature can be utilized for increasing computational speed, convenience, and modularity.
Inter-process communication (IPC) is a mechanism that allows processes to communicate with each
other and synchronize their actions. The communication between these processes can be seen as a
method of cooperation between them. The two primary models for IPC are message passing and
shared memory. Figure 1 below shows a basic structure of communication between processes via
the shared memory method and via the message passing method.

left is shared memory

and right is message passing

1. Shared Memory

Definition: In the shared memory model, multiple processes access a common memory space to
communicate. This method is suitable for processes running on the same machine.

Characteristics:

33 / 123
 Distributed Systems and Cloud Computing
Direct Access: Processes can read from and write to shared memory directly, allowing for fast
communication.
Synchronization Mechanisms: Requires synchronization to prevent conflicts when multiple
processes access the same memory location. Common techniques include semaphores,
mutexes, and condition variables.

Mechanisms:

Shared Memory Segments: Allocated regions of memory accessible by multiple processes.

Processes can attach to these segments for reading/writing data.
Memory Mapping: Processes can map files into their address space, allowing them to share
data efficiently.

Advantages:

Performance: Direct memory access allows for high-speed communication with low latency
compared to message passing.
Data Structure Flexibility: Processes can share complex data structures (e.g., arrays, linked
lists) without serialization.

Disadvantages:

Complexity of Synchronization: Requires careful management of access to shared resources

to avoid race conditions and deadlocks.
Limited to Local Processes: Typically restricted to processes on the same machine, which can
be a limitation in distributed environments.

Use Cases:

High-performance computing applications, real-time systems, and applications where low-latency

communication is critical.

Example:

Producer-Consumer problem

There are two processes: Producer and Consumer . The producer produces some items and the
Consumer consumes that item. The two processes share a common space or memory location known
as a buffer where the item produced by the Producer is stored and from which the Consumer
consumes the item if needed. There are two versions of this problem: the first one is known as the
unbounded buffer problem in which the Producer can keep on producing items and there is no limit on
the size of the buffer, the second one is known as the bounded buffer problem in which the Producer
can produce up to a certain number of items before it starts waiting for Consumer to consume it. We
will discuss the bounded buffer problem. First, the Producer and the Consumer will share some
common memory, then the producer will start producing items. If the total produced item is equal to
34 / 123
 Distributed Systems and Cloud Computing
the size of the buffer, the producer will wait to get it consumed by the Consumer. Similarly, the
consumer will first check for the availability of the item. If no item is available, the Consumer will wait
for the Producer to produce it. If there are items available, Consumer will consume them. The pseudo-
code to demonstrate is provided below:

Shared Data Between the two Processes

#define buff_max 25
#define mod %

struct item{

// different member of the produced data

// or consumed data
---------
}

// An array is needed for holding the items.

// This is the shared place which will be
// access by both process
// item shared_buff [ buff_max ];

// Two variables which will keep track of

// the indexes of the items produced by producer
// and consumer The free index points to
// the next free index. The full index points to
// the first full index.
int free_index = 0;
int full_index = 0;

Producer Process Code

item nextProduced;

while(1){

// check if there is no space

// for production.
// if so keep waiting.
while((free_index+1) mod buff_max == full_index);

shared_buff[free_index] = nextProduced;

35 / 123
 Distributed Systems and Cloud Computing
free_index = (free_index + 1) mod buff_max;
}

Consumer Process Code

item nextConsumed;

while(1){

// check if there is an available

// item for consumption.
// if not keep on waiting for
// get them produced.
while((free_index == full_index);

nextConsumed = shared_buff[full_index];
full_index = (full_index + 1) mod buff_max;
}

In the above code, the Producer will start producing again when the (free_index+1) mod buff max will
be free because if it is not free, this implies that there are still items that can be consumed by the
Consumer so there is no need to produce more. Similarly, if free index and full index point to the same
index, this implies that there are no items to consume.
The full overall C++ implementation

2. Message Passing
Definition: Message passing involves processes communicating by sending and receiving messages
through a communication channel. This model is often used in distributed systems where processes
run on different machines.

In this method, processes communicate with each other without using any kind of shared memory. If
two processes p1 and p2 want to communicate with each other, they proceed as follows:

Establish a communication link (if a link already exists, no need to establish it again.)
Start exchanging messages using basic primitives.
We need at least two primitives:
– send (message, destination) or send (message)

36 / 123
 Distributed Systems and Cloud Computing
– receive (message, host) or receive (message)

The message size can be of fixed size or of variable size. If it is of fixed size, it is easy for an OS
designer but complicated for a programmer and if it is of variable size then it is easy for a programmer
but complicated for the OS designer. A standard message can have two parts: **header and
body.** The **header part** is used for storing message type, destination id, source id, message
length, and control information. The control information contains information like what to do if runs out
of buffer space, sequence number, priority. Generally, message is sent using FIFO (First in First
out) style.

Now, We will start our discussion about the methods of implementing communication links. While
implementing the link, there are some questions that need to be kept in mind like :

How are links established?

Can a link be associated with more than two processes?
How many links can there be between every pair of communicating processes?
What is the capacity of a link? Is the size of a message that the link can accommodate fixed or
variable?
Is a link unidirectional or bi-directional?

A link has some capacity that determines the number of messages that can reside in it temporarily for
which every link has a queue associated with it which can be of zero capacity, bounded capacity, or
unbounded capacity. In zero capacity, the sender waits until the receiver informs the sender that it has
received the message. In non-zero capacity cases, a process does not know whether a message has
been received or not after the send operation. For this, the sender must communicate with the
receiver explicitly. Implementation of the link depends on the situation, it can be either a direct
communication link or an in-directed communication link.
Direct Communication links are implemented when the processes use a specific process identifier
for the communication, but it is hard to identify the sender ahead of time.
In-direct Communication is done via a shared mailbox (port), which consists of a queue of
messages. The sender keeps the message in mailbox and the receiver picks them up.

37 / 123
 Distributed Systems and Cloud Computing
Characteristics:

Asynchronous Communication: Processes can send messages without waiting for the
recipient to receive them, allowing for non-blocking communication.
Synchronous Communication: The sending process waits until the message is received by the
recipient, ensuring coordination between processes.
A process that is blocked is one that is waiting for some event, such as a resource becoming
available or the completion of an I/O operation. IPC is possible between the processes on same
computer as well as on the processes running on different computer i.e. in networked/distributed
system. In both cases, the process may or may not be blocked while sending a message or
attempting to receive a message so message passing may be blocking or non-blocking. Blocking
is considered synchronous and blocking send means the sender will be blocked until the
message is received by receiver. Similarly, blocking receive has the receiver block until a
message is available. Non-blocking is considered asynchronous and Non-blocking send has
the sender sends the message and continue. Similarly, Non-blocking receive has the receiver
receive a valid message or null. After a careful analysis, we can come to a conclusion that for a
sender it is more natural to be non-blocking after message passing as there may be a need to
send the message to different processes. However, the sender expects acknowledgment from
the receiver in case the send fails. Similarly, it is more natural for a receiver to be blocking after
issuing the receive as the information from the received message may be used for further
execution. At the same time, if the message send keep on failing, the receiver will have to wait
indefinitely. That is why we also consider the other possibility of message passing. There are
basically three preferred combinations:
Blocking send and blocking receive
Non-blocking send and Non-blocking receive
Non-blocking send and Blocking receive (Mostly used)
A much more detail in-depth deep dive into IPC is available on GFG

Mechanisms:

Send and Receive Operations: The basic operations for message passing. A process sends a
message using a "send" operation and receives it with a "receive" operation.
Message Queues: Messages can be stored in queues if the receiving process is not ready,
allowing for asynchronous communication.
Remote Procedure Calls (RPC): A higher-level abstraction that allows a program to execute a
procedure on a remote machine as if it were local.

Advantages:

Decoupling: Processes do not need to share memory space, which enhances modularity and
allows for easier maintenance.
Scalability: Message passing systems can easily be scaled by adding more processes or nodes.
Enables processes to communicate with each other and share resources, leading to increased
efficiency and flexibility.
38 / 123
 Distributed Systems and Cloud Computing
Facilitates coordination between multiple processes, leading to better overall system
performance.
Allows for the creation of distributed systems that can span multiple computers or networks.
Can be used to implement various synchronization and communication protocols, such as
semaphores, pipes, and sockets.

Disadvantages:

Overhead: Communication involves the overhead of message formatting, transmission, and

potential latency.
Complexity: Handling message delivery and synchronization can add complexity to system
design.
Increases system complexity, making it harder to design, implement, and debug.
Can introduce security vulnerabilities, as processes may be able to access or modify data
belonging to other processes.
Requires careful management of system resources, such as memory and CPU time, to ensure
that IPC operations do not degrade overall system performance.
Can lead to data inconsistencies if multiple processes try to access or modify the same data at
the same time.
Overall, the advantages of IPC outweigh the disadvantages, as it is a necessary mechanism for
modern operating systems and enables processes to work together and share resources in a
flexible and efficient manner. However, care must be taken to design and implement IPC systems
carefully, in order to avoid potential security vulnerabilities and performance issues.

Use Cases:

Distributed systems, cloud services, and microservices architectures where processes

communicate across network boundaries.

3. Comparison of Message Passing and Shared Memory

Feature Message Passing Shared Memory

Communication Indirect (via messages) Direct (via shared memory space)
Performance Higher latency due to message Lower latency due to direct access
handling
Synchronization Managed by the communication system Requires explicit synchronization
mechanisms
Scalability Easily scalable across machines Limited to processes on the same
machine
Complexity Less complex in terms of memory More complex due to synchronization
management needs

39 / 123
 Distributed Systems and Cloud Computing
Both message passing and shared memory are fundamental IPC mechanisms in distributed systems.
The choice between them depends on the specific requirements of the application, including
performance needs, scalability, and the environment in which processes operate. Understanding
these models helps in designing effective communication strategies in distributed architectures.

Distributed Objects and Remote Method Invocation (RMI)

1. Distributed Objects

Definition of Distributed Objects

Distributed objects are software entities that can exist and communicate across a network, enabling
interactions between different systems as if they were local objects. They encapsulate data and
methods, providing a clean interface for communication.

Characteristics of Distributed Objects

Encapsulation:
Distributed objects bundle both state (data) and behavior (methods) into a single unit. This
encapsulation promotes modularity and simplifies interaction.
Transparency:
Users of distributed objects are shielded from the complexities of network communication.
Interactions resemble local method calls, enhancing usability.
Location Independence:
The physical location of the object (whether on the same machine or across the network) is
abstracted away, allowing for flexible deployment and scalability.
Object Identity:
Each distributed object has a unique identifier, allowing it to be referenced across different
contexts and locations.

Types of Distributed Objects

Remote Objects:
Objects that reside on different machines but can be accessed remotely. Communication
with remote objects typically uses protocols like RMI or CORBA.
Mobile Objects:
Objects that can move between different locations in a network during their lifecycle. They
can migrate to different nodes for load balancing or fault tolerance.
Persistent Objects:
Objects that maintain their state beyond the lifecycle of the application. They can be stored
in databases or object stores and retrieved as needed.
40 / 123
 Distributed Systems and Cloud Computing
Communication Mechanisms

Distributed objects can communicate using several mechanisms, including:

Remote Method Invocation (RMI):

Allows methods of an object to be invoked on a remote machine, with the calling object
unaware of the physical separation.
Message Passing:
Objects can send and receive messages, allowing for decoupled communication. This
method is often used in distributed systems where different objects may be running on
separate machines.
Web Services:
Objects can expose their functionality through web services (SOAP or RESTful APIs),
allowing for platform-independent communication.

Object-Oriented Principles

Distributed objects leverage object-oriented principles, enhancing their design and interaction
capabilities:

Inheritance:
Allows distributed objects to inherit properties and methods from parent classes, promoting
code reuse.
Polymorphism:
Objects can be treated as instances of their parent class, allowing for dynamic method
resolution based on the object's actual type at runtime.
Abstraction:
Users interact with objects through well-defined interfaces, hiding the implementation details
and complexities.

Challenges in Distributed Objects

Network Reliability:
Communication can be affected by network failures, latency, and variable performance.
Reliable communication protocols are essential to mitigate these issues.
Security:
Exposing objects over a network introduces security vulnerabilities. Implementing
authentication, authorization, and encryption is crucial.
Serialization:
Objects must be serialized (converted to a byte stream) for transmission over the network
and deserialized on the receiving end. This process can introduce overhead and complexity.

41 / 123
 Distributed Systems and Cloud Computing
Versioning:
Maintaining compatibility between different versions of objects can be challenging,
especially when updating distributed applications.

Use Cases for Distributed Objects

Enterprise Applications:
Business applications that require interaction between various components across multiple
locations.
Cloud Computing:
Distributed objects are central to cloud services, allowing users to interact with cloud
resources as if they were local objects.
IoT Systems:
Distributed objects enable communication between IoT devices and centralized services,
facilitating data exchange and control.
Collaborative Systems:
Applications that require multiple users to interact with shared resources, such as online
editing tools and gaming platforms.

Distributed objects play a vital role in the development of distributed systems, enabling seamless
communication and interaction across diverse environments. Their encapsulation of state and
behavior, along with object-oriented principles, enhances modularity and usability. However,
challenges such as network reliability, security, and serialization need to be addressed to ensure
effective deployment.

2. Remote Method Invocation (RMI)

Definition: Remote Method Invocation (RMI) is a Java-based technology that allows a Java program
to invoke methods on an object located in another Java Virtual Machine (JVM), potentially on a
different machine. It is an API that allows an object to invoke a method on an object that exists in
another address space, which could be on the same machine or on a remote machine. Through RMI,
an object running in a JVM present on a computer (Client-side) can invoke methods on an object
present in another JVM (Server-side). RMI creates a public remote server object that enables client
and server-side communications through simple method calls on the server object.

42 / 123
 Distributed Systems and Cloud Computing
How RMI Works:

1. Remote Interfaces: Defines the methods that can be called remotely. The client interacts with
this interface rather than the implementation.
2. Stub and Skeleton: RMI generates stub (client-side) and skeleton (server-side) code. The stub
acts as a proxy for the remote object, handling the communication between the client and server.
Stub Object: The stub object on the client machine builds an information block and sends this
information to the server.
The block consists of
An identifier of the remote object to be used
Method name which is to be invoked
Parameters to the remote JVM
Skeleton Object: The skeleton object passes the request from the stub object to the
remote object. It performs the following tasks
It calls the desired method on the real object present on the server.
It forwards the parameters received from the stub object to the method.
3. RMI Registry: A naming service that allows clients to look up remote objects using a unique
identifier (name). The server registers its remote objects with the RMI registry.

These are the steps to be followed sequentially to implement Interface as defined below as
follows:

1. Defining a remote interface

2. Implementing the remote interface
3. Creating Stub and Skeleton objects from the implementation class using rmic (RMI compiler)
4. Start the rmiregistry
5. Create and execute the server application program

43 / 123
 Distributed Systems and Cloud Computing
6. Create and execute the client application program.

Step 1: Defining the remote interface

The first thing to do is to create an interface that will provide the description of the methods that can
be invoked by remote clients. This interface should extend the Remote interface and the method
prototype within the interface should throw the RemoteException.
Eg-

// Creating a Search interface

import java.rmi.*;
public interface Search extends Remote
{
// Declaring the method prototype
public String query(String search) throws RemoteException;
}

Step 2: Implementing the remote interface

The next step is to implement the remote interface. To implement the remote interface, the class
should extend to UnicastRemoteObject class of java.rmi package. Also, a default constructor needs to
be created to throw the java.rmi.RemoteException from its parent constructor in class.

// Java program to implement the Search interface

import java.rmi.*;
import java.rmi.server.*;
public class SearchQuery extends UnicastRemoteObject
implements Search
{
// Default constructor to throw RemoteException
// from its parent constructor
SearchQuery() throws RemoteException
{
super();
}

// Implementation of the query interface

public String query(String search)
throws RemoteException
{
String result;
if (search.equals("Reflection in Java"))
result = "Found";
else
result = "Not Found";

return result;
}
}

44 / 123
 Distributed Systems and Cloud Computing
Step 3: Creating Stub and Skeleton objects from the implementation class using rmic
The rmic tool is used to invoke the rmi compiler that creates the Stub and Skeleton objects. Its
prototype is rmic classname. For above program the following command need to be executed at the
command prompt
rmic SearchQuery.

Step 4: Start the rmiregistry

Start the registry service by issuing the following command at the command prompt start rmiregistry

Step 5: Create and execute the server application program

The next step is to create the server application program and execute it on a separate command
prompt.

The server program uses createRegistry method of LocateRegistry class to create rmiregistry
within the server JVM with the port number passed as an argument.
The rebind method of Naming class is used to bind the remote object to the new name.

// Java program for server application

import java.rmi.*;
import java.rmi.registry.*;
public class SearchServer
{
public static void main(String args[])
{
try
{
// Create an object of the interface
// implementation class
Search obj = new SearchQuery();

// rmiregistry within the server JVM with

// port number 1900
LocateRegistry.createRegistry(1900);

// Binds the remote object by the name

// geeksforgeeks
Naming.rebind("rmi://localhost:1900"+
"/geeksforgeeks",obj);
}
catch(Exception ae)
{
System.out.println(ae);
}
}
}

45 / 123
 Distributed Systems and Cloud Computing
Step 6: Create and execute the client application program
The last step is to create the client application program and execute it on a separate command prompt
. The lookup method of the Naming class is used to get the reference of the Stub object.

// Java program for client application

import java.rmi.*;
public class ClientRequest
{
public static void main(String args[])
{
String answer,value="Reflection in Java";
try
{
// lookup method to find reference of remote object
Search access =
(Search)Naming.lookup("rmi://localhost:1900"+

"/geeksforgeeks");
answer = access.query(value);
System.out.println("Article on " + value +
" " + answer+" at
GeeksforGeeks");
}
catch(Exception ae)
{
System.out.println(ae);
}
}
}

Note: The above client and server program is executed on the same machine so localhost is used. In
order to access the remote object from another machine, localhost is to be replaced with the IP
address where the remote object is present.

save the files respectively as per class name as : Search.java , SearchQuery.java ,

SearchServer.java & ClientRequest.java**

Important Observations:

1. RMI is a pure java solution to Remote Procedure Calls (RPC) and is used to create the
distributed applications in java.
2. Stub and Skeleton objects are used for communication between the client and server-side.

Advantages:

Simplified Communication: RMI abstracts the complexities of network communication, making

it easier for developers to work with remote objects.

46 / 123
 Distributed Systems and Cloud Computing
Language Interoperability: Although primarily Java-based, RMI can be extended to support
other languages with appropriate libraries.
Object-Oriented: Leverages Java's object-oriented features, promoting a natural design
approach.

Disadvantages:

Performance Overhead: Serialization and deserialization of objects can introduce latency and
increase overhead compared to local method calls.
Java Dependency: RMI is tightly coupled with Java, limiting its use in environments with mixed-
language components.

Use Cases:

Distributed applications in Java, such as enterprise solutions, remote service invocations, and
cloud-based applications.

Note : java.rmi package: Remote Method Invocation (RMI) has been deprecated in Java 9 and
later versions, in favour of other remote communication mechanisms like web services or
Remote Procedure Calls (RPC).

3. Comparison of Distributed Objects and RMI

Feature Distributed Objects Remote Method Invocation

(RMI)
Abstraction Encapsulates state and behavior Provides a way to invoke
methods remotely
Communication Can be implemented with various Specific to Java and uses
protocols serialization
Complexity Can involve various communication Simplifies remote method
mechanisms invocation
Language Varies based on implementation Primarily Java-dependent
Independence
Use Cases Service-oriented architectures, cloud Java-based distributed
apps applications

Distributed objects and Remote Method Invocation are critical concepts in building distributed
systems. They enable seamless interaction between components located in different environments,
promoting modular design and reusability. Understanding these concepts is essential for developing
robust and scalable distributed applications.

Remote Procedure Call (RPC)

47 / 123
 Distributed Systems and Cloud Computing
1. Definition of RPC
A remote Procedure Call (RPC) is a protocol in distributed systems that allows a client to execute
functions on a remote server as if they were local. RPC simplifies network communication by
abstracting the complexities, making it easier to develop and integrate distributed applications
efficiently.

RPC enables a client to invoke methods on a server residing in a different address space (often
on a different machine) as if they were local procedures.
The client and server communicate over a network, allowing for remote interaction and
computation.

Importance of Remote Procedural Call(RPC) in Distributed Systems

Remote Procedure Call (RPC) plays a crucial role in distributed systems by enabling seamless
communication and interaction between different components or services that reside on separate
machines or servers. Here’s an outline of its importance:

Simplified Communication
Abstraction of Complexity: RPC abstracts the complexity of network communication,
allowing developers to call remote procedures as if they were local, simplifying the

48 / 123
 Distributed Systems and Cloud Computing
development of distributed applications.
Consistent Interface: Provides a consistent and straightforward interface for invoking
remote services, which helps in maintaining uniformity across different parts of a system.
Enhanced Modularity and Reusability
Decoupling: RPC enables the decoupling of system components, allowing them to interact
without being tightly coupled. This modularity helps in building more maintainable and
scalable systems.
Service Reusability: Remote services or components can be reused across different
applications or systems, enhancing code reuse and reducing redundancy.
Facilitates Distributed Computing
Inter-Process Communication (IPC): RPC allows different processes running on separate
machines to communicate and cooperate, making it essential for building distributed
applications that require interaction between various nodes.
Resource Sharing: Enables sharing of resources and services across a network, such as
databases, computation power, or specialized functionalities.

2. How RPC Works

The RPC (Remote Procedure Call) architecture in distributed systems is designed to enable
communication between client and server components that reside on different machines or nodes
across a network. The architecture abstracts the complexities of network communication and allows
procedures or functions on one system to be executed on another as if they were local. Here’s an
overview of the RPC architecture:

1. Client and Server Components

Client: The client is the component that makes the RPC request. It invokes a procedure or
method on the remote server by calling a local stub, which then handles the details of
communication.
Server: The server hosts the actual procedure or method that the client wants to execute. It
processes incoming RPC requests and sends back responses.

2. Stubs

Client Stub: Acts as a proxy on the client side. It provides a local interface for the client to call
the remote procedure. The client stub is responsible for marshalling (packing) the procedure
arguments into a format suitable for transmission and for sending the request to the server.
Server Stub: On the server side, the server stub receives the request, unmarshals (unpacks) the
arguments, and invokes the actual procedure on the server. It then marshals the result and
sends it back to the client stub.

49 / 123
 Distributed Systems and Cloud Computing
3. Marshalling and Unmarshalling

Marshalling: The process of converting procedure arguments and return values into a format
that can be transmitted over the network. This typically involves serializing the data into a byte
stream.
Unmarshalling: The reverse process of converting the received byte stream back into the
original data format that can be used by the receiving system.

4. Communication Layer

Transport Protocol: RPC communication usually relies on a network transport protocol, such as
TCP or UDP, to handle the data transmission between client and server. The transport protocol
ensures that data packets are reliably sent and received.
Message Handling: This layer is responsible for managing network messages, including routing,
buffering, and handling errors.

5. RPC Framework

Interface Definition Language (IDL): Used to define the interface for the remote procedures.
IDL specifies the procedures, their parameters, and return types in a language-neutral way. This
allows for cross-language interoperability.
RPC Protocol: Defines how the client and server communicate, including the format of requests
and responses, and how to handle errors and exceptions.

6. Error Handling and Fault Tolerance

Timeouts and Retries: Mechanisms to handle network delays or failures by retrying requests or
handling timeouts gracefully.
Exception Handling: RPC frameworks often include support for handling remote exceptions and
reporting errors back to the client.

7. Security

Authentication and Authorization: Ensures that only authorized clients can invoke remote
procedures and that the data exchanged is secure.
Encryption: Protects data in transit from being intercepted or tampered with during transmission.

Types of Remote Procedural Call (RPC) in Distributed Systems

50 / 123
 Distributed Systems and Cloud Computing
In distributed systems, Remote Procedure Call (RPC) implementations vary based on the
communication model, data representation, and other factors. Here are the main types of RPC:

1. Synchronous RPC

Description: In synchronous RPC, the client sends a request to the server and waits for the
server to process the request and send back a response before continuing execution.
Characteristics:
Blocking: The client is blocked until the server responds.
Simple Design: Easy to implement and understand.
Use Cases: Suitable for applications where immediate responses are needed and where
latency is manageable.

2. Asynchronous RPC

Description: In asynchronous RPC, the client sends a request to the server and continues its
execution without waiting for the server’s response. The server’s response is handled when it
arrives.
Characteristics:
Non-Blocking: The client does not wait for the server’s response, allowing for other tasks
to be performed concurrently.
Complexity: Requires mechanisms to handle responses and errors asynchronously.
Use Cases: Useful for applications where tasks can run concurrently and where
responsiveness is critical.

3. One-Way RPC

Description: One-way RPC involves sending a request to the server without expecting any
response. It is used when the client does not need a return value or acknowledgment from the
server.
Characteristics:
Fire-and-Forget: The client sends the request and does not wait for a response or
confirmation.
Use Cases: Suitable for scenarios where the client initiates an action but does not require
immediate feedback, such as logging or notification services.

4. Callback RPC

Description: In callback RPC, the client provides a callback function or mechanism to the server.
After processing the request, the server invokes the callback function to return the result or notify
51 / 123
 Distributed Systems and Cloud Computing
the client.
Characteristics:
Asynchronous Response: The client does not block while waiting for the response;
instead, the server calls back the client once the result is ready.
Use Cases: Useful for long-running operations where the client does not need to wait for
completion.

5. Batch RPC

Description: Batch RPC allows the client to send multiple RPC requests in a single batch to the
server, and the server processes them together.
Characteristics:
Efficiency: Reduces network overhead by bundling multiple requests and responses.
Use Cases: Ideal for scenarios where multiple related operations need to be performed
together, reducing round-trip times.

3. Characteristics of RPC

Transparency:
RPC abstracts the complexities of network communication, allowing developers to focus on
the application logic without worrying about the underlying details.
Synchronous vs. Asynchronous:
RPC calls can be synchronous (the client waits for the server to respond) or asynchronous
(the client continues processing while waiting for the response).
Language Independence:
RPC frameworks can support multiple programming languages, allowing clients and servers
written in different languages to communicate.

4. RPC Protocols

Several protocols are used for implementing RPC, including:

HTTP/HTTPS:
Used for web-based RPC implementations. Protocols like RESTful APIs or SOAP can
facilitate remote procedure calls over HTTP.
gRPC:
A modern RPC framework developed by Google that uses HTTP/2 for transport and
Protocol Buffers (protobufs) for serialization. It supports multiple languages and features like
bidirectional streaming.
XML-RPC:

52 / 123
 Distributed Systems and Cloud Computing
A simple protocol that uses XML to encode its calls and HTTP as a transport mechanism. It
allows for remote procedure calls using a straightforward XML format.
JSON-RPC:
Similar to XML-RPC but uses JSON for encoding messages, providing a lighter-weight
alternative.

Performance and optimization of Remote Procedure Calls (RPC) in

Distributed Systems
Performance and optimization of Remote Procedure Calls (RPC) in distributed systems are crucial for
ensuring that remote interactions are efficient, reliable, and scalable. Given the inherent network
latency and resource constraints, optimizing RPC can significantly impact the overall performance of
distributed applications. Here’s a detailed look at key aspects of performance and optimization for
RPC:

Minimizing Latency
Batching Requests: Group multiple RPC requests into a single batch to reduce the
number of network round-trips.
Asynchronous Communication: Use asynchronous RPC to avoid blocking the client and
improve responsiveness.
Compression: Compress data before sending it over the network to reduce transmission
time and bandwidth usage.
Reducing Overhead
Efficient Serialization: Use efficient serialization formats (e.g., Protocol Buffers, Avro) to
minimize the time and space required to marshal and unmarshal data.
Protocol Optimization: Choose or design lightweight communication protocols that
minimize protocol overhead and simplify interactions.
Request and Response Size: Optimize the size of requests and responses by including
only necessary data to reduce network load and processing time.
Load Balancing and Scalability
Load Balancers: Use load balancers to distribute RPC requests across multiple servers or
instances, improving scalability and preventing any single server from becoming a
bottleneck.
Dynamic Scaling: Implement mechanisms to dynamically scale resources based on
demand to handle variable loads effectively.
Caching and Data Optimization
Result Caching: Cache the results of frequently invoked RPC calls to avoid redundant
processing and reduce response times.
Local Caching: Implement local caches on the client side to store recent results and
reduce the need for repeated remote calls.
Fault Tolerance and Error Handling

53 / 123
 Distributed Systems and Cloud Computing
Retries and Timeouts: Implement retry mechanisms and timeouts to handle transient
errors and network failures gracefully.
Error Reporting: Use detailed error reporting to diagnose and address issues that impact
performance.

5. Advantages of RPC

Simplified Communication:
Abstracts complex network interactions, allowing developers to invoke remote procedures
easily.
Modularity:
Encourages a modular architecture where different components can be developed, tested,
and deployed independently.
Interoperability:
Different systems can communicate regardless of the underlying technology, provided they
conform to the same RPC protocol.

6. Disadvantages of RPC

Performance Overhead:
Serialization and network communication introduce latency compared to local procedure
calls.
Complexity in Error Handling:
Handling network-related errors, timeouts, and retries can complicate the implementation.
Dependency on Network:
RPC systems are inherently reliant on network availability and performance, making them
susceptible to disruptions.

7. Use Cases for RPC

Microservices Architectures:
RPC is widely used in microservices to facilitate communication between different services
in a distributed system.
Cloud Services:
RPC enables cloud-based applications to call external services and APIs seamlessly.
Distributed Applications:
Applications that require remote interactions across different components, such as
collaborative tools and real-time systems.

54 / 123
 Distributed Systems and Cloud Computing
Remote Procedure Call (RPC) is a powerful mechanism that simplifies communication between
distributed systems by allowing remote procedure execution as if it were local. While it offers
significant advantages in terms of transparency and modularity, developers must also consider the
associated performance overhead and complexities of error handling. Understanding RPC is essential
for building efficient and scalable distributed applications.

FAQS:
Q1: How does RPC handle network failures and ensure reliability in distributed systems?

RPC mechanisms often employ strategies like retries, timeouts, and acknowledgments to handle
network failures. However, ensuring reliability requires advanced techniques such as idempotent
operations, transaction logging, and distributed consensus algorithms to manage failures and
maintain consistency.

Q2: What are the trade-offs between synchronous and asynchronous RPC calls in terms of
performance and usability?

Synchronous RPC calls block the client until a response is received, which can lead to
performance bottlenecks and reduced responsiveness. Asynchronous RPC calls, on the other
hand, allow the client to continue processing while waiting for the server’s response, improving
scalability but complicating error handling and state management.

Q3: How does RPC deal with heterogeneous environments where client and server might be
running different platforms or programming languages?

RPC frameworks use standardized protocols (e.g., Protocol Buffers, JSON, XML) and serialization
formats to ensure interoperability between different platforms and languages. Ensuring
compatibility involves designing APIs with careful attention to data formats and communication
protocols.

Q4: What are the security implications of using RPC in distributed systems, and how can they
be mitigated?

RPC can introduce security risks such as data interception, unauthorized access, and replay
attacks. Mitigation strategies include using encryption (e.g., TLS), authentication mechanisms
(e.g., OAuth), and robust authorization checks to protect data and ensure secure communication.

Q5: How do RPC frameworks handle versioning and backward compatibility of APIs in evolving
distributed systems?

Managing API versioning and backward compatibility involves strategies like defining versioned
endpoints, using feature flags, and supporting multiple API versions concurrently. RPC frameworks
often provide mechanisms for graceful upgrades and maintaining compatibility across different
versions of client and server implementations.

55 / 123
 Distributed Systems and Cloud Computing

Events and Notifications

1. Definition of Events and Notifications
Events and notifications are mechanisms used in distributed systems to enable communication and
coordination between components based on certain occurrences or conditions. They allow systems to
react to changes in state or data asynchronously.

2. Events
Events are significant occurrences or changes in state within a system that can trigger specific
actions or behaviors. An event can be generated by a user action, a system process, or a change in
the environment.

Characteristics of Events:

Asynchronous: Events can occur at any time and do not require the immediate attention of the
system or users.
Decoupling: Event producers and consumers are decoupled, allowing for greater flexibility and
scalability. Components can evolve independently as long as they adhere to the event contract.
Event Types:
Simple Events: Individual occurrences, such as a user clicking a button.
Composite Events: Aggregations of simple events that represent a more complex
occurrence, such as a user logging in.

Examples of Events:

User interactions (e.g., clicks, form submissions).

System state changes (e.g., resource availability, error conditions).
Timers and scheduled tasks.

3. Notifications

Notifications are messages sent to inform components or users of events that have occurred.
Notifications convey the information needed to react to an event without requiring the recipient to poll
for updates.

Characteristics of Notifications:

Content-Based: Notifications typically include data about the event, such as event type,
timestamp, and relevant payload.
One-to-Many Communication: Notifications can be broadcasted to multiple subscribers,
allowing for efficient dissemination of information.

56 / 123
 Distributed Systems and Cloud Computing
Types of Notifications:

Direct Notifications: Sent directly to a specific recipient or a known address.

Broadcast Notifications: Sent to multiple recipients or groups, allowing all interested parties to
receive the update.

4. Communication Mechanisms
Several mechanisms facilitate the delivery of events and notifications in distributed systems:

Publish-Subscribe Model:
In this model, components (publishers) generate events and publish them to a message
broker. Other components (subscribers) express interest in specific events and receive
notifications when those events occur.
Advantages:
Loose coupling between producers and consumers.
Scalability, as multiple subscribers can receive the same event without impacting the
publisher.
Event Streams:
A continuous flow of events generated by producers. Consumers can subscribe to the
stream and process events in real-time.
Technologies such as Apache Kafka and Amazon Kinesis are commonly used for event
streaming.
Message Queues:
Systems like RabbitMQ or ActiveMQ provide message queueing capabilities, allowing
producers to send events to queues, which can then be processed by consumers
asynchronously.

5. Advantages of Events and Notifications

Responsiveness: Systems can respond to changes and events in real-time, enhancing user
experience and system performance.
Decoupled Architecture: Encourages a more flexible architecture where components can be
developed and maintained independently.
Scalability: Facilitates scaling systems by allowing multiple producers and consumers to operate
concurrently.

6. Disadvantages of Events and Notifications

Complexity: Managing events, subscriptions, and notification flows can introduce complexity,
particularly in large systems.
57 / 123
 Distributed Systems and Cloud Computing
Event Loss: If not properly managed, events may be lost during transmission, leading to
inconsistencies in system state.
Debugging Challenges: Asynchronous communication can make it harder to trace the flow of
events and identify issues.

7. Use Cases for Events and Notifications

Real-Time Applications: Applications requiring instant updates, such as chat applications,

gaming, and financial trading platforms.
Microservices: Enabling communication between loosely coupled microservices, allowing them
to react to state changes and events.
Monitoring and Alerting Systems: Systems that need to notify administrators of issues or
status changes.

8. Conclusion

Events and notifications play a crucial role in building responsive and scalable distributed systems. By
enabling asynchronous communication and decoupled interactions between components, they
enhance system flexibility and maintainability. However, developers must carefully manage event
flows and notifications to avoid complexity and potential pitfalls.

If you have more specific aspects you’d like to explore or want to move on to the next topic, let me
know!

Case Study - Java Remote Method Invocation (RMI)

1. Introduction to Java RMI

Java Remote Method Invocation (RMI) is a powerful mechanism that allows Java programs to invoke
methods on objects located in different Java Virtual Machines (JVMs), potentially on different physical
machines. It is part of the Java Standard Edition and provides a framework for building distributed
applications.

2. Key Concepts of Java RMI

Remote Interface:
A Java interface that declares the methods that can be called remotely. Any class
implementing this interface can be invoked by remote clients.
Stub and Skeleton:
Stub: The client-side representation of the remote object, responsible for marshaling
method calls and sending them to the server.
58 / 123
 Distributed Systems and Cloud Computing
Skeleton: The server-side representation that receives requests from the stub, unmarshals
the data, and invokes the actual remote method.
RMI Registry:
A naming service that allows clients to look up remote objects by name. Servers register
their remote objects with the RMI registry, making them available for remote invocation.

3. Architecture of Java RMI

1. Client:
The application that invokes a remote method.
2. RMI Registry:
Acts as a directory service for remote objects. Clients use it to obtain references to remote
objects.
3. Server:
The application that implements the remote interface and provides the actual
implementation of the remote methods.
4. Network:
The communication medium through which the client and server interact.

4. Implementation Steps

1. Define the Remote Interface:

import java.rmi.Remote;
import java.rmi.RemoteException;

public interface Calculator extends Remote {

int add(int a, int b) throws RemoteException;
int subtract(int a, int b) throws RemoteException;
}

2. Implement the Remote Interface:

import java.rmi.server.UnicastRemoteObject;
import java.rmi.RemoteException;

public class CalculatorImpl extends UnicastRemoteObject implements Calculator {

public CalculatorImpl() throws RemoteException {
super();
}

@Override
public int add(int a, int b) throws RemoteException {
59 / 123
 Distributed Systems and Cloud Computing
return a + b;
}

@Override
public int subtract(int a, int b) throws RemoteException {
return a - b;
}
}

3. Create the Server:

import java.rmi.registry.LocateRegistry;
import java.rmi.registry.Registry;

public class CalculatorServer {

public static void main(String[] args) {
try {
CalculatorImpl calculator = new CalculatorImpl();
Registry registry = LocateRegistry.createRegistry(1099); // Default
RMI port
registry.bind("Calculator", calculator);
System.out.println("Calculator Server is ready.");
} catch (Exception e) {
e.printStackTrace();
}
}
}

4. Create the Client:

import java.rmi.registry.LocateRegistry;
import java.rmi.registry.Registry;

public class CalculatorClient {

public static void main(String[] args) {
try {
Registry registry = LocateRegistry.getRegistry("localhost", 1099);
Calculator calculator = (Calculator) registry.lookup("Calculator");
System.out.println("Addition: " + calculator.add(5, 10));
System.out.println("Subtraction: " + calculator.subtract(10, 5));
} catch (Exception e) {
e.printStackTrace();
}
}
}

5. Compile and Run:

60 / 123
 Distributed Systems and Cloud Computing
Compile the Java files and run the server first, followed by the client. Ensure that the RMI
registry is running.

5. Advantages of Java RMI

Simplicity: RMI provides a straightforward approach for remote communication, making it easier
for developers to build distributed applications.
Object-Oriented: RMI leverages Java's object-oriented features, allowing developers to work
with remote objects naturally.
Automatic Serialization: RMI handles the serialization and deserialization of objects
automatically, simplifying data exchange between client and server.

6. Disadvantages of Java RMI

Java Dependency: RMI is primarily designed for Java applications, which may limit
interoperability with non-Java systems.
Performance Overhead: The serialization process and network latency can introduce delays
compared to local method calls.
Complex Error Handling: Managing remote exceptions and communication failures can
complicate application logic.

7. Use Cases for Java RMI

Distributed Computing: Applications that require remote method execution across different
machines, such as distributed algorithms or task processing.
Enterprise Applications: Applications that need to interact with remote services or databases,
providing a modular architecture for business logic.

8. Conclusion
Java RMI is a powerful and easy-to-use framework for building distributed applications in Java. By
allowing remote method invocations to occur seamlessly, RMI simplifies the development of
applications that require interaction across different networked environments. However, developers
should be mindful of its limitations, particularly in terms of performance and interoperability.

UNIT 2 (Synchronization)
Time and Global States - Introduction

61 / 123
 Distributed Systems and Cloud Computing
1. Understanding Time in Distributed Systems
Time in distributed systems is complex due to the lack of a global clock. Unlike centralized systems,
where time is uniform and easily tracked, distributed systems consist of multiple nodes that operate
independently. This independence complicates the synchronization of actions across different nodes.

Key Concepts:

Local Time: Each node maintains its own local clock, which may drift from the clocks of other
nodes.
Logical Time: A mechanism that provides a consistent ordering of events across distributed
systems without relying on physical clocks.
Physical Time: Refers to actual time, typically synchronized using protocols like Network Time
Protocol (NTP).

2. Challenges of Time in Distributed Systems

Clock Drift: Local clocks can drift apart, leading to inconsistencies when coordinating actions.
Event Ordering: Determining the order of events across different nodes can be difficult,
especially when events are not directly related.
Causality: Understanding the cause-and-effect relationship between events is crucial for correct
system behavior.

3. Global States
A global state is a snapshot of the state of all processes and communications in a distributed system
at a particular time. Understanding global states is essential for various distributed algorithms and
protocols.

Characteristics of Global States:

Consistency: A global state must reflect a consistent view of the system, ensuring that it
adheres to the causal relationships between events.
Partial States: Global states can be viewed as partial snapshots, where not all processes may
be included.

4. Importance of Global States

Debugging: Global states provide insight into the system's behavior, aiding in debugging and
performance monitoring.
Checkpointing: Systems can save global states to recover from failures or rollback to a previous
state.

62 / 123
 Distributed Systems and Cloud Computing
Consensus Algorithms: Many distributed algorithms (like leader election) rely on the ability to
determine a consistent global state.

5. Representing Time and Global States

To handle time and global states in distributed systems, several models and concepts are employed:

Logical Clocks: Mechanisms like Lamport timestamps or vector clocks are used to impose a
logical ordering of events across distributed systems.
Lamport Timestamps: Each process maintains a counter, incrementing it for each local
event. When sending a message, it includes its current counter value, allowing receivers to
update their counters appropriately.
Vector Clocks: Each process maintains a vector of timestamps (one for each process),
providing more information about the causal relationships between events.
Snapshot Algorithms: Techniques to capture a consistent global state of a distributed system,
such as Chandy-Lamport’s algorithm. This algorithm allows processes to take snapshots of their
local state and the state of their incoming messages without stopping the system.

Time and global states are fundamental concepts in distributed systems, impacting synchronization,
coordination, and communication between processes. Understanding these concepts is essential for
designing efficient algorithms and protocols that ensure consistency and correctness in distributed
applications.

Logical Clocks
1. Introduction to Logical Clocks
Logical clocks are a concept used in distributed systems to provide a way to order events without
relying on synchronized physical clocks. Since physical clocks in different nodes of a distributed
system can drift and may not be perfectly synchronized, logical clocks ensure consistency in the order
of events across the system. In distributed systems, ensuring synchronized events across multiple
nodes is crucial for consistency and reliability. By assigning logical timestamps to events, these clocks
enable systems to reason about causality and sequence events accurately, even across network
delays and varied system clocks.

Logical clocks assign "timestamps" to events, allowing processes in a distributed system to

understand the sequence and causal relationships between events.

Differences Between Physical and Logical Clocks

Physical clocks and logical clocks serve distinct purposes in distributed systems:

63 / 123
 Distributed Systems and Cloud Computing
1. Nature of Time:
Physical Clocks: These rely on real-world time measurements and are typically
synchronized using protocols like NTP (Network Time Protocol). They provide accurate
timestamps but can be affected by clock drift and network delays.
Logical Clocks: These are not tied to real-world time and instead use logical counters or
timestamps to order events based on causality. They are resilient to clock differences
between nodes but may not provide real-time accuracy.`
2. Usage:
Physical Clocks: Used for tasks requiring real-time synchronization and precise
timekeeping, such as scheduling tasks or logging events with accurate timestamps.
Logical Clocks: Used in distributed systems to order events across different nodes in a
consistent and causal manner, enabling synchronization and coordination without strict real-
time requirements.
3. Dependency:
Physical Clocks: Dependent on accurate timekeeping hardware and synchronization
protocols to maintain consistency across distributed nodes.
Logical Clocks: Dependent on the logic of event ordering and causality, ensuring that
events can be correctly sequenced even when nodes have different physical time readings.

2. Types of Logical Clocks

1. Lamport Clocks
Lamport clocks provide a simple way to order events in a distributed system. Each node maintains a
counter that increments with each event. When nodes communicate, they update their counters based
on the maximum value seen, ensuring a consistent order of events.

Characteristics of Lamport Clocks:

Simple to implement.
Provides a total order of events but doesn’t capture concurrency.
Not suitable for detecting causal relationships between events.

Algorithm of Lamport Clocks:

1. Initialization: Each node initializes its clock LLL to 0.

2. Internal Event: When a node performs an internal event, it increments its clock LLL.
3. Send Message: When a node sends a message, it increments its clock LLL and includes this
value in the message.

64 / 123
 Distributed Systems and Cloud Computing
4. Receive Message: When a node receives a message with timestamp T: It sets
L = max(L, T ) + 1

Advantages of Lamport Clocks:

Simple to implement and understand.

Ensures total ordering of events.

2. Vector Clocks
Vector clocks use an array of integers, where each element corresponds to a node in the system.
Each node maintains its own vector clock and updates it by incrementing its own entry and
incorporating values from other nodes during communication.

Characteristics of Vector Clocks:

Captures causality and concurrency between events.

Requires more storage and communication overhead compared to Lamport clocks.

Algorithm of Vector Clocks:

1. Initialization: Each node PiP_iPi​initializes its vector clock V iV V i to a vector of zeros.

i

2. Internal Event: When a node performs an internal event, it increments its own entry in the vector
clock V i[i]V [i]V i[i].
i

3. Send Message: When a node PiP_iPi​sends a message, it includes its vector clock V iV V i in i

the message.
4. Receive Message: When a node PiP_iPi​receives a message with vector clock Vj​:
It updates each entry: V i[k] = max(V i[k], V j[k])
It increments its own entry: V i[i] = V i[i] + 1

Advantages of Vector Clocks:

Accurately captures causality and concurrency.

Detects concurrent events, which Lamport clocks cannot do.

3. Matrix Clocks

Matrix clocks extend vector clocks by maintaining a matrix where each entry captures the history of
vector clocks. This allows for more detailed tracking of causality relationships.

65 / 123
 Distributed Systems and Cloud Computing
Characteristics of Matrix Clocks:

More detailed tracking of event dependencies.

Higher storage and communication overhead compared to vector clocks.

Algorithm of Matrix Clocks:

1. Initialization: Each node PiP_iPi​initializes its matrix clock MiM_iMi​to a matrix of zeros.
2. Internal Event: When a node performs an internal event, it increments its own entry in the matrix
clock M i[i][i]M [i][i]M i[i][i].
i

3. Send Message: When a node PiP_iPi​sends a message, it includes its matrix clock MiM_iMi​in
the message.
4. Receive Message: When a node PiP_iPi​receives a message with matrix clock Mj:
It updates each entry: M i[k][l] = max(M i[k][l], M j[k][l])
It increments its own entry: M i[i][i] = M i[i][i] + 1

Advantages of Matrix Clocks:

Detailed history tracking of event causality.

Can provide more information about event dependencies than vector clocks.

4. Hybrid Logical Clocks (HLCs)

Hybrid logical clocks combine physical and logical clocks to provide both causality and real-time
properties. They use physical time as a base and incorporate logical increments to maintain event
ordering.

Characteristics of Hybrid Logical Clocks:

Combines real-time accuracy with causality.

More complex to implement compared to pure logical clocks.

Algorithm of Hybrid Logical Clocks:

1. Initialization: Each node initializes its clock HHH with the current physical time.
2. Internal Event: When a node performs an internal event, it increments its logical part of the
HLC.
3. Send Message: When a node sends a message, it includes its HLC in the message.
4. Receive Message: When a node receives a message with HLC T:

66 / 123
 Distributed Systems and Cloud Computing
It updates its H = max(H , T ) + 1

Advantages of Hybrid Logical Clocks:

Balances real-time accuracy and causal consistency.

Suitable for systems requiring both properties, such as databases and distributed ledgers.

5. Version Vectors

Version vectors track versions of objects across nodes. Each node maintains a vector of version
numbers for objects it has seen.

Characteristics of Version Vectors:

Tracks versions of objects.

Similar to vector clocks, but specifically for versioning.

Algorithm of Version Vectors:

1. Initialization: Each node initializes its version vector to zeros.

2. Update Version: When a node updates an object, it increments the corresponding entry in the
version vector.
3. Send Version: When a node sends an updated object, it includes its version vector in the
message.
4. Receive Version: When a node receives an object with a version vector:
It updates its version vector to the maximum values seen for each entry.

Advantages of Version Vectors:

Efficient conflict resolution.

Tracks object versions effectively in distributed databases and file systems.

3. Comparison of Lamport and Vector Clocks

Lamport Clocks:
Simpler and uses a single integer per process.
Only provides partial ordering of events.
Cannot detect concurrency directly.
Vector Clocks:
67 / 123
 Distributed Systems and Cloud Computing
More complex, requiring storage of a vector for each process.
Provides a total ordering of events and can detect concurrency between events.
Requires more communication overhead due to the larger size of the vector.

4. Applications of Logical Clocks

Logical clocks play a crucial role in distributed systems by providing a way to order events and
maintain consistency. Here are some key applications:

Event Ordering
Causal Ordering: Logical clocks help establish a causal relationship between events,
ensuring that messages are processed in the correct order.
Total Ordering: In some systems, it’s essential to have a total order of events. Logical
clocks can be used to assign unique timestamps to events, ensuring a consistent order
across the system.
Causal Consistency
Consistency Models: In distributed databases and storage systems, logical clocks are
used to ensure causal consistency. They help track dependencies between operations,
ensuring that causally related operations are seen in the same order by all nodes.
Distributed Debugging and Monitoring
Tracing and Logging: Logical clocks can be used to timestamp logs and trace events
across different nodes in a distributed system. This helps in debugging and understanding
the sequence of events leading to an issue.
Performance Monitoring: By using logical clocks, it’s possible to monitor the performance
of distributed systems, identifying bottlenecks and delays.
Distributed Snapshots
Checkpointing: Logical clocks are used in algorithms for taking consistent snapshots of the
state of a distributed system, which is essential for fault tolerance and recovery.
Global State Detection: They help detect global states and conditions such as deadlocks
or stable properties in the system.
Concurrency Control
Optimistic Concurrency Control: Logical clocks help detect conflicts in transactions by
comparing timestamps, allowing systems to resolve conflicts and maintain data integrity.
Versioning: In versioned storage systems, logical clocks can be used to maintain different
versions of data, ensuring that updates are applied correctly and consistently.

5. Advantages of Logical Clocks

No Need for Synchronized Physical Clocks: They eliminate the need for expensive and
potentially unreliable physical clock synchronization.

68 / 123
 Distributed Systems and Cloud Computing
Causality Preservation: They guarantee that causally related events are ordered correctly,
providing a solid foundation for building consistent distributed systems.

6. Disadvantages of Logical Clocks

No Real-Time Representation: Logical clocks only provide a relative ordering of events and do
not reflect actual physical time.
Scalability: Vector clocks require maintaining and transmitting a vector of timestamps, which can
grow large as the number of processes increases.

Challenges and Limitations with Logical Clocks

Logical clocks are essential for maintaining order and consistency in distributed systems, but they
come with their own set of challenges and limitations:

Scalability Issues
Vector Clock Size: In systems using vector clocks, the size of the vector grows with the
number of nodes, leading to increased storage and communication overhead.
Management Complexity: Managing and maintaining logical clocks across a large number
of nodes can be complex and resource-intensive.
Synchronization Overhead
Communication Overhead: Synchronizing logical clocks requires additional messages
between nodes, which can increase network traffic and latency.
Processing Overhead: Updating and maintaining logical clock values can add
computational overhead, impacting the system’s overall performance.
Handling Failures and Network Partitions
Clock Inconsistency: In the presence of network partitions or node failures, maintaining
consistent logical clock values can be challenging.
Recovery Complexity: When nodes recover from failures, reconciling logical clock values
to ensure consistency can be complex.
Partial Ordering
Limited Ordering Guarantees: Logical clocks, especially Lamport clocks, only provide
partial ordering of events, which may not be sufficient for all applications requiring a total
order.
Conflict Resolution: Resolving conflicts in operations may require additional mechanisms
beyond what logical clocks can provide.
Complexity in Implementation
Algorithm Complexity: Implementing logical clocks, particularly vector and matrix clocks,
can be complex and error-prone, requiring careful design and testing.
Application-Specific Adjustments: Different applications may require customized logical
clock implementations to meet their specific requirements.
69 / 123
 Distributed Systems and Cloud Computing
Storage Overhead
Vector and Matrix Clocks: These clocks require storing a vector or matrix of timestamps,
which can consume significant memory, especially in systems with many nodes.
Snapshot Storage: For some applications, maintaining snapshots of logical clock values
can add to the storage overhead.
Propagation Delay
Delayed Updates: Updates to logical clock values may not propagate instantly across all
nodes, leading to temporary inconsistencies.
Latency Sensitivity: Applications that are sensitive to latency may be impacted by the
delays in propagating logical clock updates.

Logical clocks are essential tools in distributed systems for ordering events and ensuring consistency
without relying on synchronized physical clocks. While Lamport timestamps are simpler to implement,
vector clocks provide a more robust solution for capturing the full causal relationships between events.
Both types of logical clocks are widely used in distributed algorithms, databases, and event-driven
systems.

FAQs for Logical Clock in Distributed System

Q 1. How do Lamport clocks work and what are their limitations?

Lamport clocks work by maintaining a counter at each node, which increments with each event.
When nodes communicate, they update their counters based on the maximum value seen,
ensuring a consistent event order. However, Lamport clocks do not capture concurrency and only
provide a total order of events, meaning they cannot distinguish between concurrent events.

Q 2. What are vector clocks and how do they differ from Lamport clocks?

Vector clocks use an array of integers where each element corresponds to a node in the system.
They capture both causality and concurrency by maintaining a vector clock that each node
updates with its own events and the events it receives from others. Unlike Lamport clocks, vector
clocks can detect concurrent events, providing a more accurate representation of event causality.

Q 3. Why are hybrid logical clocks (HLCs) used and what benefits do they offer?

Hybrid logical clocks combine the properties of physical and logical clocks to provide both real-
time accuracy and causal consistency. They are used in systems requiring both properties, such
as databases and distributed ledgers. HLCs offer the benefit of balancing the need for precise
timing with the need to maintain an accurate causal order of events.

Q 4. What is the primary use case for version vectors in distributed systems?

Version vectors are primarily used for conflict resolution and synchronization in distributed
databases and file systems. They track versions of objects across nodes, allowing the system to

70 / 123
 Distributed Systems and Cloud Computing
determine the most recent version of an object and resolve conflicts that arise from concurrent
updates. This ensures data consistency and integrity in distributed environments.

Q 5. How do logical clocks help in maintaining consistency in distributed systems?

Logical clocks help maintain consistency by providing a mechanism to order events and establish
causality across different nodes. This ensures that events are processed in the correct sequence,
preventing anomalies such as race conditions or inconsistent states. By using logical timestamps,
distributed systems can coordinate actions, detect conflicts, and ensure that all nodes have a
coherent view of the system’s state, even in the presence of network delays and asynchrony.

Synchronizing Events and Processes

1. Introduction to Synchronizing Events and Process States
In distributed systems, synchronizing events and process states is crucial to ensure consistency,
coordination, and correctness of operations across multiple processes. Events in distributed systems
refer to occurrences such as sending or receiving messages, updating variables, or invoking remote
procedures. Each process has its own state, which consists of its memory, program counter, and local
variables. Ensuring synchronization between events and process states helps maintain the overall
integrity of the system.

2. Synchronizing Events

a. Challenges of Synchronizing Events

Lack of a Global Clock: In distributed systems, there’s no single, universal clock to timestamp
events, making it difficult to determine the exact sequence of events across different processes.
Concurrency: Multiple processes can execute independently and concurrently, leading to
difficulties in event ordering.
Communication Delays: Messages between processes can be delayed, lost, or received out of
order, adding complexity to event synchronization.

b. Event Synchronization Techniques

i. Logical Clocks

Logical clocks, such as Lamport timestamps and vector clocks, are used to impose an ordering on
events in a distributed system.

Lamport Timestamps: Ensure that if event A happens before event B, the timestamp of A is less
than that of B, but it doesn’t provide information about concurrent events.
Vector Clocks: Allow the detection of causality and concurrency by maintaining a vector of
timestamps for each process, offering a more fine-grained ordering of events.
71 / 123
 Distributed Systems and Cloud Computing
ii. Causal Ordering

Causal ordering ensures that events are processed in the correct order, respecting the causal
relationships between them. If event A causally affects event B, then B must not be executed before A.

Key Concepts:

Happens-before Relation (→): If event A happens before event B in a distributed system, A

must be processed before B.
Concurrency: If two events do not affect each other and happen independently, they are
considered concurrent.

iii. Event Synchronization Protocols

Some protocols ensure event synchronization by enforcing a strict or relaxed ordering of events:

FIFO Ordering: Ensures that messages between two processes are received in the same order
they were sent.
Causal Ordering: Ensures that if one message causally affects another, the affected message is
processed only after the causally related message.
Total Ordering: All events or messages are processed in the same order across all processes.

3. Synchronizing Process States

Process Synchronization is the coordination of execution of multiple processes in a multi-process
system to ensure that they access shared resources in a controlled and predictable manner. It aims to
resolve the problem of race conditions and other synchronization issues in a concurrent system.
The main objective of process synchronization is to ensure that multiple processes access shared
resources without interfering with each other and to prevent the possibility of inconsistent data due to
concurrent access. To achieve this, various synchronization techniques such as semaphores,
monitors, and critical sections are used.
In a multi-process system, synchronization is necessary to ensure data consistency and integrity, and
to avoid the risk of deadlocks and other synchronization problems. Process synchronization is an
important aspect of modern operating systems, and it plays a crucial role in ensuring the correct and
efficient functioning of multi-process systems.

https://www.geeksforgeeks.org/introduction-of-process-synchronization/
<- more in detail here(better)

What is Process?
A process is a program that is currently running or a program under execution is called a process. It
includes the program’s code and all the activity it needs to perform its tasks, such as using the CPU,
72 / 123
 Distributed Systems and Cloud Computing
memory, and other resources. Think of a process as a task that the computer is working on, like
opening a web browser or playing a video.

Types of Process
On the basis of synchronization, processes are categorized as one of the following two types:

Independent Process: The execution of one process does not affect the execution of other
processes.
Cooperative Process: A process that can affect or be affected by other processes executing in
the system.

Process synchronization problem arises in the case of Cooperative processes also because resources
are shared in Cooperative processes.

a. Challenges in Synchronizing Process States

Distributed Nature: Each process in a distributed system has its own state, which may not be
immediately visible to other processes.
Inconsistency: Processes may operate on inconsistent views of the system’s state due to
message delays or unsynchronized clocks.
Concurrency: Concurrent changes to process states across multiple nodes can lead to race
conditions and inconsistencies.

b. Process State Synchronization Techniques

i. Global State Consistency

A global state is the collective state of all processes and the messages in transit in a distributed
system. Synchronizing process states involves capturing consistent snapshots of this global state.

Snapshot Algorithms:

Chandy-Lamport Snapshot Algorithm: This algorithm captures a consistent global state

without halting the system’s operation. It works as follows:
1. A process (the initiator) takes its local snapshot and sends a "marker" message to all its
neighbors.
2. Upon receiving the marker, the neighbors take their own local snapshots and propagate the
marker to their neighbors.
3. Each process records the state of its incoming channels (messages in transit) after
receiving the marker.

Benefits:
73 / 123
 Distributed Systems and Cloud Computing
Ensures a consistent global view without stopping the system.
Helps in debugging, checkpointing, and fault recovery.

ii. Process Checkpointing

Checkpointing involves saving the state of a process at regular intervals. In the event of a failure, the
process can be restored to the last checkpoint, ensuring that the system can recover and resume
operation.

Types of Checkpointing:

Coordinated Checkpointing: All processes synchronize to take a consistent snapshot of the

global state at the same time.
Uncoordinated Checkpointing: Each process takes checkpoints independently, which may lead
to inconsistencies but has lower overhead.

iii. Consensus Algorithms

In distributed systems, achieving consensus on the state of processes or the result of a computation is
critical for ensuring consistent decision-making across nodes. Synchronizing states often involves
reaching consensus on shared values or the outcome of an operation.

Popular Consensus Algorithms:

Paxos: Ensures that all processes in a distributed system agree on a single value, even in the
presence of faults.
Raft: Similar to Paxos, but designed to be more understandable and implementable, achieving
consensus in a fault-tolerant manner.

4. Handling Failures in Synchronization

Failures can occur at any point in a distributed system, such as message loss, process crashes, or
network partitioning. Synchronizing events and process states becomes more challenging in the
presence of failures. Techniques like:

Timeouts: Processes use timeouts to detect when a message has been lost or delayed
excessively, and they can resend the message.
Message Acknowledgment: Ensures that messages are delivered and processed successfully.
Failure Detectors: These are mechanisms that detect when processes or communication links
have failed and trigger recovery actions.

5. Advantages of Synchronizing Events and Process States

74 / 123
 Distributed Systems and Cloud Computing
Consistency: Ensures that the system remains consistent, even in the face of concurrent
operations or failures.
Correctness: Provides guarantees that events are ordered and executed correctly across
processes.
Fault Tolerance: Enables systems to detect failures and recover from them, ensuring reliable
operation.

6. Challenges of Synchronizing Events and Process States

Overhead: Synchronizing events and states adds communication overhead, especially in large
systems.
Latency: Network delays and message losses can complicate synchronization and slow down
the system.
Scalability: As the system grows, it becomes harder to maintain synchronized states and events
across many nodes.

7. Conclusion
Synchronizing events and process states is a fundamental challenge in distributed systems. Logical
clocks, causal ordering, snapshot algorithms, and consensus protocols are key tools that ensure the
correct ordering of events and consistent process states across distributed nodes. Effective
synchronization ensures that distributed systems function reliably, maintaining correctness and
consistency even in the face of network delays and process failures.

Synchronizing Physical Clocks

1. Introduction to Synchronizing Physical Clocks
In distributed systems, ensuring a consistent notion of time across all nodes is a challenging task.
Physical clocks, maintained by the nodes in a distributed system, are prone to drift and may not be
synchronized perfectly. However, in some scenarios, synchronizing physical clocks is necessary, such
as in time-sensitive applications like distributed databases, financial systems, or sensor networks.
Clock synchronization involves aligning the clocks of computers or nodes, enabling efficient data
transfer, smooth communication, and coordinated task execution.

2. The Need for Clock Synchronization

In distributed systems, physical clock synchronization is necessary for:

Consistency and Coherence:

75 / 123
 Distributed Systems and Cloud Computing
Clock synchronization ensures that timestamps and time-based decisions made across
different nodes in the distributed system are consistent and coherent. This is crucial for
maintaining the correctness of distributed algorithms and protocols.
Event Ordering:
Many distributed systems rely on the notion of event ordering based on timestamps to
ensure causality and maintain logical consistency. Clock synchronization helps in correctly
ordering events across distributed nodes.
Data Integrity and Conflict Resolution:
In distributed databases and file systems, synchronized clocks help in timestamping data
operations accurately. This aids in conflict resolution and maintaining data integrity,
especially in scenarios involving concurrent writes or updates.
Fault Detection and Recovery:
Synchronized clocks facilitate efficient fault detection and recovery mechanisms in
distributed systems. Timestamps can help identify the sequence of events leading to a fault,
aiding in debugging and recovery processes.
Security and Authentication:
Timestamps generated by synchronized clocks are crucial for security protocols, such as in
cryptographic operations and digital signatures. They provide a reliable basis for verifying
the authenticity and temporal validity of transactions and messages.

3. Challenges in Clock Synchronization

Clock synchronization in distributed systems introduces complexities compared to centralized ones
due to the use of distributed algorithms. Some notable challenges include:

Information Dispersion: Distributed systems store information on machines. Gathering and

harmonizing this information to achieve synchronization presents a challenge.
Local Decision Realm: Distributed systems rely on localized data, for making decisions. As a
result, when it comes to synchronization we have to make decisions with information, from each
node, which makes the process more complex.
Mitigating Failures: In a distributed environment it becomes crucial to prevent failures in one
node from disrupting synchronization.
Temporal Uncertainty: The existence of clocks in distributed systems creates the potential, for
time variations.

4. Clock Synchronization Techniques

Clock synchronization techniques aim to address the challenge of ensuring that clocks across
distributed nodes in a system are aligned or synchronized. Here are some commonly used
techniques:

76 / 123
 Distributed Systems and Cloud Computing
1. Network Time Protocol (NTP)

Overview: NTP is one of the oldest and most widely used protocols for synchronizing clocks
over a network. It is designed to synchronize time across systems with high accuracy.
Operation:
Client-Server Architecture: NTP operates in a hierarchical client-server mode. Clients
(synchronized systems) periodically query time servers for the current time.
Stratum Levels: Time servers are organized into strata, where lower stratum levels indicate
higher accuracy and reliability (e.g., stratum 1 servers are directly connected to a reference
clock).
Timestamp Comparison: NTP compares timestamps from multiple time servers,
calculates the offset (difference in time), and adjusts the local clock gradually to minimize
error.
Applications: NTP is widely used in systems where moderate time accuracy is sufficient, such
as network infrastructure, servers, and general-purpose computing.

2. Precision Time Protocol (PTP)

Overview: PTP is a more advanced protocol compared to NTP, designed for high-precision clock
synchronization in environments where very accurate timekeeping is required.
Operation:
Master-Slave Architecture: PTP operates in a master-slave architecture, where one node
(master) distributes its highly accurate time to other nodes (slaves).
Hardware Timestamping: PTP uses hardware timestamping capabilities (e.g., IEEE 1588)
to reduce network-induced delays and improve synchronization accuracy.
Sync and Delay Messages: PTP exchanges synchronization (Sync) and delay
measurement (Delay Request/Response) messages to calculate the propagation delay and
adjust clocks accordingly.
Applications: PTP is commonly used in industries requiring precise time synchronization, such
as telecommunications, industrial automation, financial trading, and scientific research.

3. Berkeley Algorithm

Overview: The Berkeley Algorithm is a decentralized algorithm that aims to synchronize the
clocks of distributed systems without requiring a centralized time server.
Operation:
Coordinator Election: A coordinator node periodically gathers time values from other
nodes in the system.
Clock Adjustment: The coordinator calculates the average time and broadcasts the
adjustment to all nodes, which then adjust their local clocks based on the received time
difference.

77 / 123
 Distributed Systems and Cloud Computing
Handling Clock Drift: The algorithm accounts for clock drift by periodically recalculating
and adjusting the time offset.
Applications: The Berkeley Algorithm is suitable for environments where a centralized time
server is impractical or unavailable, such as peer-to-peer networks or systems with decentralized
control.

Real-World Examples of Clock Synchronization in Distributed Systems

Below are some real-world examples of clock synchronization:

Network Time Protocol (NTP):

NTP is a widely used protocol for clock synchronization over the Internet. It ensures that
computers on a network have accurate time information, essential for tasks such as logging
events, scheduling tasks, and coordinating distributed applications.
Financial Trading Systems:
In trading systems, timestamp accuracy is critical for ensuring fair order execution and
compliance with regulatory requirements. Synchronized clocks enable precise recording
and sequencing of trade orders and transactions.
Distributed Databases:
Distributed databases rely on synchronized clocks to maintain consistency and coherence
across replicas and nodes. Timestamps help in conflict resolution and ensuring that data
operations are applied in the correct order.
Cloud Computing:
Cloud environments often span multiple data centers and regions. Synchronized clocks are
essential for tasks such as resource allocation, load balancing, and ensuring the
consistency of distributed storage systems.
Industrial Control Systems:
In industries such as manufacturing and automation, precise time synchronization (often
using protocols like Precision Time Protocol, PTP) is critical for coordinating processes,
synchronizing sensors and actuators, and ensuring timely and accurate data logging.

Challenges of Clock Synchronization in Distributed Systems

Clock synchronization in distributed systems introduces complexities compared to centralized ones
due to the use of distributed algorithms. Some notable challenges include:

Information Dispersion: Distributed systems store information on machines. Gathering and

harmonizing this information to achieve synchronization presents a challenge.
Local Decision Realm: Distributed systems rely on localized data, for making decisions. As a
result, when it comes to synchronization we have to make decisions with information, from each
node, which makes the process more complex.

78 / 123
 Distributed Systems and Cloud Computing
Mitigating Failures: In a distributed environment it becomes crucial to prevent failures in one
node from disrupting synchronization.
Temporal Uncertainty: The existence of clocks in distributed systems creates the potential, for
time variations.

Logical Time and Logical Clocks

1. Introduction to Logical Time
In distributed systems, logical time provides a way to order events without relying on physical clocks.
Since physical clocks are not perfectly synchronized across distributed systems, logical time offers a
consistent method to represent the ordering of events based on their causal relationships rather than
real-world time.

Logical time is used to understand the sequence and causality between events in a system where
there is no global clock. It is especially crucial in distributed environments to ensure consistency and
correctness of operations.

2. Logical Clocks
Logical clocks are algorithms that assign timestamps to events, allowing processes in a distributed
system to determine the order of events, even when those processes are not aware of each other’s
physical time.

Logical clocks ensure:

Causal Order: Events are ordered according to their causal relationships.

Concurrent Events Detection: The system can detect when events happen concurrently (i.e.,
independently).

3. Types of Logical Clocks

a. Lamport Clocks

Lamport Clocks (introduced by Leslie Lamport) are one of the simplest logical clock systems. They
assign a single integer value as a timestamp to each event, ensuring that if event A causally precedes
event B, then the timestamp of A is less than that of B.

Rules for Lamport Clocks:

1. Increment Rule: Each process maintains a counter (logical clock). When a process executes an
event, it increments its clock by 1.

79 / 123
 Distributed Systems and Cloud Computing
2. Message Passing Rule: When a process sends a message, it includes its current clock value
(timestamp) in the message. The receiving process sets its clock to the maximum of its own
clock and the received timestamp, and then increments its clock by 1.

Properties:

Ensures partial ordering of events. If ( A \rightarrow B ) (A happens before B), then the timestamp
of A is less than B.
Cannot detect concurrency directly, i.e., it doesn’t indicate whether two events are independent.

Example:

Process P1 executes an event and sends a message with a timestamp of 3.

Process P2 receives the message with a local clock value of 2. It updates its clock to max(2, 3) +
1 = 4 and processes the event.

b. Vector Clocks

Vector Clocks are an extension of Lamport clocks that provide more precise information about the
causal relationship between events. Instead of a single integer, each process maintains a vector of
clocks, one entry for each process in the system.

Rules for Vector Clocks:

1. Increment Rule: When a process executes an event, it increments its own entry in the vector.
2. Message Passing Rule: When a process sends a message, it attaches its vector clock. The
receiving process updates each element of its vector clock to the maximum of its own and the
sender’s corresponding clock value, then increments its own entry.

Example:

Process P1 sends a message with vector [3, 0, 0].

Process P2 receives the message with vector [0, 2, 0]. P2 updates its vector to [3, 3, 0], showing
that it now knows about events from P1 and itself.

Properties:

Allows detection of concurrent events. Two events are concurrent if their vector clocks are
incomparable (i.e., neither is strictly greater than the other).
Provides a total ordering of events if one vector clock is strictly greater than another.

4. Comparison of Lamport and Vector Clocks

80 / 123
 Distributed Systems and Cloud Computing
Feature Lamport Clocks Vector Clocks
Ordering Partial ordering Total ordering
Concurrency Detection Cannot detect Can detect concurrency
concurrency
Space Complexity Single integer per Vector with one entry per process
process
Communication Low (single integer) High (entire vector of clocks)
Overhead
Use Cases Simple causal ordering Detailed causal and concurrent event
detection

5. Logical Time vs Physical Time

Physical Time is the actual clock time (e.g., UTC time) that a system maintains. In distributed
systems, physical clocks may not be synchronized accurately due to factors like clock drift or
network delays.
Logical Time abstracts away from real-time concerns and focuses solely on the order and
causality of events. It ensures that the system’s behavior is consistent and predictable without
needing synchronized physical clocks.

Aspect Physical Time Logical Time

Accuracy Relies on accurate clock No need for synchronization
synchronization
Use Suitable for time-sensitive applications Used for event ordering and causality
Complexity High (requires synchronization) Low to medium (based on logical clock
type)
Challenges Clock drift, network delays Requires message passing overhead

6. Use Cases of Logical Time and Logical Clocks

Causal Messaging: In distributed databases or messaging systems, logical clocks are used to
ensure that causally related messages are delivered in the correct order.
Snapshot Algorithms: Logical clocks help ensure that snapshots of a distributed system’s state
are consistent.
Distributed Debugging: When debugging a distributed system, logical clocks help trace the
causal relationship between events.
Concurrency Control: In distributed databases or file systems, vector clocks help detect when
two events or transactions occurred concurrently, potentially leading to conflicts.

81 / 123
 Distributed Systems and Cloud Computing
7. Advanced Logical Time Concepts

a. Hybrid Logical Clocks (HLC)

Hybrid Logical Clocks combine the concepts of physical time and logical clocks to provide both an
accurate notion of time and the ability to capture causal relationships. HLC includes a physical
timestamp and a logical counter, providing a compromise between the advantages of physical and
logical clocks.

b. Causal Consistency

Causal consistency is a consistency model used in distributed systems where operations that are
causally related must be seen in the correct order by all processes. Logical clocks (especially vector
clocks) are essential for implementing causal consistency in distributed databases.

8. Challenges and Limitations of Logical Clocks

Increased Communication Overhead: Vector clocks require exchanging a vector of

timestamps, which can grow large as the number of processes increases.
Partial Ordering: Lamport clocks only provide a partial ordering, which might not be sufficient for
certain distributed applications.
Concurrency Detection: Detecting concurrency is not always trivial and requires more
sophisticated algorithms (like vector clocks).

Logical time and logical clocks play a critical role in maintaining consistency, correctness, and order in
distributed systems. By abstracting away from physical time, logical clocks like Lamport and vector
clocks ensure that processes can accurately determine the sequence and causality of events. Logical
clocks are foundational for distributed algorithms, causal messaging, concurrency control, and
maintaining causal consistency.

https://www.brainkart.com/article/Logical-time-and-logical-
clocks_8552/

Global States
1. Introduction to Global States
In distributed systems, the global state represents the collective state of all the processes and
messages in transit at a particular point in time. It’s a crucial concept for tasks like fault tolerance,
checkpointing, debugging, and recovery. However, capturing a consistent global state is challenging
because the state of each process is only locally available and processes can only communicate
asynchronously via message passing.

82 / 123
 Distributed Systems and Cloud Computing
Think of it like a giant puzzle where each computer holds a piece. The global state is like a snapshot
of the whole puzzle at one time. Understanding this helps us keep track of what’s happening in the
digital world, like when you’re playing games online or chatting with friends.

What is the Global State of a Distributed System?

The Global State of a Distributed System refers to the collective status or condition of all the
components within a distributed system at a specific point in time. In a distributed system, which
consists of multiple independent computers or nodes working together to achieve a common goal,
each component may have its own state or information.

The global state represents the combined knowledge of all these individual states at a given
moment.
Understanding the global state is crucial for ensuring the consistency, reliability, and correctness
of operations within the distributed system, as it allows for effective coordination and
synchronization among its components.

2. Importance of Global States in Distributed Systems

The importance of the Global State in a Distributed System lies in its ability to provide a
comprehensive view of the system’s status at any given moment. Here’s why it’s crucial:

Consistency: Global State helps ensure that all nodes in the distributed system have consistent
data. By knowing the global state, the system can detect and resolve any inconsistencies among
the individual states of its components.
Fault Detection and Recovery: Monitoring the global state allows for the detection of faults or
failures within the system. When discrepancies arise between the expected and actual global
states, it triggers alarms, facilitating prompt recovery strategies.
Concurrency Control: In systems where multiple processes or nodes operate simultaneously,
global state tracking aids in managing concurrency. It enables the system to coordinate
operations and maintain data integrity even in scenarios of concurrent access.
Debugging and Analysis: Understanding the global state is instrumental in diagnosing issues,
debugging problems, and analyzing system behavior. It provides insights into the sequence of
events and the interactions between different components.
Performance Optimization: By analyzing the global state, system designers can identify
bottlenecks, optimize resource utilization, and enhance overall system performance.
Distributed Algorithms: Many distributed algorithms rely on global state information to make
decisions and coordinate actions among nodes. Having an accurate global state is fundamental
for the proper functioning of these algorithms.

3. Challenges in Capturing Global State

83 / 123
 Distributed Systems and Cloud Computing
Determining the Global State in Distributed Systems presents several challenges due to the complex
nature of distributed environments:

Partial Observability: Nodes in a distributed system have limited visibility into the states and
activities of other nodes, making it challenging to obtain a comprehensive view of the global
state.
Concurrency: Concurrent execution of processes across distributed nodes can lead to
inconsistencies in state information, requiring careful coordination to capture a consistent global
state.
Faults and Failures: Node failures, network partitions, and message losses are common in
distributed systems, disrupting the collection and aggregation of state information and
compromising the accuracy of the global state.
Scalability: As distributed systems scale up, the overhead associated with collecting and
processing state information increases, posing scalability challenges in determining the global
state efficiently.
Consistency Guarantees: Different applications have diverse consistency requirements,
ranging from eventual consistency to strong consistency, making it challenging to design global
state determination mechanisms that satisfy these varying needs.
Heterogeneity: Distributed systems often consist of heterogeneous nodes with different
hardware, software, and communication protocols, complicating the interoperability and
consistency of state information across diverse environments.

4. Global State Components

The components of the Global State in Distributed Systems typically include:

1. Local States: These are the states of individual nodes or components within the distributed
system. Each node maintains its local state, which includes variables, data structures, and any
relevant information specific to that node’s operation.
2. Messages: Communication between nodes in a distributed system occurs through messages.
The Global State includes information about the messages exchanged between nodes, such as
their content, sender, receiver, timestamp, and delivery status.
3. Timestamps: Timestamps are used to order events in distributed systems and establish
causality relationships. Including timestamps in the Global State helps ensure the correct
sequencing of events across different nodes.
4. Event Logs: Event logs record significant actions or events that occur within the distributed
system, such as the initiation of a process, the receipt of a message, or the completion of a task.
These logs provide a historical record of system activities and contribute to the Global State.
5. Resource States: Distributed systems often involve shared resources, such as files, databases,
or hardware components. The Global State includes information about the states of these
resources, such as their availability, usage, and any locks or reservations placed on them.

84 / 123
 Distributed Systems and Cloud Computing
6. Control Information: Control information encompasses metadata and control signals used for
managing system operations, such as synchronization, error handling, and fault tolerance
mechanisms. Including control information in the Global State enables effective coordination and
control of distributed system behavior.
7. Configuration Parameters: Configuration parameters define the settings and parameters that
govern the behavior and operation of the distributed system. These parameters may include
network configurations, system settings, and algorithm parameters, all of which contribute to the
Global State.

5. Consistent Global State

A consistent global state is one where the captured local states and messages in transit do not
contradict each other. For example:

If process P1 sends a message to P2 before capturing its state, and P2 hasn’t received that
message when its state is captured, the message is considered "in transit" and part of the global
state.
If a state is inconsistent, it may appear as though a message was received without being sent,
which would be incorrect.

Consistency Condition:

The global state should satisfy the following conditions:

No messages are received without being sent.

Causal dependencies are respected: Events that depend on each other are recorded in the
correct order.

Consistency and Coordination in Global State of a Distributed System

Ensuring consistency and coordination of the Global State in Distributed Systems is crucial for
maintaining system reliability and correctness. Here’s how it’s achieved:

Consistency Models: Distributed systems often employ consistency models to specify the
degree of consistency required. These models, such as eventual consistency, strong
consistency, or causal consistency, define rules governing the order and visibility of updates
across distributed nodes.
Concurrency Control: Mechanisms for concurrency control, such as distributed locks,
transactions, and optimistic concurrency control, help manage concurrent access to shared
resources. By coordinating access and enforcing consistency protocols, these mechanisms
prevent conflicts and ensure data integrity.

85 / 123
 Distributed Systems and Cloud Computing
Synchronization Protocols: Synchronization protocols facilitate coordination among distributed
nodes to ensure coherent updates and maintain consistency. Techniques like two-phase commit,
three-phase commit, and consensus algorithms enable agreement on distributed decisions and
actions.
Global State Monitoring: Implementing monitoring systems and distributed tracing tools allows
continuous monitoring of the Global State. By tracking system operations, message flows, and
resource usage across distributed nodes, discrepancies and inconsistencies can be detected
and resolved promptly.
Distributed Transactions: Distributed transactions provide a mechanism for executing a series
of operations across multiple nodes in a coordinated and atomic manner. Techniques like
distributed commit protocols and distributed transaction managers ensure that all operations
either succeed or fail together, preserving consistency.

6. Techniques for Capturing Global States

Several techniques are employed to determine the Global State in Distributed Systems. Here are
some prominent ones:

Centralized Monitoring:
In this approach, a central monitoring entity collects state information from all nodes in the
distributed system periodically.
It aggregates this data to determine the global state. While simple to implement, this method
can introduce a single point of failure and scalability issues.
Distributed Snapshots:
Distributed Snapshot algorithms allow nodes to collectively capture a consistent snapshot of
the entire system’s state.
This involves coordinating the recording of local states and message exchanges among
nodes.
Techniques like the Chandy-Lamport and Dijkstra-Scholten algorithms are commonly used
for distributed snapshot collection.
Vector Clocks:
Vector clocks are logical timestamping mechanisms used to order events in distributed
systems. Each node maintains a vector clock representing its local causality relationships
with other nodes.
By exchanging and merging vector clocks, nodes can construct a global ordering of events,
facilitating the determination of the global state.
Checkpointing and Rollback Recovery:
Checkpointing involves periodically saving the state of processes or system components to
stable storage.
By coordinating checkpointing across nodes and employing rollback recovery mechanisms,
the system can recover to a consistent global state following failures or faults.

86 / 123
 Distributed Systems and Cloud Computing
Consensus Algorithms:
Consensus algorithms like Paxos and Raft facilitate agreement among distributed nodes on
a single value or state.
By reaching a consensus on the global state, nodes can synchronize their views and ensure
consistency across the distributed system.

7. Use Cases of Global States

The concept of Global State in Distributed Systems finds numerous applications across various
domains, including:

Distributed Computing:
Global State is fundamental in distributed computing for coordinating parallel processes,
ensuring data consistency, and synchronizing distributed algorithms.
Applications include parallel processing, distributed data processing frameworks (e.g.,
MapReduce), and distributed scientific simulations.
Distributed Databases:
In distributed databases, maintaining a consistent global state is essential for ensuring data
integrity and transaction management across distributed nodes.
Global state information helps coordinate distributed transactions, enforce consistency
constraints, and facilitate data replication and recovery.
Distributed Systems Monitoring and Management:
Global state information is utilized in monitoring and managing distributed systems to track
system health, diagnose performance issues, and detect faults or failures.
Monitoring tools analyze the global state to provide insights into system behavior and
identify optimization opportunities.
Distributed Messaging and Event Processing:
Global state information is leveraged in distributed messaging and event processing
systems to ensure reliable message delivery, event ordering, and event-driven processing
across distributed nodes.
Applications include distributed event sourcing, event-driven architectures, and distributed
publish-subscribe systems.
Distributed System Design and Testing:
Global state information is valuable in designing and testing distributed systems to simulate
and analyze system behavior under different conditions, validate system correctness, and
identify potential scalability or performance bottlenecks.

8. Global State in Distributed Databases

In distributed databases, ensuring transactional consistency across multiple nodes is a critical
problem. Techniques such as Two-Phase Commit (2PC) and Three-Phase Commit (3PC) are used
87 / 123
 Distributed Systems and Cloud Computing
to ensure that a distributed transaction either commits or aborts consistently across all participants,
forming a globally consistent state for the database.

Two-Phase Commit (2PC): A coordinator ensures that all participants either commit or abort a
transaction. The first phase (prepare phase) involves asking all participants if they are ready to
commit, and the second phase (commit phase) ensures that either all participants commit or all
abort.
Three-Phase Commit (3PC): Extends 2PC by adding a third phase to make the system more
resilient to failures.

9. Global State and Distributed System Models

The concept of global state is tightly linked to different models of distributed systems:

Synchronous Systems: In synchronous systems (where message delays and process

execution times are bounded), capturing the global state is relatively easier, as there is a
predictable upper bound on message transmission.
Asynchronous Systems: In asynchronous systems (where there are no such bounds),
capturing a consistent global state becomes more challenging. Snapshot algorithms like Chandy-
Lamport are particularly useful in such scenarios.

10. Conclusion
Understanding the Global State of a Distributed System is vital for keeping everything running
smoothly in our interconnected digital world. From coordinating tasks in large-scale data processing
frameworks like MapReduce to ensuring consistent user experiences in multiplayer online games, the
Global State plays a crucial role. By capturing the collective status of all system components at any
given moment, it helps maintain data integrity, coordinate actions, and detect faults.

Global state is essential for maintaining consistency, ensuring that distributed systems remain fault-
tolerant, and providing the foundation for advanced tasks like distributed debugging, checkpointing,
and consensus.

Distributed Debugging
1. Introduction to Distributed Debugging
Distributed debugging refers to the process of detecting, diagnosing, and fixing bugs or issues in
distributed systems. Since distributed systems consist of multiple processes running on different
machines that communicate via messages, debugging such systems is significantly more complex
than debugging centralized systems.

88 / 123
 Distributed Systems and Cloud Computing
It involves tracking the flow of operations across multiple nodes, which requires tools and
techniques like logging, tracing, and monitoring to capture and analyze system behavior.
Issues such as synchronization errors, concurrency bugs, and network failures are common
challenges in distributed systems. Debugging aims to ensure that all parts of the system work
correctly and efficiently together, maintaining overall system reliability and performance.

Common Sources of Errors and Failures in Distributed Systems

When debugging distributed systems, it’s crucial to understand the common sources of errors and
failures that can complicate the process. Here are some key sources:

Network Issues: Problems such as latency, packet loss, jitter, and disconnections can disrupt
communication between nodes, causing data inconsistency and system downtime.
Concurrency Problems: Simultaneous operations on shared resources can lead to race
conditions, deadlocks, and livelocks, which are difficult to detect and resolve.
Data Consistency Errors: Ensuring data consistency across multiple nodes can be challenging,
leading to replication errors, stale data, and partition tolerance issues.
Faulty Hardware: Failures in physical components like servers, storage devices, and network
infrastructure can introduce errors that are difficult to trace back to their source.
Software Bugs: Logical errors, memory leaks, improper error handling, and bugs in the code
can cause unpredictable behavior and system crashes.
Configuration Mistakes: Misconfigured settings across different nodes can lead to
inconsistencies, miscommunications, and failures in the system’s operation.
Security Vulnerabilities: Unauthorized access and attacks, such as Distributed Denial of
Service (DDoS), can disrupt services and compromise system integrity.
Resource Contention: Competing demands for CPU, memory, or storage resources can cause
nodes to become unresponsive or degrade in performance.
Time Synchronization Issues: Discrepancies in system clocks across nodes can lead to
coordination problems, causing errors in data processing and transaction handling.

2. Challenges in Distributed Debugging

a. Concurrency Issues

Concurrency issues, such as race conditions and deadlocks, are harder to identify because they
may occur only in specific execution sequences or under certain timing conditions. Events happening
in parallel make it difficult to track and reproduce bugs.

b. Nondeterminism

89 / 123
 Distributed Systems and Cloud Computing
Due to the nondeterministic behavior of message passing and process scheduling, the same
sequence of inputs may produce different outputs during different runs. This makes debugging
distributed systems less predictable.

c. Incomplete Observability

In distributed systems, each process has only a partial view of the overall system's state. As a result,
observing and logging every event in the system is challenging.

d. Delayed Failures

Sometimes, failures in distributed systems occur after a considerable delay due to message losses or
late arrivals. These delayed failures make it hard to identify the root cause and trace it back to the
point of failure.

3. Techniques for Distributed Debugging

a. Logging and Monitoring, Tracing and Distributed Tracing

Logging and monitoring are essential techniques for debugging distributed systems, offering vital
insights into system behavior and helping to identify and resolve issues effectively.
Tracing and distributed tracing are critical techniques for debugging distributed systems, providing
visibility into the flow of requests and operations across multiple components.

What is Logging?

Logging involves capturing detailed records of events, actions, and state changes within the system.
Key aspects include:

Centralized Logging: Collect logs from all nodes in a centralized location to facilitate easier
analysis and correlation of events across the system.
Log Levels: Use different log levels (e.g., DEBUG, INFO, WARN, ERROR) to control the
verbosity of log messages, allowing for fine-grained control over the information captured.
Structured Logging: Use structured formats (e.g., JSON) for log messages to enable better
parsing and searching.
Contextual Information: Include contextual details like timestamps, request IDs, and node
identifiers to provide a clear picture of where and when events occurred.
Error and Exception Logging: Capture stack traces and error messages to understand the root
causes of failures.
Log Rotation and Retention: Implement log rotation and retention policies to manage log file
sizes and storage requirements.

90 / 123
 Distributed Systems and Cloud Computing
What is Monitoring?

Monitoring involves continuously observing the system’s performance and health to detect anomalies
and potential issues. Key aspects include:

Metrics Collection: Collect various performance metrics (e.g., CPU usage, memory usage, disk
I/O, network latency) from all nodes.
Health Checks: Implement regular health checks for all components to ensure they are
functioning correctly.
Alerting: Set up alerts for critical metrics and events to notify administrators of potential issues in
real-time.
Visualization: Use dashboards to visualize metrics and logs, making it easier to spot trends,
patterns, and anomalies.
Tracing: Implement distributed tracing to follow the flow of requests across different services and
nodes, helping to pinpoint where delays or errors occur.
Anomaly Detection: Use machine learning and statistical techniques to automatically detect
unusual patterns or behaviors that may indicate underlying issues.

What is Tracing?

Tracing involves following the execution path of a request or transaction through various parts of a
system to understand how it is processed. This helps in identifying performance bottlenecks, errors,
and points of failure. Key aspects include:

Span Creation: Breaking down the request into smaller units called spans, each representing a
single operation or step in the process.
Span Context: Recording metadata such as start time, end time, and status for each span to
provide detailed insights.
Correlation IDs: Using unique identifiers to correlate spans that belong to the same request or
transaction, allowing for end-to-end tracking.

What is Distributed Tracing?

Distributed Tracing extends traditional tracing to distributed systems, where requests may traverse
multiple services, databases, and other components spread across different locations. Key aspects
include:

Trace Propagation: Passing trace context (e.g., trace ID and span ID) along with requests to
maintain continuity as they move through the system.
End-to-End Visibility: Capturing traces across all services and components to get a
comprehensive view of the entire request lifecycle.
Latency Analysis: Measuring the time spent in each service or component to identify where
delays or performance issues occur.
91 / 123
 Distributed Systems and Cloud Computing
Error Diagnosis: Pinpointing where errors happen and understanding their impact on the overall
request.

b. Event Ordering and Logical Clocks

Logical clocks, such as Lamport clocks and vector clocks, are crucial for debugging distributed
systems because they help order events based on causal relationships.

By assigning timestamps to events, logical clocks allow developers to understand the causal
sequence of events, helping to detect race conditions, deadlocks, or inconsistencies that result from
improper event ordering.

For example:

Causal Ordering of Messages: Ensuring that messages are delivered in the same order in
which they were sent is critical for debugging message-passing errors.
Detecting Concurrency: Vector clocks can be used to detect whether two events happened
concurrently, helping to uncover race conditions.

c. Global State Inspection and Snapshot Algorithms

Global state inspection allows developers to observe the state of multiple processes and
communication channels to detect issues. As discussed earlier, the Chandy-Lamport snapshot
algorithm is an effective way to capture a consistent global state. This captured global state can then
be analyzed to identify anomalies such as deadlocks or inconsistent states.

Snapshot algorithms are especially useful in:

Deadlock Detection: By capturing the global state, the system can identify circular
dependencies between processes waiting for each other, indicating a deadlock.
System Recovery: Snapshots can be used to restore a system to a consistent state during
debugging sessions.

d. Message Passing Assertions

Message passing assertions are rules or conditions that can be applied to the messages exchanged
between processes. These assertions specify the expected sequence or content of messages and
can be checked during execution to detect violations.

For example, an assertion might specify that process P1 must always receive a response from
process P2 within a certain time frame. If the assertion fails, it indicates a potential bug, such as
message loss or an unresponsive process.

92 / 123
 Distributed Systems and Cloud Computing
e. Consistent Cuts

A consistent cut is a snapshot of a system where the recorded events form a consistent view of the
global state. It is used to analyze the system's behavior across different processes. By dividing the
system's execution into a series of consistent cuts, developers can step through the execution and
examine the system's state at different points in time.

4. Common Bugs in Distributed Systems

a. Race Conditions

A race condition occurs when two or more processes attempt to access shared resources or perform
operations simultaneously, leading to unexpected or incorrect results. In distributed systems, race
conditions are difficult to detect due to the lack of global synchronization.

Debugging Race Conditions:

Use vector clocks to detect concurrent events that should not happen simultaneously.
Employ distributed tracing tools to visualize the order of events and interactions between
processes.

b. Deadlocks

A deadlock occurs when two or more processes are stuck waiting for resources held by each other,
forming a circular dependency. This can lead to the system halting.

Debugging Deadlocks:

Use global state inspection or snapshot algorithms to detect circular waiting conditions.
Analyze logs to identify resources that are being locked and not released.

c. Message Loss and Delays

In distributed systems, messages can be delayed, lost, or arrive out of order. This can lead to incorrect
states or inconsistent data across processes.

Debugging Message Loss:

Use logging and message passing assertions to verify that all sent messages are correctly
received.
Implement mechanisms like acknowledgements or timeouts to detect and handle lost
messages.

93 / 123
 Distributed Systems and Cloud Computing
d. Inconsistent Replication

In systems that use replication for fault tolerance or performance, ensuring that all replicas remain
consistent is a major challenge. Inconsistent replication can lead to scenarios where different
replicas of the same data have different values, causing errors.

Debugging Inconsistent Replication:

Use versioning and vector clocks to track the causal order of updates to replicated data.
Compare logs from all replicas to identify inconsistencies.

5. Debugging Tools and Techniques

a. Distributed Tracing Tools

Jaeger: An open-source distributed tracing system that helps developers monitor and
troubleshoot transactions in distributed systems.
Zipkin: A distributed tracing system that helps developers trace requests as they propagate
through services, allowing them to pinpoint where failures occur.

b. Debuggers for Distributed Systems

GDB for Distributed Systems: Some variations of traditional debuggers like GDB have been
adapted for distributed systems, allowing developers to set breakpoints and step through the
execution of processes running on different machines.
DTrace: A powerful debugging and tracing tool that can dynamically instrument code to collect
data for analysis.

c. Visualizing Causal Relationships

Tools like ShiViz help visualize the causal relationships between events in distributed systems. ShiViz
analyzes logs generated by distributed systems and creates a graphical representation of the causal
order of events, allowing developers to trace the flow of events and detect potential issues.

6. Advanced Techniques

a. Deterministic Replay

In deterministic replay, the system is designed to record all nondeterministic events (e.g., message
deliveries, process schedules) during execution. Later, the system can be replayed in a deterministic
manner to reproduce the same execution and allow for easier debugging.

94 / 123
 Distributed Systems and Cloud Computing
Challenges: It can be difficult and resource-intensive to capture all nondeterministic events.
Benefits: It ensures that even bugs that occur due to timing issues or specific event orders can
be reproduced consistently.

b. Model Checking

Model checking involves creating a formal model of the distributed system and systematically
exploring all possible execution paths to check for bugs. This approach helps detect race conditions,
deadlocks, and other concurrency-related bugs.

Tools: TLA+ (Temporal Logic of Actions) is a popular formal specification language and model-
checking tool used to verify the correctness of distributed systems.

c. Dynamic Instrumentation

In dynamic instrumentation, tools like DTrace or eBPF (Extended Berkeley Packet Filter)
dynamically instrument running code to collect runtime data, helping developers debug live systems
without stopping them.

7. Best Practices for Distributed Debugging

Debugging distributed systems is a complex task due to the multiple components and asynchronous
nature of these systems. Adopting best practices can help in identifying and resolving issues
efficiently. Here are some key best practices for debugging in distributed systems:

**Detailed Logs:** Ensure that each service logs detailed information about its operations,
including timestamps, request IDs, and thread IDs.
**Consistent Log Format:** Use a standardized log format across all services to make it easier
to correlate logs.
**Trace Requests:** Implement distributed tracing to follow the flow of requests across multiple
services and identify where issues occur.
**Tools:** Use tools like Jaeger, Zipkin, or OpenTelemetry to collect and visualize trace data.
**Real-Time Monitoring:** Monitor system metrics (e.g., CPU, memory, network usage),
application metrics (e.g., request rate, error rate), and business metrics (e.g., transaction rate).
**Dashboards:** Use monitoring tools like Prometheus and Grafana to create dashboards that
provide real-time insights into system health.
**Simulate Failures:** Use fault injection to simulate network partitions, latency, and node
failures.
**Chaos Engineering:** Regularly practice chaos engineering to identify weaknesses in the
system and improve resilience.
**Unit Tests:** Write comprehensive unit tests for individual components.

95 / 123
 Distributed Systems and Cloud Computing
**Integration Tests:** Implement integration tests that cover interactions between services.

8. Conclusion

Distributed debugging is a complex but critical task in ensuring the correctness and reliability of
distributed systems. Using techniques like logging, tracing, snapshot algorithms, and event ordering
(with logical clocks), developers can analyze and troubleshoot issues such as race conditions,
deadlocks, and message inconsistencies. Leveraging advanced tools and techniques like distributed
tracing, deterministic replay, and model checking can greatly enhance the debugging process,
ensuring that distributed systems function as expected even under challenging conditions.

https://www.geeksforgeeks.org/debugging-techniques-in-distributed-
systems/ <- More here

Coordination and Agreement : Distributed Mutual Exclusion

1. Introduction to Distributed Mutual Exclusion

In distributed systems, mutual exclusion ensures that only one process can access a shared
resource at a time, preventing conflicts and ensuring consistency. In centralized systems, mutual
exclusion is typically implemented using locks or semaphores, but in a distributed environment,
achieving mutual exclusion is more complex due to the lack of shared memory, global clocks, and
synchronized state.

Mutual exclusion is a concurrency control property which is introduced to prevent race conditions. It
is the requirement that a process can not enter its critical section while another concurrent process is
currently present or executing in its critical section i.e only one process is allowed to execute the
critical section at any given instance of time.

Mutual exclusion in single computer system Vs. distributed system: In single computer system,
memory and other resources are shared between different processes. The status of shared resources
and the status of users is easily available in the shared memory so with the help of shared variable
(For example: Semaphores) mutual exclusion problem can be easily solved. In Distributed systems,
we neither have shared memory nor a common physical clock and therefore we can not solve mutual
exclusion problem using shared variables. To eliminate the mutual exclusion problem in distributed
system approach based on message passing is used. A site in distributed system do not have
complete information of state of the system due to lack of shared memory and a common physical
clock.

Distributed mutual exclusion (DME) algorithms aim to coordinate processes across different nodes
to ensure exclusive access to a shared resource without central coordination or direct memory
access.

96 / 123
 Distributed Systems and Cloud Computing
2. Key Requirements for Distributed Mutual Exclusion
A robust distributed mutual exclusion algorithm must satisfy the following conditions:

No Deadlock: Two or more site should not endlessly wait for any message that will never arrive.
No Starvation: Every site who wants to execute critical section should get an opportunity to
execute it in finite time. Any site should not wait indefinitely to execute critical section while other
site are repeatedly executing critical section
Fairness: Each site should get a fair chance to execute critical section. Any request to execute
critical section must be executed in the order they are made i.e Critical section execution
requests should be executed in the order of their arrival in the system.
Fault Tolerance: In case of failure, it should be able to recognize it by itself in order to continue
functioning without any disruption.

Some points are need to be taken in consideration to understand mutual exclusion fully :

1. It is an issue/problem which frequently arises when concurrent access to shared resources by

several sites is involved. For example, directory management where updates and reads to a
directory must be done atomically to ensure correctness.
2. It is a fundamental issue in the design of distributed systems.
3. Mutual exclusion for a single computer is not applicable for the shared resources since it involves
resource distribution, transmission delays, and lack of global information.

3. Categories of Distributed Mutual Exclusion Algorithms

Distributed mutual exclusion algorithms can be classified into two categories:

1. Token-Based Algorithms: A unique token circulates among the processes. Possession of the
token grants the process the right to enter the critical section.
2. Permission-Based Algorithms: Processes must request permission from other processes to
enter the critical section. Only when permission is granted by all relevant processes can the
process enter the CS.

4. Token-Based Algorithms

a. Token Ring Algorithm

In the Token Ring Algorithm, processes are logically arranged in a ring. A unique token circulates in
the ring, and a process can only enter its critical section if it holds the token. Once a process finishes
its execution, it passes the token to the next process in the ring.

Mechanism:
1. The token is passed from one process to another in a circular fashion.

97 / 123
 Distributed Systems and Cloud Computing
2. A process enters the critical section when it holds the token.
3. After using the CS, it passes the token to the next process.
Properties:
Fairness: Every process gets the token in a round-robin manner.
No starvation: Every process is guaranteed to get the token eventually.
Simple implementation: No complex permission handling is needed.
Disadvantages:
Failure Handling: If a process holding the token crashes, the token is lost, and the system
must recover.
Inefficient in high loads: Token passing may involve high communication overhead,
especially if the critical section requests are sparse.

b. Raymond’s Tree-Based Algorithm

Raymond's tree-based algorithm organizes processes into a logical tree structure. The token is
passed between processes according to their tree relationship, making it more efficient in systems
with many processes.

Mechanism:
1. The processes form a spanning tree.
2. The token is held at a particular process node, which serves as the root.
3. Processes request the token by sending a message up the tree.
4. The token moves up or down the tree until it reaches the requesting process.
Properties:
Efficiency: Fewer messages are exchanged compared to a token ring, especially in
systems with large numbers of processes.
Scalability: The tree structure reduces the overhead compared to simpler token-based
systems.
Disadvantages:
Fault Tolerance: Like the token ring, the system needs a recovery mechanism if the
process holding the token crashes.

5. Permission-Based Algorithms

a. Lamport’s Mutual Exclusion Algorithm

Lamport's Algorithm is based on the concept of logical clocks and relies on permission from other
processes to enter the critical section. This is one of the earliest permission-based distributed mutual
exclusion algorithms.

Mechanism:

98 / 123
 Distributed Systems and Cloud Computing
1. When a process wants to enter the critical section, it sends a request to all other processes.
2. Each process replies to the request only after ensuring that it does not need to enter the
critical section itself (based on timestamps).
3. Once the process has received permission (replies) from all other processes, it can enter
the critical section.
4. After exiting the CS, the process informs other processes, allowing them to proceed.
Properties:
Mutual exclusion: No two processes enter the critical section at the same time.
Ordering: The system uses logical timestamps to ensure that requests are processed in
order.
Disadvantages:
Message Overhead: Every process needs to communicate with all other processes,
leading to (O(N^2)) messages in a system with (N) processes.
Fairness: Processes must wait for responses from all other processes, which may slow
down the system under high load or failure scenarios.

b. Ricart-Agrawala Algorithm

Ricart-Agrawala's Algorithm is an optimization of Lamport’s algorithm, reducing the number of

messages required for mutual exclusion. This algorithm also relies on permission but simplifies the
communication pattern.

Mechanism:
1. When a process wants to enter the critical section, it sends a request to all other processes.
2. Other processes respond either immediately or after they have finished their critical section.
3. The requesting process can enter the critical section once all processes have replied with
permission.
Properties:
Improved Efficiency: The algorithm requires (2(N-1)) messages per critical section request
(request and reply messages).
Mutual Exclusion: Like Lamport’s algorithm, it guarantees mutual exclusion using logical
timestamps.
Disadvantages:
Message Overhead: Although reduced, it still requires communication with all processes in
the system.
Handling Failures: The algorithm can face issues if processes fail to respond, requiring
additional mechanisms for fault tolerance.

6. Comparison of Token-Based and Permission-Based Algorithms

99 / 123
 Distributed Systems and Cloud Computing
Feature Token-Based Algorithms Permission-Based Algorithms
Message Generally fewer Higher message complexity (all-to-all)
Complexity messages (single token)
Fairness Token rotation ensures Depends on timestamps and request order
fairness
Deadlock and Deadlock is possible if the No deadlock, but requires replies from all
Starvation token is lost processes
Fault Tolerance Difficult (token loss needs Better fault tolerance but requires additional
recovery) handling for non-responsive nodes
Efficiency in High Efficient for high loads May result in message overhead in high loads
Loads (few CS requests)
Examples Token Ring, Raymond’s Lamport’s Algorithm, Ricart-Agrawala
Tree Algorithm

7. Advanced Techniques

a. Maekawa’s Algorithm

Maekawa’s Algorithm reduces the number of messages required by grouping processes into smaller,
overlapping subsets (or voting sets). Each process only needs permission from its subset (not all
processes) to enter the critical section.

Mechanism:
1. The system divides processes into voting sets.
2. A process requests permission from its voting set to enter the critical section.
3. Once a process has obtained permission from all members of its voting set, it enters the
critical section.
Properties:
Message Reduction: The algorithm reduces the number of messages compared to
permission-based algorithms like Lamport’s.
Deadlock Prone: It can lead to deadlocks, requiring additional mechanisms like timeouts to
resolve.

b. Quorum-Based Approaches

Quorum-based algorithms optimize permission-based algorithms by requiring processes to get

permission from a quorum (subset) of processes rather than all processes. This reduces the number
of messages exchanged, making the system more efficient.

Instead of requesting permission to execute the critical section from all other sites, Each site
requests only a subset of sites which is called a **quorum**.

100 / 123
 Distributed Systems and Cloud Computing
Any two subsets of sites or Quorum contains a common site.
This common site is responsible to ensure mutual exclusion

8. Fault Tolerance in Distributed Mutual Exclusion

Fault tolerance is crucial in distributed systems, as process or communication failures can leave the
system in an inconsistent state. Fault tolerance in DME algorithms includes:

Token Regeneration: In token-based algorithms, if a process holding the token crashes, the
system needs a mechanism to regenerate the token.
Timeouts and Retransmission: In permission-based algorithms, processes can use timeouts to
detect non-responsiveness and retry sending requests.

9. Conclusion
Distributed mutual exclusion is a critical aspect of ensuring correct coordination in distributed systems.
Whether token-based or permission-based, each algorithm has its advantages and drawbacks. The
choice of algorithm depends on system requirements such as message complexity, fairness, fault
tolerance, and scalability. Understanding the trade-offs between these approaches is essential for
implementing efficient and reliable distributed systems.

FAQs:

How does the token-based algorithm handle the failure of a site that possesses the token?

If a site that possesses the token fails, then the token is lost until the site recovers or another site
generates a new token. In the meantime, no site can enter the critical section.

What is a quorum in the quorum-based approach, and how is it determined?

A quorum is a subset of sites that a site requests permission from to enter the critical section. The
quorum is determined based on the size and number of overlapping subsets among the sites.

How does the non-token-based approach ensure fairness among the sites?

The non-token-based approach uses a logical clock to order requests for the critical section. Each
site maintains its own logical clock, which gets updated with each message it sends or receives.
This ensures that requests are executed in the order they arrive in the system, and that no site is
unfairly prioritized.

https://www.geeksforgeeks.org/mutual-exclusion-in-distributed-
system/ <- more here

101 / 123
 Distributed Systems and Cloud Computing

Elections in Distributed Systems

Distributed Algorithm is an algorithm that runs on a distributed system. Distributed system is a
collection of independent computers that do not share their memory. Each processor has its own
memory and they communicate via communication networks. Communication in networks is
implemented in a process on one machine communicating with a process on another machine. Many
algorithms used in the distributed system require a coordinator that performs functions needed by
other processes in the system.

Election algorithms are designed to choose a coordinator.

1. Introduction to Election Algorithms

In distributed systems, election algorithms are used to select a coordinator or leader process among a
group of processes. The coordinator typically handles specific tasks such as resource allocation,
synchronization, or decision-making in the system. If the coordinator process crashes due to some
reasons, then a new coordinator is elected on other processor. Election algorithm basically determines
where a new copy of the coordinator should be restarted. Election algorithm assumes that every
active process in the system has a unique priority number. The process with highest priority will be
chosen as a new coordinator. Hence, when a coordinator fails, this algorithm elects that active
process which has highest priority number. Then this number is send to every active process in the
distributed system. We have two election algorithms for two different configurations of a distributed
system.

Election algorithms ensure:

A single leader (coordinator) is elected.

The leader is chosen in a fair and efficient manner.
All nodes in the system agree on the selected leader.

2. Key Requirements for Election Algorithms

For an election algorithm to be effective, it must meet the following requirements:

Unique Leader: At the end of the election process, only one process should be designated as
the leader.
Termination: The election must eventually terminate, meaning a leader must be chosen after a
finite number of steps.
Agreement: All processes in the system must agree on the same leader.
Fault Tolerance: The election process should handle failures, especially if the current
coordinator or other processes crash.

102 / 123
 Distributed Systems and Cloud Computing
3. Election Triggers
An election is typically triggered by:

The failure or crash of the current coordinator.

The coordinator becoming unreachable due to network partitions.
Voluntary withdrawal of the current leader.
Initial system startup (when no leader exists).

4. Common Election Algorithms

a. Bully Algorithm

The Bully Algorithm is one of the simplest and most widely used algorithms in distributed systems. It
assumes that processes are numbered uniquely, and the process with the highest number becomes
the coordinator. This algorithm applies to system where every process can send a message to every
other process in the system

Algorithm – Suppose process P sends a message to the coordinator.

1. If the coordinator does not respond to it within a time interval T, then it is assumed that
coordinator has failed.
2. Now process P sends an election messages to every process with high priority number.
3. It waits for responses, if no one responds for time interval T then process P elects itself as a
coordinator.
4. Then it sends a message to all lower priority number processes that it is elected as their new
coordinator.
5. However, if an answer is received within time T from any other process Q,
(I) Process P again waits for time interval T’ to receive another message from Q that it has
been elected as coordinator.
(II) If Q doesn’t responds within time interval T’ then it is assumed to have failed and
algorithm is restarted.

Properties:
Efficiency: The process with the highest ID always wins, making the election deterministic.
Message Complexity: The algorithm may involve multiple messages, especially in large
systems with many processes.
Failure Handling: The system tolerates the failure of any process except the one initiating
the election.
Drawbacks:
Aggressive Nature: The algorithm can be inefficient when there are many processes since
each lower-ID process keeps starting new elections.

103 / 123
 Distributed Systems and Cloud Computing
Message Overhead: Multiple election and response messages can create a
communication overhead in larger systems.

b. Ring-Based Election Algorithm

The Ring-Based Election Algorithm assumes that all processes are organized into a logical ring.
Each process has a unique ID, and the goal is to elect the process with the highest ID as the leader. In
this algorithm we assume that the link between the process are unidirectional and every process can
message to the process on its right only. Data structure that this algorithm uses is active list, a list
that has a priority number of all active processes in the system.

Algorithm –

1. If process P1 detects a coordinator failure, it creates new active list which is empty initially. It
sends election message to its neighbour on right and adds number 1 to its active list.
2. If process P2 receives message elect from processes on left, it responds in 3 ways:
(I) If message received does not contain 1 in active list then P1 adds 2 to its active list and
forwards the message.
(II) If this is the first election message it has received or sent, P1 creates new active list with
numbers 1 and 2. It then sends election message 1 followed by 2.
(III) If Process P1 receives its own election message 1 then active list for P1 now contains
numbers of all the active processes in the system. Now Process P1 detects highest priority
number from list and elects it as the new coordinator.

Properties:
Efficiency: Each process only communicates with its neighbor, reducing the message
complexity.
Simple Structure: Since the algorithm uses a logical ring, it is easy to implement.
Drawbacks:
Single Point of Failure: If a process in the ring fails, the entire ring can become
disconnected, requiring mechanisms to handle failures.
Long Delay: Election messages must traverse the entire ring, which can lead to delays in
large systems.

5. Other Election Algorithms (potentially out of syllabus)

a. Chang and Roberts’ Algorithm

This is an optimized version of the ring-based election algorithm designed to minimize the number of
messages exchanged.

Mechanism:

104 / 123
 Distributed Systems and Cloud Computing
1. Processes are arranged in a logical ring.
2. When a process starts an election, it sends an election message to its successor.
3. If the receiver’s ID is higher, it continues to forward the message with its ID. If it is lower, it
forwards the original message unchanged.
4. Once a message returns to the initiator with a higher ID, that ID is declared the new
coordinator.
Efficiency:
The algorithm ensures that only the highest-ID process circulates messages, thus reducing
unnecessary message forwarding.

b. Paxos-Based Leader Election

The Paxos algorithm is a consensus algorithm that can also be used for leader election. It focuses
on achieving consensus in distributed systems and can elect a leader in a fault-tolerant way.

Mechanism:
1. Processes propose themselves as leaders.
2. The system must reach consensus on which process becomes the leader.
3. Paxos ensures that even if multiple processes propose themselves as leaders, only one will
be chosen by the majority.
Properties:
Fault Tolerance: Paxos is resilient to process and communication failures.
Complexity: Paxos is more complex than simple election algorithms like Bully or Ring
algorithms.
Use Cases: Paxos is used in distributed databases and consensus-based systems where fault
tolerance is critical.

6. Optimizations in Election Algorithms

a. Hierarchical Election Algorithms

In large distributed systems, a hierarchical election structure can be used to reduce message
complexity and improve efficiency. In these systems:

The system is divided into smaller groups or clusters.

Each cluster elects its own local coordinator.
Local coordinators then participate in a higher-level election to choose a global leader.

b. Election with Failure Detection

105 / 123
 Distributed Systems and Cloud Computing
In many distributed systems, failure detectors are used to automatically initiate elections when the
current coordinator becomes unreachable or fails. This reduces the delay in detecting failures and
starting the election process.

7. Comparison of Election Algorithms

Feature Bully Algorithm Ring Algorithm Paxos-Based

Election(potentially out of
syllabus)
Message High (all-to-all Low (message Moderate (requires consensus)
Complexity communication) passes in ring)
Failure Handles failures but Single point of Highly fault-tolerant
Handling needs recovery failure (ring)
Fairness Highest-ID process Highest-ID Achieves consensus-based
always wins process always fairness
wins
Time to Elect Fast but with message Longer delays in Depends on consensus process
overhead large systems
Use Cases Simple, small-scale Systems with Fault-tolerant critical systems
systems structured rings

8. Best Practices in Election Algorithms

Efficient Message Passing: In large-scale systems, minimizing message complexity is crucial.

Ring-based and quorum-based election algorithms reduce the communication overhead.
Fault Tolerance: Election algorithms must handle process failures. Paxos-based or failure
detector-based elections are ideal for systems that require high availability and reliability.
Fairness and Termination: Ensure that the algorithm guarantees a unique leader and that it
eventually terminates, even in the presence of failures.

9. Applications of Election Algorithms

Election algorithms are widely used in:

Distributed Databases: To elect a leader or master node for managing transactions and
consistency.
Distributed Consensus Systems: For example, in consensus algorithms like Paxos or Raft, a
leader is elected to coordinate the agreement.
Cloud Services: In systems like Apache ZooKeeper and Google’s Chubby, leader election is
used to maintain consistency and coordination among distributed nodes.
Peer-to-Peer Systems: Used to coordinate tasks or services in decentralized networks.
106 / 123
 Distributed Systems and Cloud Computing
10. Conclusion
Election algorithms are fundamental to ensuring coordination and consistency in distributed systems.
Whether using simple approaches like the Bully and Ring algorithms or more complex consensus-
based algorithms like Paxos, each algorithm has its strengths and weaknesses. The choice of
algorithm depends on the system’s size, fault tolerance requirements, and communication efficiency.
The goal is always to ensure that the system agrees on a unique leader to maintain the proper
functioning of the distributed system.

Multicast Communication in Distributed Systems

1. Introduction to Multicast Communication
In distributed systems, multicast communication allows a process to send a message to a group of
processes simultaneously. It is essential for efficient data dissemination, coordination, and event
notification among multiple processes or nodes. Instead of sending a separate message to each
process, multicast enables a process to communicate with all interested parties at once, reducing
communication overhead and ensuring consistency.

Multicast communication is widely used in:

Distributed databases for data replication.

Consensus algorithms like Paxos and Raft.
Group communication systems to synchronize processes.

Group communication is critically important in distributed systems due to several key reasons:

Coordination and Synchronization:

Distributed systems often involve multiple nodes or entities that need to collaborate and
synchronize their activities.
Group communication mechanisms facilitate the exchange of information, coordination of
tasks, and synchronization of state among these distributed entities.
This ensures that all parts of the system are aware of the latest updates and can act in a
coordinated manner.
Efficient Information Sharing:
In distributed systems, different nodes may generate or process data that needs to be
shared among multiple recipients.
Group communication allows for efficient dissemination of information to all relevant parties
simultaneously, reducing latency and ensuring consistent views of data across the system.
Fault Tolerance and Reliability:
Group communication protocols often include mechanisms for ensuring reliability and fault
tolerance.

107 / 123
 Distributed Systems and Cloud Computing
Messages can be replicated or acknowledged by multiple nodes to ensure that
communication remains robust even in the face of node failures or network partitions.
This enhances the overall reliability and availability of the distributed system.
Scalability:
As distributed systems grow in size and complexity, the ability to scale effectively becomes
crucial.
Group communication mechanisms are designed to handle increasing numbers of nodes
and messages without compromising performance or reliability.
They enable the system to maintain its responsiveness and efficiency as it scales up.

2. Types of Multicast

There are three main types of multicast communication in distributed systems:

a. Unicast

Unicast communication refers to the point-to-point transmission of data between two nodes in a
network. In the context of distributed systems:

Definition: Unicast involves a sender (one node) transmitting a message to a specific receiver
(another node) identified by its unique network address.
Characteristics:
One-to-One: Each message has a single intended recipient.
Direct Connection: The sender establishes a direct connection to the receiver.
Efficiency: Suitable for scenarios where targeted communication is required, such as
client-server interactions or direct peer-to-peer exchanges.

108 / 123
 Distributed Systems and Cloud Computing
Use Cases:
Request-Response: Common in client-server architectures where clients send requests to
servers and receive responses.
Peer-to-Peer: Direct communication between two nodes in a decentralized network.
Advantages:
Efficient use of network resources as messages are targeted.
Simplified implementation due to direct connections.
Low latency since messages are sent directly to the intended recipient.
Disadvantages:
Not scalable for broadcasting to multiple recipients without sending separate messages.
Increased overhead if many nodes need to be contacted individually.

b. Broadcast

Broadcast communication involves sending a message from one sender to all nodes in the network,
ensuring that every node receives the message:

Definition: A sender transmits a message to all nodes within the network without the need for
specific recipients.
Characteristics:
One-to-All: Messages are delivered to every node in the network.
Broadcast Address: Uses a special network address (e.g., IP broadcast address) to reach
all nodes.
Global Scope: Suitable for disseminating information to all connected nodes
simultaneously.
Use Cases:

109 / 123
 Distributed Systems and Cloud Computing
Network Management: Broadcasting status updates or configuration changes.
Emergency Alerts: Disseminating critical information to all recipients in a timely manner.
Advantages:
Ensures that every node receives the message without requiring explicit recipient lists.
Efficient for scenarios where global dissemination of information is necessary.
Simplifies communication in small-scale networks or LAN environments.
Disadvantages:
Prone to network congestion and inefficiency in large networks.
Security concerns, as broadcast messages are accessible to all nodes, potentially leading
to unauthorized access or information leakage.
Requires careful network design and management to control the scope and impact of
broadcast messages.

c. Multicast

Multicast communication involves sending a single message from one sender to multiple receivers
simultaneously within a network. It is particularly useful in distributed systems where broadcasting
information to a group of nodes is necessary:

Definition: A sender transmits a message to a multicast group, which consists of multiple

recipients interested in receiving the message.
Characteristics:
One-to-Many: Messages are sent to multiple receivers in a single transmission.
Efficient Bandwidth Usage: Reduces network congestion compared to multiple unicast
transmissions.
Group Membership: Receivers voluntarily join and leave multicast groups as needed.

110 / 123
 Distributed Systems and Cloud Computing
Use Cases:
Content Distribution: Broadcasting updates or notifications to subscribers.
Collaborative Systems: Real-time collaboration tools where changes made by one user
need to be propagated to others.
Advantages:
Saves bandwidth and network resources by transmitting data only once.
Simplifies management by addressing a group rather than individual nodes.
Supports scalable communication to a large number of recipients.
Disadvantages:
Requires mechanisms for managing group membership and ensuring reliable delivery.
Vulnerable to network issues such as packet loss or congestion affecting all recipients.

3. Characteristics of Multicast Communication

Efficiency: Multicast reduces the overhead of sending multiple individual messages by sending
a single message to a group of recipients.
Scalability: It is scalable for large distributed systems, as the number of messages sent grows
minimally compared to unicast.
Reliability: In some multicast protocols, reliability is a concern. Reliable multicast ensures that all
intended recipients receive the message, while unreliable multicast does not guarantee message
delivery.
Ordering Guarantees: Different multicast protocols provide various levels of ordering
guarantees (e.g., FIFO, causal, total ordering).

4. Reliable Multicast vs. Unreliable Multicast

a. Unreliable Multicast

In unreliable multicast communication, the sender transmits the message without ensuring that all
recipients successfully receive it. This is similar to the UDP (User Datagram Protocol) in networking,
where packet loss may occur, and no acknowledgment is required.

Use Cases: Suitable for scenarios where occasional message loss is tolerable, such as in real-
time media streaming, where it is more important to keep transmitting data quickly rather than
ensuring every packet is received.

b. Reliable Multicast

In reliable multicast communication, the sender ensures that all members of the multicast group
receive the message. This is critical in distributed systems where consistency and correctness are

111 / 123
 Distributed Systems and Cloud Computing
important. Reliable multicast requires additional protocols to detect lost messages and retransmit
them.

Use Cases: Distributed databases, distributed file systems, or consensus algorithms where
message loss can result in system inconsistencies.

Key Protocols for Reliable Multicast:

TCP-Based Multicast: Some reliable multicast protocols use mechanisms like

acknowledgments (ACKs) from recipients or error recovery methods to ensure reliable delivery.
NACK-Based Protocols: In NACK-based protocols (Negative Acknowledgments), receivers
send a NACK if they detect a missing message, prompting the sender to retransmit the lost
message.

5. Multicast Communication Protocols in Distributed Systems

There are several well-known multicast communication protocols designed to provide different
guarantees in distributed systems:

a. IP Multicast

Definition: A network-layer protocol used for sending a message to multiple recipients on an IP

network. IP multicast uses group addresses to identify a set of receivers that will receive the
message.
Features:
Efficient in transmitting the same message to multiple receivers.
Works across large networks, such as the internet, without duplication of messages.
Limitations: Does not inherently provide reliability or message ordering.

b. Reliable Multicast Transport Protocol (RMTP)

Definition: RMTP is a protocol designed for reliable multicast delivery. It ensures that all
recipients receive the message by using hierarchical acknowledgments from receivers.
Mechanism:
1. Receivers are organized hierarchically into sub-groups, with one receiver in each group
acting as a local repair node.
2. Local repair nodes acknowledge messages to the sender and handle retransmissions within
their group if any receiver reports a lost message.
Advantages: More scalable than traditional ACK-based reliable multicast as it reduces the
overhead of having every receiver send acknowledgments.

112 / 123
 Distributed Systems and Cloud Computing
c. Totally Ordered Multicast

In some distributed systems, it is necessary to maintain a strict ordering of messages, such as when
multiple processes update a shared resource.

Mechanism: Totally ordered multicast ensures that all processes receive messages in the same
order, regardless of when or from whom the message originated. This is crucial for consistency in
replicated state machines or distributed databases.
Protocol Examples:
Paxos Consensus Algorithm: Ensures totally ordered message delivery as part of the
consensus process.
Atomic Broadcast Protocols: Guarantee total ordering of messages and are often used in
fault-tolerant distributed systems.

6. Ordering in Multicast Communication

Ordering is crucial in distributed systems where multiple processes may send messages concurrently.
Different multicast protocols provide different levels of ordering guarantees:

a. FIFO Ordering

Definition: Messages from a single process are received by all other processes in the same
order they were sent.
Use Case: Ensures consistency in systems where the order of messages from a particular
process is important, such as in message logging systems.

b. Causal Ordering

Definition: Messages are delivered in an order that respects the causal relationships between
them. If message A causally precedes message B (e.g., A was sent before B as a result of some
action), then all recipients will receive A before B.
Use Case: Suitable for systems with interdependent events, such as collaborative editing
systems where one user’s action may depend on another’s.

c. Total Ordering

Definition: All processes receive all messages in the same order, regardless of the sender. Total
ordering is stronger than causal ordering and guarantees that no two processes see the
messages in a different order.
Use Case: Critical in distributed databases or replicated state machines, where the order of
operations must be consistent across all replicas.
113 / 123
 Distributed Systems and Cloud Computing
7. Use Cases for Multicast Communication in Distributed Systems
Multicast communication plays a key role in several types of distributed systems:

Distributed Databases: Multicast is used to ensure that all replicas of the database receive
updates consistently and in the correct order.
Distributed File Systems: For example, in Google’s GFS or Hadoop’s HDFS, multicast ensures
that data is replicated across different nodes consistently.
Consensus Algorithms: Protocols like Paxos and Raft rely on multicast to disseminate
proposals and decisions to all participants in the consensus process.
Event Notification Systems: Multicast is commonly used to send events to multiple subscribers,
ensuring that all subscribers receive updates simultaneously.
Multimedia Broadcasting: Applications like video streaming or online gaming platforms use
multicast to efficiently distribute data to multiple clients.

8. Multicast in Cloud Computing

In cloud environments, where services are distributed across multiple data centers and nodes,
multicast communication is used to:

Replicate data: Across different geographic regions for fault tolerance and high availability.
Synchronize services: Among microservices and container-based applications.
Enable event-driven architectures: In serverless or event-driven models, multicast is used to
send notifications and events to multiple services that need to act on the data.

9. Fault Tolerance in Multicast Communication

Multicast communication must be fault-tolerant to handle message loss, node failures, and network
partitioning:

Redundancy: Sending multiple copies of critical messages ensures delivery even if some
messages are lost.
Timeouts and Retransmissions: Reliable multicast protocols use timeouts to detect lost
messages and trigger retransmissions.
Backup Nodes: In group-based multicast protocols, backup nodes take over if the primary
sender fails, ensuring the message is delivered.

10. Conclusion

Multicast communication is an essential component in distributed systems, enabling efficient and

scalable communication between processes. Whether for replicating data, synchronizing state, or
enabling distributed consensus, multicast communication reduces message overhead and ensures
114 / 123
 Distributed Systems and Cloud Computing
consistent delivery of messages across distributed nodes. Depending on the system’s needs, various
multicast protocols provide different levels of reliability, ordering guarantees, and fault tolerance.

https://www.geeksforgeeks.org/group-communication-in-distributed-
systems/ < more details

Consensus and Related Problems in Distributed Systems

1. Introduction to Consensus

What is the Distributed Consensus in Distributed Systems?

Distributed consensus in distributed systems refers to the process by which multiple nodes or
components in a network agree on a single value or a course of action despite potential failures or
differences in their initial states or inputs. It is crucial for ensuring consistency and reliability in
decentralized environments where nodes may operate independently and may experience delays or
failures. Popular algorithms like Paxos and Raft are designed to achieve distributed consensus
effectively.

Importance of Distributed Consensus in Distributed Systems

Below are the importance of distributed consensus in distributed systems:

Consistency and Reliability:

Distributed consensus ensures that all nodes in a distributed system agree on a common
state or decision. This consistency is crucial for maintaining data integrity and preventing
conflicting updates.
Fault Tolerance:
Distributed consensus mechanisms enable systems to continue functioning correctly even if
some nodes experience failures or network partitions. By agreeing on a consistent state, the
system can recover and continue operations smoothly.
Decentralization:
In decentralized networks, where nodes may operate autonomously, distributed consensus
allows for coordinated actions and ensures that decisions are made collectively rather than
centrally. This is essential for scalability and resilience.
Concurrency Control:
Consensus protocols help manage concurrent access to shared resources or data across
distributed nodes. By agreeing on the order of operations or transactions, consensus
ensures that conflicts are avoided and data integrity is maintained.
Blockchain and Distributed Ledgers:
In blockchain technology and distributed ledgers, consensus algorithms (e.g., Proof of
Work, Proof of Stake) are fundamental. They enable participants to agree on the validity of
115 / 123
 Distributed Systems and Cloud Computing
transactions and maintain a decentralized, immutable record of transactions.

2. Key Challenges in Consensus

Achieving consensus in distributed systems presents several challenges due to the inherent
complexities and potential uncertainties in networked environments. Some of the key challenges
include:

Network Partitions:
Network partitions can occur due to communication failures or delays between nodes.
Consensus algorithms must ensure that even in the presence of partitions, nodes can
eventually agree on a consistent state or outcome.
Node Failures:
Nodes in a distributed system may fail or become unreachable, leading to potential
inconsistencies in the system state. Consensus protocols need to handle these failures
gracefully and ensure that the system remains operational.
Asynchronous Communication:
Nodes in distributed systems may communicate asynchronously, meaning messages may
be delayed, reordered, or lost. Consensus algorithms must account for such communication
challenges to ensure accurate and timely decision-making.
Byzantine Faults:
Byzantine faults occur when nodes exhibit arbitrary or malicious behavior, such as sending
incorrect information or intentionally disrupting communication. Byzantine fault-tolerant
consensus algorithms are needed to maintain correctness in the presence of such faults.

3. Types of Consensus Problems

Value Consensus: Processes must agree on a proposed value, often referred to as the
"decision value."
Leader Election: A subset of consensus where processes must agree on a leader or coordinator
among themselves.
Distributed Agreement: All processes must agree on a sequence of actions or operations that
have occurred in the system.

4. Consensus Algorithms

Several algorithms have been developed to solve the consensus problem in distributed systems:

a. Paxos Algorithm

116 / 123
 Distributed Systems and Cloud Computing

Overview: Paxos is a widely used consensus algorithm designed for fault tolerance in distributed
systems. Paxos is a classic consensus algorithm which ensures that a distributed system can
agree on a single value or sequence of values, even if some nodes may fail or messages may be
delayed. Key concepts of paxos algorithm include:
Roles:
Proposer: Initiates the proposal of a value.
Acceptor: Accepts proposals from proposers and communicates its acceptance.
Learner: Learns the chosen value from acceptors.
Phases:

1. Prepare Phase: A proposer selects a proposal number and sends a "prepare" request to a
majority of acceptors.
2. Promise Phase: Acceptors respond with a promise not to accept lower-numbered proposals and
may include the highest-numbered proposal they have already accepted.
3. Accept Phase: The proposer sends an "accept" request with the chosen value to the acceptors,
who can accept the proposal if they have promised not to accept a lower-numbered proposal.

Working:
Proposers: Proposers initiate the consensus process by proposing a value to be agreed
upon.
Acceptors: Acceptors receive proposals from proposers and can either accept or reject
them based on certain criteria.

117 / 123
 Distributed Systems and Cloud Computing
Learners: Learners are entities that receive the agreed-upon value or decision once
consensus is reached among the acceptors.
Properties:
Fault Tolerance: Paxos can tolerate failures of up to (n-1)/2 processes in a system with n
processes.
Safety: Guarantees that a value is only chosen once and that all processes agree on that
value.
Liveness: Guarantees that if the network is functioning, a decision will eventually be
reached.
Safety and Liveness:
Paxos ensures safety (only one value is chosen) and liveness (a value is eventually chosen)
properties under normal operation assuming a majority of nodes are functioning correctly.
Use Cases:
Paxos is used in distributed databases, replicated state machines, and other systems where
achieving consensus among nodes is critical.

b. Raft Algorithm

Overview: Raft is another consensus algorithm designed to be easier to understand than Paxos
while still providing strong consistency guarantees. It operates in terms of a leader and follower
model. It simplifies the complexities of traditional consensus algorithms like Paxos while
providing similar guarantees. Raft operates by electing a leader among the nodes in a cluster,
where the leader manages the replication of a log that contains commands or operations to be
executed.
Key Concepts:

118 / 123
 Distributed Systems and Cloud Computing
Leader Election: Nodes elect a leader responsible for managing log replication and
handling client requests.
Log Replication: Leader replicates its log entries to followers, ensuring consistency across
the cluster.
Safety and Liveness: Raft guarantees safety (log entries are consistent) and liveness (a
leader is elected and log entries are eventually committed) under normal operation.
Phases:
Leader Election: Nodes participate in leader election based on a term number and leader’s
heartbeat.
Log Replication: Leader sends AppendEntries messages to followers to replicate log
entries, ensuring consistency.
Properties:
Simplicity: Raft is designed to be more intuitive than Paxos, making it easier to implement
and reason about.
Strong Leader: The leader manages communication, simplifying the consensus process.
Efficient Log Management: Uses efficient mechanisms for log replication and
management.
Use Cases:
Raft is widely used in modern distributed systems such as key-value stores, consensus-
based replicated databases, and systems requiring strong consistency guarantees.

c. Byzantine Fault Tolerance (BFT)

119 / 123
 Distributed Systems and Cloud Computing

Overview: BFT algorithms are designed to handle the presence of malicious nodes (Byzantine
faults) that may behave arbitrarily, such as sending conflicting information. BFT algorithms
require a higher number of processes to ensure agreement.
Byzantine Fault Tolerance (BFT) algorithms are designed to address the challenges posed by
Byzantine faults in distributed systems, where nodes may fail in arbitrary ways, including sending
incorrect or conflicting information. These algorithms ensure that the system can continue to
operate correctly and reach consensus even when some nodes behave maliciously or fail
unexpectedly.
Key Concepts:
Byzantine Faults: Nodes may behave arbitrarily, including sending conflicting messages or
omitting messages.
Redundancy and Voting: BFT algorithms typically require a 2/3 or more agreement among
nodes to determine the correct state or decision.
Example:
Practical Byzantine Fault Tolerance (PBFT): Used in systems where safety and liveness
are crucial, such as blockchain networks and distributed databases.
Simplified Byzantine Fault Tolerance (SBFT): Provides a simpler approach to achieving
BFT with reduced complexity compared to PBFT.
Properties:
Robustness: Can tolerate up to (n-1)/3 faulty processes in a system with n processes.
120 / 123
 Distributed Systems and Cloud Computing
Communication Complexity: Higher overhead due to the need for more messages among
processes to reach agreement.
Use Cases:
BFT algorithms are essential in environments requiring high fault tolerance and security,
where nodes may not be fully trusted or may exhibit malicious behavior.

A practical Byzantine Fault Tolerant system can function on the condition that the maximum
number of malicious nodes must not be greater than or equal to one-third of all the nodes in the
system. As the number of nodes increase, the system becomes more secure. pBFT consensus
rounds are broken into 4 phases.

The client sends a request to the primary(leader) node.

The primary(leader) node broadcasts the request to the all the secondary(backup) nodes.
The nodes(primary and secondaries) perform the service requested and then send back a reply
to the client.
The request is served successfully when the client receives ‘m+1’ replies from different nodes in
the network with the same result, where m is the maximum number of faulty nodes allowed.

5. Consensus Problem Variants

Leaderless Consensus: In systems where a leader is not viable, algorithms like Chandra-
Toueg can achieve consensus without a designated leader.
Weak Consistency Models: Systems may use relaxed consistency models like eventual
consistency, which allows for more flexible consensus mechanisms but sacrifices strong
consistency guarantees.

6. Applications of Consensus
Below are some practical applications of distributed consensus in distributed systems:

Distributed Databases: Ensuring data consistency and synchronization across replicas.

Blockchain Technology:
Use Case: Blockchain networks rely on distributed consensus to agree on the validity and
order of transactions across a decentralized ledger.
Example: Bitcoin and Ethereum use consensus algorithms (like Proof of Work and Proof of
Stake) to achieve decentralized agreement among nodes.
Distributed Databases:
Use Case: Consensus algorithms ensure that distributed databases maintain consistency
across nodes, ensuring that updates and transactions are applied uniformly.
Example: Google Spanner uses a variant of Paxos to replicate data and ensure
consistency across its globally distributed database.

121 / 123
 Distributed Systems and Cloud Computing
Cloud Computing:
Use Case: Cloud providers use distributed consensus to manage resource allocation, load
balancing, and fault tolerance across distributed data centers.
Example: Amazon DynamoDB uses quorum-based techniques for replication and
consistency among its distributed database nodes.

Challenges and Considerations:

Network Partitions and Delays: Algorithms must handle network partitions and communication
delays, ensuring that nodes eventually reach consensus.
Scalability: As the number of nodes increases, achieving consensus becomes more challenging
due to increased communication overhead.
Performance: Consensus algorithms should be efficient to minimize latency and maximize
system throughput.
Understanding and Implementation: Many consensus algorithms, especially BFT variants, are
complex and require careful implementation to ensure correctness and security.

Achieving consensus in distributed systems presents several challenges due to the inherent
complexities and potential uncertainties in networked environments. Some of the key challenges
include:

Network Partitions:
Network partitions can occur due to communication failures or delays between nodes.
Consensus algorithms must ensure that even in the presence of partitions, nodes can
eventually agree on a consistent state or outcome.
Node Failures:
Nodes in a distributed system may fail or become unreachable, leading to potential
inconsistencies in the system state. Consensus protocols need to handle these failures
gracefully and ensure that the system remains operational.
Asynchronous Communication:
Nodes in distributed systems may communicate asynchronously, meaning messages may
be delayed, reordered, or lost. Consensus algorithms must account for such communication
challenges to ensure accurate and timely decision-making.
Byzantine Faults:
Byzantine faults occur when nodes exhibit arbitrary or malicious behavior, such as sending
incorrect information or intentionally disrupting communication. Byzantine fault-tolerant
consensus algorithms are needed to maintain correctness in the presence of such faults.

7. Consensus in Real-World Systems

122 / 123
 Distributed Systems and Cloud Computing
Google’s Chubby: A distributed lock service that uses Paxos for consensus to manage
distributed locks and configuration data.
Apache ZooKeeper: Provides distributed coordination and consensus services based on ZAB
(ZooKeeper Atomic Broadcast).
Cassandra and DynamoDB: Use consensus mechanisms to handle data consistency across
distributed nodes, often employing techniques inspired by Paxos and other consensus
algorithms.

8. Conclusion
Consensus is a vital aspect of distributed systems, ensuring that multiple processes can agree on
shared values or states despite failures and network complexities. With various algorithms available,
including Paxos, Raft, and Byzantine fault-tolerant solutions, systems can achieve reliable consensus
tailored to their specific needs. Understanding these algorithms and their applications is crucial for
designing robust and fault-tolerant distributed systems.

UNIT 3
tbd

123 / 123