[Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

UNIT III: INTRODUCTION TO DISTRIBUTED SYSTEMS

Introduction – Characterization of Distributed Systems – Distributed Shared
Memory – Message Passing – Programming using the Message Passing
Paradigm – Group Communication – Case Study (RPC and JAVA RMI)

1. INTRODUCTION TO DISTRIBUTED SYSTEMS

Distributed System is a collection of autonomous computer systems that are
physically separated but are connected by a centralized computer network that is
equipped with distributed system software. The autonomous computers will
communicate among each system by sharing resources and files and performing the
tasks assigned to them.
Example of Distributed System:
Any Social Media can have its Centralized Computer Network as its Headquarters
and computer systems that can be accessed by any user and using their services will
be the Autonomous Systems in the Distributed System Architecture.

Fig. 3.1 Representation of Distributed System

Distributed System Software: This Software enables computers to coordinate their

activities and to share the resources such as Hardware, Software, Data, etc.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

Database: It is used to store the processed data that are processed by each
Node/System of the Distributed systems that are connected to the Centralized
network.

Fig. 3.2 Working of Distributed System

 As we can see that each Autonomous System has a common Application that
can have its own data that is shared by the Centralized Database System.
 To Transfer the Data to Autonomous Systems, Centralized System should be
having a Middleware Service and should be connected to a Network.
 Middleware Services enable some services which are not present in the local
systems or centralized system default by acting as an interface between the
Centralized System and the local systems. By using components of
Middleware Services systems communicate and manage data.
 The Data which is been transferred through the database will be divided into
segments or modules and shared with Autonomous systems for processing.
 The Data will be processed and then will be transferred to the Centralized
system through the network and will be stored in the database.

Characteristics of Distributed System:

Resource Sharing: It is the ability to use any Hardware, Software, or Data anywhere
in the System.
Openness: It is concerned with Extensions and improvements in the system (i.e.,
How openly the software is developed and shared with others)

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

Concurrency: It is naturally present in Distributed Systems, that deal with the same
activity or functionality that can be performed by separate users who are in remote
locations. Every local system has its independent Operating Systems and Resources.
Scalability: It increases the scale of the system as a number of processors
communicate with more users by accommodating to improve the responsiveness of
the system.
Fault tolerance: It cares about the reliability of the system if there is a failure in
Hardware or Software, the system continues to operate properly without degrading
the performance the system.
Transparency: It hides the complexity of the Distributed Systems to the Users and
Application programs as there should be privacy in every system.
Heterogeneity: Networks, computer hardware, operating systems, programming
languages, and developer implementations can all vary and differ among dispersed
system components.
Advantages of Distributed System:
 Applications in Distributed Systems are Inherently Distributed Applications.
 Information in Distributed Systems is shared among geographically
distributed users.
 Resource Sharing (Autonomous systems can share resources from remote
locations).
 It has a better price performance ratio and flexibility.
 It has shorter response time and higher throughput.
 It has higher reliability and availability against component failure.
 It has extensibility so that systems can be extended in more remote locations
and also incremental growth.
Disadvantages of Distributed System:
 Relevant Software for Distributed systems does not exist currently.
 Security possesses a problem due to easy access to data as the resources are
shared to multiple systems.
 Networking Saturation may cause a hurdle in data transfer.
 In comparison to a single user system, the database associated with
distributed systems is much more complex and challenging to manage.
 If every node in a distributed system tries to send data at once, the network
may become overloaded.
Applications Area of Distributed System:

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

 Finance and Commerce: Amazon, eBay, Online Banking, E-Commerce

websites.
 Information Society: Search Engines, Wikipedia, Social Networking, Cloud
Computing.
 Cloud Technologies: AWS, Salesforce, Microsoft Azure, SAP.
 Entertainment: Online Gaming, Music, youtube.
 Healthcare: Online patient records, Health Informatics.
 Education: E-learning.
 Transport and logistics: GPS, Google Maps.
 Environment Management: Sensor technologies.
Challenges of Distributed Systems:
While distributed systems offer many advantages, they also present some challenges
that must be addressed. These challenges include:
Network latency: The communication network in a distributed system can introduce
latency, which can affect the performance of the system.
Distributed coordination: Distributed systems require coordination among the
nodes, which can be challenging due to the distributed nature of the system.
Security: Distributed systems are more vulnerable to security threats than
centralized systems due to the distributed nature of the system.
Data consistency: Maintaining data consistency across multiple nodes in a
distributed system can be challenging.

1.2 CHARACTERIZATION OF DISTRIBUTED SYSTEMS

A distributed system is one in which components located at networked computers
communicate and coordinate their actions only by passing messages. This definition
leads to the following especially significant characteristics of distributed systems:
concurrency of components, lack of a global clock and independent failures of
components.
We look at several examples of modern distributed applications, including web
search, multiplayer online games and financial trading systems, and also examine the
key underlying trends driving distributed systems today: the pervasive nature of
modern networking, the emergence of mobile and ubiquitous computing, the
increasing importance of distributed multimedia systems, and the trend towards
viewing distributed systems as a utility. Resources may be managed by servers and

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

accessed by clients or they may be encapsulated as objects and accessed by other

client objects.
The challenges arising from the construction of distributed systems are the
heterogeneity of their components, openness (which allows components to be added
or replaced), security, scalability – the ability to work well when the load or the
number of users increases – failure handling, concurrency of components,
transparency and providing quality of service.

Networks of computers are everywhere. The Internet is one, as are the many
networks of which it is composed. Mobile phone networks, corporate networks,
factory networks, campus networks, home networks, in-car networks – all of these,
both separately and in combination, share the essential characteristics that make
them relevant subjects for study under the heading distributed systems.
Computers that are connected by a network may be spatially separated by any
distance. They may be on separate continents, in the same building or in the same
room. Our definition of distributed systems has the following significant
consequences:
Concurrency: In a network of computers, concurrent program execution is the norm. I
can do my work on my computer while you do your work on yours, sharing
resources such as web pages or files when necessary. The capacity of the system to
handle shared resources can be increased by adding more resources (for example.
computers) to the network. We will describe ways in which this extra capacity can be
usefully deployed at many points in this book. The coordination of concurrently
executing programs that share resources is also an important and recurring topic.
No global clock: When programs need to cooperate they coordinate their actions by
exchanging messages. Close coordination often depends on a shared idea of the time
at which the programs’ actions occur. But it turns out that there are limits to the
accuracy with which the computers in a network can synchronize their clocks – there
is no single global notion of the correct time. This is a direct consequence of the fact
that the only communication is by sending messages through a network.
Independent failures: All computer systems can fail, and it is the responsibility of
system designers to plan for the consequences of possible failures. Distributed systems
can fail in new ways. Faults in the network result in the isolation of the computers that
are connected to it, but that doesn’t mean that they stop running. In fact, the programs
on them may not be able to detect whether the network has failed or has become
unusually slow. Similarly, the failure of a computer, or the unexpected termination of
a program somewhere in the system (a crash), is not immediately made known to the
other components with which it communicates. Each component of the system can fail

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

independently, leaving the others still running. The consequences of this characteristic
of distributed systems will be a recurring theme throughout the book.
The key challenges faced by the designers of distributed systems: heterogeneity,
openness, security, scalability, failure handling, concurrency, transparency and
quality of service.
Challenges
In this section we describe the main challenges.
Heterogeneity
The Internet enables users to access services and run applications over a
heterogeneous collection of computers and networks. Heterogeneity (that is, variety
and difference) applies to all of the following:
• networks;
• computer hardware;
• operating systems;
• programming languages;
• implementations by different developers.
Middleware • The term middleware applies to a software layer that provides a
programming abstraction as well as masking the heterogeneity of the underlying
networks, hardware, operating systems and programming languages. The Common
Object Request Broker (CORBA) is an example. Some middleware, such as Java
Remote Method Invocation (RMI), supports only a single programming language.
Most middleware is implemented over the Internet protocols, which themselves
mask the differences of the underlying networks, but all middleware deals with the
differences in operating systems and hardware.
Heterogeneity and mobile code • The term mobile code is used to refer to program code
that can be transferred from one computer to another and run at the destination –
Java applets are an example. Code suitable for running on one computer is not
necessarily suitable for running on another because executable programs are
normally specific both to the instruction set and to the host operating system.
Openness
The openness of a computer system is the characteristic that determines whether the
system can be extended and re-implemented in various ways. The openness of
distributed systems is determined primarily by the degree to which new resource-
sharing services can be added and be made available for use by a variety of client
programs.
To summarize:

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

• Open systems are characterized by the fact that their key interfaces are
published.
• Open distributed systems are based on the provision of a uniform
communication mechanism and published interfaces for access to shared
resources.
• Open distributed systems can be constructed from heterogeneous hardware
and software, possibly from different vendors. But the conformance of each
component to the published standard must be carefully tested and verified if
the system is to work correctly.
Security
Many of the information resources that are made available and maintained in
distributed systems have a high intrinsic value to their users. Their security is
therefore of considerable importance. Security for information resources has three
components: confidentiality (protection against disclosure to unauthorized
individuals), integrity (protection against alteration or corruption), and availability
(protection against interference with the means to access the resources).
In a distributed system, clients send requests to access data managed by servers,
which involves sending information in messages over a network. For example:
 A doctor might request access to hospital patient data or send additions to that
data.
 In electronic commerce and banking, users send their credit card numbers
across the Internet.
In both examples, the challenge is to send sensitive information in a message over a
network in a secure manner. But security is not just a matter of concealing the
contents of messages – it also involves knowing for sure the identity of the user or
other agent on whose behalf a message was sent. In the first example, the server
needs to know that the user is really a doctor, and in the second example, the user needs
to be sure of the identity of the shop or bank with which they are dealing. The second
challenge here is to identify a remote user or other agent correctly. Both of these
challenges can be met by the use of encryption techniques developed for this
purpose.
However, the following two security challenges have not yet been fully met:
 Denial of service attacks
 Security of mobile code
Scalability
Distributed systems operate effectively and efficiently at many different scales,

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

ranging from a small intranet to the Internet. A system is described as scalable if it

will remain effective when there is a significant increase in the number of resources
and the number of users. The number of computers and servers in the Internet has
increased dramatically.
The design of scalable distributed systems presents the following challenges:
Controlling the cost of physical resources: As the demand for a resource grows, it
should be possible to extend the system, at reasonable cost, to meet it.
Controlling the performance loss: Consider the management of a set of data whose
size is proportional to the number of users or resources in the system – for example,
the table with the correspondence between the domain names of computers and their
Internet addresses held by the Domain Name System, which is used mainly to look
up DNS names such as www.amazon.com.
Preventing software resources running out: An example of lack of scalability is
shown by the numbers used as Internet (IP) addresses (computer addresses in the
Internet). In the late 1970s, it was decided to use 32 bits for this purpose, the supply
of available Internet addresses is running out. For this reason, a new version of the
protocol with 128-bit Internet addresses is being adopted, and this will require
modifications to many software components.
Avoiding performance bottlenecks: In general, algorithms should be decentralized
to avoid having performance bottlenecks. We illustrate this point with reference to
the predecessor of the Domain Name System, in which the name table was kept in a
single master file that could be downloaded to any computers that needed it. That was
fine when there were only a few hundred computers in the Internet, but it soon
became a serious performance and administrative bottleneck.
Failure handling
Computer systems sometimes fail. When faults occur in hardware or software, programs
may produce incorrect results or may stop before they have completed the intended
computation. We shall discuss and classify a range of possible failure types that can
occur in the processes and networks that comprise a distributed system in Chapter 2.
Failures in a distributed system are partial – that is, some components fail while
others continue to function. Therefore the handling of failures is particularly
difficult. The following techniques for dealing with failures are discussed
throughout the book:
Detecting failures: Some failures can be detected. For example, checksums can be
used to detect corrupted data in a message or a file. Chapter 2 explains that it is
difficult or even impossible to detect some other failures, such as a remote crashed

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

server in the Internet. The challenge is to manage in the presence of failures that
cannot be detected but may be suspected.
Masking failures: Some failures that have been detected can be hidden or made less
severe. Two examples of hiding failures:
1. Messages can be retransmitted when they fail to arrive.
2. File data can be written to a pair of disks so that if one is corrupted, the
other may still be correct.
Tolerating failures: Most of the services in the Internet do exhibit failures – it would
not be practical for them to attempt to detect and hide all of the failures that might
occur in such a large network with so many components. Their clients can be
designed to tolerate failures, which generally involves the users tolerating them as
well. For example, when a web browser cannot contact a web server, it does not make
the user wait forever while it keeps on trying – it informs the user about the problem,
leaving them free to try again later. Services that tolerate failures are discussed in the
paragraph on redundancy below.
Recovery from failures: Recovery involves the design of software so that the state of
permanent data can be recovered or ‘rolled back’ after a server has crashed. In
general, the computations performed by some programs will be incomplete when a
fault occurs, and the permanent data that they update (files and other material stored
in permanent storage) may not be in a consistent state.
Redundancy: Services can be made to tolerate failures by the use of redundant
components. The design of effective techniques for keeping replicas of rapidly
changing data up- to-date without excessive loss of performance is a challenge.
Distributed systems provide a high degree of availability in the face of hardware
faults. The availability of a system is a measure of the proportion of time that it is
available for use. When one of the components in a distributed system fails, only the
work that was using the failed component is affected. A user may move to another
computer if the one that they were using fails; a server process can be started on
another computer.
Concurrency
Both services and applications provide resources that can be shared by clients in a
distributed system. There is therefore a possibility that several clients will attempt
to access a shared resource at the same time.
Transparency
Transparency is defined as the concealment from the user and the application
programmer of the separation of components in a distributed system, so that the

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

system is perceived as a whole rather than as a collection of independent

components. The implications of transparency are a major influence on the design of
the system software.
The ANSA Reference Manual [ANSA 1989] and the International Organization for
Standardization’s Reference Model for Open Distributed Processing (RM-ODP)
[ISO 1992] identify eight forms of transparency.
Access transparency enables local and remote resources to be accessed using
identical operations.
Location transparency enables resources to be accessed without knowledge of their
physical or network location (for example, which building or IP address).
Concurrency transparency enables several processes to operate concurrently using
shared resources without interference between them.
Replication transparency enables multiple instances of resources to be used to
increase reliability and performance without knowledge of the replicas by users or
application programmers.
Failure transparency enables the concealment of faults, allowing users and
application programs to complete their tasks despite the failure of hardware or
software components.
Mobility transparency allows the movement of resources and clients within a system
without affecting the operation of users or programs.
Performance transparency allows the system to be reconfigured to improve
performance as loads vary.
Scaling transparency allows the system and applications to expand in scale without
change to the system structure or the application algorithms.
The two most important transparencies are access and location transparency; their
presence or absence most strongly affects the utilization of distributed resources.
They are sometimes referred to together as network transparency.
Quality of service
Once users are provided with the functionality that they require of a service, such as
the file service in a distributed system, we can go on to ask about the quality of the
service provided. The main nonfunctional properties of systems that affect the
quality of the service experienced by clients and users are reliability, security and
performance. Adaptability to meet changing system configurations and resource
availability has been recognized as a further important aspect of service quality.

1.3 Distributed Shared Memory (DSM)

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

DSM is a mechanism that manages memory across multiple nodes and makes inter-
process communications transparent to end-users. The applications will think that
they are running on shared memory. DSM is a mechanism of allowing user
processes to access shared data without using inter-process communications. In
DSM every node has its own memory and provides memory read and writes
services and it provides consistency protocols. The distributed shared memory
(DSM) implements the shared memory model in distributed systems but it doesn’t
have physical shared memory. All the nodes share the virtual address space
provided by the shared memory model. The Data moves between the main
memories of different nodes.

Types of Distributed shared memory

On-Chip Memory:
 The data is present in the CPU portion of the chip.
 Memory is directly connected to address lines.
 On-Chip Memory DSM is expensive and complex.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

Bus-Based Multiprocessors:
 A set of parallel wires called a bus acts as a connection between CPU and
memory.
 accessing of same memory simultaneously by multiple CPUs is prevented by
using some algorithms
 Cache memory is used to reduce network traffic.

Ring-Based Multiprocessors:
 There is no global centralized memory present in Ring-based DSM.
 All nodes are connected via a token passing ring.
 In ring-bases DSM a single address line is divided into the shared area.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

Advantages of Distributed shared memory

1. Easy Abstraction - Since the address space is the same, data migration is not
an issue for programmers, making it simpler to build than RPC.
2. Easier Portability - Sequential to distributed system migration is made easy
by the access protocols employed in DSM. Because they make use of a
common programming interface, DSM programmes are portable.
3. Locality of data - Data that is being fetched in large blocks, or data that is
close to the memory address being fetched, may be needed in the future.
4. Larger memory space - Large virtual memory space is provided, paging
operations are decreased, and the total memory size is the sum of the memory
sizes of all the nodes.
5. Better Performance - It speeds up the data access in order to increase the
performance of the system.
6. Flexible communication Environment
7. Simplified process Migration - Since they all share the same address space,
moving one process to a different computer is simple
Some more basic advantages of the distributed shared memory (DSM) system are
listed below:
1. It is less expensive than using multiprocessing systems
2. Data access is done smoothly

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

3. It provides better scalability as several nodes can access the memory.

Disadvantages of the distributed shared memory:
1. Accessing is faster in a non-distributed shared memory system than in a
distributed system.
2. Simultaneous access to data has always been a topic of discussion. It should
ensure some additional protection to it.
3. It generates some programmer control over the actual message
4. In order to write correct programs, programmers must learn the consistency
model
5. It is not much efficient as the message-passing implementation as it uses the
asynchronous message-passing implementations

1.4 Message Passing

Message passing in distributed systems refers to the communication medium used by
nodes (computers or processes) to commute information and coordinate their actions.
It involves transferring and entering messages between nodes to achieve various
goals such as coordination, synchronization, and data sharing.

Message passing is a flexible and scalable method for inter-node communication in

distributed systems. It enables nodes to exchange information, coordinate activities,
and share data without relying on shared memory or direct method invocations.
Models like synchronous and asynchronous message passing offer different
synchronization and communication semantics to suit system requirements.
Synchronous message passing ensures sender and receiver synchronization, while
asynchronous message passing allows concurrent execution and non-blocking
communication.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

Types of Message Passing

 Synchronous message passing
 Asynchronous message passing
 Hybrids
1. Synchronous Message Passing
Synchronous message passing is a communication mechanism in existing
programming where processes or threads change messages in a synchronous manner.
The sender blocks until the receiver has received and processed the message,
ensuring coordination and predictable execution. This approach is generally enforced
through blocking method calls or procedure invocations, where a process or thread
blocks until the called system returns a result or completes its prosecution. This
blocking behavior ensures that the caller waits until the message is processed before
proceeding. However, synchronous message passing has potential downsides, such
as delays or halts in the system if the receiver takes too long to process the message
or gets stuck. To ensure the proper functioning of synchronous message passing in
concurrent systems, it’s crucial to precisely design and consider potential backups
and error handling.
2. Asynchronous Message Passing
Asynchronous message passing is a communication mechanism in concurrent and
distributed systems that enable processes or factors to change messages without
demanding synchronization in time. It involves sending a message to a receiving
process or component and continuing execution without waiting for a response. Key
characteristics of asynchronous message passing include its asynchronous nature,
which allows the sender and receiver to operate singly without waiting for a
response. Communication occurs through the exchange of messages, which can be
one-way or include a reply address for the receiver to respond. Asynchronous
message passing also allows for loose coupling between the sender and receiver, as
they can be running on separate processes, threads, or different machines.
Message buffering is frequently used in asynchronous message passing, allowing the
sender and receiver to operate at their own pace. Asynchronous message passing is
extensively used in scenarios like distributed systems, event-driven architectures,
message queues, and actor models, enabling concurrency, scalability, and fault
tolerance.

3. Hybrids

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

Hybrid message passing combines elements of both synchronous and asynchronous

message ends. It provides flexibility to the sender to choose whether to block and
hold on for a response or continue execution asynchronously. The choice between
synchronous or asynchronous actions can be made based on the specific
requirements of the system or the nature of the communication. Hybrid message
passing allows for optimization and customization based on different scenarios,
enabling a balance between synchronous and asynchronous paradigms.
Advantages of Message Passing Model
 Easier to implement.
 Quite tolerant of high communication latencies.
 Easier to build massively parallel hardware.
 It is more tolerant of higher communication latencies.
 Message passing libraries are faster and give high performance.
Disadvantages of Message Passing Model
 Programmer has to do everything.
 Connection setup takes time that’s why it is slower.
 Data transfer usually requires cooperative operations which can be difficult to
achieve.
 It is difficult for programmers to develop portable applications using this
model because message-passing implementations commonly comprise a
library of subroutines that are embedded in source code. Here again, the
programmer has to do everything on his own.
Difference between Shared Memory and Message Passing Model

S.No Shared Memory Model Message Passing Model

The shared memory region is used for A message passing facility is

1.
communication. used for communication.

It is used for communication between

It is typically used in a
processes on a single processor or
distributed environment where
multiprocessor systems where the
2. communicating processes reside
communicating processes reside on the
on remote machines connected
same machine as the communicating
through a network.
processes share a common address

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

S.No Shared Memory Model Message Passing Model

space.

No such code required here as

the message passing facility
The code for reading and writing the
provides mechanism for
data from the shared memory should
3. communication and
be written explicitly by the
synchronization of actions
Application programmer.
performed by the communicating
processes.

It provides a maximum speed of

It is time-consuming as message
computation as communication is done
passing is implemented through
4. through shared memory so system
kernel intervention (system
calls are made only to establish the
calls).
shared memory.

Here the processes need to ensure that It is useful for sharing small
5. they are not writing to the same amounts of data as conflicts need
location simultaneously. not to be resolved.

Relatively slower
6. Faster communication strategy.
communication strategy.

7. No kernel intervention. It involves kernel intervention.

It can be used in exchanging larger It can be used in exchanging

8.
amounts of data. small amounts of data.

Example- Example-
 Data from a client process may  Web browsers
9.
need to be transferred to a server  Web Servers
process for modification before  Chat program on WWW

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

S.No Shared Memory Model Message Passing Model

being returned to the client. (World Wide Web)

1.5 Programming Using the Message Passing

A message passing architecture uses a set of primitives that allows processes to
communicate with each other. i.e., send, receive, broadcast, and barrier.
Numerous programming languages and libraries have been developed for explicit
parallel programming. These differ in
 their view of the address space that they make available to the programmer,
 the degree of synchronization imposed on concurrent activities, and the
multiplicity of programs.
Some links; Scientific Applications on Linux, Parallel Programming Laboratory.
Principles of Message-Passing Programming
There are two key attributes that characterize the message-passing programming
paradigm.
1. the first is that it assumes a partitioned address space,
2. the second is that it supports only explicit parallelization.
Each data element must belong to one of the partitions of the space;
 hence, data must be explicitly partitioned and placed.
 Adds complexity, encourages data locality, NUMA architecture.
All interactions (read-only or read/write) require cooperation of two
processes (the process that has the data and the process that wants to access
the data).
 process that has the data must participate in the interaction,
 for dynamic and/or unstructured interactions, the complexity of the
code can be very high,
 primary advantage of explicit two-way interactions is that the
programmer is fully aware of all the costs of non-local interactions
more likely to think about algorithms (and mappings) that minimize
interactions.
The programmer is responsible for analyzing the underlying serial
algorithm/application.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

Identifying ways by which he or she can decompose the computations and

extract concurrency.
As a result, programming using the message-passing paradigm tends to be
hard and intellectually demanding.
However, on the other hand, properly written message-passing programs can
often achieve very high performance and scale to a very large number of
processes.

Structure of Message-Passing Programs

Message-passing programs are often written using the asynchronous or loosely
synchronous paradigms.
In the asynchronous paradigm, all concurrent tasks execute asynchronously.
 However, such programs can be harder and can have non-
deterministic behavior due to race conditions.
Loosely synchronous programs are a good compromise between two extremes.
 In such programs, tasks or subsets of tasks synchronize to perform
interactions.
 However, between these interactions, tasks execute completely
asynchronously.
 In its most general form, the message-passing paradigm supports execution of
a different program on each of the processes.
 This provides the ultimate flexibility in parallel programming, but makes the
job of writing parallel programs effectively unscalable.
 For this reason, most message-passing programs are written using the single
program multiple data (SPMD).
 In SPMD programs the code executed by different processes is identical
except for a small number of processes (e.g., the ”root” process).
 In an extreme case, even in an SPMD program, each process could execute a
different code (many case statements).
 But except for this degenerate case, most processes execute the same code.
 SPMD programs can be loosely synchronous or completely asynchronous.

The Building Blocks: Send and Receive Operations
Since interactions are accomplished by sending and receiving messages, the basic
operations in the message-passing programming paradigm are send and receive.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

In their simplest form, the prototypes of these operations are defined as follows:

– send buf points to a buffer that stores the data to be sent,

– recv buf points to a buffer that stores the data to be received,
– nelems is the number of data units to be sent and received,
– dest is the identifier of the process that receives the data,
– source is the identifier of the process that sends the data.

•Process P0 sends a message to process P1which receives and prints the message.
• The important thing to note is that process P0 changes the value of a to 0
immediately following the send.
•The semantics of the send operation require that the value received by process
P1must be 100 (not 0).
•That is, the value of a at the time of the send operation must be the value that is
received by processP1.
•It may seem that it is quite straight forward to ensure the semantics of the send and
receive operations.
•However, based on how the send and receive operations are implemented this may
not be the case.
•Most message passing platforms have additional hardware support for sending and
receiving messages.
•They may support DMA (direct memory access) and asynchronous message transfer
using network interface hardware.
•Network interfaces allow the transfer of messages from buffer memory to desired
location without CPU intervention.
•Similarly, DMA allows copying of data from one memory location to another (e.g.,
communication buffers) without CPU support (once they have been programmed).

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

•As a result, if the send operation programs the communication hardware and returns
before the communication operation has been accomplished, process P1 might
receive the value 0 in a instead of 100!

Blocking Message Passing Operations

•A simple solution to the dilemma presented in the code fragment above is for the
send operation to return only when it is semantically safe to do so.
•Note that this is not the same as saying that the send operation returns only after the
receiver has received the data.
•It simply means that the sending operation blocks until it can guarantee that the
semantics will not be violated on return irrespective of what happens in the program
subsequently.
•There are two mechanisms by which this can be achieved.
1. Blocking Non-Buffered Send/Receive
2. Blocking Buffered Send/Receive
3.
1. Blocking Non-Buffered Send / Receive
•The send operation does not return until the matching receive has been encountered
at the receiving process.
•When this happens, the message is sent and the send operation returns upon
completion of the communication operation.
•Typically, this process involves a handshake between the sending and receiving
processes (see Fig.1).
•The sending process sends a request to communicate to the receiving process.
•When the receiving process encounters the targets receive, it responds to the
request.
•The sending process upon receiving this response initiates a transfer operation.
•Since there are no buffers used at either sending or receiving ends, this is also
referred to as a non-buffered blocking operation.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

•Idling Overheads in Blocking Non-Buffered Operations: It is clear from the figure

that a blocking non-buffered protocol is suitable when the send and receive are
posted at roughly the same time (middle in the figure).

Figure1: Handshake for a blocking non-buffered send/receive operation.

•However, in an asynchronous environment, this may be impossible to predict.
•This idling over head is one of the major drawbacks of this protocol.
•Deadlocks in Blocking Non-Buffered Operations: Consider the following simple
exchange of messages that can lead to a deadlock:

•The code fragment makes the values of a available to both processes P0 and P1.
•However, if the send and receive operations are implemented using a blocking non-
buffered protocol,
 The send at P0 waits for the matching receive at P1
 whereas the send at process P1waits for the corresponding receive at
P0 ,
 resulting in an infinite wait.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

• Deadlocks are very easy in blocking protocols and care must be taken to
break cyclic waits.

2 Blocking Buffered Send / Receive

•A simple solution to the idling and deadlocking problems outlined above is to rely
on buffers at the sending and receiving ends.
•On a send operation, the sender simply copies the data into the designated buffer
and returns after the copy operation has been completed.
•The sender process can now continue with the program knowing that any changes to
the data will not impact program semantics.
•If the hardware supports asynchronous communication (independent of the CPU),
then a network transfer can be initiated after the message has been copied into the
buffer.

Figure 2: Blocking buffered transfer protocols: Left: in the presence of

communication hardware with buffers at send and receive ends; and Right: in
the absence of communication hardware, sender interrupts receiver and
deposits data in buffer at receiver end.

•Note that at the receiving end, the data cannot be stored directly at the target
location since this would violate program semantics.
•Instead, the data is copied into a buffer at the receiver as well.
•When the receiving process encounters a receive operation, it checks to see if the
message is available in its receive buffer. If so, the data is copied into the target
location.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

In Fig.2 Left, buffers are used at both sender and receiver and communication is
handled by dedicated hardware.
•Sometimes machines do not have such communication hardware.
•In this case, some of the overhead can be saved by buffering only on one side.
•For example, on encountering a send operation, the sender interrupts the receiver,
both processes participate in a communication operation and the message is
deposited in a buffer at the receiver end.
•When the receiver eventually encounters a receive operation, the message is copied
from the buffer into the target location.
•In general, if the parallel program is highly synchronous, non-buffered sends may
perform better than buffered sends.
•However, generally, this is not the case and buffered sends are desirable unless
buffer capacity becomes an issue.
•Impact of finite buffers in message passing; consider the following code fragment:

•In this code fragment, process P0 produces 1000 data items and process P1 consumes
them.
•However, if process P1 was slow getting to this loop, process P0 might have sent all
of its data.
•If there is enough buffer space, then both processes can proceed;
•However, if the buffer is not sufficient (i.e., buffer overflow), the sender would have
to be blocked until some of the corresponding receive operations had been posted,
thus freeing up buffer space.
•This can often lead to unforeseen overheads and performance degradation.
•In general, it is a good idea to write programs that have bounded buffer
requirements.
•Dead locks in Buffered Send and Receive Operations:
•While buffering relieves many of the deadlock situations, it is still possible to write
code that deadlocks.
•This is due to the fact that as in the non-buffered case, receive calls are always
blocking (to ensure semantic consistency).

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

•Thus, a simple code fragment such as the following deadlocks since both processes
wait to receive data but nobody sends it.
•Once again, such circular waits have to be broken.
•However, deadlocks are caused only by waits on receive operations in this case.

Non-Blocking Message Passing Operations

•In blocking protocols, the overhead of guaranteeing semantic correctness was paid
in the form of idling (non-buffered) or buffer management (buffered).
•It is possible to require the programmer
 to ensure semantic correctness,
 to provide a fast send / receive operation that incurs little overhead.
•This class of non-blocking protocols returns from the send or receive operation
before it is semantically safe to do so.
•Consequently, the user must be careful not to alter data that may be potentially
participating in communication.
•Non-blocking operations are generally accompanied by a check-status operation,
•Which indicates whether the semantics of a previously initiated transfer may be
violated or not.
•Upon return from a non-blocking operation, the process is free to perform any
computation that does not depend upon the completion of the operation.
•Later in the program, the process can check whether or not the non- blocking
operation has completed,
•and, if necessary, wait for its completion.
•Non-blocking operations can be buffered or non-buffered.
•In the non-buffered case, a process wishing to send data to another simply posts a
pending message and returns to the user program.
•The program can then do other useful work.
•At some point in the future, when the corresponding receive is posted, the
communication operation is initiated.
•When this operation is completed, the check status operation indicates
That it is safe to touch this data.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

• This transfer is indicated in Fig.3 Left.

• The benefits of non-blocking operations are further enhanced by the presence
of dedicated communication hardware.
• In this case, the communication overhead can be almost entirely masked by
non-blocking operations.
• However, the data being received is unsafe for the duration of the receive
operation.
• This is illustrated in Fig.3 Right.

Figure 3: Non-blocking non-buffered send and receive operations Left: in

absence of communication hardware; Right: in presence of communication
hardware.

•Comparing Figures 3Left and 1a, it is easy to see that the idling time when the
process is waiting for the corresponding receive in a blocking operation can now be
utilized for computation (provided it does not update the data being sent).
•This removes the major bottleneck associated with the former at the expense of
some program restructuring.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

•Typical message-passing libraries such as Message Passing Interface (MPI) and

Parallel Virtual Machine (PVM) implement both blocking and non-blocking
operations.
•Blocking operations facilitate safe and easier programming.
•Non-blocking operations are useful for performance optimization by masking
communication overhead.
•One must, however, be careful using non-blocking protocols since errors can result
from unsafe access to data that is in the process of being communicated.

Figure 4: Space of possible protocols for send and receive operations.

1.6 Group Communication in distributed Systems

Communication between two processes in a distributed system is required to
exchange various data, such as code or a file, between the processes. When one
source process tries to communicate with multiple processes at once, it is
called Group Communication. A group is a collection of interconnected processes

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

with abstraction. This abstraction is to hide the message passing so that the
communication looks like a normal procedure call. Group communication also helps
the processes from different hosts to work together and perform operations in a
synchronized manner, therefore increasing the overall performance of the system.

Types of Group Communication in a Distributed System:

Broadcast Communication: When the host process tries to communicate with every
process in a distributed system at same time. Broadcast communication comes in
handy when a common stream of information is to be delivered to each and every
process in most efficient manner possible. Since it does not require any processing
whatsoever, communication is very fast in comparison to other modes of
communication. However, it does not support a large number of processes and
cannot treat a specific process individually.

A broadcast Communication: P1 process communicating with every process in

the system
Multicast Communication: When the host process tries to communicate with a
designated group of processes in a distributed system at the same time. This
technique is mainly used to find a way to address problem of a high workload on
host system and redundant information from process in system. Multitasking can
significantly decrease time taken for message handling.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

A multicast Communication: P1 process communicating with only a group of

the process in the system
Unicast Communication: When the host process tries to communicate with a
single process in a distributed system at the same time. Although, same
information may be passed to multiple processes. This works best for two
processes communicating as only it has to treat a specific process only. However,
it leads to overheads as it has to find exact process and then exchange
information/data.

A unicast Communication: P1 process communicating with only P3 process

1.7 Case Study - Remote Procedure Call
Remote Procedure Call (RPC) is a communication technology that is used by one
program to make a request to another program for utilizing its service on a
network without even knowing the network’s details. A function call or a
subroutine call are other terms for a procedure call.

It is based on the client-server concept. The client is the program that makes the
request, and the server is the program that gives the service. An RPC, like a local

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

procedure call, is based on the synchronous operation that requires the requesting
application to be stopped until the remote process returns its results. Multiple
RPCs can be executed concurrently by utilizing lightweight processes or threads
that share the same address space. Remote Procedure Call program as often as
possible utilizes the Interface Definition Language (IDL), a determination
language for describing a computer program component’s Application
Programming Interface (API). In this circumstance, IDL acts as an interface
between machines at either end of the connection, which may be running
different operating systems and programming languages.

Working Procedure for RPC Model:

 The process arguments are placed in a precise location by the caller when the
procedure needs to be called.
 Control at that point passed to the body of the method, which is having a
series of instructions.
 The procedure body is run in a recently created execution environment that
has duplicates of the calling instruction’s arguments.
 At the end, after the completion of the operation, the calling point gets back
the control, which returns a result.
 The call to a procedure is possible only for those procedures that are not
within the caller’s address space because both processes (caller and callee)
have distinct address space and the access is restricted to the caller’s
environment’s data and variables from the remote procedure.
 The caller and callee processes in the RPC communicate to exchange
information via the message-passing scheme.
 The first task from the server-side is to extract the procedure’s parameters
when a request message arrives, then the result, send a reply message, and
finally wait for the next call message.
 Only one process is enabled at a certain point in time.
 The caller is not always required to be blocked.
 The asynchronous mechanism could be employed in the RPC that permits the
client to work even if the server has not responded yet.
 In order to handle incoming requests, the server might create a thread that
frees the server for handling consequent requests.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

Working Procedure for RPC Model

Types of RPC:
Callback RPC: In a Callback RPC, a P2P (Peer-to-Peer)paradigm opts between
participating processes. In this way, a process provides both client and server
functions which are quite helpful. Callback RPC’s features include:
 The problems encountered with interactive applications that are handled
remotely
 It provides a server for clients to use.
 Due to the callback mechanism, the client process is delayed.
 Deadlocks need to be managed in callbacks.
 It promotes a Peer-to-Peer (P2P) paradigm among the processes involved.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

RPC for Broadcast: A client’s request that is broadcast all through the network and
handled by all servers that possess the method for handling that request is known as a
broadcast RPC. Broadcast RPC’s features include:
 You have an option of selecting whether or not the client’s request message
ought to be broadcast.
 It also gives you the option of declaring broadcast ports.
 It helps in diminishing physical network load.
Batch-mode RPC: Batch-mode RPC enables the client to line and separate RPC
inquiries in a transmission buffer before sending them to the server in a single batch
over the network. Batch-mode RPC’s features include:
 It diminishes the overhead of requesting the server by sending them all at
once using the network.
 It is used for applications that require low call rates.
 It necessitates the use of a reliable transmission protocol.
Local Procedure Call Vs Remote Procedure Call:
 Remote Procedure Calls have disjoint address space i.e. different address
space, unlike Local Procedure Calls.
 Remote Procedure Calls are more prone to failures due to possible processor
failure or communication issues of a network than Local Procedure Calls.
 Because of the communication network, remote procedure calls take longer
than local procedure calls.
Advantages of Remote Procedure Calls
 The technique of using procedure calls in RPC permits high-level languages
to provide communication between clients and servers.
 This method is like a local procedure call but with the difference that the
called procedure is executed on another process and a different computer.
 The thread-oriented model is also supported by RPC in addition to the
process model.
 The RPC mechanism is employed to conceal the core message passing
method.
 The amount of time and effort required to rewrite and develop the code is
minimal.
 The distributed and local environments can both benefit from remote
procedure calls.
 To increase performance, it omits several of the protocol layers.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

 Abstraction is provided via RPC. To exemplify, the user is not known about
the nature of message-passing in network communication.
 RPC empowers the utilization of applications in a distributed environment.

Disadvantages of Remote Procedure Calls:

 In Remote Procedure Calls parameters are only passed by values as pointer
values are not allowed.
 It involves a communication system with another machine and another
process, so this mechanism is extremely prone to failure.
 The RPC concept can be implemented in a variety of ways, hence there is no
standard.
 Due to the interaction-based nature, there is no flexibility for hardware
architecture in RPC.
 Due to a remote procedure call, the process’s cost has increased.

1.8 Remote Method Invocation

RMI stands for Remote Method Invocation. It is a mechanism that allows an object
residing in one system (JVM) to access/invoke an object running on another JVM.
RMI is used to build distributed applications; it provides remote communication
between Java programs. It is provided in the package java.rmi.
Architecture of an RMI Application
In an RMI application, we write two programs, a server program (resides on the
server) and a client program (resides on the client).
 Inside the server program, a remote object is created and reference of that
object is made available for the client (using the registry).
 The client program requests the remote objects on the server and tries to
invoke its methods.
The following diagram shows the architecture of an RMI application.
Let us now discuss the components of this architecture.
 Transport Layer − This layer connects the client and the server. It manages
the existing connection and also sets up new connections.
 Stub − A stub is a representation (proxy) of the remote object at client. It
resides in the client system; it acts as a gateway for the client program.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

 Skeleton − This is the object which resides on the server

side. stub communicates with this skeleton to pass request to the remote
object.
 RRL(Remote Reference Layer) − It is the layer which manages the
references made by the client to the remote object.

Working of an RMI Application

The following points summarize how an RMI application works −
 When the client makes a call to the remote object, it is received by the stub
which eventually passes this request to the RRL.
 When the client-side RRL receives the request, it invokes a method
called invoke() of the object remoteRef. It passes the request to the RRL on
the server side.
 The RRL on the server side passes the request to the Skeleton (proxy on the
server) which finally invokes the required object on the server.
 The result is passed all the way back to the client.
Marshalling and Unmarshalling
Whenever a client invokes a method that accepts parameters on a remote object, the
parameters are bundled into a message before being sent over the network. These
parameters may be of primitive type or objects. In case of primitive type, the
parameters are put together and a header is attached to it. In case the parameters are
objects, then they are serialized. This process is known as marshalling.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.
 [Unit III - INTRODUCTION TO DISTRIBUTED SYSTEMS]

At the server side, the packed parameters are unbundled and then the required
method is invoked. This process is known as unmarshalling.
RMI Registry
RMI registry is a namespace on which all server objects are placed. Each time the
server creates an object, it registers this object with the RMI registry
(using bind() or reBind() methods). These are registered using a unique name
known as bind name.
To invoke a remote object, the client needs a reference of that object. At that time,
the client fetches the object from the registry using its bind name
(using lookup() method).
The following illustration explains the entire process −

Goals of RMI
Following are the goals of RMI −
 To minimize the complexity of the application.
 To preserve type safety.
 Distributed garbage collection.
 Minimize the difference between working with local and remote objects.

Prepared By: Dr. K. Suresh, Assistant Professor / CSE Department, KPRIET, Coimbatore.