Distribut

ed
Systems Chapter 2

Communication
in Distributed
systems
By Prof. Dr.
Amany Sarhan
(2025)
1
 Introduction
 The single most important difference between a
distributed system and a uniprocessor system is the
interposes communication.
 In a uniprocessor system, most interprocess
communication implicitly assumes the existence of
shared memory.
 In a distributed system there is no shared memory
whatsoever, so the entire nature of interprocess
communication must be completely rethought from
scratch.
Interprocessor Communication
 Where there are N processors each with its own
individual data stream i.e. SIMD and MIMD, it is usually
necessary to communicate data between processors.

2
3
 1. Using a SHARED MEMORY and
SHARED VARIABLES

 This consists of a global address space which is accessible by all N

processors.
 A processor can communicate to another by writing into the global
memory where the second processor can read it.
 Shared memory solves the interprocessor communication problem
but introduces the problem of simultaneous accessing of the same
location in the memory.

Shared
variables in
shared
memory

4
 How to communicate in a
Shared Memory?

 Here each processor has

access to any variable
residing in the shared
memory. (global variable vs
local variable???)

5
 Importance of
synchronizatio
n in shared
memory
access
 Consider. If x is a shared variable accessible by P1 and P2.
Depending on certain factors, x=1 or x=2 or x=3.
 if P1 executes and completes x=x+1 before P2 reads the value of
x from memory then x=3 similarly if P2 executes and completes
x=x+2 before P1 reads the value of x from memory then x=3
 if P1 and P2 read the value of x before either has updated it then
the processor which finishes last will determine the value of x.
 if P1 finishes last the value is x=1
 if P2 finishes last the value is x=2
 In a multiuser, real time environment the processor which
finishes last would vary from run to run - so the final value would
vary.(NON-DETERMINANCY ) 6
 How the processors gain access
to the same memory location
simultaneously?
 We can organize the read or write access to the
shared memory in two approaches:
1. Concurrent: all processors can perform read or write
at the same time
2. Exclusive: only one processor is allowed to read or
write at a time

 Conceptually, each of the several processors reading

from that location makes a copy of its contents and
stores it in its own register ( RAM )
 Problems arise however, with concurrent write
access.
7
 How the processors gain access
to the same memory location
simultaneously?
 If several processors are trying to simultaneously
store ( potentially different ) data at the same
address, which of them should succeed ?
i.e. we need a deterministic way of specifying the
contents of a memory location after a concurrent
write operation.
 Therefore depending on whether 2 or more
processors can gain access to the same memory
location simultaneously, we have 4 subclasses of
shared memory computers :-

8
 Possible read-write
configuratin
1. Exclusive Read, Exclusive Write (EREW) SM Computers
 Access to memory locations is exclusive i.e. no 2 processors
are allowed to simultaneously read from or write into the
same location.
2. Concurrent Read, Exclusive Write (CREW) SM Computers
 Multiple processors are allowed to read from the same
location but write is still exclusive. .i.e. no 2 processors are
allowed to write into the same location simultaneously
3. Exclusive Read, Concurrent Write (ERCW) SM Computers
 Multiple processors are allowed to write into the same
memory location but read access remains exclusive.
4. Concurrent Read, Concurrent Write (CRCW) SM Computers
 Both multiple read and multiple write privileges are allowed.

9
 Some ways of resolving write conflicts include :-

1. Assign priorities to the processors and accept value from

highest priority processor
2. All the processors are allowed to write, provided that the
quantities they are attempting to store are equal, otherwise
access is denied to ALL processors.
3. The max / min / sum / average of the value is stored (numeric
data ).

 Despite this, it is only feasible to allow P processors to access P

memory locations simultaneously for relatively small P (< 30 ).
Usually because the cost of the communication hardware is too
high and physical size of the device used for a memory location.
So shared memory SIMD machines are unrealistic and no
commercial machines exist with this design.

10
 2. INTERCONNECTION NETWORK and
MESSAGE PASSING
 Here each processor has its own private (local)
memory and there is no global, shared memory.
 Therefore, the processors need to be connected in
some way to allow communication of data to take
place.

11
Example:
 if a processor requires data contained in a separate
processor then it must be explicitly passed by using
instructions supplied for communication e.g. send and
receive functions.
 P1 P2
send (x, P1) receive (x,P2)
 the value of x is explicitly passed from P2 to P1.
 This is known as message passing.
 In addition to the extreme cases of shared memory and
distributed memory there are possibilities for hybrid
designs that combine features of both.
 E.g. clusters of processeors, where a high speed bus
serves for intracluster communication and an
interconnection network is used for intercluster
communication.

12
Architectural models of
multicomputers

13
 Architectural models of
multicomputers
 1- Client-server model
 2- Peer-to-Peer model
 3- possible variations to relief load of server
A- Multiple servers
B- Proxy servers
C- Mobile code
D- Mobile agents
E- Thin clients and computers servers
 Each architecture is differs in the following cc/s:
 The placement of components (data and workload distribution)
across the network of computers
 The interrelationship between the components (the role of each
one and pattern of communication)

14
15
 1- Client/Server
Model

 Many distributed systems today are constructed out of interacting

clients/servers. Most widely employed (used)
 Server:
 service software running on a single machine
 a process on a networked computer that accepts requests from other
(local or remote) processes to perform a service and responds
appropriately.
 Client:
 process that can invoke a service using a set of operations that forms its
client interface
 requesting process in the above is referred to as the client.
 Request and response are in the form of messages.
 Client is said to invoke an operation on the server.
16
 Client process interact with individual server processes
in separate host computers to access the shared
resources that the hosts manage.
 Servers may be clients for other servers: e.g. a web
server is a client of a local file server that manages the
files in which the web pages are stored, web servers and
most other internet services are clients for the database
server to access protected stored data.

17
Advantages of client
server model

18
 Client/server
Disadvantages
 Dependency on Central Server: Server-based networks
are dependent on a central server, which can cause
problems if the server goes down or becomes
overloaded.
 Higher Installation Cost: Server-based networks are
more expensive to install than peer-to-peer networks
because they require a dedicated server.
 Higher Maintenance Cost: Server-based networks
require more maintenance than peer-to-peer networks
because they have a dedicated server.

19
 2- Peer 2 Peer Model
 All the processes play similar roles, interacting cooperatively as
peers, without difference between server and client processes
or the computer they run on, to perform tasks.
 The problem with client/server is its scale up problem
 Centralizing the services at the server makes it overload under
large number of clients
 The overload value is determined by the server’s capability and
the network bandwidth.
 So, there is a need to distribute shared resources in order to
share computing and communication loads to large numbers of
computers and network links
 The hardware capacity of today’s PC is much larger than
yesterday's servers, so we can make use of these PC.
20
 The aim of P2P arch. is to make use of the resources
(data or HW) of participating computers to fulfill a
given task or activity.
 The most famous P2P instance is Napster 21
22
 P2P Disadvantages
 Unequal Resource Distribution: P2P networks can suffer
from unequal resource distribution, which means that
some peers may have more resources than others.
 Security Concerns: P2P networks are typically less secure
than client-server networks because security is managed
by individual computers, not on the network as a whole.

 Performance Limitations: P2P networks can suffer from

performance limitations because traffic is distributed
across multiple computers.

23
24
Examples of peer to
peer model from real
life

25
Napster

26
27
28
 Napster
 It is an application that facilitates the sharing of music and video
recordings among its users around the world.
Client
 Connect to a Napster server
 Upload list of music files that you want to share (names only, not
the files themselves!)
 Server maintains list of
<filename, ip_address, portnum> tuples
Search from a client:
 Send server keywords to search with
 Server searches its directory with the keywords
 Server returns a list of matching hosts - <ip_address, portnum>
tuples to client
 Client pings each host in the list to find transfer rates
 Client fetches file from best host 29
 Napster Structure
Store a directory, i.e.,
Filename Info about
filenames with peer
pointers
PennyLane.mp3 Beatles, @
128.84.92.23:1006
napster.com Servers
…..
S S

S
Client machines
(“Peers”)

P P

P P Store their own

P P files

30
31
 Gnutella
 Another protocol for managing peer communication
 Based on no central node approach
 Uses flooding idea for neighboring nodes for
searching

32
Gnutella is P2P system based on query flooding
• Peer Join: – On startup, a node (peer) contacts at least
one node
• Asks who its friends are. These become its “connected nodes”
• Search for file X: – Ask connected nodes. If they don’t know,
they will ask their connected nodes, and so on…
– Once/if the reply is found, it is returned to the sender

33
34
35
36
37
38
 Kaza
 Merges between directory-based idea and flooding-
based idea
 Having supernodes that is responsible for several
nodes
 Only supernodes communicate as flooding

39
Kazaa is a P2P system that uses Selective flooding of
queries
• Node Join – A peer contacts a supernode to join the system

Supernodes

nodes

40
• Publish files
– Peer sends a list of files to a supernode

41
Search
– Send a query to the supernode
– Supernodes flood the query to other
supernodes

42
43
 3-A Multiple
Severs
 Services may be implemented as several server processes
in separate host computers interacting as necessary to
provide service to client processes.
 The server may partition the set of objects on which the
service is based and distribute them between themselves
or they may make different copies (replicas) of the
objects on several host but they have to take care of any
changes made to them on any host.
 These two options are illustrated through the following
examples:
1- The web is a good example of partitioned data which is
distributed on different hosts with each server can
manage its own set of data files.
2- The NIS (Network Information Service) is an example of
replicated data. NIS is used by computers on a LAN to
check users login. A copy (replica) of usernames and
passwords is stored on each NIS server.
44
Partitioning and Replication on
multiple servers
PARTITION REPLICATE
Tasks (or workload) are partitioned The same tasks(services) are copied
and distributed to different servers. and the copies are distributed to
different servers.
Achieves speed up for task
execution in parallel. Achieves availability and fault
tolerance.
Also load balance on different
servers. Avoids contentions and delay on
one server, thus increases the
Some kind of fault tolerance (partial
throughput
fault)
Problems: - how to maintain the
Problems: - how to partition,
replicated data consistency.
number of servers, heterogeneous
server speed, workload balance.

45
Passive replication
Active replication 46
Task allocation
to servers

Services
partitioning to
servers

47
Cluster:
A group of standard, autonomous servers configured so
they appear on the network as a single machine (Single
system image)

48
 Service

Server
Client

Server

Client
Server

•A cluster is an example of multiple-server

architecture which is used for highly scalable web
services.
• Service processing can be partitioned or
replicated between them.
49
 Client-Server Architecture
Clients Clients
Service
Server
invocation invocation

Results Results

… invocation … invocation

Results Results

Single-Server Multiple-Server

50
51
52
 3-B Proxy Server
 Web proxy servers provide a shared cache of web resources
for client machines at a site or across several sites.
 A cache is a store of recently used data objects (for example:
recently visited web pages) that is closer to one client or a
particular set of clients than the objects themselves.
 When a new object is received at a computer it is added to
cache store. (if it is full, the object will replace other objects).
 When an object is needed by a client process, the caching
service first checks the cache and supplies it if it is available
and up-to-date.
 If not, an up-to-date object is fetched.
 Caches is located at a proxy server that can be shared by
several clients.

53
 Client Web
server
Proxy
server

Client Web
server

• The purpose of proxy servers is to increase

availability and performance of services by
reducing the load on WAN and web servers.
• Proxy servers can take on other roles, for
example, they may be used to access remote web
servers through firewall.

54
55
 3-C Mobile Code
 The user running a browser selects a link to a service whose code is stored on a
web server.
 The code is then downloaded to the user’s browser and runs on the user’s side.
 Examples: cookies, scripts, activex, java applets
 Sometimes it is called code migration. Why?
 Moving from heavily loaded to lightly loaded system (server and network).
a) client request results in the downloading of applet code

Client Web
server
Applet code

b) client interacts with the applet

Web
Client Applet
server
56
 Mobile Code Example:
Applet
 Moving code to data, rather than data to code.
 Advantage of mobile code running locally on the user’s
side is that it gives good interactive response since it does
not suffer from delays or variability of BW due to network
variable loads.
 An example of an applet usage that we are familiar with is
the up-to-date notifiers of new versions of services such as
virus scan programs.
 The code in the applet automatically communicates the
server and displays if there is new versions of the program.
 This is different than the normal way of interaction
between the client/server as the client should have
initiated the communication with the server each time.
 This type of interaction is called push model.

57
 Another example, a stockbroker might provide his
client with a service that automatically shows the
current status of his shares.
 Of course, the client has to download the special
applet that receives the updates from the broker’s
server and display them for the client.
 However, this rises the security problem !!!!!!
 As this applet is very dangerous to the local
resources of the client’s computer.
 Therefore browsers give applets limited access to
local resources such as not allowing the applet to
access local files, printers or network sockets.

58
 3-D Mobile agent
 Merges between DS and AI.
 A mobile agent is a running program (including code and
data this time) that travels from one computer to another in
a network carrying out a task on someone’s behalf (client)
and returning with the result to him.
 Examples of tasks are: collecting information or search.
 A mobile agent may use the local resources at each site
visited (for example accessing a DB).
 If we compare this model with a static client that remotely
invokes the resources that may include transferring data,
then we will find a mobile agent converts the remote
invocation into local invocation which means reduction in
communication cost and time.
59
60
 Mobile agents can be used to:
 - install and maintain SW on the computers within an
organization.
 - compare the prices of products from a number of
vendors by visiting the site of each vendor and
performing a series of DB operations.
 An old version of the mobile agent was presented at
Xerox called worm program.
 It was designed to make use of the idle computers in
order to carry out intensive computations.
 Just as the mobile code, the mobile agent rises the
problem of security of the local resources on each
visited computers.
 Thus, each of the visited computer that will receive the
mobile agent should decide which local resource it
should access according to the identity of the user
initiating this agent.
61
 The user’s identity must be included in a secure way
with the code and data of the mobile agent.
 We have to note that a mobile agent may fail to
complete its work if it is denied of access to
resources it needs.
 For this reason, the applicability of mobile agents
are limited.

62
 E Thin Client/Compute-3
Server
• Thin client = software layer that supports
windows-based user interface on a computer who is
local to the user while executing application
programs on a remote computer.
•Instead of downloading the code of applications
into the user’s computer, it runs them on a compute
server
•A Compute server is a powerful computer that has
the capacity to run large numbers of applications
simultaneously. Compute server
Network computer or PC
•It could be a multiprocessor or cluster computer.

Thin network Application

Client Process

63
 The main disadvantages of this model is when it is
used in highly interactive graphical activities (such
as CAD and image processing) where the user
suffers from delay due to the need to transfer image
and vector information between the thin client and
the application process at the server.

64