Chapter 4 Communication

Fundamentals
Layered Protocols

Open Systems Interconnection (OSI) Reference Model

Dadhi R. Ghimire [email protected] Patan Multiple Campus

Dadhi R. Ghimire [email protected] Patan Multiple Campus
Dadhi R. Ghimire [email protected] Patan Multiple Campus
Types of Communication

Dadhi R. Ghimire [email protected] Patan Multiple Campus

Remote Procedure Call
Issue:
 Many distributed systems have been based on explicit message exchange between processes.
 These procedures do not conceal communication mitigating access transparency in distributed
systems.

Solution:
 Allowing programs to call procedures located on other machines.
 When a process on machine A calls a procedure on machine B, the calling process on A is
suspended, and execution of the called procedure takes place on B.
 Information can be transported from the caller to the callee in the parameters and be returned in the
procedure result.
 No message passing is visible to the programmer.
 This method is known as Remote Procedure Call, or often just RPC.

Dadhi R. Ghimire [email protected] Patan Multiple Campus

 If the call is made from the main program, the stack will be as shown in Figure 1 (a) before the call.
 To make the call, the caller pushes the parameters onto the stack in order, last one first, as shown in
Figure 1 (b).
 After the read procedure has finished running, it puts the return value in a register, removes the return
address, and transfers control back to the caller.
 The caller then removes the parameters from the stack, returning the stack to the original state it had
before the call.

Figure 1: (a) Parameter passing in a local procedure call: the stack before the call to read and (b) The
stack while the called procedure is active.

Dadhi R. Ghimire [email protected] Patan Multiple Campus

Client and Server Stubs

Figure 2: Principle of RPC between a client and server program.

Dadhi R. Ghimire [email protected] Patan Multiple Campus

A remote procedure call occurs in the following steps:
1. The client procedure calls the client stub in the normal way.
2. The client stub acts a proxy and builds a message and calls the local operating system.
3. The client's OS sends the message to the remote OS (via TCP/UDP).
4. The remote OS gives the message to the server stub.
5. The server stub unpacks the parameters and calls the server.
6. The server does the work and returns the result to the stub.
7. The server stub packs it in a message and calls its local OS.
8. The server's OS sends the message to the client's OS.
9. The client's OS gives the message to the client stub.
10. The client stub unpacks the result and returns to the client.

Dadhi R. Ghimire [email protected] Patan Multiple Campus

 Figure 3: The steps involved in calling a remote procedure doit(a,b)

 When the message arrives at the server, the stub examines the message to see which procedure is
needed and then makes the appropriate call.
 The actual call from the stub to the server looks like the original client call, except that the parameters
are variables initialized from the incoming message.
 When the server has finished, the server stub gains control again.
 It takes the result sent back by the server and packs it into a message.
 This message is sent back to the client stub, which unpacks it to extract the result and returns the value
to the waiting client procedure.

Parameter Passing

Dadhi R. Ghimire [email protected] Patan Multiple Campus

 Figure 4: Copy/ Restore Mechanism

Message-Oriented Communication

Dadhi R. Ghimire [email protected] Patan Multiple Campus

 Figure 5: Connection-oriented communication pattern using sockets.

Message-oriented persistent communication

Dadhi R. Ghimire [email protected] Patan Multiple Campus

An important class of message-oriented middleware service is known as message-queuing systems, or just
Message-Oriented Middleware (MOM). Message-queuing systems provide extensive support for persistent
asynchronous communication. The essence of these systems is that they offer intermediate-term storage
capacity for messages, without requiring either the sender or receiver to be active during message
transmission. An important difference with sockets and MPI is that message-queuing systems are typically
targeted to support message transfers that are allowed to take minutes instead of seconds or milliseconds.

Message-queuing model

The basic idea behind a message-queuing system is that applications communicate by inserting messages in
specific queues. These messages are forwarded over a series of communication servers and are eventually
delivered to the destination, even if it was down when the message was sent. In practice, most
communication servers are directly connected to each other. In other words, a message is generally
transferred directly to a destination server. In principle, each application has its own private queue to which
other applications can send messages. A queue can be read only by its associated application, but it is also
possible for multiple applications to share a single queue.

An important aspect of message-queuing systems is that a sender is generally given only the guarantees that
its message will eventually be inserted in the recipient's queue. No guarantees are given about when, or
even if the message will actually be read, which is completely determined by the behavior of the recipient.

Figure 6: Four Cases of loosely coupled Communication using Queues

Messages can, in principle, contain any data. The only important aspect from the perspective of middleware
is that messages are properly addressed. In practice, addressing is done by providing a system wide unique
name of the destination queue. In some cases, message size may be limited, although it is also possible that
the underlying system takes care of fragmenting and assembling large messages in a way that is completely

Dadhi R. Ghimire [email protected] Patan Multiple Campus

transparent to applications. An effect of this approach is that the basic interface offered to applications can
be extremely simple, as shown in Figure 7.
The put operation is called by a sender to pass a message to the underlying system that is to be appended to
the specified queue. The get operation is a blocking call by which an authorized process can remove the
longest pending message in the specified queue. The process is blocked only if the queue is empty.
Variations on this call allow searching for a specific message in the queue, for example, using a priority, or
a matching pattern.

Figure7: Basic Interface to a queue in a message queueing system

Dadhi R. Ghimire [email protected] Patan Multiple Campus

Multicast Communication
Sending data to multiple receivers is known as multicast communication. For many years, this topic has
belonged to the domain of network protocols, where numerous proposals for network-level and transport-
level solutions have been implemented and evaluated. A major issue in all solutions was setting up the
communication paths for information dissemination. In practice, this involved a huge management effort,
in many cases requiring human intervention.

With the advent of peer-to-peer technology, and notably structured overlay management, it became easier
to set up communication paths. As peer-to-peer solutions are typically deployed at the application layer,
various application level multicasting techniques have been introduced.

Application Level Tree Based Multicasting

The basic idea in application-level multicasting is that nodes organize into an overlay network, which is
then used to disseminate information to its members. An important observation is that network routers are
not involved in group membership. As a consequence, the connections between nodes in the overlay
network may cross several physical links, and as such, routing call messages within the overlay may not be
optimal in comparison to what could have been achieved by network-level routing.

A crucial design issue is the construction of the overlay network. In essence, there are two approaches:
First, nodes may organize themselves directly into a tree, meaning that there is a unique (overlay) path
between every pair of nodes. An alternative approach is that nodes organize into a mesh network in which
every node will have multiple neighbors and, in general, there exist multiple paths between every pair of
nodes. The main difference between the two is that the latter generally provides higher robustness: if a
connection breaks (e.g., because a node fails there will still be an opportunity to disseminate information
without having to immediately reorganize the entire overlay network.

Flooding-based multicasting
Multicasting refers to sending a message to a subset of all the nodes, that is, a specific group of nodes. A
key design issue when it comes to multicasting is to minimize the use of intermediate nodes for which the
message is not intended. To make this clear, if the overlay is organized as a multi-level tree, yet only the
leaf nodes are the ones who should receive a multicast message, then clearly there may be quite some nodes
who need to store and subsequently forward a message that is not meant for them.

One simple way to avoid such inefficiency is to construct an overlay network per multicast group. As a
consequence, multicasting a message m to a group G is the same as broadcasting m to G. The drawback of
this solution is that a node belonging to several groups, will, in principle, need to maintain a separate list of
its neighbors for each group of which it is a member.

If we assume that an overlay corresponds to a multicast group, and thus that we need to broadcast a
message, a naive way of doing so is to apply flooding. In this case, each node simply forwards a message m
to each of its neighbors, except to the one from which it received m. Furthermore, if a node keeps track of
the messages it received and forwarded, it can simply ignore duplicates. We will roughly see twice as many
messages being sent as there are links in the overlay network, making flooding quite inefficient.

Dadhi R. Ghimire [email protected] Patan Multiple Campus

Gossip-based data dissemination
An important technique for disseminating information is to rely on epidemic behavior, also referred to as
gossiping. Observing how diseases spread among people, researchers have since long investigated whether
simple techniques could be developed for spreading information in very large-scale distributed systems.
The main goal of these epidemic protocols is to rapidly propagate information among a large collection of
nodes using only local information. In other words, there is no central component by which information
dissemination is coordinated. It is assumed that all updates for a specific data item are initiated at a single
node.

Epidemic algorithms are based on the theory of epidemics, which studies the spreading of infectious
diseases. In the case of large scale distributed systems, instead of spreading diseases, they spread
information. Designers of epidemic algorithms for distributed systems will try to “infect” all nodes with
new information as fast as possible.

A node that is part of a distributed system is called infected if it holds data that it is willing to spread to
other nodes. A node that has not yet seen this data is called susceptible. Finally, an updated node that is not
willing or able to spread its data is said to have been removed. Note that we assume we can distinguish old
from new data, for example, because it has been time-stamped or versioned. In this light, nodes are also
said to spread updates.

A popular propagation model is that of anti-entropy. In this model, a node P picks another node Q at
random, and subsequently exchanges updates with Q. There are three approaches to exchanging updates:

 P only pulls in new updates from Q

 P only pushes its own updates to Q
 P and Q send updates to each other (i.e., a push-pull approach)

In a pure push-based approach, updates can be propagated only by infected nodes. However, if many nodes
are infected, the probability of each one selecting a susceptible node is relatively small. Consequently,
chances are that a particular node remains susceptible for a long period simply because it is not selected by
an infected node.

In contrast, the pull-based approach works much better when many nodes are infected. In that case,
spreading updates is essentially triggered by susceptible nodes. Chances are big that such a node will
contact an infected one to subsequently pull in the updates and become infected as well. If only a single
node is infected, updates will rapidly spread across all nodes using either form of anti-entropy, although
push-pull remains the best strategy

Dadhi R. Ghimire [email protected] Patan Multiple Campus

Case Study: Java RMI (Remote Method Invocation) and Message passing Interface (Task for
Students)

Dadhi R. Ghimire [email protected] Patan Multiple Campus