Communication

Basics, RPC & RMI

CS403/534
Distributed Systems
Erkay Savas
Sabanci University

1
 Communication Models
1. Remote Procedure Call (RPC)
• Client/Server application
2. Remote Method Invocation (RMI)
• Distributed objects
3. Message-Oriented Middleware (MOM)
• High-level message-queuing
4. Streams
• Communication of continuous media (having timing
constraints)

2
 Layered Protocols (1)
application protocol
application 7

presentation protocol
presentation 6
session protocol
session 5
transport protocol
transport 4
network protocol
network 3
data link protocol
data link 2

physical protocol
physical 1

network

Layers, interfaces, and protocols in the OSI model. 3

 Layered Protocols (2)

data link layer header

network layer header
transport layer header
session layer header
presentation layer header
application layer header
data link
message layer trailer

Bits that actually appear on the network

A typical message as it appears on the network.

4
 Lower-Level Protocols (1)
• Physical layer:
– contains the specification and
implementations of bits, and their
transmission between sender and receiver.
– standardizing electrical, mechanical and
signaling interfaces
– Example: RS-232-C
• Data link layer:
– prescribes the transmission of a group of
bits, called frames, with error correction and
flow control.

5
 Data Link Layer: Example
time A B Event
0 data 0 A sends data message 0

1 data 0 B gets 0, sees bad checksum

2 cntrl 0 B complains about the checksum

3 cntrl 0 message arrives correctly

4 data 0 A re-transmits data 0

5 data 0 B finally gets data 0

Discussion between a receiver and a sender in the data

link layer.
6
 Lower-Level Protocols (2)
Network layer: describes how packets in a
network of computers are to be routed.
– Connectionless IP (Internet Protocol)
– Connection-oriented virtual channel in ATM
networks.

Observation: for many distributed systems, the

lowest-level interface (of any interest) is that of
network layer.

7
 Transport Layer
Important: The transport layer provides the
actual communication facilities for most
distributed systems.
Responsibility: deliver messages without loss;
– The messages are broken into smaller pieces, and each piece is
assigned a sequence number.
– if necessary retransmit messages.
Standard Internet protocols:
– TCP: connection-oriented, reliable, stream-oriented
– UDP: connectionless, unreliable (best-effort) datagram
communication; essentially IP with minor additions.

8
 Client-Server TCP

a) Normal operation of TCP

b) Transactional TCP (T/TCP) (for C/S interaction) 9
 Application Layer
Observation: Many application protocols are directly
implemented on top of transport protocols (bypassing
session and presentation layers); hence doing a lot of
application independent work.
News FTP WWW
Protocol NNTP FTP HTTP
Encoding 7-bit + 7-bit text + 8-bit binary 8-bit + content
MIME type
Naming Newsgroup Host + path URL
Distribution Push Pull Pull
Replication Flooding Caching + DNS tricks Caching + DNS
tricks
Security None (PGP) Username + password Username +
password
10
 Middleware Layer
Middleware is invented to provide common services
and protocols that can be used by many different
applications:
– A rich set of communication protocols
– Marshalling and unmarshalling data
– Naming protocols, so that different applications can
easily share resources
– Security protocols
– Scaling mechanisms (support for replication & caching)
– Commit protocols
– Locking protocol
– Reliable multicasting
– Many others
11
 Middleware Protocols Application
independent

application protocol
application 6

middleware protocol
Middleware 5
transport protocol
transport 4
network protocol
network 3
data link protocol
data link 2

physical protocol
physical 1

network
An adapted reference model for networked communication.
12
 Remote Procedure Call (RPC)
The procedures such as send and receive do
not conceal the communication
More sophisticated is allowing programs to call
procedures located on other machines.

• Basic RPC operation

• Parameter passing
• Variations

13
 Conventional Procedure Call
int read (int fd, char *buf, int bytes);

Main program’s Main program’s

local variables local variables
Call-by-value
bytes
SP buf Call-by reference
fd
return address
Call-by-copy/restore
procedure read’s
local variables

SP
(a) (b)

a) Parameter passing in a local procedure call: the stack before the

call to read 14
b) The stack while the called procedure is active
 Local Procedure Call Principles
1. Push parameter values of the procedure on
stack
2. Call procedure
3. Use stack for local variables
4. Pop results (parameters)

Principle: “communication” with local procedure is

handled by copying data to/from the stack.

15
 Basic RPC Operation
Goal: to make a remote procedure call look as
much as possible like a local one
Observations:
– Application developers are familiar with simple local
procedure call model
– Well-engineered procedures operate in isolation
(black box)
– There is no fundamental reason not to execute
procedures on a remote machine

Conclusion: communication between caller & callee can be

hidden by using procedure-call mechanism

16
 Client and Server Model

Principle of RPC between a client and server program.

17
 Key Components in RPC
Client stub:
A piece of code that transforms the procedure
call to a request message that is to be delivered
to the receiver by the underlying network
services
– It blocks until the reply comes back

Server stub:
A piece of code that transforms requests coming
in over the network into local procedure calls
– The server stub is blocked waiting for incoming
messages.
18
 Steps of a Remote Procedure Call
1. Client procedure calls client stub in normal way
2. Client stub builds message, calls local OS
3. Client's OS sends message to remote OS
4. Remote OS gives message to server stub
5. Server stub unpacks parameters, calls server
6. Server does work, returns result to the stub
7. Server stub packs it in message, calls local OS
8. Server's OS sends message to client's OS
9. Client's OS gives message to client stub
10. Stub unpacks result, returns to client
19
 RPC: Parameter Passing
client machine server machine

server process
client process implementation of add
k = add(i, j)
k = add(i, j)

proc: “add” proc: “add”

int: val(i) int: val(i)
int: val(j) int: val(j)
client stub server stub
client OS client OS

Steps involved in doing remote computation through RPC 20

 RPC: Parameter Marshalling
Parameter marshalling: Packing parameters into a
message. However, There is more than just wrapping
parameters into a message:
– Client and server machines may have different data
representations (think of byte ordering)
– Wrapping a parameter means transforming it into a sequence of
bytes
– Client and server have to agree on the same encoding:
• How are basic data values represented (integers, floats,
characters)
• How are complex data values represented (arrays, unions)
– Client and server need to properly interpret messages,
transforming them into machine-dependent representations

21
 Passing Value Parameters
Two parameters to the RPC:
• int 5;
• string “VELI”;
address increases

5 5 0
0 0 0
0 0 0
0 0 5
V V I
E E L
L L E
I I V

a) Pentium b) on SPARC c) after

after inversion
receipt 22
Something to Think About

How to pass pointers?

23
 RPC Protocol

• RPC protocol governs two things:

1. The format of the parameters and return
values
2. The actual communication protocol the
caller & the callee agree
• For example, TCP/IP or UDP

• Once the protocol is defined, clients and

server stubs needs to be implemented.

24
 Interface
• An interface consists of a collection of procedures that
can be called by a client, and which are implemented by
a server.
• An interface is generally available in the same
programming language as the one in which the client or
server is written (not necessarily though).
• It, often, looks like a ANSI function declaration
• Interfaces are often specified by means of an
Interface Definition Language (IDL).
• An interface specified in an IDL, is then subsequently
compiled into a client and server stub (e.g. rmic
generates client stub given an interface in Java RMI)

25
 Variations: Asynchronous RPC
Essence: client needs not to block when there is nothing
to return.

2-12

a) traditional RPC
b) asynchronous RPC 26
 Asynchronous RPC with Return Value
Deferred synchronous RPC: A client and server
interacting through two asynchronous RPCs

One-way RPC: Client does not even wait for an

acknowledgement of server’s acceptance of the request.
Naturally, unreliable. 27
 RPC in practice
• Goal: Let the developer concentrate on only the
client- and server-specific code; let the RPC
system (generators and libraries) do the rest.
• Distributed Computing Environment (DCE) is
developed by OSF (The Open Group).
• DCE is a true middleware system.
• It uses client-server model.
• DCE Services:
– Distributed file service
– Directory service (to keep track of resources)
– Security service (for protection of resources)
– Distributed time service (for global synchronization of system
clocks) 28
 Reading Assignments

1. Read the pages 77-88 of the textbook for

Doors

2. Read the pages 80-85 of the textbook

for DCE RPC

29
 Remote Method Invocation

• Distributed objects
• Remote method invocation
• Parameter passing

30
 Distributed Objects (1)
• Data(state) and operations are encapsulated in an
object.
• Operations on object data are implemented as methods,
and are accessible through interfaces
• Object offers only interfaces to its clients
• Object server is responsible for a collection of objects
• Client stub (proxy) implements interface on the client
side
• Server stub (skeleton) handles un/marshalling and
method invocation (probably returning results)
• The state of the objects are not distributed
remote
objects
31
 Distributed Objects (2)

Common organization of a remote object with client-

side proxy 32
 Object References
Systemwide object reference: Having an object
reference allows a client to bind to a remote object:
– Binding is the resolving the object reference to an
actual object
– Distributed objects support systemwide object
references
– Object references can be freely passed between
processes, for example, as parameter to method
invocations.
– Reference denotes a server, object, and communication
protocol
– Binding results in a proxy being placed in the process’s
address space
– Client loads associated stub code
– Stub is instantiated and initialized for specific object
33
 Client-to-Object Binding: Implicit

distr_object * obj_ref; // Declare a systemwide object

// reference
obj_ref = …; // Initialize the reference to
// a distributed object
obj_ref->do_something(); // Implicitly bind and invoke
// a method

• Implicit binding: invoke methods directly on

the referenced object

34
 Client-to-Object Binding: Explicit
distr_object* obj_ref; // Declare a systemwide
// object reference
local_object* obj_ptr; // Declare a pointer to
// local objects
obj_ref = …; // Initialize the
// reference to a
// distributed object
obj_ptr = bind(obj_ref); // Explicitly bind and
// obtain a pointer to the
// local proxy
obj_ptr->do_something(); // Invoke a method on the
// local proxy

• Explicit binding: Client must first explicitly bind to

object before invoking its method
35
 Implementing Object References (1)
• Object reference must contain sufficient
information for a client to bind to an object.
• A simple object reference would include
network address of the machine and endpoint
(port) + which object.
• Not good if the machine crashes or the object
relocates.
• A local daemon per machine listening to a well-
known port and that keeps track of server-to-
endpoint assignments
– only good when the endpoint is changed
– server ID in object reference is necessary
36
 Implementing Object References (2)
• Solution:
• A location server keeps track of the machine
where an object’s server is currently running
– Object reference contains the network address of
the location server + systemwide identifier for the
server

37
 Implementing Object References (3)

• Object reference may even contain protocol

information
– TCP or UDP,
– protocols for marshalling parameters.
• Reference may contain an URL pointing to an
implementation file (implementation handle)
– May refer to a complete implementation of a proxy
that the client can dynamically load
– We need only a standard protocol for dynamically
loading, installing and subsequently instantiating code.
– Flexible
38
 Remote Method Invocation
Static RMI: Uses a predefined interface definitions
(Java)
– Interfaces of an object are known when the client application is
being developed.
– If interfaces change, the client application must be recompiled
– Object-specific stubs are used.
Dynamic RMI: Application selects at runtime which
method it will invoke at a remote object
– Interfaces can be inspected at runtime, method invocation can
be dynamically constructed.
– Application must be developed to support any possible interface.
– Dynamic invocation look like

invoke (object, method, input_parameters,

output_parameters);

39
 Dynamic vs. Static RMI
Example:

Static Invocation:
fobject.append(int i)

Dynamic Invocation:
invoke(fobject, id(append), int i)

40
 Parameter Passing (1)
Object as a parameter: Two cases:
• Object-by-reference: when it refers to a remote
object, an object reference is passed when invoking a
remote method.
• Object-by-value: reference refers to a local object
– It is in the address space of the client
– The state and the methods of the object are to be marshaled
– Server unmarshalls the object; creating a copy of the original
object
• This distinction may lead to some problems due to the
transparency
• In Java, reference to a local object and reference to a
remote object are of different data types. But similarly
treated. 41
 Parameter Passing (2)
Machine A Machine B
Local remote
ref L1 Local object ref R1 Remote object
O1 O2

client code with Machine C copy of R1

RMI to server at C to O2
(proxy) Copy of O1

Remote invocation
with L1 and R1 as server code
parameters (method imp)
• The situation when passing an object by reference or
by value 42
 Java RMI: Object model
• Distributed objects are integral part of the language
• Distributed object is defined as remote object, whose
state resides on a single remote machine
• Interfaces are implemented by means of a proxy
• A proxy appear as a local object in the client’s address
space
• Remote objects cannot be cloned (neither their proxies
at each client) (can be cloned only by the server)
• Each object can be constructed as a monitor, by
declaring a method to be synchronized
– Access to object’s internal data is completely serialized.
– protecting remote objects through synchronized methods is
generally not possible.
43
 Java RMI: Remote Object Invocation
• Any primitive or object type can be passed as a
parameter to an RMI, provided that the type can be
marshaled
• In Java terminology, objects must be serializable
• Platform dependent objects (e.g. file descriptors,
sockets) are not serializable
• Local objects are passed by value, remote objects by
reference
• Object reference contains
– network address,
– endpoint of the server
– local identifier for the actual object in the server’s address
space
– protocol stack for communication
44
 Components of Remote Object
• Each object in Java is an instance of a class which
contains an implementation of one or more interfaces
• Server-class: implementation of server-side code
– Contains implementation of the object (description of the
object’s state, implementation of the methods on the state)
– Server side stub, skeleton, is generated from the interface
specification of the object (obsolete now)
• Client-class: implementation of client-side code
– Contains the client-side stub, called proxy, generated from the
interface specification of the remote object
– Remote object reference (i.e. network address of the server,
the port number etc.) is always stored as part of the state of a
proxy.
– Proxies are serializable since they are, in essence, local objects
45
 Marshalling a Proxy
• A proxy can be used as a reference to a remote
object
• Marshalling a proxy may be inefficient since it
can lead to very large references
• In Java, when marshalling a proxy, an
implementation handle is generated
• Implementation handle specifies precisely which
classes are needed to construct the proxy
• Some of these classes may first need to be
downloaded from a remote site
• Since Java code is completely portable passing
proxies is possible
46
 Java RMI: Example (1)
---- Hello.java File (interface) ------

import java.rmi.Remote;
import java.rmi.RemoteException;

public interface Hello extends Remote {

String sayHello() throws RemoteException;
}

47
 Java RMI: Example (2)
--- HelloImpl.java File ------

import java.rmi.Naming;
import java.rmi.RemoteException;
import java.rmi.RMISecurityManager;
import java.rmi.server.UnicastRemoteObject;

public class HelloImpl

extends UnicastRemoteObject implements Hello {

public HelloImpl() throws RemoteException {

super();
}
public String sayHello() {
return "Hello World!";
}

48
 Java RMI: Example (3)
--- HelloImpl.java File (cont.) ------
public static void main(String args[]) {
// Create and install a security manager
if (System.getSecurityManager() == null) {
System.setSecurityManager(new RMISecurityManager());
}
try {
HelloImpl obj = new HelloImpl();
// Bind this object instance to the name
// "HelloServer"
Naming.rebind("//localhost/HelloServer", obj);

System.out.println("HelloServer bound in registry");

} catch (Exception e) {
System.out.println("HelloImpl err: " + e.getMessage());
e.printStackTrace();
} } }
49
 Java RMI: Example (4)
---- HelloImpl_stub.java File ------
// Stub class generated by rmic, do not edit.
// Contents subject to change without notice.
public final class HelloImpl_Stub
extends java.rmi.server.RemoteStub
implements hello.Hello, java.rmi.Remote
{
private static final long serialVersionUID = 2;
private static java.lang.reflect.Method $method_sayHello_0;
static {
try {
$method_sayHello_0 =
hello.Hello.class.getMethod("sayHello", new
java.lang.Class[] {});
} catch (java.lang.NoSuchMethodException e) {
throw new java.lang.NoSuchMethodError(
"stub class initialization failed");
}}
50
 Java RMI: Example (5)
---- HelloImpl_stub.java File (cont.)------
// constructors
public HelloImpl_Stub(java.rmi.server.RemoteRef ref) {
super(ref);
}
// methods from remote interfaces implementation of sayHello()
public java.lang.String sayHello()
throws java.rmi.RemoteException
{
try {
Object $result = ref.invoke(this, $method_sayHello_0, null,
6043973830760146143L);
return ((java.lang.String) $result);
} catch (java.lang.RuntimeException e) {
throw e;
} catch (java.rmi.RemoteException e) {
throw e;
} catch (java.lang.Exception e) {
throw new java.rmi.UnexpectedException("undeclared checked
exception", e);
}}} 51
 Java RMI: Example (6)
---- HelloClient.java File (Client-side code) ------

import java.rmi.*;
import java.rmi.registry.*;
import java.rmi.server.*;
import java.util.Date;

public class HelloClient

{

public static void main(String[] argv) {

String serverName = "";
String message = "blank";

Hello obj = null;

52
 Java RMI: Example (7)
---- HelloClient.java File (cont.) ------

System.setSecurityManager(new RMISecurityManager());
if (argv.length != 1) {
try {
serverName =java.net.InetAddress.getLocalHost().getHostName();
}
catch(Exception e) {
e.printStackTrace();
}
}
else {
serverName = argv[0];
}
if (serverName == "") {
System.out.println("usage: java SimpleRMIClient <IP
address of host running RMI server>");
System.exit(0);
}
53
 Java RMI: Example (8)
---- HelloClient.java File (cont.) ------
try {
//explicitly bind server object to object in client
obj = (Hello) Naming.lookup("//" + serverName + "/HelloServer");
//Print success message
System.out.println("RMI connection successful");
//Call method on server and put result into String message
message = obj.sayHello();
//Print message on server
System.out.println("Message on server is " + message);
}
catch(Exception e) {
System.out.println("Exception occurred: " + e);
System.exit(0);
}
}
}

54
 Java RMI: Example (9)
Server side:
– HelloImpl.java
– Hello.java (interface)
– HelloImpl_stub.java (needed for registry)
– Skeleton is deprecated in version 1.2

Client side:
– HelloClient.java (client object invoking remote
method)
– HelloImpl_stub.java (proxy) – can be downloaded
remotely
– Hello.java (interface)

55