5 - Communication Primitives
5 - Communication Primitives
Primitives
1
Process Communication
• The single most important difference between a distributed
system and a uniprocessor system is the interprocess
communication.
3
Process Communication
• A and B have to agree on the meaning of the bits being sent.
For example:
• How many volts should be used to signal a 0-bit? 1-bit?
• How does the receiver know which is the last bit of the message?
• How can it detect if a message has been damaged or lost?
• What should it do if it finds out?
• How long are numbers, strings, and other data items? And how are they
represented?
4
Types of Communication
Primitives
Widely used communication primitives in Distributed Operating
Systems
• Message Passing
• Using Send() and Receive() Primitives
• Remote Procedure Calls
• An extension of conventional procedure call (used
for transfer of
control and data within a single process)
5
Message Passing Model
• Two basic communication primitives
• Send() and Receive()
• Example:
send(&x, destination_id)
receive(&y, source_id)
• Found in Client/Server computing models
6
Message
passing
• Synchronous and asynchronous send routines
7
Client blocked
Client running
Client running
Message Trap to
Return from kernel,
process released
passing
kernel,
Process blocked
Client
blocked
Client running
Client running
Trap Return
9
Group communication routines
11
Group communication routines
12
Group communication routines
13
Example
14
Example
15
Remote
Procedure
• The idea behind RPCCall
is to make a remote procedure call look as much as
possible like a local one.
• A remote procedure call occurs in the following steps:
• The client procedure calls the client stub in the normal way.
• The client stub builds a message and traps to the kernel.
• The kernel sends the message to the remote kernel.
• The remote kernel gives the message to the server stub.
• The server stub unpacks the parameters and calls the server.
• The server does the work and returns the result to the stub.
• The server stub packs it in a message and traps to the kernel.
• The remote kernel sends the message to the client’s kernel.
• The client’s kernel gives the message to the client stub.
• The stub unpacks the result and returns to the client.
16
Remote Procedure Call
Client machine Client stub Server stub Server machine
Kernel Kernel
Kernel Kernel
Kernel Kernel
Kernel Kernel
Kernel Kernel
Kernel Kernel
• The server does the work and returns the Message transport
result to the stub. over the network
17
Remote Procedure Call
Client machine Client stub Server stub Server machine
Kernel Kernel
Message transport
• The server stub packs it in a message and
over the network
traps to the kernel. 17
Remote Procedure Call
Client machine Client stub Server stub Server machine
Kernel Kernel
Kernel Kernel
Kernel Kernel
19
RPC Communication
2
0
RPC System Components
• Message module
– IPC module of Send/Receive/Reply
– responsible for exchanging messages
• Client stub
– packs the arguments with the procedure name or ID into a
message
– sends the msg to the server and then awaits a reply msg
– unpacks the results and returns them to the client
• Server stub
– receives a request msg
– unpacks the arguments and calls the appropriate server
procedure
– when it returns, packs the result and sends a reply msg back to 22
RPC System Components and Call Flows
23
Parameter Passing
• little endian: bytes are numbered from right to left
(lsb at the lowest address)
The number is
2 1 0 LLIJ0005
0 3 0 0 5
7 6 5 4 In Little Endian
L L I J
LSB at the least address:
• Use native format and indicate in the first byte of the message which
format this is.
25
How are pointers passed?
• not to use pointers. Highly undesirable.
• copy the array into the message and send it to the server. When the
server finishes, the array can be copied back to the client.
27
Dynamic
Binding
•Server: exports the server interface.
•The server registers with a binder (a program), that is,
give the binder
its name, its version number, a unique identifier, and a
handle.
•The server can also deregister when it is no longer
prepared to offer service.
28
Remote Procedure Call
Binding
Server Register
Receive Services
Local Proc. Stub Query Stub Remote Proc.
Local Query
Return Execute
call binding Unpack
Server procedure
server
Address
Params Local
packing call
Wait
Pack
Return Unpack
results Return
result
Caller Callee
29
How the client locates the server?
• When the client calls one of the remote procedure “read” for the first time, the
client stub sees that is not yet bound to a server.
• The client stub sends message to the binder asking to import version 3.1 of the
file-server interface.
• The binder checks to see if one or more servers have already exported an
interface with this name and version number.
• If no server is willing to support this interface, the “read” call fails; else if a
suitable server exists, the binder gives its handle and unique identifier to
the client stub.
• The client stub uses the handle as the address to send the request message
to.
3
0
Advant
ages
• It can handle multiple servers that support the same interface
• The binder can spread the clients randomly over the servers to even the
load (load balancing for the server).
• It can also poll the servers periodically, automatically deregistering any
server that fails to respond, to achieve a degree of fault tolerance
• It can also assist in authentication. Because a server could specify: if it
only wished to be used by a specific list of users (clients).
Disadvantages
• the extra overhead of exporting and importing interfaces cost time.
31
Server Crashes
32
Server Crashes
• The server can crash before the execution or after the execution
• The client cannot distinguish these two.
• The client can:
• Wait until the server reboots and try the operation again (at least once
semantics).
• Zero or one execution can take place, if the remote procedure succeeds,
exactly one computation has taken place otherwise none (Exactly
once semantics)
• Gives up immediately and reports back failure (at most once
semantics). Guarantees that RPC has been carried out at most one time
• Client gets no help. Guarantees nothing (RPC may have been carried out
anywhere from 0 to a large number). Easy to implement.
32
Server Crashes
At-least-once call semantics: On timeout, the RPC layer on the client’s
machine resends the same request repeatedly until it gets a reply, in
order to be sure that the remote procedure is called at least once.
When an RPC returns, the client cannot tell how many times it has been
executed.
At-most-once call semantics: implemented by the “Never
retransmit" policy of the client.
Client Crashes
• If a client sends a request to a server and crashes before the server
replies, then a computation is active and no parent is waiting for
the result. Such an unwanted computation is called an orphan.
33
Problems with orphans
• They waste CPU cycles
• They can lock files or tie up valuable resources
• If the client reboots and does the RPC again, but the reply from the
orphan comes back immediately afterward, confusion can result
34
What to do with
orphans?
• Extermination: Before a client stub sends an RPC message, it makes a
log entry telling what it is about to do. After a reboot, the log
is checked and the orphan is explicitly killed off.
• Disadvantage: the expense of writing a disk record for every RPC; it may not
even work, since orphans themselves may do RPCs, thus
creating grandorphans or further descendants that are impossible to
locate.
• Reincarnation: Divide time into epochs between failures and delete
computations from old epochs . When a client reboots, it
broadcasts a message to all machines declaring the start of a
new epoch. When such a broadcast comes in, all remote
computations are killed.
35
What to do with
orphans?
• Gentle reincarnation: upon a new epoch broadcast, try to locate
owner first (delete only if no owner). when an epoch broadcast
comes in, each machine checks to see if it has any remote
computations, and if so, tries to locate their owner. Only if the
owner cannot be found is the computation killed.
• Expiration:Each RPC is given a standard amount of time, T, to do the
job. If it cannot finish, it must explicitly ask for another quantum.
On the other hand, if after a crash the server waits a time T
before rebooting, all orphans are sure to be gone.
• None of the above methods are desirable.
36
Implementation Issues
• The choice of the RPC protocol: connection-oriented
or connectionless protocol?
• General-purpose protocol or specifically designed protocol for RPC?
• Packet and message length
• Acknowledgements
• Flow control
overrun error: with some designs, a chip cannot accept two back-to-
back packets because after receiving the first one, the chip is
temporarily disabled during the packet-arrived interrupt, so it misses
the start of the second one.
37
How to deal with overrun
error?
• If the problem is caused by the chip being disabled temporarily while
it is processing an interrupt, a smart sender can insert a delay
between packets to give the receiver just enough time.
• If the problem is caused by the finite buffer capacity of the network
chip, say n packets, the sender can send n packets, followed by a
substantial gap.
38