Communication

Chapter 2

Chapter 2  Communication 1
 Communication
 Layered protocols
o Usual networking approach (client/server)
 Remote Procedure Call (RPC)
o Hide message passing details (client/server)
 Remote Method Invocation (RMI)
o Improved RPC (client/server)

Chapter 2  Communication 2
 Communication
 Message-Oriented Communications
o Message Passing  Low level, efficient
o Message-Oriented Middleware (MOM)  Non
client/server
 Streams
o Continuous flow subject to timing constraints

Chapter 2  Communication 3
 Layered Protocols

 OSI reference model

o Each layer provides service to layer above
o Implementation of service can change

Chapter 2  Communication 4
 Layer Services
 Transport layer
o Logical connection between hosts
o Reliable communication between hosts
 Network layer
o Route packet thru network
 Data link layer
o Get packet over each hop
 Physical layer
o Put the bits on the “wire”

Chapter 2  Communication 5
 Layered Protocols

 Layered message
o Add headers when msg sent (down protocol stack)
o Peel the onion when msg received (up the protocol stack)

Chapter 2  Communication 6
 Data Link Layer

 Communication at data link layer

o Above, A tries to send msgs 0 and 1 to B
Chapter 2  Communication 7
 Network Layer
 On a LAN
o Have a shared media
o Put the packet out, recipient picks it up
 On a WAN
o Have point-to-point communication
o Many possible routes
o Finding best route is difficult

Chapter 2  Communication 8
 Transport Layer
 UDP for unreliable delivery
o Better performance possible with UDP
 TCP for reliable delivery
o May be easier to build app with TCP

Chapter 2  Communication 9
 Client-Server TCP
Normal TCP Transactional TCP

Chapter 2  Communication 10
 Middleware Protocols

 Reference model for middleware based

distributed communication
Chapter 2  Communication 11
 What is Middleware?
 Logically at application layer
 General purpose protocols
 Independent of an application
 We’ll distinguish between
o High level application and
o Middleware

Chapter 2  Communication 12
 Middleware Example
 Often, must authenticate users
o Require users prove identity
 Spse you build authentication system
 Any app can use your auth system
 Your authentication “application”
o Is at application layer in OSI
o Is also at middleware layer in our view

Chapter 2  Communication 13
 Middleware
 Remainder of this chapter
 4 middleware communication services
o RPC
o RMI
o Message oriented communication
o Streaming

Chapter 2  Communication 14
 Remote Procedure Call
 Distributed systems can be built on explicit
message passing
o For example, send and receive
 What’s wrong with this approach?
o It’s not transparent to users
 Why should we care?
o Recall that transparency is one of primary goals
in distributed systems

Chapter 2  Communication 15
 Remote Procedure Call
 RPC is a simple idea
o Make remote operation seem like a (local)
procedure call
o “All the great things are simple”  Winston
Churchill
 Much better transparency compared to
primitive message passing
 Can we make remote operation seem local?

Chapter 2  Communication 16
 Conventional Procedure Call

 Consider C function: count = read(fd, buf, bytes)

a) Stack before call to read
b) Stack while called procedure is active
Chapter 2  Communication 17
 Parameter Passing
 Consider again
o C function: count = read(fd, buf, bytes)
 In C, parameters can be
o Passed by value: bytes
o Passed by reference: the array buf
 Usually not important whether pass by value
or pass by reference is used
 But it’s a big deal in RPC!
o Since procedure will execute at remote location

Chapter 2  Communication 18
 RPC between Client/Server

 We say that this is synchronous

o Since client waits for result

Chapter 2  Communication 19
 Stubs
 On client side, stub marshalls
parameters and send to server
o Pack parameters into message(s)
 On server side, stub converts to local
procedure call, sends back results
 Stubs increase transparency

Chapter 2  Communication 20
 Passing Value Parameters

 Suppose add(i,j) returns i + j

 Remote computation via RPC

Chapter 2  Communication 21
 Client Stub

a) Procedure
b) Stub marshalls params

Chapter 2  Communication 22
 Steps in RPC
1. Client procedure calls client stub in normal way
2. Client stub builds message, calls local OS
3. Client's OS sends message to remote OS
4. Remote OS gives message to server stub
5. Server stub unpacks parameters, calls server
6. Server does work, returns result to the stub
7. Server stub packs it in message, calls local OS
8. Server's OS sends message to client's OS
9. Client's OS gives message to client stub
10. Stub unpacks result, returns to client

Chapter 2  Communication 23
 Additional RPC Topics
 Doors
o Caller and sender on same machine
 Asynchronous RPC
o Client does something while server works
on procedure
 DCE RPC
o Specific implementation of RPC

Chapter 2  Communication 24
 Doors

 If client and server on same machine

o Use interprocess communication (IPC)
o More efficient than network protocols

Chapter 2  Communication 25
 The Doors
 Doors are not to be confused with
“The Doors”

Chapter 2  Communication 26
 Asynchronous RPC

a) Usual (synchronous) RPC

b) Asynchronous RPC

Chapter 2  Communication 27
 Asynchronous RPC

 Client and server interact via two

asynchronous RPCs
 More efficient, if applicable

Chapter 2  Communication 28
 DCE RPC
 Distributed Computing Environment (DCE)
 Read this section
 A couple of interesting items…
 DCE semantic options
o At-most-once  no call done more than once,
even if system crash
o Idempotent  calls can be repeated multiple
times (e.g., read)

Chapter 2  Communication 29
 DCE RPC

 Client-to-serverbinding in DCE
 Note directory service

Chapter 2  Communication 30
 RMI
 Remote Method Invocation
o Distributed objects
 Objects hide internals
o Provides transparency
o Also desirable in distributed systems
 RMI can increase transparency
compared to RPC
 Chapter 10 has real systems

Chapter 2  Communication 31
 Objects
 Object encapsulates data, the state
 Object encapsulated methods,
operations on the data
 Methods are made available thru well-
defined interfaces
 In distributed environment
o Interface can be on one machine and
o Corresponding object on another machine

Chapter 2  Communication 32
 Distributed Objects

 Interface on client
o Proxy  like client stub in RPC
 Object on server
o Skeleton  like server stub in RPC
Chapter 2  Communication 33
 Compile-time vs Runtime
 Compile-time objects
o Objects analogous to those in Java, C++
o Pluses: easy to implement
o Minuses: depends on specific language
 Runtime objects
o Implementation is open, use adapter
(wrapper) to hide implementation
o Plus and minus opposite of those above

Chapter 2  Communication 34
 RMI and Parameter Passing

 Makes sense to treat local and remote

objects differently
o Lots of overhead to remote objects
o Pass by reference gets complicated
Chapter 2  Communication 35
 Java RMI
 Distributed objects are an integral
part of Java
o Aims for high degree of transparency
o For example client proxy has same
interface as remote object
 There are subtle differences
between local and remote objects…

Chapter 2  Communication 36
 Java RMI
 Cloning
o Cloning a local object results in exact copy
o Only server can clone remote object
o In Java, proxies not cloned
o So must bind (again) to cloned object
 Can declare method to be synchronized
o Ensures access to data is serialized
 Blocking
o Clients blocked

Chapter 2  Communication 37
 Java RMI
 Read the details
 A preview of Chapter 5…
 Spse multiple clients want to access a
method on server (method is synchronized)
o Block all but one client  lots of overhead
o Block at the server  what if client crashes?
 Java restricts blocking to proxies
o Simplifies things
o But then can’t prevent simultaneous access of
remote objects simply by synchronized

Chapter 2  Communication 38
 Message-Oriented Comm.
 RPC and RMI enhance transparency
 But RPC and RMI are “inherently
synchronous”
 Consider an email system where
o Messages stored on email servers when
in transit and before read
o Stored locally after read
 Example of persistent communication

Chapter 2  Communication 39
 Message-Oriented Comm.
 In email example
o Sender need not continue executing
after sending msg
o Receiver need not be executing when
msg sent
 Comparable to the Pony Express!
 The more things change, the more
they stay the same…

Chapter 2  Communication 40
 Pony Express

 Persistent comm and the Pony Express

Chapter 2  Communication 41
Transient and Asynchronous
 Transient
o Msg is stored only as long as sender and
receiver are alive
o If msg can’t be delivered, discard it
 Asynchronous
o Sender does not wait for response
before continuing
 Recallpersistent and synchronous
 Four possible combinations…

Chapter 2  Communication 42
 Examples
 Transient asynchronous
o UDP
 Transient synchronous
o Synchronous RPC
 Persistent asynchronous
o email
 Persistent synchronous
o Msg can only be stored at receiving host

Chapter 2  Communication 43
 Persistence and Synchronicity

a) Persistent asynchronous communication

b) Persistent synchronous communication

Chapter 2  Communication 44
 Persistence and Synchronicity

c) Transient asynchronous communication

d) Receipt-based transient synchronous communication

Chapter 2  Communication 45
 Persistence and Synchronicity

e) Delivery-based transient synchronous communication

at message delivery
f) Response-based transient synchronous
communication

Chapter 2  Communication 46
 Message-Oriented Comm.
 Message-oriented systems take
transient asynchronous as baseline
o Like UDP
 But persistence sometimes needed
o Especially if geographically distributed
o Network or process failures likely
 Message passing like transport layer

Chapter 2  Communication 47
 Message-Oriented Comm.
 Transient
o Berkeley sockets
o Message Passing Interface (MPI)
 Persistent
o Message queuing model, MOM
o Message brokers

Chapter 2  Communication 48
 Berkeley Sockets
Primitive Meaning
Socket Create a new communication endpoint
Bind Attach a local address to a socket
Listen Announce willingness to accept connections
Accept Block caller until a connection request arrives
Connect Actively attempt to establish a connection
Send Send some data over the connection
Receive Receive some data over the connection
Close Release the connection

 Socket primitives for TCP/IP

Chapter 2  Communication 49
 Berkeley Sockets

 Connection-oriented communication pattern

using sockets
 Note “synchronization point”

Chapter 2  Communication 50
Message-Passing Interface (MPI)
Primitive Meaning
MPI_bsend Append outgoing message to a local send buffer
MPI_send Send a message and wait until copied to local or remote buffer
MPI_ssend Send a message and wait until receipt starts
MPI_sendrecv Send a message and wait for reply
MPI_isend Pass reference to outgoing message, and continue
Pass reference to outgoing message, and wait until receipt
MPI_issend
starts
MPI_recv Receive a message; block if there are none
MPI_irecv Check if there is an incoming message, but do not block

 A few (of the many) MPI primitives

 Emphasis here is on efficiency
 Big parallel machines use MPI
Chapter 2  Communication 51
Message-Passing Interface (MPI)
Primitive Meaning
MPI_bsend Append outgoing message to a local send buffer
MPI_send Send a message and wait until copied to local or remote buffer
MPI_ssend Send a message and wait until receipt starts
MPI_sendrecv Send a message and wait for reply
MPI_isend Pass reference to outgoing message, and continue
MPI_issend Pass reference to outgoing message, and wait until receipt starts
MPI_recv Receive a message; block if there are none
MPI_irecv Check if there is an incoming message, but do not block

 Transient asynchronous: MPI_bsend

 Transient synchronous: MPI_ssend
 “Stronger” form of synchronous: MPI_sendrecv
 Many more possibilities (read the book…)

Chapter 2  Communication 52
 Message Queuing
 Persistent
o Message-Queuing Systems or MOM
 Insert msgs into queues
o Delivered via a series of servers
o Can be delivered even if server down
o No guarantee msg will be delivered
o No assurance msg will be read, etc.
 For systems where communications takes
minutes instead of milliseconds

Chapter 2  Communication 53
 Message-Queuing Model

 Loosely-coupled communications using queues

Chapter 2  Communication 54
 Message-Queuing Model
Primitive Meaning

Put Append a message to a specified queue

Get Block until the specified queue is nonempty, and remove the first message

Poll Check a specified queue for messages, and remove the first. Never block.
Install a handler to be called when a message is put into the specified
Notify
queue.

 Simple interface to message-queuing system

o Put is non-blocking
o Get blocks only if queue is empty
o Poll is non-blocking form of Get

Chapter 2  Communication 55
 Message-Queuing System

 Addressing in message-queuing system

Chapter 2  Communication 56
 Message-Queuing System

 Routing in message-queuing system

Chapter 2  Communication 57
 Message Brokers

 Message broker in message-queuing system

 Translates between msg formats

Chapter 2  Communication 58
 Example: IBM MQSeries
 Read it

Chapter 2  Communication 59
 Streams
 Other methods based on more-or-less
independent units of data
o Timing does not affect correctness
 In multimedia, timing is critical
o Audio and video
o Can tolerate loss, but not “jitter”
 Temporal relationship is important

Chapter 2  Communication 60
Stream Transmission Modes
 Asynchronous transmission mode
o Data sent one after another
o No other timing constraints
 Synchronous transmission mode
o Max end-to-end delay for each unit
 Isochronous transmission mode
o Max and min end-to-end delay

Chapter 2  Communication 61
 Stream

 Stream from process to process

 Stream can be viewed as a virtual connection
between source and sink

Chapter 2  Communication 62
 Stream

 Stream sent directly between two devices

Chapter 2  Communication 63
 Stream

 Multicasting a stream
 Different requirements for receivers?

Chapter 2  Communication 64
 Specifying QoS
Characteristics of the Input Service Required

Maximum data unit size (bytes) Loss sensitivity (bytes)

Token bucket rate (bytes/sec) Loss interval (sec)
Toke bucket size (bytes) Burst loss sensitivity (data units)
Maximum transmission rate Minimum delay noticed (sec)
(bytes/sec) Maximum delay variation (sec)
Quality of guarantee

 A flow specification

Chapter 2  Communication 65
 Specifying QoS

 A token bucket algorithm

 Don’t want bucket to be empty of overflowing
 Then can feed out at precise time intervals

Chapter 2  Communication 66
 Setting Up a Stream

 RSVP for resource reservation

 Purpose is to try to insure QoS
 Highly dependent on data link layer

Chapter 2  Communication 67
 Synchronization

 Explicit synchronization for data units

 Read and write incoming stream units
 App is responsible for sync., only low-level utilities

Chapter 2  Communication 68
 Synchronization

 Synchronization supported by high-level interfaces

 A middleware approach
Chapter 2  Communication 69
 Summary
 Communication is a fundamental issue
in distributed systems
 Networking overview
 RPC
o Goal is transparency
 RMI
o Transparency and objects
 RPC and RMI are synchronous

Chapter 2  Communication 70
 Summary
 Recall
o Synchronous  block until msg delivered
(or until response received)
o Asynchronous  sender continues
immediately after sending
o Persistent  msg stored until delivered
o Transient  msg delivered now or never

Chapter 2  Communication 71
 Summary
 Message-Oriented Communication
o Message passing
 For transient asynchronous (MPI)
 Good for big parallel machines
o Message-oriented middleware (MOM)
 Designed for persistent asynchronousStreams
 Streams
o Primarily for video and audio
o Temporal relationship is critical

Chapter 2  Communication 72