BCS515D:

Distributed
Systems

MODULE 1
Topics to be covered

CHARACTERIZATION OF DISTRIBUTED SYSTEMS

∙ Introduction,
∙ Focus on resource sharing,
∙ Challenges.
REMOTE INVOCATION
∙ Introduction,
∙ Request-reply protocols,
∙ Remote procedure call,
∙ Introduction to Remote Method Invocation.
 CHAPTER 1

CHARACTERIZATION OF
DISTRIBUTED SYSTEMS
1.1 Introduction
∙ Networks of computers are everywhere.
∙ The Internet is one, as are the many networks of which it is composed.
∙ Mobile phone networks, corporate networks, factory networks, campus networks,
home networks, in-car networks – all of these, both separately and in
combination, share the essential characteristics that make them relevant subjects
for study under the heading distributed systems.

∙ We define a distributed system as one in which hardware or software components

located at networked computers, communicate and coordinate their actions only
by passing messages.
1.1 Introduction
∙ Computers that are connected by a network may be spatially separated by any distance.
They may be on separate continents, in the same building or in the same room.
∙ Our definition of distributed systems has the following significant consequences:
∙ Concurrency
• Tasks Carry out Independently
∙ No global clock
• Tasks Coordinate their actions by exchanging messages.
∙ Independent failures
• Faults in the networks results in the isolation of of the computers that are
connected to it.
• Each component of the system can fail independently leaving the others still
working
∙ The prime motivation for constructing and using distributed systems stems from a desire
to share resources.
1.5. Challenges
∙ Designers of Distributed systems need to take the following challenges
into account:
∙ Heterogeneity
Heterogenous components must be able to interoperate.
∙ Openness
Interfaces should allow components to be added or replaced.
∙ Security
The system should only be used in the way intended.
∙ Scalability
System should work efficiently with an increasing number of
users.
System performance should increase with inclusion of additional
resource
1.5. Challenges
∙ Failure handling
Failure of component ( Partial failure) should not result in failure
of the whole system
∙ Concurrency
∙ Transparency
Distribution should be hidden from the user as much as possible
∙ Quality of service
Heterogeneity
∙ Heterogeneity (that is, variety and difference) applies to all of
the following:
∙ networks;
∙ computer hardware;
∙ operating systems;
∙ programming languages;
Heterogeneity
Middleware

∙ The term middleware applies to a software layer

∙ Middleware provides a programming abstraction
∙ Middleware masks the heterogeneity of the underlying networks, hardware,
operating systems and programming languages.

∙ The Common Object Request Broker (CORBA)

Mobile code

∙ The term mobile code is used to refer to program code that can be
transferred from one computer to another and run at the destination –
Java applets are an example

∙ The virtual machine approach provides a way of making code executable

on a variety of host computers: the compiler for a particular language
generates code for a virtual machine instead of a particular hardware
order code
Openness

∙ Distributed System's must be extensible.

∙ The openness of a computer system is the characteristic that determines
whether the system can be extended and reimplemented in various ways.

∙ The first step in openness is publishing the documentation of software

components and interfaces of the components to make them available to
software developers.
Security
∙ Many of the information resources that are made available and maintained
in distributed systems have a high intrinsic value to their users.

∙ Their security is therefore of considerable importance. Security for

information resources has three components:

∙ Confidentiality (protection against disclosure to unauthorized individuals),

∙ Integrity (protection against alteration or corruption),
∙ Availability (protection against interference with the means to access the
resources).
Security
Security mechanism includes:

• Encryption

• Authentication

• Authorization
Security
Security Challenges

∙ Denial of service attacks: DoS attack is an attempt to make to make

a computer resource unavailable to its intended users.

∙ Security of mobile code: Mobile code needs to be handled with care.

Scalability
∙ Distributed systems operate effectively and efficiently at many different
scales, ranging from a small intranet to the Internet.

∙ A system is described as scalable if it will remain effective when there is a

significant increase in the number of resources and the number of users.

∙ The design of scalable distributed systems presents the following challenges:

Controlling the cost of physical resources: Cost should linearly

increase with the system size.

Controlling the performance loss: An example of lack of scalability is

shown by the numbers used as Internet (IP) addresses (computer
addresses in the Internet).
Scalability
Preventing software resources running out : In the late 1970s, it was
decided to use 32 bits as IP address, but the supply of available Internet
addresses has run out. For this reason, a new version of the protocol with
128-bit Internet addresses is being adopted, and this will require
modifications to many software components.

∙ Avoiding performance bottlenecks : In general, algorithms should be

decentralized to avoid having performance bottlenecks
Failure Handling
Failures in a distributed system are partial – that is, some components fail while
others continue to function.

The following are the techniques for dealing with failures:

Detecting failures:
∙ For example, checksums can be used to detect corrupted data in a message or a
file
Masking failures:
∙ Messages can be retransmitted when they fail to arrive.
∙ File data can be written to a pair of disks so that if one is corrupted, the other
may still be correct.
Failure Handling
Tolerating failures:
∙ Exception handling
∙ Timeouts
Recovery from failures:
∙ Rollback Mechanism
Redundancy:
∙ Services can be made to tolerate failures by the use of redundant components
Distributed systems provide a high degree of availability in the face of hardware
faults.
The availability of a system is a measure of the proportion of time that it is
available for use.
Concurrency
∙ With Concurrency services and applications provide resources that can be shared
by clients in a distributed system.

∙ For an object to be safe in a concurrent environment, its operations must be

synchronized in such a way that its data remains consistent.

∙ This can be achieved by standard techniques such as semaphores, which are used
in most operating systems.
Transparency
∙ Transparency is defined as the concealment from the user and the application
programmer of the separation of components in a distributed system, so that the
system is perceived as a whole rather than as a collection of independent
components

∙ Access transparency enables local and remote resources to be accessed using

identical operations.

∙ Location transparency enables resources to be accessed without knowledge of their

physical or network location (for example, which building or IP address).

∙ Concurrency transparency enables several processes to operate concurrently using

shared resources without interference between them.
Transparency
∙ Replication transparency enables multiple instances of resources to be used to
increase reliability and performance without knowledge of the replicas by users or
application programmers.
∙ Failure transparency enables the concealment of faults, allowing users and
application programs to complete their tasks despite the failure of hardware or
software components.
∙ Mobility transparency allows the movement of resources and clients within a
system without affecting the operation of users or programs.
∙ Performance transparency allows the system to be reconfigured to improve
performance as loads vary.
∙ Scaling transparency allows the system and applications to expand in scale without
change to the system structure or the application algorithms.
Quality of Service
The main nonfunctional properties of systems that affect the quality of the service
experienced by clients and users are
∙ Reliability
∙ Security
∙ Performance.
 CHAPTER 5

REMOTE INVOCATION
 Figure 5.1: Middleware layers

Applications
This chapter
Remote invocation, indirect communication
(and Chapter 6)
Middleware
Underlying interprocess communication primitives: layers

Sockets, message passing, multicast support, overlay networks

UDP and TCP

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012
Remote Invocation
∙ request-reply communication: most primitive; minor improvement over underlying
IPC primitives
• 2 way exchange of messages as in client-server computing
∙ RPC, RMI: mechanisms enabling a client to invoke a procedure/method from the
server via communication between client and server
∙ Remote Procedure Call (RPC): extension of conventional procedural
programming model
• allow client programs to transparently call procedures in server programs
running in separate processes, and in separate machines from the client
• i.e. to the caller, the procedures appear to be in local address space
• RPC system hides encoding/decoding of parameters and results, message
passing, and preserves required invocation semantics
Remote Invocation
∙ Remote Method Invocation (RMI): extension of conventional object oriented
programming model
• allows objects in different processes to communicate i.e. an object in one JVM
is able to invoke methods in an object in another JVM
• extension of local method invocation allows object in one process to invoke
methods of an object living in another process
Pros and Cons of RPC, RMI
∙ pros:
• high transparency: caller works with remote procedure/object as if it were
local
• general purpose, flexible
∙ cons:
• high overhead for communication, marshalling/unmarshalling: latency
Figure 5.2 : Request-reply communication

Client Server

Request
doOperation
message getRequest
select object
(wait) execute
Reply method
message sendReply
(continuation)

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012
Request-reply Communication

Most common exchange protocol for remote invocation

Operations

• doOperation(): send request to remote object, and returns the reply received

• getRequest(): acquire client request at server port

• sendReply(): sends reply message from server to client

Request-reply Communication
∙ Design issues
∙ timeouts: what to do when a request times out? how many retries?
∙ duplicate messages: how to discard?
• e.g. recognise successive messages with the same request ID and filter them

∙ lost replies: dependent on idempotency of server operations

∙ history: do servers need to send replies without re-execution? then history needs
to be maintained

∙ Design decisions
∙ retry policy
• how many times to retry?

∙ duplicate filter mechanism

∙ retransmission policy
Request-reply Communication
Exchange protocols
∙ Different flavours of exchange protocols:
∙ request (R): no value to be returned from remote operation
• client needs no confirmation operation has been executed

• e.g. sensor producing large amounts of data: may be acceptable for some loss

∙ request-reply (RR): useful for most client-server exchanges. Reply regarded as

acknowledgement of request

• subsequent request can be considered acknowledgement of the previous reply

∙ request-reply-acknowledge (RRA): acknowledgement of reply contains request id, allowing

server to discard entry from history
Request-reply Communication
TCP vs UDP
∙ limited length of datagrams may affect transparency of RMI/RPC systems which should
be able to accept data of any size

∙ TCP can be chosen to avoid multipacket protocols, avoiding this issue

∙ TCP additional overheads: acknowledgements, connection establishmen
∙ TCP also ensures reliable delivery
∙ no need to filter duplicates or use histories

∙ TCP therefore simplifies implementation of request-reply protocol

∙ if application doesn’t require all of TCP facilities, more efficient, tailored protocol can be
implemented over UDP
Request-reply Communication
Invocation semantics
∙ maybe: RPC may be executed once or not at all
• unless call receives result, it is unknown whether RPC was called
∙ at-least-once: either
• remote procedure was executed at least once and caller received a response, or
• caller received exception to indicate remote procedure was not executed at all
∙ at-most-once: RPC was either
• executed exactly once, in which case caller received response, or
• not executed at all, and caller receives an exception
∙ level of transparency provided depends on design choices and objectives
∙ Java RMI supports at-most-once invocation semantics
∙ Sun RPC supports at-least-once
Request-reply Communication
Invocation semantics
∙ maybe: RPC may be executed once or not at all
• unless call receives result, it is unknown whether RPC was called
∙ at-least-once: either
• remote procedure was executed at least once and caller received a response, or
• caller received exception to indicate remote procedure was not executed at all
∙ at-most-once: RPC was either
• executed exactly once, in which case caller received response, or
• not executed at all, and caller receives an exception
∙ level of transparency provided depends on design choices and objectives
∙ Java RMI supports at-most-once invocation semantics
∙ Sun RPC supports at-least-once
Figure 5.3
Operations of the request-reply protocol

public byte[] doOperation (RemoteRef s, int operationId, byte[] arguments)

sends a request message to the remote server and returns the reply.
The arguments specify the remote server, the operation to be invoked and the
arguments of that operation.
public byte[] getRequest ();
acquires a client request via the server port.
public void sendReply (byte[] reply, InetAddress clientHost, int clientPort);
sends the reply message reply to the client at its Internet address and port.

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012
Figure 5.4
Request-reply message structure

messageType int (0=Reque st, 1= Reply)

requestId int
remoteReference Rem o te Ref
operationId int or Ope ratio n
arguments array of byte s

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012
Figure 5.5
RPC exchange protocols

Name Messages sent by

Client Server Client
R Request
RR Request Reply
RRA Request Reply Acknowledge reply

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012
Figure 5.6
HTTP request message

method URL or pathname HTTP version headers message body

GET //www.dcs.qmw.ac.uk/index.html HTTP/ 1.1

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012
Figure 5.7
HTTP Reply message

HTTP version status code reason headers message body

HTTP/1.1 200 OK resource data

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012
Design Issues in RPC

Three issues that are important in understanding this concept:

• the style of programming promoted by RPC – programming with

interfaces;

• the call semantics associated with RPC;

• the key issue of transparency and how it relates to remote procedure

calls.
Figure 5.9
Call semantics

Fault tolerance measures Call

semantics

Retransmit request Duplicate Re-execute procedure

message filtering or retransmit reply
No Not applicable Not applicable Maybe
Yes No Re-execute procedure At-least-once

Yes Yes Retransmit reply At-most-once

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012
Figure 5.10
Role of client and server stub procedures in RPC

client process server process

Request

Reply
client stub server stub
procedure procedure
client service
program Communication Communication procedure
module module dispatcher

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012
Design Issues in MI

RMI shares the same design issues as RPC in terms of programming interfaces, call
semantics and level of transparency.

1. The object model

2. Distributed object

3. The distributed object model

4. Actions in a distributed object system.

Figure 5.12
Remote and local method invocations

remote local C
invocation invocation local E
remote
invocation invocation F
B local
A
invocation D

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012
Figure 5.13
A remote object and its remote interface

remoteobject

Data
remote
interface
m1 implementation m4

{ m2
m3 of methods
m5
m6

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012
Figure 5.14
Instantiation of remote objects

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012
Figure 5.15
The role of proxy and skeleton in remote method invocation

Instructor’s Guide for Coulouris, Dollimore, Kindberg and Blair, Distributed Systems: Concepts and Design Edn. 5
© Pearson Education 2012