Distributed Systems

Temesgen Mekonnen

[email protected]

Fall
2021
Literature

Textbook

George Coulouris, Jean Dollimore, Tim Kindberg, Gordon Blair, Distributed

Sys- tems: Concepts and Design (5th Edition), Addison-Wesley 2012.

References

1. Sape Mulleuder, Distributed Systems, Addison Wesley, 2009

2. Albert Flieshman, Distributed Systems, Software Design and

Implementation, Springer Verlag, 2008
 Chapter Two
System Models
 Outline
1. Physical Models
2. Architectural Models
3. Fundamental Models
System models
Three basic ways to describe design of Distributed systems are

• Physical Models – consider the types of computers and devices that

constitute a system and their interconnectivity, without details of
specific technologies

• Architectural Models – describe a system in terms of the computation

and communication tasks performed by its computational elements.
Client-server and peer-to-peer most commonly used.

• Fundamental Models – take an abstract perspective in order to describe

solutions to individual issues faced by most distributed systems.

There is no global time in a distributed system, so message communication

over a computer network can be affected by delays, can suffer from a variety
of failures and is vulnerable to security attacks.
System models

These issues are addressed by three models:

–Interaction Models
–Failure Models
– Security Models

1, Physical models
• Baseline physical model – minimal physical model of a
distributed system as an extensible set of computer nodes
interconnected by a computer network for the required
passing of messages.
System models

There are three generations of distributed systems beyond the baseline

model

• Early distributed systems

Such systems emerged in the late 1970s and early 1980s in response to the
emergence of LAN technology
10 and 100 nodes interconnected by a local area network
limited Internet connectivity
Supported a small range of services e.g.
∗ shared local printers
∗ file servers
∗ email services
∗ file transfer services

• Internet-scale distributed systems

Extensible set of nodes interconnected by a network of networks (the

System models

• Contemporary DS with hundreds of thousands nodes + emergence of:

– Mobile computing
∗ laptops or smart phones may move from location to location
– need for added capabilities (service discovery; support for
spontaneous interoperation)

– Ubiquitous computing

∗ computers are embedded everywhere

– Cloud computing

∗ pools of nodes that together provide a given service

• Distributed systems of systems (ultra-large-scale (ULS) distributed

systems)

It describes todays global DS configurations

System models

Figure 2.1 Generations of distributed systems

System models

2, Architectural Models
The architecture of a system is its structure in terms of separately
specified components and their interrelationships.

Major concerns: make the system reliable, manageable, adaptable and

cost- effective.
1. Architectural Elements
Four key questions to understand building blocks of DS
• What are the entities that are communicating in the distributed
system?

• How do they communicate, or, more specifically, what

communication paradigm is used?

• What (potentially changing) roles and responsibilities do they have

in the overall architecture?

• How are they mapped on to the physical distributed infrastructure

System models

Communicating Entities

• From system oriented perspective: processes

Distributed system as processes coupled with appropriate interprocess
communication paradigms.

– in some cases we can say that:

∗ nodes (sensors)
∗ threads (endpoints of communication)

• From programming (problem oriented) perspective

– Objects
∗ computation consists of a number of interacting objects
representing natural units of decomposition for the given
problem domain
∗ Objects are accessed via interfaces, with an associated
interface defi- nition language (or IDL)
System models

– Components – emerged due to some weaknesses with

distributed objects
∗ offer problem-oriented abstractions for building distributed
systems
∗ accessed through interfaces
Components specify not only their (provided) interfaces but
also assumptions to components/interfaces that must be
present (i.e. making all dependencies explicit and providing a
more complete contract for system construction.)
– Web services

∗ closely related to objects and components

∗ intrinsically integrated into the World Wide Web
· using web standards to represent and discover services
System models

The World Wide Web consortium (W3C) defines web service

as:
Web service is a software application identified by a URI, whose
interfaces and bindings are capable of being defined, described and
discovered as XML artefacts. A Web service supports direct
interactions with other software agents using XML-based message
exchanges via Internet-based protocols.
System models

Communication Paradigms

The three types of communication paradigms are

• Interprocess communication

• Remote invocation

• Indirect communication

Interprocess communication – refers low-level support for communication

between processes in distributed systems, including message-passing
primitives, direct access to the API offered by Internet protocols (socket
programming) and support for multicast communication

Remote invocation – calling of a remote operation, procedure or method.

Request-reply protocols – a pattern with message-passing service to

support client-server computing.
System models

Remote procedure call (RPC)

• Procedures in processes on remote computers can be called as if they

are procedures in the local address space

• Supports client-server computing with servers offering a set of

operations through a service interface and clients calling these
operations directly as if they were available locally

– What type of transparency do RPC systems offer?

∗ RPC systems offer (at a minimum) access and location
transparency

Remote method invocation (RMI)

• Strongly resemble RPC but in a world of distributed objects

• With this approach, a calling object can invoke a method in a remote

object.
System models

In RPC and RMI

• Senders and Receivers of messages

– Coexist at the same time

– Are aware of each other’s identities

Indirect communication

• Senders do not need to know who they are sending to (space

uncoupling)

• Senders and receivers do not need to exist at the same time (time
uncoupling)

Key techniques in indirect communication:

• Group communication (1 to many with group identifier)

System models

– (publish-subscribe systems – sometimes also called distributed

event- based systems)

– Publishers distribute information items of interest (events) to

possibly a large number of consumers (or subscribers)

• Message queues:

– (publish-subscribe systems offer a one-to-many style of

communication), message queues offer a point-to-point
indirection service

– Producer processes can send messages to a specified queue

– Consumer processes can

∗ receive messages from the queue or

∗ be notified
System models

• Tuple spaces (also known as generative communication)

– Processes can place arbitrary items of structured data, called

tuples, in a persistent tuple space

– Other processes can either read or remove such tuples from the
tuple space by specifying patterns of interest

– Readers and writers do not need to exist at the same time (Since
the tuple space is persistent)

• Distributed shared memory (DSM):

– Abstraction for sharing data between processes that do not share

physical memory
System models

Figure 2.2 Communication entities and communication paradigms

System models

Roles and responibilities

• Client-server
Figure 2.3 Clients invoke individual
servers
System models
• Peer-to-peer
Figure 2.4a Peer-to-peer
architecture

– same set of interfaces to each and every node in the

model
System models

Placement
• crucial in terms of determining the DS
properties:
– Performance

– Reliability

– Security

• Mapping of services to multiple Figure 2.4b A service provided by

Possible placement strategies:
servers multi- ple servers

– Mapping distributed
objects between servers,
or

– Replicating copies on
several hosts

– More closely coupled

System models
• Caching

– A cache is a s to re of recently used data objects that is closer

to one client or a particular set of clients than the objects
themselves

Figure 2.5 Web proxy

server
 System models
• Mobile code

– Applets are an example of mobile

code Figure 2.6 Web Applets

–Another possibility – push model: server initiates interaction (e.g. stockbroker

might provide a customized service to notify customers of changes in the
prices of shares)
System models
• Mobile Agents

–Running program (including both code and data) that travels from
one computer to another in a network carrying out a task on
someone’s behalf (e.g. collecting information), and eventually
returning with the results

– could be used for

∗ software maintenance
∗ collecting information from different vendors’ databases
of prices

Possible security threats with mobile code and mobile agents...

System models

2 Architectural Patterns

Architectural Pattern: Layering

Layered approach – complex system partitioned into a number of layers:

• Vertical organisation of services

• Given layer making use of the services offered by the layer below

• Software abstraction

• Higher layers unaware of implementation details of any other layers

beneath them
System models

Figure 2.7 Software and hardware service layers in distributed systems

• Platform for distributed systems and applications consists of

the lowest-level hardware and software layers

• Middleware – a layer of software whose purpose is to mask

hetero- geneity and to provide a convenient programming
model to application programmers
System models

Architectural Pattern: Tiered Architecture

Tiering is a technique to organize functionality of a given layer and place

this functionality into appropriate servers and, as a secondary
consideration, on to physical nodes

Example: two-tier and three-tier architecture

functional decomposition of a given application, as follows:

• Presentation logic

• Application logic

• Data logic
System models
Figure 2.8 Two-tier and three-tier
• Three aspects par-
architectures
titioned into two
processes

• (+) low latency

• (-) splitting applica-
tion logic

• (+) one-to-one map-

ping from logical
elements to physical
servers

• (-) added complexity,

network traffic and
la- tency
System models

Architectural Pattern: Thin Clients

• Enabling access to sophisticated networked services (e.g. cloud

services) with few assumptions to client device

• software layer that supports a window-based user interface (local) for

executing remote application programs or accessing services on
remote computer

Figure 2.10 Thin clients and computer servers

System models

Architectural Patterns: Other Commonly Occurring Patterns

• proxy pattern

– Designed to support location transparency in RPC or RMI

– Proxy created in local address space, with same interface as the

remote object

• Brokerage in web services

– supporting interoperability in potentially complex distributed

infrastruc- tures

∗ interoperability - ability of different information technology

systems and software applications to communicate,
exchange data, and use the information that has been
exchanged

– service provider, service requestor and service broker

System models

Figure 2.11 The web service architectural pattern

• Reflection pattern

– a means of supporting both: introspection and intercession

∗ introspection – the dynamic discovery of properties of the
system
∗ intercession – the ability to dynamically modify
structure or be- haviour
System models

– ability to intercept incoming messages or invocations

– dynamically discover interface offered by a given object

– discover and adapt the underlying architecture of the system

System models

3, Fundamental Models
What is:

• Interaction model?

• Failure model?

• Security model?

1. Interaction model

• Processes interact by passing messages

– communication (information flow) and

– coordination (synchronization and ordering of activities) between

pro- cesses
System models

• Communication takes place with delays of considerable duration

– Coordination accuracy of independent processes is limited by

these delays and by difficulty of maintaining the same notion of
time across all the computers in a distributed system

Behaviour and state of DS can be described by a distributed algorithm:

• It specifies steps to be taken by each interacting process

• and transmission of messages between them

State belonging to each process is completely private!

System models

Performance of communication channels

• latency – delay between the start of message’s transmission from one

process and the beginning of receipt by another

• bandwidth of a computer network – the total amount of information

that can be transmitted over it in a given time

• Jitter – the variation in the time taken to deliver a series of messages

Computer clocks and timing events

• clock drift rate – rate at which a computer clock deviates from a

perfect refer- ence clock
System models

Two variants of DS based on their

interaction model

Synchronous distributed systems: Asynchronous distributed

systems:
• The time to execute each step
of a process has known lower
No bounds on:
and upper bounds

• Each message transmitted over • Process execution speeds

a channel is received within a
• Message transmission
known bounded time delays

• Each process has a local clock • Clock drift rates

whose drift rate from real time
has a known bound
System models

Event ordering

Figure 2.13 Real-time ordering of

events

• Logical time – based on event

ordering
System models
2. Failure Model
• faults occur in:

– any of the computers (including software faults)

– or in the network

• Failure model defines and classifies the faults

Omission Failures

• Process or communication channel fails to perform actions it is

supposed to do

Process omission failures

• Cheaf omission failure of a process is crash

Crash is called fail-stop if other processes can detect certainly that the
process has crashed
System models

Communication omission failures

• Communication channel does not transport a message from
p’s outgoing message buffer to q’s incoming message buffer

These are known as dropping messages

∗ send-omission failures
∗ receive-omission failures
∗ channel-omission failures

Figure 2.14 Processes and channels

All failures so far: benign failures

System models

Arbitrary Failures
Arbitrary or Byzantine failure is used to describe the worst possible failure,
in which any type of error may occur
Figure 2.15 Omission and arbitrary failures
System models

Timing Failures

• Applicable in synchronous distributed

systems Figure 2.16 Timing failures

Failure Masking

• knowledge of the failure can enable a new service to be designed to

mask the failure of the components on which it depends.
System models

Reliability of one-to-one communication

• Reliable Communication meets:

– Validity: Any message in the outgoing message buffer is

eventually deliv- ered to the incoming message buffer

– Integrity: The message received is identical to one sent, and no

messages are delivered twice
System models
3. Security Model

• Modular nature of distributed systems and their openness exposes

them to attack by

– both external and internal agents

• Security model defines and classifies attack forms, this enables

– Providing a basis for the analysis of threats

– Basis for design of systems that are able to resist them

The security of a distributed system can be achieved by securing the

processes and the channels used for their interactions and by protecting
the objects that they encapsulate against unauthorized access.
System models

Protecting Objects by
• Users with access rights

• Association of each invocation and each result with the authority on

which it is issued

– such an authority is called a principal

∗ principal may be a user or a process

Figure 2.17 Objects and principals

System models

Securing processes and their interactions

• Securing communications over open
channels and open service interfaces
from

The enemy or also: adversary

Figure 2.18 The enemy
System models

Threats To Processes

• Lack of knowledge of true source of a

message

– Problem for both server and client

side

– example: spoofing a mail server

Threats to communication channels

• Threat to the privacy and integrity of

messages

• Can be defeated using secure channels

System models

Defeating security threats

Cryptography and shared secrets

• Cryptography is the science of keeping messages secure

• Encryption is the process of scrambling a message in such a way

as to hide its contents

Authentication

• Based on shared secrets authentication of messages – proving the

identities sup plied by their senders
System models

Secure channels

Figure 2.19 Secure

channels

Properties of a secure channel:

• Each of the processes knows reliably the identity of the principal on

whose behalf the other process is executing

• A secure channel ensures the privacy and integrity (protection against

tamper- ing) of the data transmitted across it
System models

• Each message includes a physical or logical timestamp to prevent

messages from being replayed or reordered

Other possible threats from an enemy

• Denial of service:

– By overloading of physical resources (network bandwidth, server

processing capacity)

Example: with infinite invocations

System models

The uses of security models

Performing security analysis

Security analysis involves

• the construction of a threat model:

– listing all the forms of attack to which the system is

exposed

– an evaluation of the risks and consequences of each

attacks