Tanta University

Faculty of science

Mathematics Dept.

Division of computer science

Distributed Systems

Prepared by

Dr. Aya Abdelaziz Ismail

Lecturer in computer science,

Faculty of science

Tanta University

2024
 Distributed Systems

Chapter 01: Introduction

Introduction: What is a distributed system?

Distributed System

Definition
A distributed system is a collection of autonomous computing elements
(each being able to behave independently of each other) that appears
to its users as a single coherent system.

Characteristic features
1. Collection of Autonomous computing elements
► Autonomous computing elements, also referred to as nodes, be
they hardware devices or software processes.
2. Single coherent system: users or applications perceive a single
system (believe they are dealing with a single system) ⇒ nodes
need to collaborate.

3 / 56
Introduction: What is a distributed system? Characteristic 1: Collection of autonomous computing elements

1. Collection of autonomous nodes

Independent behavior
Each node is autonomous and will thus have its own notion of time: there is no
global clock. Leads to fundamental synchronization and coordination problems.

Collection of nodes
► Who are the members of the group (open or closed group)? How to
manage group membership and organization of that collection ?
► How to know that you are indeed communicating with an authorized
(non)member?
- In an open group, any node is allowed to join the distributed system, i.e., it can send
messages (communicate) to any other node in the system without any authorization
process.
- In contrast, with a closed group, only the members of that group can communicate
with each other, and a separate mechanism is needed to let a node join (it needs to go
through an authorization process) or leave the group.
- we may need to register which nodes may or may not belong to the system
- we provide each member with a list of nodes it can directly communicate with.
4 / 56
Introduction: What is a distributed system? Characteristic 1: Collection of autonomous computing elements

** Organization: collection of nodes are organized as Overlay network.

- Overlay networks are a common way to organize distributed systems.
- The overlay network provides a logical structure allowing nodes to
communicate with each other nodes in the system, its neighbors.
efficiently.
- i.e., between any two nodes there is always a communication path
allowing those nodes to route messages from one to the other
** Overlay network types:
1. Structured: each node has a well-defined set of neighbors with whom it
can communicate. Nodes are organized in the form of: (ex: tree, ring).
•Tree: In a tree overlay network, nodes are organized in a hierarchical structure where
each node has a parent (except for the root node) and a set of child nodes.
Communication typically follows the tree structure, with messages flowing from parent
nodes to child nodes or vice versa.

•Ring: In a ring overlay network, nodes are connected in a circular manner, forming a
ring structure. Each node has a neighbor to which it can send messages. Messages can
be passed from one node to the next in a clockwise or counterclockwise direction
around the ring until they reach the intended destination.
5 / 56
Introduction: What is a distributed system? Characteristic 1: Collection of autonomous computing elements

Overlay network types (Cont.):

2. Unstructured: each node has references to randomly selected other

nodes from the system.
-Each node maintains a list of its neighbors, which can be selected
randomly or using some other criteria. The connections between
nodes in an unstructured overlay network can be highly dynamic
and may change frequently.

6 / 56
Introduction: What is a distributed system? Characteristic 2: Single coherent system
2. Single Coherent system
Essence:
- End users should not even notice that they are dealing with the fact that
processes, data, and control are dispersed across a computer network. Ex: as
on google search box, we write text on search box, and we get result
-The collection of nodes as a whole operates the same, no matter where,
when, and how interaction between a user and the system takes place.
*Offering Single Coherent view is challenging: Examples: it requires:
► An end user cannot know where a computation is taking place (on which
computer a process is currently executing, or even perhaps that part of a
task has been transmitted to another process executing somewhere else)
► Where data is exactly stored should be irrelevant to an application.
► If or not data has been replicated to enhance performance is completely
hidden
The Keyword is distribution transparency (important design goal of distributed
system)

*** partial failures: As distributed system consists of multiple, networked nodes,

it is inevitable that at any time only a part of the distributed system fails
(halt). 7 / 56
Introduction: What is a distributed system? Middleware and distributed systems

Middleware: the OS of distributed systems

-To assist the development of distributed applications, distributed systems are
often organized to have a separate layer of software; middleware, that is logically placed
on top of the respective operating systems (OS) of the computers that are part of the
system.

Figure: A distributed system organized in a middleware layer, which extends over multiple
machines. Each application offers the same interface.
Same interface everywhere

Computer 1 Computer 2 Computer 3 Computer 4

Application B
Appl. A Appl. C
(distributed on 2 computers)

Distributed-system layer (middleware)

Local OS 1 Local OS 2 Local OS 3 Local OS 4

Network
What does middleware layer contain?
Commonly used components and functions that need not be
implemented by applications separately. 8 / 56
Middleware services are offered in a network environment: middleware is
the same to a distributed system as what an operating system is to a computer

1. Resource management (a manager of resources

offering its applications to efficiently share and
deploy those resources across a network )
2. Facilities for interapplication communication
(send & receive mechanism).
3. Security services (authentication).
4. Accounting services (Pay).
5. Masking of (hide) and recovery from failures.
Introduction: Design goals

What do we want to achieve?

Design goals of distributed system:
1. Support sharing of resources: An important goal of a distributed
system is to make it easy for users (and applications) to access and share
remote resources. Resources Ex: peripherals, storage facilities, data, files,
services, and networks, etc.
2. Distribution transparency: it tries to make the distribution of processes
and resources transparent, that is, invisible, to end users and applications.
3. Openness (How being open?): An open distributed system is essentially
a system that offers components that can easily be used by, or
integrated into other systems. At the same time, an open distributed
system itself will often consist of components that originate from
elsewhere.
4. Scalability (How to scale up the system?) : (size, Geographical,
Administrative)
10 / 56
Introduction: Design goals Supporting resource sharing

1. Sharing resources
- An important goal of a distributed system is: Make it easy for users
(and applications) to access and share remote resources. Resources Ex:
peripherals, storage facilities, data, files, services, and networks, etc.
examples
► Cloud storage services like Google Drive allow users to store their files and
access them from anywhere with an internet connection. Users can easily
upload, download, and share files with others by granting them permission to
access specific folders or files.
- Imagine a team of employees working on a project. They can store
project-related documents, spreadsheets, and presentations in a shared
folder on Google Drive. Each team member can access and edit these files,
facilitating seamless collaboration and eliminating the need for sending
files back and forth via email.
► File-sharing peer-to-peer networks like BitTorrent (BitTorrent is a popular
peer-to-peer file-sharing protocol that allows users to share files across the
internet without relying on a central server).
11 /
Introduction: Design goals Supporting resource sharing

1. Sharing resources
► File-sharing peer-to-peer networks like BitTorrent.

- Ex, if a user wants to download a large file using BitTorrent, they connect to the
network and start downloading different pieces of the file from various peers who
already have it. At the same time, the user also uploads the parts they have to other
users who are requesting the same file. This decentralized approach allows for
faster and more efficient file sharing.

► Peer-to-peer networks are often associated with distribution of

media files such as audio and video.

- Ex: when a user sends a video on WhatsApp, the application breaks the
video into smaller pieces and sends them to several peers in the network. The
received device retrieves these pieces from different sources simultaneously,
allowing for faster streaming and reducing the load on any single server

Observation: “The network is the computer” (quote from John Gage, then at
Sun Microsystems)
12 /
Introduction: Design goals Making distribution transparent

2. Distribution transparency: it tries to make the distribution of processes and resources

transparent, that is, invisible, to end users and applications.

Types of distribution transparency:

Transparency Description
Access Hide differences in data representation and the way that an object
is accessed (Ex: a distributed system may have computer systems
that run different OS, each having their own file-naming
conventions. Differences in naming conventions, or differences in
file operations, are examples of access issues that should
preferably be hidden from users and applications.)
Location Hide where an object is physically located in system (achieved by
assigning only logical names to resources.) EX of a such a name is:
(URL) http://www.prenhall.com/index.html, which gives no clue about
the actual location of Prentice Hall’s main Web server.
Relocation Hide that an object may be moved to another location while in use
(The URL also gives no clue as to whether the file index.html has
always been at its current location or was recently moved there. Ex:
the entire site may have been moved by the distributed system from
one data center to another, yet users should not notice. )
**An object can be a resource or a process 13 / 56
Types of distribution transparency
Types of distribution transparency (Continues)

Replication Hide that an object is replicated (by make all replicas have the same name). ---
Resources may be replicated to increase availability or to improve performance by
placing a copy close to the place where it is accessed.
“Ex: Imagine a distributed database system that stores customer information for an
e-commerce platform. To improve performance and increase availability, the
database is replicated across multiple nodes in different geographical locations.
Each replica contains the same data, allowing clients to access the nearest replica
for faster response times.”
-Replication transparency deals with hiding the fact that several copies of a
resource exist, or that several processes are operating in some form of lockstep
mode so that one can take over when another fails.
Migration Hide that an object may move to another location (EX: communication between
mobile phones: regardless whether two people are actually moving, mobile phones
will allow them to continue their conversation.)
Concurrency Hide that an object may be shared by several independent users
(Two independent users may each have stored their files on the same file server or
may be accessing the same tables in a shared database. In such cases, it is
important that each user does not notice that the other is making use of the same
resource.)
Types of distribution transparency (Continues)

Failure Hide the failure and recovery of an object (user or application does not notice that
some piece of the system fails to work properly, and that the system subsequently
(and automatically) recovers from that failure.)

Masking failures and transparently recovering from failures is one of the hardest
issues in distributed systems and is even impossible. Because the inability to
distinguish between a dead process and slowly responding one.
Ex., when contacting a busy Web server, a browser will eventually time out and
report that the Web page is unavailable. At that point, the user cannot tell whether
the server is actually down or that the network is badly congested.
Introduction: Design goals Making distribution transparent

Degree of distribution transparency:

Observation: Aiming at full distribution transparency may be too much.

► 1. There are communication latencies that cannot be hidden (In a wide-

area distributed system, sending a message from one process in Egypt to the
other in America takes several hundred milliseconds using a computer
network. Signal transmission is not only limited by the speed of light, but also
by limited processing capacities and delays in the intermediate switches. )

► 2. Completely hiding failures of networks and nodes is (theoretically

and practically) impossible
► You cannot distinguish a slow computer from a failing one
► You can never be sure that a server actually performed an
operation before a crash

Degree of distribution transparency 10 / 56

Degree of distribution transparency(Cont.)
► 3. Full transparency will cost performance. There is a trade-off
between a high degree of transparency and the
performance of a system.

► Keeping replicas exactly up-to-date with the master

takes time (where we need to guarantee that several replicas,
located on different continents, must be consistent all the time. i.e.,
if one copy is changed, that change should be propagated to all
copies before allowing any other operation. It is clear that a single
update operation may now even take seconds to complete,
something that cannot be hidden from users. )
Introduction: Design goals Making distribution transparent

Degree of distribution transparency

Although distribution transparency is generally considered preferable for any distributed
system, there are situations in which attempting to hide all distribution aspects
from users is not a good idea (Exposing distribution may be good)
► Making use of location-based services, which can be found on
mobile phones (such: finding the nearest Chinese takeaway or
checking whether any of your friends are nearby.)
► When dealing with users in different time zones
► When it makes it easier for a user to understand what’s going on
(when e.g., a server does not respond for a long time, report it as
failing).

Conclusion
Distribution transparency is a nice a goal, but achieving it is a different
story, and it should often not even be aimed at.

Degree of distribution transparency 11 / 56

 Introduction: Design goals Being open

3. Openness of distributed systems

A system that offers components that can easily be used by, or integrated into
other systems. At the same time, an open distributed system itself will often consist
of components that originate from elsewhere.
What are we talking about?
Be able to interact with services from other open systems, irrespective of
the underlying environment:
► Systems should conform to well-defined interfaces (clear and complete
functionality)
► Systems should easily interoperate (components from different
manufacturers (or systems) can co-exist and work together)

Ex: consider a messaging system where different applications need to

exchange messages. If the messaging system follows a standard protocol or
interface, applications developed by different manufacturers can interact with
the messaging system without any compatibility issues. The well-defined
interface ensures that the applications can send and receive messages,
regardless of their specific implementations or origins.

Interoperability, composability, and extensibility 12 / 56

 Introduction: Design goals Being open

3. Openness of distributed systems (cont.)

What are we talking about? (cont.)
Be able to interact with services from other open systems, irrespective of
the underlying environment:

► Systems should support portability of applications (the ability to run

(execute) an application developed for a distributed system A, without
modification, on a different distributed system B that implements the same
interfaces as A. This enables flexibility and ease of deployment across
different systems).
► Systems should be easily extensible (it should be easy to add new
components or replace existing ones without affecting those components that
stay in place. EX: in an extensible system, it should be relatively easy to add
parts that run on a different operating system, or even to replace an entire file
system. It allows the system to evolve and adapt to changing
requirements or technologies.)

Interoperability, composability, and extensibility 12 / 56

Introduction: Design goals Being open

Policies versus mechanisms (Separating policy from mechanism)

**Implementing openness: policies (rules “objectives, requirements, and
constraints” that govern the system's behavior and we follow to build the open
distributed system)

- It typically determined by the system designers or administrators based on

factors such as security, performance, reliability, and user requirements.
► What level of consistency do we require for client-cached data (ex: in
web browser) (-> store data locally (take a copy) on a client device, such as
a computer, smartphone, or tablet, for quick access and improved performance,
instead of access network each time we need data)?
Ex: the policy could state that: the data should be globally consistent,,
i.e., any updates to the data should be immediately visible to all clients.
Alternatively, the policy might allow for: eventual consistency, where
updates propagate gradually and clients may observe temporary
inconsistencies.

Separating policy from mechanism 13 / 56

Introduction: Design goals Being open

Policies versus mechanisms (Separating policy from mechanism)

**Implementing openness: policies (Cont.)

Which operations or actions are permitted for code downloaded and

executed by clients?

Ex: the policy might: restrict downloaded code from accessing

certain system resources or performing potentially harmful
operations to ensure security and prevent unauthorized actions.

Separating policy from mechanism 13 / 56

Introduction: Design goals Being open

Policies versus mechanisms (Separating policy from mechanism)

**Implementing openness: policies (cont.)

► Which Quality of Service (QoS) adjustments in the face of varying
bandwidth? The policy may specify how the system adjusts its QoS
parameters in response to varying bandwidth conditions

Ex: the policy might dictate that: the system should prioritize certain
types of traffic or adjust the level of data compression based on
available bandwidth to optimize performance.

► What level of secrecy or encryption required for communication

between different components or nodes in the distributed system?

Ex: The policy could mandate: the use of strong encryption

algorithms and secure communication protocols to protect sensitive
data from unauthorized access.

Separating policy from mechanism 13 / 56

 Introduction: Design goals Being open

Policies versus mechanisms (Separating policy from mechanism )

**Implementing openness: mechanisms (Make implementation of policy
(rules) via mechanism-->

- Mechanisms: are the technical implementations or procedures that

execute the policies. They represent the underlying functionality,
algorithms, protocols, or techniques used to achieve the desired behavior
according to the defined policies.)

- Offer different techniques, algorithms, or procedures to:

► Allow (dynamic) setting of the desired level of consistency for
caching policies
► Support different levels of trust for mobile code to restrict the
operations and ensure that the code adheres to the defined
security policies.
► Adjust the Quality of Service (QOS) parameters based on the available
bandwidth, aligning with the policy requirements.
- Offer different encryption algorithms to protect the secrecy of
communication channels. 13 / 56
Introduction: Design goals Being scalable

4. Scale in distributed systems

Observation
Many developers of modern distributed systems easily use the
adjective “scalable” without making clear why their system actually
scales.

Scalability dimensions : At least three components

a. Number of users and/or processes (size scalability)
b. Maximum distance between nodes (geographical scalability)
c. Number of administrative domains (administrative scalability)

Observation
Most systems account only, to a certain extent, for size scalability. Often a
solution: multiple powerful servers operating independently in parallel.
Today, the challenge still lies in geographical and administrative scalability.

Scalability dimensions 15 / 56
Introduction: Design goals Being scalable

Problems of scalability,
and how being scalable?

--- In next Lecture

Scalability dimensions 21 / 56