Distributed Systems

Lec 01
Introduction
 Outline
Introduction

Defining Distributed Systems

Characteristics of Distributed Systems

Examples of Distributed Systems

Challenges and Problems

Summary
4/10/2024 Distributed Systems 2
 This Course
Week Topic Hours
1 Introduction of models of distributed systems 2
2 Basic principles and models 2
3-4 Communication and Operations 4
5 Names of distributed systems 2
6 Synchronization of distributed systems 2
7-8 Compatibility of distributed systems 4
9 Midterm Exam
10 Replication of distributed systems 2
11 Fault detection 2
12 Explaining different models including object systems 2
13 Distributed file systems 2
14 Document systems 2
15 Discussion + Oral Exam
16 Final Exam
4/10/2024 Distributed Systems 3
 Assessment Strategy
Assessment Method Degrees
Attendance 2
Assignments 2
Quiz 1 6 40
Quiz 2 6
Midterm 24
Final Exam 60 60
Total 100
4/10/2024 Distributed Systems 4
 Introduction
As soon as computers are interconnected and communicating
we have a "distributed system“
This approach of interconnectivity has been known by several
names:
Multi-computers.
Parallel computers.
Cluster.
Computational grid.
Cloud.

4/10/2024 Distributed Systems 5

 Goals and Motivation
The Need For High Performance Computers
Solving technology problems using computer
modeling, simulation and analysis

Mechanical Design Geographic

Life Sciences Aerospace & Analysis Information
(CAD/CAM) Systems
4/10/2024 Distributed Systems 6
 Multiprocessor
vs.
Mainframe
Economic, Speed,
Distribution,
Reliability, Growth,
Software,
Networking, Security
4/10/2024 Distributed Systems 7
What is a Distributed System?
A collection of independent computers
(H/W) that appear to the users of the system
as a single computer (S/W) [Tanenbaum]

task (job) S/W DS

(Pool of processors)

S/W maps the job to different computers (processors).

So that the user feels that the system is a single computer (transparency).
4/10/2024 Distributed Systems 8
What is a Distributed System?
A distributed system is several computers
doing something together.
Thus, a distributed system has three primary
characteristics:
multiple computers, interconnections, and
shared state. [Michael Schroeder]

4/10/2024 Distributed Systems 9

What is a Distributed System?
A system in which components located on
networked computers communicate and
coordinate their actions by passing
messages.
The components interact with each other in
order to achieve a common goal.

4/10/2024 Distributed Systems 10

What is a Distributed System?
Independent
components or elements
connected by a network
communicate by passing
messages
to achieve a common
goal, appearing as a
single coherent system.

4/10/2024 Distributed Systems 11

 Examples of DS
World Wide Web
A cluster of nodes on the cloud
(AWS, Azure, GCP)
Multi-player games
BitTorrent
Online banking

4/10/2024 Distributed Systems 12

 Why DS?
Nature of the • Multiplayer games, P2P file sharing,
application client requesting a service.

Availability despite • A service shouldn’t fail when one

unreliable
components computer does.

• More CPU cycles, more memory, more

Scale up capacity
storage, etc.

Customize computers
for specific tasks
• E.g. for storage, email, backup.

4/10/2024 Distributed Systems 13

 Typical Design Goals
Heterogeneity – can the system handle a large variety of types of machines and
devices?
Robustness – is the system resilient to host crashes and failures, and to the
network dropping messages?
Availability – are data + services always there for clients?
Transparency – can the system hide its internal workings from the users?
(warning: term means the opposite of what the name implies!)
Concurrency – can the server handle multiple clients simultaneously?
Efficiency – is the service fast enough? Does it utilize 100% of all resources?
Scalability – can it handle 100 million nodes without degrading service?
(nodes=clients and/or servers) How about 6 B? More?
Security – can the system withstand hacker attacks?
Openness – is the system extensible?
Distributed Systems
levels
 Centralized
A mainframe and dumb terminals

All of the computation is done on the mainframe.

Each line or keystroke is sent from the terminal to
the mainframe.
4/10/2024 Distributed Systems 16
Moving Towards Distribution
In a client-server system, the clients are
workstations or computers in their own right and
perform computations and formatting of the data.

However, the data and the application which

manipulates it ultimately resides on the server.
4/10/2024 Distributed Systems 17
 More Decentralization
In Distributed-with-Coordinator, the nodes or sites depend on
a coordinator node with extra knowledge or processing abilities

Coordinator might be used only in case of failures or other problems

4/10/2024 Distributed Systems 18
 True Decentralization
A true Distributed system
has no distinguished node
which acts as a coordinator
and all nodes or sites are
equals.

The nodes may choose to

elect one of their own to act
as a temporary coordinator
or leader
4/10/2024 Distributed Systems 19
Flynn’s Taxonomy of
computer
architectures
 Flynn’s Taxonomy
SISD • Single Instruction/Single Data

SIMD • Single Instruction/Multiple Data

MISD • Multiple Instruction/Single Data

MIMD • Multiple Instruction/Multiple Data

4/10/2024 Distributed Systems 21
 Single Instruction/Single Data
Your desktop, before the spread of dual core
CPUs

PU – Processing Unit

4/10/2024 Distributed Systems 22

 SIMD
Processors that execute same instruction on
multiple pieces of data

NVIDIA GPUs

4/10/2024 Distributed Systems 23

 SIMD
Each core runs the same set of instructions
on different data

Example:
NVIDIA: processes pixels of an image in parallel.
Array Processor.
4/10/2024 Distributed Systems 24
 MISD
Example: Pipeline Architecture

4/10/2024 Distributed Systems 25

 MIMD

4/10/2024 Distributed Systems 26

 MIMD
Shared Memory Distributed Memory
Processors are all Each processor has its
connected to a own individual
memory location.
"globally available"
Each processor has no
memory direct knowledge about
Via either software or other processor's
hardware means. memory.
4/10/2024 Distributed Systems 27
 DS
Challenges &
Problems
 Challenges
Increasing the (inter-processor communications).
Number of processors available to execute on, and
Processors synchronization.
Memory size and the bandwidth of interconnection.
Analyzing the problem to suit distributed computing.
Infrastructure to handle distribution and
communication.
HW & SW combined to provide a tool to solve specific
problems.
4/10/2024 Distributed Systems 30
 Challenges
Multiple computers
Concurrent execution.
Independent failure.
Autonomous administration.
Heterogeneous.
Large numbers.
Networked communication
Asynchronous
Unreliable
Insecure
Common goal
Consistency
Transparency
4/10/2024 Distributed Systems 31
 Problems
Hardware
Software Problems
Problems
Memory, Data Memory size and
Management bandwidth

Detection of Bandwidth of
parallelism interconnection

Processes Scheduling No. of processors

4/10/2024 Distributed Systems 32

 Thank you

Questions?
4/10/2024 Distributed Systems 33