Unit-IV

❑ Distributed Computing Systems

❑ Distributed Computing models
 Parallel v/s Distributed Systems

Parallel Systems Distributed Systems

Memory Tightly coupled shared memory Distributed memory

UMA, NUMA Message passing, RPC, and/or used
of distributed shared memory
Control Global clock control No global clock control
SIMD, MIMD Synchronization algorithms needed

Processor Order of Tbps Order of Gbps

interconnection Bus, mesh, tree, mesh of tree, and Ethernet(bus), token ring and SCI
hypercube (-related) network (ring), myrinet(switching network)
Main focus Performance Performance(cost and scalability)
Scientific computing Reliability/availability
Information/resource sharing
2
 What is a distributed system?
• Tanenbaum’s definition of a distributed
system: “
A distributed system is a collection of
independent computers that appear to the
users of the system as a single coherent
system.”
 An example of a Distributed System
• Nationalized Bank with multiple Branch Offices
 Requirements of Distributed systems
• Security and reliability.
• Consistency of replicated data.
• Concurrent transactions (operations which
involve accounts in different banks;
simultaneous access from several users, etc)
• Fault tolerance
 What is a Distributed Computing
System?
• "a collection of processors interconnected by a
communication network in which each processor
has its own local memory and other peripherals
and the communication between any two
processors of the system takes place by message
passing over the communication
network" (Sinha97)

6
 Distributed Computing System

1945-1950s Loading monitor

1950s-1960s Batch system
1960s Multiprogramming
1960s-1970s Time sharing systems Multics, IBM360
1969-1973 WAN and LAN ARPAnet, Ethernet
1960s-early1980s Minicomputers PDP, VAX
Early 1980s Workstations Alto
1980s – present Workstation/Server models Sprite, V-system
1990s Clusters Beowulf
Late 1990s Grid computing Globus, Legion
7
Distributed Computing System Models
• Minicomputer model
• Workstation model
• Workstation-server model
• Processor-pool model
• Cluster model
• Grid computing

8
 Minicomputer Model

Mini-
computer

ARPA
Mini- net Mini-
computer computer

• Extension of Time sharing system

– User must log on his/her home minicomputer.
– Thereafter, he/she can log on a remote machine by telnet.
• Resource sharing
– Database
– High-performance devices
9
The U.S. Advanced Research Projects Agency Network (ARPANET) was the first public packet-switched computer network.
 Minicomputer Model
• The minicomputer model is a simple extension of the
centralized time-sharing system.

• A distributed computing system based on this model

consists of a few minicomputers interconnected by a
communication network were each minicomputer
usually has multiple users simultaneously logged on
to it.

• Several interactive terminals are connected to each

minicomputer. Each user logged on to one specific
minicomputer has remote access to other
minicomputers.
 Minicomputer Model

• The network allows a user to access remote

resources that are available on some machine
other than the one on to which the user is
currently logged. The minicomputer model may
be used when resource sharing with remote
users is desired.

• The early ARPA net is an example of a

distributed computing system based on the
minicomputer model.
 Workstation Model
Workstation

Workstation 100Mbps Workstation

LAN

Workstation Workstation
• Process migration
– Users first log on his/her personal workstation.
– If there are idle remote workstations, a heavy job may migrate to
one of them.
• Problems:
– How to find an idle workstation?
– How to migrate a job?
– What if a user log on the remote machine?
12
 Workstation Model
• A distributed computing system based on the workstation model
consists of several workstations interconnected by a communication
network.

• An organization may have several workstations located throughout

an infrastructure were each workstation is equipped with its own
disk & serves as a single-user computer.

• In such an environment, at any one time a significant proportion of the

workstations are idle which results in the waste of large amounts of CPU
time.

• Therefore, the idea of the workstation model is to interconnect all these

workstations by a high-speed LAN so that idle workstations may be used
to process jobs of users who are logged onto other workstations & do
not have sufficient processing power at their own workstations to get
their jobs processed efficiently.

• Example:Sprite system & Xerox PARC.

 Workstation-Server Model
•Client workstations
Workstation
– Diskless
– Graphic/interactive applications processed in local
– All file, print, http and even cycle computation
Workstation
requests are sent to servers.
Workstation
• Server minicomputers
– Each minicomputer is dedicated to one or more
100Gbps different types of services (print, database,…etc).
LAN • Client-Server model of communication
– RPC (Remote Procedure Call)
– RMI (Remote Method Invocation)
• A Client process calls a server process’ function.
Mini- Mini- Mini- • No process migration invoked
Computer Computer Computer
• Example: NFS
file server http server print server

• Disadvantage : capability of idle workstations is

not utilized.
14
 Workstation-Server Model
• The workstation model is a network of personal workstations
having its own disk & a local file system.

• A workstation with its own local disk is usually called a diskful

workstation & a workstation without a local disk is called a diskless
workstation. Diskless workstations have become more popular in
network environments than diskful workstations, making the
workstation-server model more popular than the workstation
model for building distributed computing systems.

• A distributed computing system based on the workstation-server

model consists of a few minicomputers & several workstations
interconnected by a communication network.
 Processor-Pool Model

• Clients:
– They log in one of terminals (diskless
workstations or X terminals)
– All services are dispatched to
servers.
100Mbps • Servers:
LAN
– Necessary number of processors are
allocated to each user from the pool.
• Better utilization but less interactivity
Server 1 Server N

16
 Processor-Pool Model
• The processor-pool model is based on the observation that
most of the time a user does not need any computing power
but once in a while the user may need a very large amount of
computing power for a short time.

• Therefore, unlike the workstation-server model in which a

processor is allocated to each user, in processor-pool model
the processors are pooled together to be shared by the users
as needed.

• The pool of processors consists of a large number of

microcomputers & minicomputers attached to the network.
 Processor-Pool Model
• Each processor in the pool has its own memory to load & run a system
program or an application program of the distributed computing
system.
• In this model no home machine is present & the user does not log onto
any machine.
• This model has better utilization of processing power & greater
flexibility.
 Cluster Model

Workstation
• Client
– Takes a client-server model
Workstation Workstation • Server
– Consists of many
100Mbps PC/workstations connected
LAN
http server2 to a high-speed network.
http server1 http server N – Puts more focus on
performance: serves for
Master Slave Slave Slave requests in parallel.
node 1 2 N

1Gbps SAN

19
 Grid Computing
• Goal
Workstation – Collect computing power of supercomputers
and clusters sparsely located over the nation
and make it available as if it were the electric
grid
Super- Mini- • Distributed Supercomputing
computer computer – Very large problems needing lots of CPU,
memory, etc.
Cluster
High-speed • High-Throughput Computing
Information high way – Harnessing many idle resources
• On-Demand Computing
Super- – Remote resources integrated with local
Cluster computation
computer
• Data-intensive Computing
– Using distributed data
• Collaborative Computing
– Support communication among multiple parties
Workstation Workstation

20
 Why are Distributed Computing
Systems Gaining Popularity?
• Inherently distributed applications
– Distributed DB, worldwide airline reservation, banking system
• Information sharing among distributed users
– CSCW (Computer Supported Cooperative Works )or groupware
• Resource sharing
– Sharing DB/expensive hardware and controlling remote lab. devices
• Better cost-performance ratio / Performance
– Emergence of Gbit network and high-speed/cheap MPUs
– Effective for coarse-grained or embarrassingly parallel applications
• Reliability
– Non-stopping (availability) and voting features.
• Scalability, Extensibility and Incremental Growth
– Loosely coupled connection and hot plug-in
• Flexibility
– Reconfigure the system to meet users’ requirements

21