CISC 525

Unit-3 Distributed Computing

By Sangwhan Cha, PhD.
Harrisburg University of Science and Technology
Mainly coming from
Tanenbaum, Raghu
www.masterraghu.com/subjects/DC
Computing?
• Computing is usually defined as the
activity of using and improving
computer technology, computer
hardware and software.
-
wikipedia
Types of Computing
• Monolithic
• Distributed
• Parallel
• Cooperative
Monolithic Computing
• The simplest form of computing, a single computer,
such as a personal computer(PC), is used for
computing.

• The computer is not connected to any network,

and thus it may use only the resources within its
immediate access. This form of computing is called
Monolithic computing.
Monolithic Computing – Cont1.
• Examples: Use of applications such as word
processing program or a spreadsheet on a pc.

• Mainframe can be used a Monolithic Machine.

• Multiple users can engage in monolithic computing.

• The resources can be shared by concurrent users

using technique known as “Time Sharing”.
Monolithic Computing – Cont2.

Terminals Main frame

Distributed Computing
• Distributed Computing is performed in a
distributed system.

• Distributed System is a collection of independent

computers, interconnected via a network, that are
capable of collaborating on a task. Computers are
considered independent if they do not share
memory or program execution space.
Distributed Computing –Contd1.
• Distributed computing involves computing
performed among multiple network-connected
computers, each of which has its own processor(s)
and other resources.
Distributed Computing ? –contd2.
• Example: The world wide web is an excellent
example of this type of computing.

• When you use a browser to visit a web site, a

program such as Internet Explorer runs on local
system and interacts with a program(known as web
server) which runs on remote system to fetch a file
or resource.
Parallel Computing
• Parallel computing is typically performed on a
single computer that has multiple CPU’s.

• It is also possible to compute by connecting the

computers in a network but requires special
software.
Parallel computing – Contd1.
• Example: weather forecasting, biology, etc.
Parallel computing – Contd2.
• Traditionally, software has been written for serial
computation:
• To be run on a single computer having a single Central
Processing Unit (CPU);
• A problem is broken into a discrete series of instructions.
• Instructions are executed one after another.
• Only one instruction may execute at any moment in
time.
Parallel computing – Contd3.
• In the simplest sense, parallel computing is the simultaneous use of multiple
compute resources to solve a computational problem.
• To be run using multiple CPUs
• A problem is broken into discrete parts that can be solved concurrently
• Each part is further broken down to a series of instructions
• Instructions from each part execute simultaneously on different CPUs
Parallel computing – Contd4.
• The compute resources can include:
• A single computer with multiple processors;
• A single computer with (multiple) processor(s) and some
specialized computer resources (GPU, FPGA …)
• An arbitrary number of computers connected by a
network;
• A combination of both.
Cooperative Computing
• Cooperative Computing is done using multiple
computers for processing a task by sharing a
problem into units between multiple computers.
Cooperative Computing –Contd1.
• Example: Hadoop System
Strengths of Distributed Computing
• Availability
• Resource Sharing
• Scalability
• Fault tolerance
Availability
• Computers connected to internet has become
universally available and generally affordable, the
large number of interconnected computers makes
for an ideal community for distributed computing.
Resource Sharing
• The architecture of distributed computing mirrors
the computing architectures of modern
organizations.
• Each organization independently maintains
computers and resources that are local to the
organization while sharing resources over the
network.
Scalability
• Distributed computing provides scalability in that
increasing demand for resources can be addressed
effectively with addition of resource required.
Fault tolerance
• Distributed computing provides the opportunity for
fault tolerance in that a resource can be replicated
to sustain its availability in the presence of failures.
Weaknesses of Distributed
Computing
• Multiple points of failure
• Security concerns
Multiple points of failure
• There are more points of failure in distributed
computing.
• Since multiple computers are involved, all of which
depend on the network for communication, the
failure of one or more computers or one or more
network links can cause trouble for distributed
computing.
Security Concerns
• In distributed computing there are more
opportunities for unauthorized attack.
• The decentralization of control makes it difficult to
implement and enforce security policies, hence
distributed computing is vulnerable for security
breaches and unauthorized access.
Operating Systems Concepts

• Computer Programs and process

• Concurrent Programming
Program & Process
• A software program is an artifact constructed by a
software developer using some form of
programming languages.

• When a program is “run” or executed, on a

computer it is represented as a process.
Process State Transistion
Concurrent Programming
• Distributed computing involves concurrent
programming, which is programming that involves
the simultaneous execution of processes.
• Concurrent Processes executed on multiple computers
• Concurrent Processes executed on a single computer.
• Concurrent Programming in a process.
Parent and Child Processes
• At runtime a process may spawn subordinate
processes or child process.

• The process that spwans subordinate process is

called Parent Process.

• A Child process is a complete process, consisting of

an executing program, its own current values, and
state information, some of which is inherited from
the parent process.
Threads
• A process may spawn threads, also known as
lightweight processes.

• Threads carry a minimum of state information, but

otherwise behave the same as processes.

• They incur less overhead, threads are preferred

over child processes.
Architecture of Distributed Applications
• The idea of using a multilayer architecture to
organize the functionalities of a data network can
be applied to distributed applications.
Functionalities
• The functionalities of the distributed applications
can be classified in three layers:

• Presentation Layer
• Application Logic Layer
• Service Layer
Presentation Layer
• It provides the user interface.
• Example:
• If the application is a shopping cart, this layer generates
the set of web pages that are viewable by a shooper
using a browser
Application Logic Layer
• It provides the computation for the application.
• Example:
• This layer is responsible for such tasks as credit card
verification and computing the dollar amounts of the
orders,sales tax and delivery costs.
Service Layer
• It provides the underlying services needed to
support the functionalities of the top two layers.
Services may include data access facilities such as
DBMS,DNS,IPC
 Types of Distributed Systems
• Distributed Computing Systems
– Clusters
– Grids
– Clouds
• Distributed Information Systems
– Transaction Processing Systems
– Enterprise Application Integration
• Distributed Embedded Systems
– Home systems
– Health care systems
– Sensor networks
 Cluster Computing
• A collection of similar processors (PCs,
workstations) running the same operating
system, connected by a high-speed LAN.
• Parallel computing capabilities using
inexpensive PC hardware
• Replace big parallel computers (MPPs)
 Cluster Types & Uses
• High Performance Clusters (HPC)
– run large parallel programs
– Scientific, military, engineering apps; e.g., weather
modeling
• Load Balancing Clusters
– Front end processor distributes incoming requests
– server farms (e.g., at banks or popular web site)
• High Availability Clusters (HA)
– Provide redundancy – back up systems
– May be more fault tolerant than large mainframes
Clusters
• Linux-based
• Master-slave paradigm
– One processor is the master; allocates tasks to
other processors, maintains batch queue of
submitted jobs, handles interface to users
– Master has libraries to handle message-based
communication or other features (the
middleware).
 Cluster Computing Systems
• Figure 1-6. An example of a cluster
computing system.

Figure 1-6. An example of a (Beowolf) cluster

computing system
 Grid Computing Systems
• Modeled loosely on the electrical grid.
• Highly heterogeneous with respect to
hardware, software, networks, security
policies, etc.
• Grids support virtual organizations: a
collaboration of users who pool resources
(servers, storage, databases) and share them
• Grid software is concerned with managing
sharing across administrative domains.
 Grids
• Similar to clusters but processors are more loosely
coupled, tend to be heterogeneous, and are not all
in a central location.
• Can handle workloads similar to those on
supercomputers, but grid computers connect over
a network (Internet?) and supercomputers’ CPUs
connect to a high-speed internal bus/network
• Problems are broken up into parts and distributed
across multiple computers in the grid – less
communication between parts than in clusters.
 Cloud Computing
• Provides scalable services as a utility over
the Internet.
• Often built on a computer grid
• Users buy services from the cloud
 Summary
• Cluster/grid/cloud distinctions