MODULE I

• Parallel Computing
• Distributed Computing, Interprocess
Communication
• Web 2.0
• Utility Computing
1.Introduction to cloud computing
• Cloud computing is a new technological trend and it supports better
utilization of IT infrastructures, services, and applications.
• It adopts a service-delivery model based on pay-per-use approach,
where users do not own infrastructure, platform, and applications but
use them for the time they need them. These IT assets are owned and
maintained by service providers who make them accessible through
the Internet.
ERAS OF COMPUTING
2.ELEMENTS OF PARALLEL COMPUTING(15mark)
• Processing of multiple tasks simultaneously on multiple processors is
called parallel processing. The parallel program consists of multiple
active processes (tasks) simultaneously solving a given problem. A
given task is divided into multiple subtasks using divide-and-conquer
technique, and each one of them is processed on different CPUs.
Programming on multi–processor system using divide-and-conquer
technique is called parallel programming.
• Many applications today require more computing power than a
traditional sequential computer can offer. Parallel processing provides
a cost-effective solution to this problem by increasing the number of
CPUs in a computer and by adding an efficient communication system
between them.
Hardware Architectures for Parallel
Processing
• The core elements of parallel processing are CPUs. Based on a
number of instruction and data streams that can be processed
simultaneously, computing systems are classified into the following
four categories:
● Single Instruction Single Data (SISD
● Single Instruction Multiple Data (SIMD)
● Multiple Instruction Single Data (MISD)
● Multiple Instruction Multiple Data (MIMD)
1. Single Instruction Single Data (SISD)

The speed of processing element in SISD model is limited .

2. Single Instruction Multiple Data (SIMD)
3. Multiple Instruction Single Data (MISD)
4. Multiple Instruction Multiple Data (MIMD)
• A MIMD computing system is a multiprocessor machine capable of
executing multiple instructions on multiple data sets (see Fig. 2.5).
Each PE in the MIMD model has separate instructions and data
streams, and hence machines built using this model are well suited
for any kind of application. Unlike SIMD and MISD machines, PEs in
MIMD machines work asynchronously. MIMD machines are broadly
categorized into shared-memory MIMD and distributed-memory
MIMD based on how PEs are coupled to the main memory.
(a) Shared Memory MIMD Machine. In the shared memory MIMD
model, all the PEs are connected to a single global memory and they all
have access to it (see Fig. 2.6). Systems based on this model are also
called tightly-coupled multiprocessor systems.
(b) Distributed Memory MIMD Machine. In distributed memory MIMD
model, all PEs have a local memory. Systems based on this model are
also called loosely-coupled multiprocessor systems. The
communication between PEs in this model takes place through the
interconnection network (IPC-inter-process communication channel).
Approaches to Parallel Programming
A wide variety of parallel programming approaches are available. The most
prominent among them are the following:
● Data Parallelism ● Process Parallelism ● Farmer and Worker Model
In case of data parallelism, divide-and-conquer technique is used to split data into
multiple sets, and each data set is processed on different PEs by using the same
instruction. This approach is highly suitable for processing on machines based on
the SIMD model.
In the case of process parallelism, a given operation has multiple (but distinct)
activities, which can be processed on multiple processors.
In case of farmer and worker model, a job distribution approach is used; one
processor is configured as master and all other remaining PEs are designated as
slaves; the master assigns a job to slave PEs, and they on completion inform the
master which in turn collects the results. The above approaches can be utilized in
different levels of parallelism
Levels of Parallelism
Laws of Caution
3.ELEMENTS OF DISTRIBUTED COMPUTING(15mark)
2.1.Definition

• Distributed computing refers to a system where processing and data storage is

distributed across multiple devices or systems, rather than being handled by a
single central device. In a distributed system, each device or system has its own
processing capabilities and may also store and manage its own data. These devices
or systems work together to perform tasks and share resources, with no single
device serving as the central hub.
• One example of a distributed computing system is a cloud computing system,
where resources such as computing power, storage, and networking are delivered
over the Internet and accessed on demand. In this type of system, users can access
and use shared resources through a web browser or other client software.
2. 2 Components of a Distributed System
• Fig. 2.10 provides an over view of the different layers that areinvolved
in providing the services of a distributed system.
• At the very bottom layer, computer and network hardware constitute
the physical infrastructure; these components are directly managed
by the operating system that provides the basic services for: inter-
process communication, process scheduling and management, and
resource management in terms of file system and local devices. Taken
together, these two layers become the platform on top of which
specialized software is deployed to turn a set of networked
computers into a distributed system.
2.3Architectural Styles for Distributed Computing
a. Components and Connectors(2 mark)
components and connectors, since these are the basic building blocks
with which architectural styles are defined.
A component represents a unit of software that encapsulates a function
or a feature of the system. Examples of components can be programs,
objects, processes, pipes, and filters.
A connector is a communication mechanism that allows the cooperation
and coordination among components.
We organize the architectural styles into two major classes:
● Software architectural styles ● System architectural styles
The first class relates to the logical organization of the software, while
the second class includes all those styles that describe the physical
organization of distributed software systems in terms of their major
components.
b. Software Architectural Styles
• Software architectural styles are based on the logical arrangement of software components. They
are helpful because they provide an intuitive view of the whole system, despite its physical
deployment.
1.Data- Centered Architectures
These architectures identify the data as the fundamental element of the software system and
access to shared data is the core characteristic of the data centered architectures.
The Repository architectural style is the most relevant reference model in this category. It is
characterized by two main components: the central data structure, which represents the current
state of the system, and a collection of independent components, which operate on the central
data.
The Blackboard architectural style is characterized by three main components:
2.Data-Flow Architectures
Data-flow styles explicitly incorporate the pattern of data flow, since their design is
determined by an orderly motion of data from component to component, which is the
form of communication between them.
Batch-Sequential Style: The batch-sequential style is characterized by an ordered
sequence of separate programs executing one after the other. These programs are
chained together by providing as input for the next program the output generated by the
last program after its completion, which is most likely in the form of a file.
Pipe-and-Filter Style:Each component of the processing chain is called filter. Filters,
generally, do not state neither know the identity of the previous or the next filter, and they
are connected within memory data structures such as FIFO buffers or other structures. This
particular sequencing is called pipelining and introduces concurrency in the execution of
the filters.
3.Virtual Machine Architectures
This class of architectural styles is characterized by the presence of an abstract
execution environment (generally referred as virtual machine) that simulates
features that are not available in the hardware or software.
Rule-Based Style:
This architecture is characterized by representing the abstract execution
environment as an inference engine. Programs are expressed in the form of rules or
predicates.
Interpreter Style:
The core feature of the interpreter style is the presence of an engine that is used to
interpret a pseudo program expressed in a format acceptable for the interpreter.
4.Call and Return Architectures
This category identifies all those systems that are composed by components mostly connected
together by method calls.
Top-Down Style : This architectural style is quite representative for systems developed with
imperative programming, which leads to a divide-and-conquer approach for problem resolution.
Systems developed according to this style are composed of one large main program that
accomplishes its tasks by invoking sub-programs or procedures. The components in this style are
procedures and sub-programs, and connections are method calls or invocation. The calling program
passes information with parameters and receives data from return values or parameters.
Object-Oriented Style : This architectural style encompasses a wide range of systems that have
been designed and implemented by leveraging the abstractions of object-oriented programming.
Systems are specified in terms of classes and implemented in terms of objects. Classes define the
type of components by specifying the data that represent their state and the operations that can be
done over this data.
Layered Style: The layered system style allows the design and the
implementation of software systems in terms of layers, which provide a different
level of abstraction of the system. Each layer generally operates with at most two
layers: the one that provides a lower abstraction level, and the one that provides a
higher abstraction layer. Specific protocols and interfaces define how adjacent
layers interact. It is possible to model such systems as a stack of layers—one for
each level of abstraction.
5.Architectural Styles based on Independent Components

This class of architectural styles model systems in terms of independent

components, having their own life cycle, which interact to each other in
order to perform their activities.
Communicating Processes : In this architectural style, components are
represented by in dependent processes that leverage inter-process
communication (IPC) facilities for coordination management.
Event Systems:each component also publishes (or announces) a collection of
events that other components can register with. In general, other
components provide a callback that will be executed when the event is
activated.
C. System architectural styles
• System architectural styles cover the physical organization of components
and processes over a dis tributed infrastructure.
• Two fundamental reference styles: client-server and peer-to-peer.
(a) Client-Server:
Fig. 2.12, the client-server model features two major components: a server
and a client. These two components interact with each other through a
network connection by using a given protocol.
The communication is unidirectional: the client issues a request to the
server, and server after processing the request returns a response. There
could be multiple client components issuing requests to the server that is
passively waiting for them. Hence, the important operations in the client-
server paradigm are request, accept (client side), and listen and response
(server side).
Two-tier Architecture. This architecture partitions the systems into two
tiers, which are located one in the client component and the other on
the server. The client is responsible for the presentation tier by
providing a user interface, while the server concentrates the
application logic and the data store into a single tier.
Three-tier Architecture/N-tier Architecture. The three-tier architecture
separates the presentation of data, the application logic, and the data
storage into three tiers. This architecture is generalized into an N-tier
model, in case it is necessary to further divide the stages composing
the application logic and storage tiers.
(b) Peer-to-Peer.
• This model introduces a symmetric architecture where all the
components, called peers.
• Peers play the same role and incorporate both the client and server
capabilities of the previous model.
• More precisely, each peer acts as a server when it processes requests
from other peers and as a client when it issues requests to other
peers.
• The disadvantage of this approach is that the management of the
implementation of algorithms is more complex if compared to the
client-server model (Fig. 2.13).
MODELS FOR INTERPROCESS COMMUNICATION
• Distributed systems are composed of a collection of concurrent processes
interacting with each other by means of a network connection. Therefore,
inter-process communication (IPC) is a fundamental aspect of distributed
systems design and implementation.
• IPC is used to either exchange data and information or to coordinate the
activity of processes.
• Two reference model for architecting the communication among
processes:
1. Message-based Communication
2. Models for Message-based Communication
1. Message-based Communication
(a) Message Passing: This paradigm introduces the concept of message
as the main abstraction of the model. The entities exchanging
information explicitly encode, in the form of message, the data to be
exchanged. The structure and the content of a message vary
according to the model. Examples of this model are Message Passing
Interface (MPI ) and OpenMP.
(b) Remote Procedure Call (RPC):This paradigm extends the concept of
procedure call beyond the boundaries of a single process, thus triggering
the execution of code in remote processes. A remote process hosts a
server component, thus allowing client processes to request the
invocation of methods and returns the result of the execution.
• (c) Distributed Objects:This is an implementation of the RPC model
for the object-oriented paradigm, and contextualizes this feature for
the remote invocation of methods exposed by objects. Each process
registers a set of interfaces that are accessible remotely. Client
processes can request a pointer to these interfaces and invoke the
methods available through them.
2. Models for Message-based Communication
(a) Point-to-Point Message Model
• This model organizes the communication among single components. Each
message is sent from one component to another, and there is a direct ad
dressing to identify the message receiver.
• In a point-to-point communication model, it is necessary to know the
location or how to address another component in the system.
• There is no central infrastructure that dispatches the messages and the
communication is initiated by the sender of the message.
• It is possible to identify two major sub-categories: direct communication
and queue-based communication.
• In the former, the message is sent directly to the receiver and processed at
the time of reception.
• In the latter, the receiver maintains a message queue where the messages
received are placed for later processing.
(b) Publish-Subscribe Message Model:
• which is based on notification among components.
• There are two major roles: the publisher and the subscriber.
• The former provides facilities for the latter to register its interest in a
specific topic or event. Specific conditions holding true on the publisher
side can trigger the creation of messages which are attached to a specific
event. This message will be available to all the subscribers that registered
for the corresponding event.
• There are two major strategies for dispatching the event to the
subscribers:
Push Strategy: In this case, it is the responsibility of the publisher to notify all
the subscribers.
Pull Strategy :In this case, the publisher simply makes available the message
for a specific event, and it is the responsibility of the subscribers to check
whether there are messages on the events that are registered.
(c) Request-Reply Message Model.
• The request-reply message model identifies all those communication
models, where for each of the message sent by a process, there is a
reply.
• Point-to point message models are more likely to be based on a
request-reply interaction especially in the case of direct
communication.
• Publish-subscribe models are less likely to be based on request-reply
since they rely on notifications.