Scribd
Upload
21 views20 pages

Cloud Computing: Mr. Ajay B. Kapase

Uploaded by

shivamkashid70
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
21 views20 pages

Cloud Computing: Mr. Ajay B. Kapase

Uploaded by

shivamkashid70
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

Cloud Computing

Lecture 3
13-8-2024
Mr. Ajay B. Kapase
Principles of Parallel and Distributed Computing
• Terms parallel and distributed computing used interchangeably.

• There is a slight difference between these two terms.

• Parallel Computing: Tightly Coupled System involving single computer


with multiple processors/cores.

• Distributed Computing: Loosely Coupled system involving multiple


computers (nodes) connected through network.
Parallel Computing
• A single computation is divided in to several units.

• Each unit is handled by separate processor / separate core.

• Has single physical shared memory.

• Different processors and can communicate with each other by means of the
shared memory

• Components belong to single computer, Hence Homogeneous (each processor is


of same type and same capacity)
Parallel Computing

P1 P2 P3 P4

Shared Memory

IO

Single Computer
Distributed Computing
• A single computing task is divided in several units.

• Every unit is executed concurrently on different computing elements.

• Mostly processors on different nodes.

• Each node has its own processor and its own memory.

• These nodes communicates with each other over network.

• Promotes heterogeneous characteristics


Distributed vs. Parallel Computing
Elements of parallel computing

• Parallel Processing
• Processing of multiple tasks simultaneously on multiple processors.

• A given task is divided into multiple subtasks using a divide-and-conquer technique.

• Each subtask is processed on a different central processing unit (CPU).

• Programming on a multiprocessor system using the divide-and-conquer technique is


called parallel programming.
Elements of parallel computing
• Need of Parallel Processing
• Computational requirements are ever increasing in the areas of both scientific and
business computing.

• Such as Technical computing problems, require high-speed computational power, are


related to life sciences, aerospace, mechanical design and analysis.

• Sequential architectures are reaching physical limitations as they are constrained by


the speed of light and thermodynamics laws.

• The technology of parallel processing is mature there is already significant R&D work
done on development tools and environments
Elements of parallel computing
• Hardware architectures for parallel processing
• Based on the number of instruction and data streams that can be processed
simultaneously, computing systems are classified as

1. Single-instruction, single-data (SISD) systems


2. Single-instruction, multiple-data (SIMD) systems
3. Multiple-instruction, single-data (MISD) systems
4. Multiple-instruction, multiple-data (MIMD) systems
Single-instruction, single-data (SISD) systems
• This is the most basic form of computer architecture, typically found in traditional
sequential computers.

• a single instruction operates on a single data stream at a time.

• CPU fetches an instruction from memory, executes it on a single piece of data, and
then moves to the next instruction.

• Example: Traditional single-core processors like those in early computers or basic


microcontrollers.

• Use Case: SISD is used in simple applications where parallelism is not required.
Single-instruction, single-data (SISD) systems
Single-instruction, multiple-data (SIMD) systems
• Multiprocessor machine capable of executing the same instruction on all
the CPUs but operating on different data streams.

• A single instruction is applied to multiple data streams at the same time.

• Example: Graphics Processing Units (GPUs) SIMD systems, where the same
operation is applied to multiple pixels simultaneously.

• Use Case: SIMD is commonly used in multimedia, scientific computing, and


data parallel applications like image processing and matrix operations.
Single-instruction, multiple-data (SIMD)
systems
Multiple-instruction, single-data (MISD) systems
• Involves multiple instructions operating on a single data stream.

• Different processing units perform different operations on the same


data.

• Architecture is rare and not commonly used

• Example: Used for read only operations (checking data)


Multiple-instruction, Single-data (MISD)
systems
Multiple-instruction, multiple-data (MIMD) systems
• Most flexible and powerful architecture where multiple instructions operate on
multiple data streams.

• Different processors execute different instructions on different data.

• This architecture allows for true parallel processing.

• Example: supercomputers often use MIMD architecture, each core or processor works
on a different part of a problem simultaneously.

• MIMD is used in complex, large-scale applications like scientific, engineering


applications
Multiple-instruction, multiple-data (MIMD)
systems
Approaches of Parallel Programming
• Data parallelism:
• divide-and-conquer technique
• split data into multiple sets
• each data set is processed on different Processor using same instruction.
• Uses SMID Model
• Process parallelism:
• a given operation has multiple (but distinct) activities.
• Each activity is executed by distinct processors.
• Farmer-and-worker model:
• one processor is configured as master and all remaining are as worker.
• the master assigns jobs to worker processors
• Upon completion of task master collects and combines the results.
Elements of distributed computing
• A distributed system is one in which components located at networked
computers communicate and coordinate their actions only by passing
messages.
• Components of a distributed system:
• Hardware
• OS
• Middleware
• Distributed Applications
Layered View of Distributed System

You might also like