Cloud Computing: Mr. Ajay B. Kapase
Cloud Computing: Mr. Ajay B. Kapase
Lecture 3
13-8-2024
Mr. Ajay B. Kapase
Principles of Parallel and Distributed Computing
• Terms parallel and distributed computing used interchangeably.
• Different processors and can communicate with each other by means of the
shared memory
P1 P2 P3 P4
Shared Memory
IO
Single Computer
Distributed Computing
• A single computing task is divided in several units.
• Each node has its own processor and its own memory.
• Parallel Processing
• Processing of multiple tasks simultaneously on multiple processors.
• The technology of parallel processing is mature there is already significant R&D work
done on development tools and environments
Elements of parallel computing
• Hardware architectures for parallel processing
• Based on the number of instruction and data streams that can be processed
simultaneously, computing systems are classified as
• CPU fetches an instruction from memory, executes it on a single piece of data, and
then moves to the next instruction.
• Use Case: SISD is used in simple applications where parallelism is not required.
Single-instruction, single-data (SISD) systems
Single-instruction, multiple-data (SIMD) systems
• Multiprocessor machine capable of executing the same instruction on all
the CPUs but operating on different data streams.
• Example: Graphics Processing Units (GPUs) SIMD systems, where the same
operation is applied to multiple pixels simultaneously.
• Example: supercomputers often use MIMD architecture, each core or processor works
on a different part of a problem simultaneously.