Scribd
Upload
3 views

Lecture 04 - Parallel Algorithm Models

Uploaded by

mabdulllahbhatti9193
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
3 views

Lecture 04 - Parallel Algorithm Models

Uploaded by

mabdulllahbhatti9193
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 18

Parallel And Distributed

Computing
OUTLINE

PARALLEL ALGORITHM
MODELS
Parallel algorithm models
The models of parallel algorithm are developed by considering
a strategy for dividing the data and processing method and
also applying a suitable strategy to reduce interaction.
Following are some parallel algorithm models are
• Data parallel models
• Task Graph Model
• Work Pool Model
• Master Slave Model
• Pipeline Model /Producer-Consumer Model
• Hybrid Model
Data Parallel Model

Tasks are assigned to processes and each task


performs similar types of operations, so that data
parallelism is achieved by applying single operation on
multiple data
Overhead is reduced by overlapping computation and
interaction
EffIcient for parallelizing data-intensive tasks.
Data Parallel Model

Example: MapReduce, where data is split into chunks,


and each worker node processes its chunk in parallel
Data Parallel Model
1. Simple algorithm model
2. Tasks are statistically mapped into processors and
each task perform similar operations
3. Identical operations being operate on different data
item so called data parallelism
4. The degree of parallelism is increase as the size of
problem size increase hence effective for big
problems
TASK GRAPH MODEL
Parallelism is expressed by a task graph which is either trivial
or non trivial.
Divides problem into separate tasks or subproblems, with each
processing unit responsible for executing specific task
Tasks are mapped statistically to in order to reduce interaction
Different problems are divided into different tasks to implement
a graph, each Problem is represented by vertex/task
Advantages:
• Effective for complex
applications with
interdependent tasks.
• Allows for better load
balancing.
Disadvantages:
• Complexity in managing
dependencies and
synchronization.
Work Pool Model
Tasks are assigned among processes to balance the load
so any process execute any process.
In this model operations are large but data is small
No pre assigning of tasks to process so assigning can be
centralized or decentralized
Task may be available in the beginning or may be
generated dynamically.
If the task is generated dynamically and decentralized
assigning of task is done the termination detection
algorithm is required.
Advantages:
• Simplifies load balancing,
as available workers can
pick tasks from the pool.
Disadvantages:
• May not be suitable for
tasks with complex
dependencies.
• Careful management of
the task pool is required to
avoid contention.
Example:
• Web server handling
multiple client requests
concurrently
Master/Slave Model
One or more process that generate tasks and
assign those tasks to slave for processes
Tasks are assigned before if:
1. Master can estimate the number of operations
2. Slaves are assigned smaller tasks
This model is suitable in share memory system
Master/Slave Model
Advantages:
• Effective for tasks with
complex dependencies or
coordination
• Allows centralized control
and monitoring of parallel
computations.
Disadvantages:
• Master can become
performance bottleneck
incase of high
coordination.
Master/Slave Model - Precautions

Master processes generate work and allocate to


worker processors hence no pre-mapping is needed
Care should be taken to assure that the master does
not become a congestion point. It may happen if tasks
are too small or processors are comparatively fast.
Tasks should be selected in a way that the cost of
performing a task dominates the cost of
communications.
Pipeline Model / Producer Consumer Model

Stream of data is passed on through processors


Data is passed through a series of processes
New data generate new tasks that is to be processed by
process in the queue
A queue Is built, by using either array, tree graph
It divides computation into stages, where each stage is
executed in parallel, and output of one stage becomes
input of next.
Simply, pipeline is like assembly line for processing tasks
step by step
Hybrid model

• When more than one model is required


for solving a problem, hybrid algorithm
model is used.
• A hybrid model consists of multiple
models either applied hierarchically or
multiple models applied sequentially to
different phases of the parallel
algorithm.

You might also like