Scribd
Upload
28 views

Lecture 13 - Programming Models

The document discusses various parallel programming models that abstract hardware and memory architectures, including Data Parallel, Task Parallel, Process-Centric, and Shared/Distributed Memory models. Each model has distinct characteristics for sharing information and dividing work among processors, with examples illustrating their application. The document emphasizes the importance of understanding these models for effective parallel computation in software implementation.

Uploaded by

noor.semab.butt
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
28 views

Lecture 13 - Programming Models

The document discusses various parallel programming models that abstract hardware and memory architectures, including Data Parallel, Task Parallel, Process-Centric, and Shared/Distributed Memory models. Each model has distinct characteristics for sharing information and dividing work among processors, with examples illustrating their application. The document emphasizes the importance of understanding these models for effective parallel computation in software implementation.

Uploaded by

noor.semab.butt
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 15

CS-423 Parallel and Distributed Computing

Lecture 13
Programming Models
Parallel Programming Model
• Parallel programming models exist as an abstraction of hardware and
memory architectures.
• In fact, these models are not specific and do not refer to particular types
of machines or memory architectures.
• They can be implemented (at least theoretically) on any kind of
machines.
• Compared to the previous subdivisions, these programming models are
made at a higher level and represent the way in which the software
must be implemented to perform a parallel computation.
• Each model has its own way of sharing information with other
processors in order to access memory and divide the work.
Parallel Programming Model
• Most widely used models are discussed here
• Data Parallel Model
• Task Parallel Model
• Process-Centric Model
• Shared/Distributed Memory Model
Data Parallel Model
• In data parallel model, tasks are assigned to processes and each task
performs similar types of operations on different data.
• Data parallelism is a consequence of single operations that is being
applied on multiple data items.
• Many programs apply the same operation on different parts of the
common structure
Data Parallel Model
Data Parallel Model
Example
• Suppose the task is to add two arrays of 100 elements store the result in
another array. If there are four processor then each processor can do 25
additions.
Data Parallel Model
• The primary characteristic of data-parallel model problems is that the
intensity of data parallelism increases with the size of the problem,
which in turn makes it possible to use more processes to solve larger
problems.

• Example − Dense matrix multiplication


Task Parallel Model
• Parallel programs that emphasize the concurrent execution of different
tasks on the same or different data
• It is a form of parallelization of computer code across multiple
processors in parallel computing environments.
• Task parallelism focuses on distributing tasks —concurrently performed
by processes or threads—across different processors.
• Used for modularity reasons.
• Also known as function parallelism and control parallelism
Task Parallel Model
• In contrast to data parallelism which involves running the same task on
different components of data, task parallelism is distinguished by
running many different tasks at the same time on the same data or
different data.

• Example: Pipelining, which consists of moving a single set of data


through a series of separate tasks where each task can execute
independently of the others
Process Centric Model
• Process-oriented programming can be defined as a paradigm in which
the process structure and communication between processes of a
system are the primary concerns.
• It separates the concerns of data structures and the concurrent
processes that act upon them.
• The model allows the creation of large scale applications that partially
share common data sets.
• Programs are functionally decomposed into parallel processes that
create and act upon logically shared data.
Shared/Distributed Memory Model
Shared Memory Model
• In this model the tasks share a single shared memory area, where the
access (reading and writing data) to shared resources is asynchronous.
• There are mechanisms that allow the programmer to control the access
to the shared memory, for example, locks or semaphores.
• This model offers the advantage that the programmer does not have to
clarify the communication between tasks.
• An important disadvantage in terms of performance is that it becomes
more difficult to understand and manage data locality; keeping data
local to the processor that works on it conserves memory accesses,
cache refreshes, and bus traffic that occur when multiple processors use
the same data.
Shared/Distributed Memory Model
Shared Memory Model
Shared/Distributed Memory Model
Distributed Memory Model
• Different processes may be present on single multiple processors. Every
process has its own set of data
• The data transfer between the processes is achieved by send and receive
message requires co-operation between every process.
• There must be a receive operation for every send operation.
• Advantages
• The data can be stored anywhere.
• Communication between processes is simple.
• Many Message Passing Interfaces (MPI) are available.
• Disadvantage
• Programmers should take care that there is a receive function for every send
function.
Shared/Distributed Memory Model
Distributed Memory Model

You might also like