Scribd
Upload
13 views

Lecture-4 Parallel Programming Model

Uploaded by

Saad Tayef
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
13 views

Lecture-4 Parallel Programming Model

Uploaded by

Saad Tayef
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 14

Course No: CSE 4117

Course Name: Parallel and


Distributed Processing
Parallel Programming Model

 Parallel programming model exists an abstraction above hardware and memory


architectures. It’s a set of program abstractions for fitting parallel activities from the
application to the underlying parallel hardware. It spans over different layers: applications,
programming languages, compilers, libraries, network communication, and I/O systems.

Shared Memory Model: Multiple processes share common memory space

Distributed Memory Model :The user makes call to libraries to explicitly share information
between processors.
Threads Model: A single process having multiple (concurrent) execution paths

Data Parallel Model: Data partitioning determines parallelism


Parallel Programming Model
Shared Memory Model (Without Thread)
• In this programming model, processes/tasks share a common address space, which they read
and write to asynchronously.
• Various mechanisms such as locks / semaphores are used to control access to the shared
memory, resolve contentions and to prevent race conditions and deadlocks.
• This is perhaps the simplest parallel programming model.
Parallel Programming Model
Shared Memory Model (Without
Thread)
• The processor may not have a private program or
data memory. A common program and data are
stored in the main memory. This is accessed by all
processors.
• Each processor is assigned a different part of
the program and the data. The main program
creates separate process for each processor.
• The process is allocated to the processor along
with the required data. These process are executed
indecently on different processors.
• After the execution, all the processors have to
rejoin the main program
Parallel Programming Model
Advantages of shared memory model
• Data "ownership" is lacking, so there is no need to specify explicitly the communication of
data between tasks.
• All processes have equal access to shared memory.

• Program development becomes simple.

Disadvantages of shared memory model


• Different techniques need to use to block the access of the same data among different
processors.
• Users will not understand where his variables use stored.
Parallel Programming Model
Distributed Memory Model
• A set of tasks that use their own local memory during computation.
• Multiple tasks can reside on the same physical machine and/or across an arbitrary number of machines.
• Tasks exchange data through communications by sending and receiving messages.
• Data transfer usually requires cooperative operations to be performed by each process. For example, a
send operation must have a matching receive operation.
Parallel Programming Model
Distributed Memory Model
Parallel Programming Model

Advantages of Distributed memory model


• Data can be stored anywhere.

• Communication between processes are simple.

• Many Manage Passing Interfaces (MPI) are available.

Disadvantages of Distributed memory model


• Programmers should take care that there is a receive function for every send function.

• If any dependent process is quit or stop then the other processes will stop working.
• What are the advantages and disadvantages of using a shared memory
model compared to a distributed memory model?

• How can race conditions and deadlocks be avoided in a shared


memory model?
Parallel Programming Model
Thread Model
• A thread is defined as a short sequence of
instructions, within a process.
• This programming model is a type of shared memory
programming.
• In the threads model of parallel programming, a single
"heavy weight" process can have multiple "light
weight", concurrent execution paths called threads.
• Each thread has local data, but also, shares the
common data.
• Threads communicate with each other through global
memory (updating address locations).
• This requires synchronization constructs to ensure
that more than one thread is not updating the same
global address at any time.
Parallel Programming Model
Thread Model
• Different threads can be
executed on same processor or
on different processors.

• If the threads are executed on


same processor, then the
processor switches between the
threads in a random fashion.

• If the threads are executed on


different processors, they are
executed simultaneously.
Parallel Programming Model

Advantages of Thread Model


• Programmer need not have to worry about parallel processing.

Disadvantages of Thread Model


• Care must be taken that no two threads update the shared resources
simultaneously.
Parallel Programming Model
Data Parallel Model
• Data parallelism is one of the simplest forms of parallelism. Here data set is organized into a common structure.
It could be an array. It may also be referred to as the Partitioned Global Address Space (PGAS) model.

• The data parallel model demonstrates the following characteristics:

• Address space is treated globally

• Most of the parallel work focuses on performing operations on a data set. The data set is typically organized
into a common structure, such as an array.

• A set of tasks work collectively on the same data structure, however, each task works on a different partition
of the same data structure.

• On shared memory architectures, all tasks may have access to the data structure through global memory.

• On distributed memory architectures, the global data structure can be split up logically and/or physically across
tasks.
Parallel Programming Model
 Example of Data Parallel Model
Int x [100],
For(i=0;i<100; i++)
y [100], z[i] = x[i] + y[i]
z [100]

Processor 1 Processor 2 Processor 3 Processor 4

For(i=0;i<25; i++) For(i=25;i<50; i++) For(i=50;i<75; i++) For(i=75;i<100; i++)


z[i] = x[i] + y[i] z[i] = x[i] + y[i] z[i] = x[i] + y[i] z[i] = x[i] + y[i]

You might also like