Parallel And Distributed
Computing
OUTLINE
Programming Models
aka Inter Process Communication
Programming Models
There are two primary types of Programming Models
1. Share Memory Model 2. Message Passing Model
Share Memory Model
• Programmer views his program as collection of
processes which use common or share variables.
• Processors may not have a private program or data
memory
• common program and data are stored in main
memory, which is then accessed by all processors
• i.e. parallel processes share global address space
Share Memory Model
• Each processor is assigned a different part of program
and the data
• the main program creates separate process for each
processor
• After execution, all processors have to rejoing main
program
Share Memory Model
Share Memory Model-Advantages
Ease of programming, traditional programming
Efficient data sharing, when multiple thread of same
process need to access data,
Faster communication as they direct read/write to share
memory
Share Memory Model-Disadvantages
Complex synchronization, incorrect synchronization
can lead to race condition
Limited Scalability
Security if one process has malicious intent, it can alter
the data
Message Passing
Provide mechanism to allow processes to communicate
and to synchronize without sharing same address
space.
Processes exchange data by sending or receiving
messages through communication channel
Message can be either synchornous and asynchronous
Message Passing
Message Passing-Advantages
Security, more secure than Share Memory Model,
because messages are send directly between
processes,
Flexibility allows processes to communicate even if
they are not running on same OS
Error Handling each message is processed
independently and doesn’t affect other message or
entire system
Message Passing-Disadvantages
More Overhead than share memory because message
must be copied and queue for transmission
Latency is more, message must be processed by OS
Complexity it requires explicit coding to send and
receive message and nedd to handle message queues
and buffering
Summary
Both mechanisms have their advantages and
disadvantages, and the choice of mechanism depends
on the specific requirements of the application. Shared
memory is faster and more efficient for sharing large
amounts of data, while message passing is more
secure and flexible.
Share Memory Message Pasing
• Use for data • Use for data
communication communication
• Use to communicate b/w • Use to communicate in
single/Multiprocessor distributed environment
• Faster because • take more time, because
communication is done via performed via kernel
shared memory (System calls)
• Synchoronization is done • Synchronization is done
through locks and through message passing
semaphores primitives
