Parallel Processing
Parallel Processing
• It is an efficient form of information processing
which emphasizes the exploitation of concurrent
events in the computing process.
• Purpose of parallel processing
– To speed up the computer processing capability
– To increase its throughput
• Advantages over sequential processing
– Cost effective
– Improved performance ,i.e., faster execution time
Concurrency implies parallelism, simultaneity,
and pipelining
– Parallel events occur in multiple resources
during the same time interval
– Simultaneous events may occur at the same
time instant
– Pipelined events may occur in overlapped
time spans
• Note that parallel processing differs from
multitasking, in which a single CPU
executes several programs at once.
• Parallel processing is also called parallel
computing.
4 levels of parallel processing
• Job or Program level
• Task or procedure level
• Interinstruction level
• Intrainstruction level
Distributed Processing is a form of parallel
processing in a special environment.
Parallelism in Uniprocessor System
Basic Uniprocessor System
• A typical Uniprocessor (super
minicomputer) consists of 3 major
components:
– Main memory
– CPU
– I/O subsystem
• Uniprocessor : Mainframe IBM
– Main memory: 4 LSUs, storage controller
– Peripherals are connected to the I/O channels
– Operation with the CPU is asynchronous
Parallel Processing Mechanisms
• Multiplicity of functional units
• Parallelism and pipelining within the CPU
• Overlapped CPU and I/O operations
• Use of hierarchical memory system
• Balancing of subsystem bandwidth
• Multiprogramming and Time-sharing
1.Multiplicity of functional units
• Use of multiple processing elements
under one controller
• Many of the ALU functions can be
distributed to multiple specialized units
• These multiple Functional Units are
independent of each other
Example:
• IBM 360/91
– 2 parallel execution units
• Fixed point arithmetic
• Floating point arithmetic(2 Functional units)
– Floating point add-sub
– Floating point multiply-div
2.Parallelism & pipelining within the
CPU
• Parallelism is provided by building parallel
adders in almost all ALUs
• Pipelining
– Each task is divided into subtasks which can
be executed in parallel
3.Overlapped CPU and I/O operations
• I/O operations can be performed
simultaneously with the CPU
computations by using
– separate I/O controllers
– I/O channels
– I/O processors
DMA-Cycle-stealing
• Perform I/O transaction between the
device and the main memory without the
intervention of the CPU
– I/O controller will generate a sequence of
memory addresses
– CPU is responsible for initiating the block
transfer
– CPU is notified, when the transfer is complete
Use of I/O Processor : achieves maximum
concurrency
• I/O subsys is facilitated by an IOP
• CPU is free to proceed with its primary task
• IOP consists of:
– I/O processor
– I/O channels
• I/O processor is attached directly to system bus
• I/O processor is capable of executing I/O
requests
4.Use of hierarchical memory system
• Speed of CPU = 1000 times speed of
Main memory
• hierarchical memory structure is used
to close up the speed gap
– Cache memory
– Virtual memory
– Parallel memories for array processors
5.Balancing of subsystem bandwidth
• Balancing bandwidth between main
memory and CPU
• Balancing bandwidth between main
memory and I/O
6.Multiprogramming and Time-sharing
• Multiprogramming
– Mix the execution of various types of
programs (I/o bound, CPU bound)
– The interleaving of CPU and I/O operations
across several programs
• Time-sharing
– Time-sharing OS is used to avoid high-priority
programs occupying the CPU for long
– Fixed or variable time-slices are used
– Creates a concept of virtual processors
Parallel Computer Models
• Pipeline Computers
• Array Computers
• Multiprocessor Computers
Performance of parallel computers
• Parallel computers with n identical
processors do not have speeds n times
faster than a single processor computer,
but less than n
• Speeds range from log2n to n/ln n
