Microprocessor & Computer
Architecture (μpCA)
Unit 4: Parallel Computer Memory Architecture
and Flynn’s Taxonomy
UE22CS251B
Session : 4.2
Microprocessor & Computer Architecture (μpCA)
Parallel Computer Memory Architectures
Shared Memory Architecture Distributed Memory Architecture
Hybrid Architecture
Microprocessor & Computer Architecture (μpCA)
With Graphical Processing Unit
Microprocessor & Computer Architecture (μpCA)
Parallel Programming Languages
OpenMP: (Open Multi Processing):
API that Support multiprocessing in C, C++, Fortran. Now with Python also.
MPI: (Message Passing Interface):
C, C++, Fortran, Java, Python, Ocaml, R……etc
CILK: Customized C Language
CUDA (Computer Unified Device Architecture): for Nvidia GPU
Pthreads:
Microprocessor & Computer Architecture (μpCA)
Flynn's Taxonomy of Computer Architecture
• One of the more widely used classifications, in use since 1966
• View Computer Architecture in two independent dimensions
• Instruction
• Data
Microprocessor & Computer Architecture (μpCA)
Analogy of Flynn’s Classifications
• An analogy of Flynn’s classification is the check-in desk at an airport
SISD: A single desk.
SIMD: Many desks and a supervisor with a megaphone giving instructions
that every desk obeys.
MIMD: Many desks working at their own pace, synchronized through a
central database.
Microprocessor & Computer Architecture (μpCA)
SISD: Single Instruction Single Data
Single instruction: only one instruction stream is being acted on by the CPU
during any one clock cycle.
Single data: only one data stream is being used as input during any one
clock cycle.
Deterministic Execution .
Very few are found in recent days.
• Few representatives are Intel Atom Family
(Silverthorne, Lincroft, Diamondville, Pineview)
Microprocessor & Computer Architecture (μpCA)
SIMD: Single Instruction Multiple Data
A type of Parallel computer
Best suited for specialized problems characterized by a high degree
of regularity, such as image processing.
Two varieties: Processor Arrays and Vector Pipelines
Representatives:
Processor Arrays: Connection Machine CM-2,
Maspar MP-1, MP-2
Vector Pipelines: IBM 9000, Cray C90, Fujitsu VP, NEC SX-2,
Hitachi S820
Microprocessor & Computer Architecture (μpCA)
MIMD: Multiple Instruction Multiple Data
• Currently, the most common type of parallel computer. Most modern computers fall
into this category.
• Execution can be synchronous or asynchronous, deterministic or non-deterministic
• Representatives: Most current supercomputers, networked parallel computer "grids"
and multi-processor SMP computers - including some types of PCs.
Microprocessor & Computer Architecture (μpCA)
MISD: Multiple Instruction Single Data (Does Not exist)
• Few actual examples of this class of parallel computer have ever existed. One is the
experimental Carnegie-Mellon University.
• A single data stream is fed into multiple processing units.
• Representatives: Systolic Arrays
Microprocessor & Computer Architecture (μpCA)
Systolic Arrays :
Motivation:
• Design an accelerator that is
• Simple and Regular design : # of Unique parts small and regular
• High Concurrency : High Performance
• Balanced Computation and I/O (bandwidth management).
• Idea: Replace a single Processing Element with a regular array of PEs
and carefully orchestrate flow of data between PEs.
Microprocessor & Computer Architecture (μpCA)
Systolic Arrays : Matrix Representation
Microprocessor & Computer Architecture (μpCA)
Systolic Arrays : Matrix Representation
Microprocessor & Computer Architecture (μpCA)
SIMD - Array processor
• Single Computer with Multiple parallel processors
• Processing Units are designed to work together under the supervision of a single control unit.
• Results in a single instruction stream and multiple data streams.
Ex : An Array Processor general block diagram is shown.
• It comprises several identical processing elements (PEs),
each with its local memory M
• An ALU and registers are included in each processor
element.
• The master control unit controls the processing elements'
actions
• It also decodes instructions and determines how they should
be carried out
THANK YOU
Team MPCA
Department of Computer Science and Engineering
