Scribd
Upload
22 views

Lecture-3 Parallel Computer Memory Architecture

Parallel Computer Memory Architecture

Uploaded by

Meherab Hasan
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
22 views

Lecture-3 Parallel Computer Memory Architecture

Parallel Computer Memory Architecture

Uploaded by

Meherab Hasan
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 14

Parallel Computer Memory Architectures

❖Shared Memory

❖Distributed Memory

❖Hybrid Shared-Distributed Memory


Parallel Computer Memory Architectures
❖Shared Memory
❖ General Characteristics
▪ Shared memory parallel computers have the ability for all processors to access memory
as global address space.

▪ Multiple processors can operate independently but share the same memory resources.

▪ Changes in a memory location effected by one processor are visible to all other
processors.

▪ Shared memory machines have been classified as UMA and NUMA, based upon memory
access times.
Parallel Computer Memory Architectures
❖ Shared Memory
❖ Advantages
▪ Data sharing between processes is both fast and uniform due to the proximity of memory to
CPUs.

▪ Global address space provides a user-friendly programming perspective to memory.

❖ Disadvantages
▪ Primary disadvantage is the lack of scalability between memory and CPUs.

▪ Adding more CPUs can geometrically increases traffic on the shared memory-CPU path, and
for cache coherent systems, geometrically increase traffic associated with cache/memory
management.

▪ Programmer responsibility for synchronization constructs that insure "correct“ access of


global memory, is also increased.
Parallel Computer Memory Architectures
❖ Uniform Memory Access (UMA)
▪ Most commonly represented today by Symmetric Multiprocessor (SMP) machines.

▪ Identical processors.

▪ Equal access and access times to memory.

▪ Sometimes called CC-UMA - Cache Coherent UMA. Cache coherent means if one processor updates a location in
shared memory, all the other processors know about the update.
Parallel Computer Memory Architectures
❖ Non-Uniform Memory Access-
(NUMA)-
▪ Often made by physically linking two or more SMPs.

▪ One SMP can directly access memory of another SMP.

▪ Not all processors have equal access time to all memories.

▪ Memory access across link is slower (a processor can


access its own local memory faster than non-local
memory).

▪ If cache coherency is maintained, then may also be called


CC-NUMA - Cache Coherent NUMA.
Parallel Computer Memory Architectures

Basis for UMA NUMA


comparison
Basic Uses a single memory Multiple memory controller
controller
Memory Access time Equal Changes according to the
distance of microprocessor
Suitable for General purpose and Real-time and time-critical
time-sharing application applications
Bandwidth Limited More than UMA
Parallel Computer Memory Architectures
❖ Distributed Memory

❖ General Characteristics

▪ Distributed memory systems require a communication network to connect inter-processor memory.

▪ Processors have their own local memory. Memory addresses in one processor do not map to another
processor, so there is no concept of global address space across all processors.

▪ Because each processor has its own local memory, it operates independently. Changes it makes to
its local memory have no effect on the memory of other processors. Hence, the concept of cache
coherency does not apply.

▪ When a processor needs access to data in another processor, it is usually the task of the
programmer to explicitly define how and when data is communicated. Synchronization between
tasks is likewise the programmer's responsibility.
Parallel Computer Memory Architectures
❖ Distributed Memory

❖ Advantages

• Memory is scalable with the number of processors.


Increase the number of processors and the size of
memory increases proportionately.

• Each processor can rapidly access its own memory


without

• interference of other processor.


• The overhead of cache coherency
Parallel Computer Memory Architectures
❖ Distributed Memory
❖ Disadvantages
• The programmer is responsible for many of the details
associated with data communication between processors.

• It may be difficult to map existing data structures, based


on global memory, to this memory organization.

•Need distributed data structure.

• Non-uniform memory access times

•Data residing on a remote node takes longer to access


than node-local data.
Parallel Computer Memory Architectures
❖ Hybrid Shared-Distributed Memory
❖ General Characteristics
▪ The largest and fastest computers in the world today employ both shared and
distributed memory architectures.

▪ The shared memory component can be a shared memory machine and/or


graphics processing units (GPU).

▪ The distributed memory component is the networking of multiple shared


memory/GPU machines, which know only about their own memory - not the
memory on another machine. Therefore, network communications are required to
move data from one machine to another.
Parallel Computer Memory Architectures
❖ Hybrid Shared-Distributed Memory

❖Advantages and Disadvantages


▪Increased scalability is an important advantage.
▪Increased programmer complexity is an important disadvantage.
• What is SMP?
• Identify the memory architecture
• In which type of memory architecture does each processor have its own
local memory?
a) Shared memory
b) Distributed memory
• In which type of memory architecture does processor have its own local
memory?
a) Shared memory
b) Distributed memory
• Which one provides a user-friendly programming perspective to memory?
a) Shared memory
b) Distributed memory
• Which type of memory architecture requires the use of message passing to
exchange data between processors?
a) Shared memory
b) Distributed memory

You might also like