The Big Picture: Where are We Now?

Memory Hierarchy Technology

° The Five Classic Components of a Computer ° Random Access:
Processor • “Random” is good: access time is the same for all locations
Input • DRAM: Dynamic Random Access Memory
Control
- High density, low power, cheap, slow
Memory
- Dynamic: need to be “refreshed” regularly
Datapath • SRAM: Static Random Access Memory
Output
- Low density, high power, expensive, fast
- Static: content will last “forever”(until lose power)

° Today’s Topics: ° “Non-so-random” Access Technology:

• Memory technologies • Access time varies from location to location and from time to time
• Technology trends • Examples: Disk, CDROM
• Impact on performance
° Sequential Access Technology: access time linear in
• Memory Hierarchy location (e.g.,Tape)
• The principle of locality
• Memory hierarchy terminology ° Main memory: DRAMs + Caches: SRAMs
cs 152 L1 6 .1 DAP Fa97,  U.CB cs 152 L1 6 .2 DAP Fa97,  U.CB
 Technology Trends Who Cares About the Memory Hierarchy?
Capacity Speed (latency) Processor-DRAM Memory Gap (latency)
Logic:2x in 3 years 2x in 3 years
DRAM: 4x in 3 years 2x in 10 years 1000 CPU
µProc
60%/yr.
“Moore’s Law”

Performance
Disk: 4x in 3 years 2x in 10 years (2X/1.5yr)
100 Processor-Memory
DRAM
Performance Gap:
Year Size Cycle Time
1000:1! 64 Kb 2:1! (grows 50% / year)
1980 250 ns 10
1983 256 Kb 220 ns DRAM
1986 1 Mb 190 ns
DRAM
9%/yr.
1989 4 Mb 165 ns 1 (2X/10 yrs)

1980
1981
1982

1985
1986
1987
1988

1991

1993
1994

1997
1998
1999
2000
1983
1984

1989
1990

1992

1995
1996
1992 16 Mb 145 ns
1995 64 Mb 120 ns

Time
cs 152 L1 6 .3 cs 152 L1 6 .4 DAP Fa97,  U.CB
DAP Fa97,  U.CB

DRAM Statistics
 Impact on Performance The Goal: illusion of large, fast, cheap memory
Inst Miss
° Suppose a processor executes at (0.5) Ideal CPI
16% (1.1)
• Clock Rate = 200 MHz (5 ns per cycle) 35% ° Fact: Large memories are slow, fast memories are
small
• CPI = 1.1
• 50% arith/logic, 30% ld/st, 20% control ° How do we create a memory that is large, cheap and
DataMiss
(1.6)
fast (most of the time)?
° Suppose that 10% of memory 49% • Hierarchy
operations get 50 cycle
miss penalty • Parallelism
° CPI = ideal CPI + average stalls per instruction
= 1.1 + ( 0.30 (memory access/ins)
x 0.10 (miss/memory access) x 50 (cycle/miss) )
= 1.1 cycle + 1.5 cycle
= 2. 6
° 58 % of the time the processor
is stalled waiting for memory!
° a 1% instruction miss rate would add
an additional 0.5 cycles to the CPI!

cs 152 L1 6 .5 DAP Fa97,  U.CB cs 152 L1 6 .6 DAP Fa97,  U.CB

 An Expanded View of the Memory System Why hierarchy works

° The Principle of Locality:

• Program access a relatively small portion of the address space at
any instant of time.
Processor

Control
Memory Probability
of reference
Memory
Memory

Memory

Datapath Memory
0 Address Space 2^n - 1

Speed: Fastest Slowest

Size: Smallest Biggest
Cost: Highest Lowest

cs 152 L1 6 .7 DAP Fa97,  U.CB cs 152 L1 6 .8 DAP Fa97,  U.CB

 Memory Hierarchy: How Does it Work? Memory Hierarchy: Terminology

° Hit: data appears in some block in the upper level

° Temporal Locality (Locality in Time): (example: Block X)
=> Keep most recently accessed data items closer to the processor • Hit Rate: the fraction of memory access found in the upper level
° Spatial Locality (Locality in Space): • Hit Time: Time to access the upper level which consists of
RAM access time + Time to determine hit/miss
=> Move blocks consists of contiguous words to the upper levels
° Miss: data needs to be retrieve from a block in the
lower level (Block Y)
• Miss Rate = 1 - (Hit Rate)
Lower Level • Miss Penalty: Time to replace a block in the upper level +
To Processor Upper Level Memory
Memory
Time to deliver the block the processor
Blk X ° Hit Time << Miss Penalty Lower Level
From Processor Blk Y
To Processor Upper Level Memory
Memory
Blk X
From Processor Blk Y

cs 152 L1 6 .9 DAP Fa97,  U.CB cs 152 L1 6 .10 DAP Fa97,  U.CB

We take advantage of the principle of locality by
implementing the memory of a computer as a memory
hierarchy. A memory hierarchy consists of multiple levels of
memory with different speeds and sizes. The faster
memories are more expensive per bit than the slower
memories and thus are smaller.

Memory Hierarchy of a Modern Computer System How is the hierarchy managed?

° By taking advantage of the principle of locality:

• Present the user with as much memory as is available in the ° Registers <-> Memory
cheapest technology.
• by compiler (programmer?)
• Provide access at the speed offered by the fastest technology.
° cache <-> memory
• by the hardware
Processor
° memory <-> disks
Control Tertiary • by the hardware and operating system (virtual memory)
Secondary Storage
Storage • by the programmer (files)
Second Main (Disk)
(Disk)
On-Chip
Registers

Level Memory
Cache

Datapath Cache (DRAM)

(SRAM)

Speed (ns): 1s 10s 100s 10,000,000s 10,000,000,000s

Size (bytes): 100s (10s ms) (10s sec)
Ks Ms Gs Ts
cs 152 L1 6 .11 DAP Fa97,  U.CB cs 152 L1 6 .12 DAP Fa97,  U.CB
 Static RAM (SRAM) 4 X 2 SRAM
15
D latch Address
Din[1] Din[0]

C Chip select D Dı D Dı
Q SRAMı 8 Write enable C latch Q C latch Q
Output enable Dout[7– 0]
32K × 8 Enable Enable

_
Write enable 0
Q
D 8 2-to-4ı D D
Din[7– 0] Dı Dı
decoder
C latch Q C latch Q
Enable Enable
1

D Dı D Dı
Address
C latch Q C latch Q
Select 0 Enable Enable Enable
In Out 2
Data 0

D Dı D Dı
Select 1 Enable C latch Q C latch Q
In Out Enable Enable
Data 1
Three-state buffers 3

Select 2 Enable Output

In Out Dout[1] Dout[0]

Data 2

Select 3 Enable
In Out
Data 3

cs 152 L1 6 .13 DAP Fa97,  U.CB cs 152 L1 6 .14 DAP Fa97,  U.CB
 32K X 8 SRAM Dynamic RAM (DRAM)

DRAM cell
Word line

Pass transistor
9-to-512ı 512 × 64ı 512 × 64ı 512 × 64ı 512 × 64ı 512 × 64ı 512 × 64ı 512 × 64ı 512 × 64ı
decoder 512 SRAM SRAM SRAM SRAM SRAM SRAM SRAM SRAM Capacitor
4M X 1 DRAM
Addressı
[14– 6]
Bit line

64 Rowı
Addressı 2048 × 2048ı
ı decoderı
array
[5– 0] 11-to-2048

Mux Mux Mux Mux Mux Mux Mux Mux

Address[10– 0] Column latches

Dout7 Dout6 Dout5 Dout4 Dout3 Dout2 Dout1 Dout0

Mux

Dout

cs 152 L1 6 .15 DAP Fa97,  U.CB cs 152 L1 6 .16 DAP Fa97,  U.CB