Module 3: Memory System

Part-1

Jay Prakash Maurya

DoSCSE
VIT University, Bhopal
Content
• What is Computer Memory?
• Why is memory important or needed for a
computer?
• Types of Computer Memory
• Primary Memory
• Secondary Memory
• Primary Vs Secondary Memory
• Memory Units
• References
What is Computer
Memory
• Computer memory is any physical device capable of
storing information temporarily, like RAM (random access
memory), or permanently, like ROM (read-only memory).
• Memory devices utilize integrated circuits and are used
by operating systems, software, and hardware.
Why is memory important
or needed for a computer?
• Each device in a computer operates at different speeds
and computer memory gives your computer a place to
quickly access data.
• If the CPU had to wait for a secondary storage device,
like a hard disk drive, a computer would be much
slower.
Types of Computer Memory

• Cache Memory
• Primary Memory/Main Memory
• Secondary Memory
 Primary Memory
• This is the main memory of the computer. CPU can
directly read or write on this memory. It is fixed on
the motherboard of the computer.
• Primary memory is further divided in two types:
1.RAM(Random Access Memory)
2.ROM(Read Only Memory)
RAM(Random Access Memory)

• RAM is a temporary memory. The information stored in

this memory is lost as the power supply to the
computer is turned off. That’s why it is also called
Volatile Memory.
• It stores the data and instruction given by the user and
also the results produced by the computer temporarily
 ROM(Read only
Memory)
• Information stored in ROM is permanent in nature, i.e., it
holds the data even if the system is switched off.
• It holds the starting instructions for the computer. ROM
cannot be overwritten by the computer. It is also called Non-
Volatile Memory.
SRAM V/S DRAM
 Secondary Memory
• This memory is permanent in nature. It is used to store
the different programs and the information
permanently (which were temporarily stored in RAM).
It holds the information till we erase it.
• Different types of secondary storage devices are:
1. Hard Disc, Compact Disc,
2. DVD, Pen Drive,
3. Flash Drive, etc.
 Hard Disc Drive
• Hard disk drive is made up of a series of circular disks
called platters arranged one over the other almost ½
inches apart around a spindle.
• Disks are made of non-magnetic material like aluminum
alloy and coated with 10-20 nm of magnetic material.
 CD Drive
• CD stands for Compact Disk. CDs are circular disks that
use optical rays, usually lasers, to read and write data.
• They are very cheap as you can get 700 MB of storage
space for less than a dollar. CDs are inserted in CD drives
built into CPU cabinet.
• They are portable as you can eject the drive, remove the
CD and carry it with you.
 DVD Drive
• DVD stands for Digital Video Display. DVD are optical
devices that can store 15 times the data held by CDs.
• They are usually used to store rich multimedia files
that need high storage capacity. DVDs also come in
three varieties – read only, recordable and rewritable.
 Pen Drive
• Pen drive is a portable memory device that uses solid
state memory rather than magnetic fields or lasers to
record data.
• It uses a technology similar to RAM, except that it is
nonvolatile. It is also called USB drive, key drive or
flash memory.
 Blu Ray Disk
• Blu Ray Disk (BD) is an optical storage media used to
store high definition (HD) video and other multimedia
filed.
• BD uses shorter wavelength laser as compared to
CD/DVD. This enables writing arm to focus more tightly
on the disk and hence pack in more data. BDs can store
up to 128 GB data.
Primary Vs Secondary Memory

Parameter Primary memory Secondary memory

The primary memory is
The secondary memory is
Nature categorized as volatile &
always a non-volatile memory.
nonvolatile memories.
Secondary memory is known
These memories are also called as a Backup memory or
Alias
internal memory. Additional memory or Auxiliary
memory.
Data cannot be accessed
directly by the processor. It is
Data is directly accessed by the
Access first copied from secondary
processing unit.
memory to primary memory.
Only then CPU can access it.
It's a volatile memory meaning It's a non-volatile memory so
Formation data cannot be retained in case that that data can be retained
of power failure. even after power failure.
Memory Units
• Data in the computer’s memory is represented by the
two digits 0 and 1.
• These two digits are called Binary Digits or Bits.
• A bit is the smallest unit of computer’s memory.
• Bits=0,1
1 Byte= 8 bits(e.g,11001011)
1 KB(kilobyte) = 1024 Bytes
1 MB(megabyte) = 1024 KB
1 GB(Gigabyte) = 1024 MB
1 TB(Terabyte) = 1024 GB
Memory Structure and SRAM
Output enable
Chip select

Storage
Write enable cells
Data in / D Q / / Data out
g g g
Address / FF
h C Q
0

D Q /
g
FF
C Q
Address
1
decoder
.
. WE
. D Q / D in
g D out
FF Addr
C Q CS OE
2h –1

Conceptual inner structure of a 2h  g SRAM chip and its

shorthand representation.
Data
Multiple-Chip SRAM
in

32
WE WE WE WE
Address D in D in D in D in
D out D out D out D out
/ / Addr Addr Addr Addr
18 17 CS OE CS OE CS OE CS OE

MSB

WE WE WE WE
D in D in D in D in
D out D out D out D out
Addr Addr Addr Addr
CS OE CS OE CS OE CS OE

Data out, Data out, Data out, Data out,

byte 3 byte 2 byte 1 byte 0

Fig. 17.2 Eight 128K  8 SRAM chips forming a 256K  32

memory unit.
SRAM with Bidirectional Data Bus

Output enable
Chip select

Write enable

Data in/out /
Address g
/
h Data in Data out

When data input and output of an SRAM chip are shared or connected
to a bidirectional data bus, output must be disabled during write
operations.
DRAM and Refresh Cycles
DRAM vs. SRAM Memory Cell Complexity
Word line Word line Vcc

Pass
transistor

Capacitor
Compl.
Bit Bit
bit
line line
line
(a) DRAM cell (b) Typical SRAM cell
Single-transistor DRAM cell, which is considerably simpler than SRAM cell,
leads to dense, high-capacity DRAM memory chips.
 DRAM Refresh Cycles and Refresh Rate

Voltage 1 Written Refreshed Refreshed Refreshed

for 1

Threshold
voltage
10s of ms
0 Stored before needing
Voltage refresh cycle Time
for 0

Variations in the voltage across a DRAM cell capacitor after writing a 1

and subsequent refresh operations.
 DRAM Packaging

24-pin dual in-line package (DIP)

Vss D4 D3 CAS OE A9 A8 A7 A6 A5 A4 Vss Legend:

24 23 22 21 20 19 18 17 16 15 14 13 Ai Address bit i
CAS Column address strobe
Dj Data bit j
NC No connection
OE Output enable
1 2 3 4 5 6 7 8 9 10 11 12 RAS Row address strobe
WE Write enable
Vcc D1 D2 WE RAS NC A10 A0 A1 A2 A3 Vcc

Typical DRAM package housing a 16M  4 memory.

DRAM 1000
Evolutio Computer class

n Memory size
Super-
computers 1
TB
Number of memory chips
256
100 Servers GB
64
GB
16
Work- GB
stations 4
GB
Large 1
PCs GB
256
10 MB
Small 64
PCs MB
16
MB
4
MB
Trends in 1
MB
DRAM main
memory. 1
1980 1990 2000 2010
Calendar year
Hitting the Memory Wall
10 6
Relative performance

Processor
10 3

Memory
1
1980 1990 2000 2010
Calendar year
Memory density and capacity have grown along with the CPU power
and complexity, but memory speed has not kept pace.
Bridging the CPU-Memory Speed Gap
Idea: Retrieve more data from memory with each access

Wide- . . Narrow bus Wide- . .

Wide bus
access
. .
Mux to access . to . Mux
memory processor memory
. . . processor .

(a) Buffer and mult iplex er (a) Buffer and mult iplex er
at the memory side at the processor side

Two ways of using a wide-access memory to bridge the speed

gap between the processor and memory.
Nonvolatile Memory
S u p p ly vo l t a g e
ROM
Word contents
PROM
EPROM 1010

1001
Word
lines
0010

1101

B i t li nes
Read-only memory organization, with the fixed
contents shown on the right.
Flash Memory
S o u r c e l i n es
Control gate
Floating gate
Source
Word
lines
n

p subs-
trate

n+

B i t li nes Drain

EEPROM or Flash memory organization. Each memory

cell is built of a floating-gate MOS transistor.
The Need for a Memory Hierarchy
The widening speed gap between CPU and main memory

Processor operations take of the order of 1 ns

Memory access requires 10s or even 100s of ns

Memory bandwidth limits the instruction execution rate

Each instruction executed involves at least one memory access

Hence, a few to 100s of MIPS is the best that can be achieved
A fast buffer memory can help bridge the CPU-memory gap
The fastest memories are expensive and thus not very large
A second (third?) intermediate cache level is thus often used
 Typical Levels in a Hierarchical Memory
Capacity Access latency Cost per GB
100s B ns
Reg’s $Millions
10s KB a few ns
Cache 1
$100s Ks
MBs 10s ns
Cache 2 $10s Ks
100s MB 100s ns Speed
Main $1000s
gap
10s GB 10s ms Secondary $10s

TBs min+ Tertiary $1s

Fig. 17.14 Names and key characteristics of levels in a memory hierarchy.

 Memory Price Trends
100K
■ DRAM  Flash
10K

1K
$ / GByte

100

Hard disk drive

10

0.1

Source: https://www1.hitachigst.com/hdd/technolo/overview/chart03.html
 Cache Memory Organization
Processor speed is improving at a faster rate than memory’s
• Processor-memory speed gap has been widening
• Cache is to main as desk drawer is to file cabinet

Organization Type
• Independent
• Hierarchical
Cache, Hit/Miss Rate, and Effective Access
Time
Cache is transparent to user;
transfers occur automatically Line
Word

Main
Reg Cache
CPU file
(slow)
(fast)
memory
memory

Data is in the cache

fraction h of the time
(say, hit rate of 98%)
Go to main 1 – h of the time
(say, cache miss rate of 2%)
Two level
Tavg= h1.t1+(1-h1).(t1+t2) [Hierarchical ]
Tavg= h1.t1+(1-h1).t2 [Independent]

Three level
Tavg= h1.t1+(1-h1).h2.(t1+t2)+(1-h1).(1-h2).(t1+t2+t3) [Hierarchical]

Tavg= h1.t1+(1-h1).h2.t2+(1-h1).(1-h2).t3 [Independent]

Questions
• Suppose a cache is 30 times faster than main memory and cache can be used 90% of
time. How much speed up do this system gain by using cache organization.
• In three level memory hierarchy, the access time of cache, main, and virtual memory is 5
nano-second, 100 nano-second, and 10 milli second respectively. If the hit ratio is 80%
for cache, and 99.5% for main memory. Find average access time for this memory
hierarchy.
• A computer with virtual memory has an access time to main memory 50 ns, the time to
transfer a block from the virtual into main memory is 10 ms. The probability for the
page-fault is 10-6. What is the average access time, if the page-table is in the main
memory?
• The computer has a main memory access time of 50 ns. We want to reduce this time to
20 ns by adding cache. Determine how fast the cache must be (access time) if we can
expect a 90% probability of a hit.
Multiple Cache Levels

Cleaner and
CPU
easier to analyze CPU
CPU CPU
registers registers

Level-1 Level-2 Main Level-2 Level-1 Main

cache cache memory cache cache memory

(a) Level 2 between level 1 and main (b) Level 2 connected to “backside” bus

Cache memories act as intermediaries between the

superfast processor and the much slower main memory.
Typical Cache Organization
Mapping Function

• Cache of 64kByte
• Cache block of 4 bytes
• i.e. cache is 16k (214) lines of 4 bytes
• 16MBytes main memory
• 24 bit address
• (224=16M)
Direct Mapping
• Each block of main memory maps to only one cache line
• i.e. if a block is in cache, it must be in one specific place
• Address is in two parts
• Least Significant w bits identify unique word
• Most Significant s bits specify one memory block
• The MSBs are split into a cache line field r and a tag of s-r (most
significant)
Mapping Function is (Which Block is placed in which line)
Block will be placed in Line No = (Block No) Mod (No of Line in cache)
 Direct Mapping
Address Structure

Tag s-r Line or Slot r Word w

8 14 2

• 24 bit address
• 2 bit word identifier (4 byte block)
• 22 bit block identifier
• 8 bit tag (=22-14)
• 14 bit slot or line
• No two blocks in the same line have the same Tag field
• Check contents of cache by finding line and checking Tag
Direct Mapping from Cache to Main Memory
 Direct Mapping
Cache Line Table
Cache line Main Memory blocks held
0 0, m, 2m, 3m…2s-m

1 1,m+1, 2m+1…2s-m+1

…
m-1 m-1, 2m-1,3m-1…2s-1
Direct Mapping Cache Organization
Direct
Mapping
Example
Direct Mapping Summary

• Address length = (s + w) bits

• Number of addressable units = 2s+w words or bytes
• Block size = line size = 2w words or bytes
• Number of blocks in main memory = 2s+ w/2w = 2s
• Number of lines in cache = m = 2r
• Size of tag = (s – r) bits
Direct Mapping pros & cons

• Simple
• Inexpensive
• Fixed location for given block
• If a program accesses 2 blocks that map to the same line repeatedly, cache
misses are very high
Victim Cache
• Lower miss penalty
• Remember what was discarded
• Already fetched
• Use again with little penalty
• Fully associative
• 4 to 16 cache lines
• Between direct mapped L1 cache and next memory level
Associative Mapping

• A main memory block can load into any line of cache

• Memory address is interpreted as tag and word
• Tag uniquely identifies block of memory
• Every line’s tag is examined for a match
• Cache searching gets expensive
Associative Mapping from
Cache to Main Memory
Fully Associative Cache Organization
Associative
Mapping
Example
 Associative Mapping
Address Structure
Word
Tag 22 bit 2 bit

• 22 bit tag stored with each 32 bit block of data

• Compare tag field with tag entry in cache to check for hit
• Least significant 2 bits of address identify which 16 bit word is
required from 32 bit data block
• e.g.
• Address Tag Data Cache line
• FFFFFC FFFFFC 24682468 3FFF
Associative Mapping Summary

• Address length = (s + w) bits

• Number of addressable units = 2s+w words or bytes
• Block size = line size = 2w words or bytes
• Number of blocks in main memory = 2s+ w/2w = 2s
• Number of lines in cache = undetermined
• Size of tag = s bits
Set Associative Mapping

• Cache is divided into a number of sets

• Each set contains a number of lines
• A given block maps to any line in a given set
• e.g. Block B can be in any line of set i
• e.g. 2 lines per set
• 2 way associative mapping
• A given block can be in one of 2 lines in only one set
Set Associative Mapping
Example
• 13 bit set number
• Block number in main memory is modulo 213
• 000000, 00A000, 00B000, 00C000 … map to same set
Mapping From Main Memory to Cache:
v Associative
Mapping From Main Memory to Cache:
k-way Associative
K-Way Set Associative Cache
Organization
 Set Associative Mapping
Address Structure
Word
Tag 9 bit Set 13 bit 2 bit

• Use set field to determine cache set to look in

• Compare tag field to see if we have a hit
• e.g
• Address Tag Data Set number
• 1FF 7FFC1FF 12345678 1FFF
• 001 7FFC 001 11223344 1FFF
Two Way Set Associative Mapping
Example
Set Associative Mapping Summary

• Address length = (s + w) bits

• Number of addressable units = 2s+w words or bytes
• Block size = line size = 2w words or bytes
• Number of blocks in main memory = 2d
• Number of lines in set = k
• Number of sets = v = 2d
• Number of lines in cache = kv = k * 2d
• Size of tag = (s – d) bits
Direct and Set Associative Cache
Performance Differences

• Significant up to at least 64kB for 2-way

• Difference between 2-way and 4-way at 4kB much less than 4kB to
8kB
• Cache complexity increases with associativity
• Not justified against increasing cache to 8kB or 16kB
• Above 32kB gives no improvement
• (simulation results)
Replacement Algorithms for
Direct mapping
• No choice
• Each block only maps to one line
• Replace that line
Replacement Algorithms for
Associative & Set Associative
• Hardware implemented algorithm (speed)
• Least Recently used (LRU)
• Replace block which was not used for a longer time in past .
• First in first out (FIFO)
• replace block that has been in cache longest
• Least frequently used (recently used)[Most recently used]
• replace block which has been used recently
• Random
LRU Examples
Most Recently Used
IMPROVING CACHE PERFORMANCE

• There are three ways to improve cache performance:

1. Reduce the miss rate.
2. Reduce the miss penalty.
3. Reduce the time to hit in the cache.
Virtual memory
TLB
Cache Writing Policies
Write Through and Write Back
Cache Coherence and problem:
Error Detection and Correction
• Single Bit Error
• Burst Error
Length of Burst Error
Parity Codes (Even)
Parity Code ( Odd Parity)
Longitudinal Redundancy Check
VRC
Cont..
CRC (Cyclic redundancy Check)
CRC…
Checksum
Hamming Codes
• Hamming code is useful for both detection and correction of error
present in the received data. This code uses multiple parity bits and
we have to place these parity bits in the positions of powers of 2.
7 Bit Hamming Code Example
Cont….
Hamming Code for 7 Bit