• Memory Hierarchy

• Main Memory

• Auxiliary Memory

• Associative Memory

• Cache Memory

• Virtual Memory

• Memory Management Hardware

Memory Hierarchy is to obtain the highest possible
access speed while minimizing the total cost of the memory system
Auxiliary memory
Magnetic
tapes I/O Main
processor memory
Magnetic
disks

CPU Cache
memory

Register

Cache

Main Memory

Magnetic Disk

Magnetic Tape
 Main Memory

RAM and ROM Chips

Typical RAM chip
Chip select 1 CS1
Chip select 2 CS2
Read RD 128 x 8 8-bit data bus
RAM
Write WR
7-bit address AD 7

CS1 CS2 RD WR Memory function State of data bus

0 0 x x Inhibit High-impedence
0 1 x x Inhibit High-impedence
1 0 0 0 Inhibit High-impedence
1 0 0 1 Write Input data to RAM
1 0 1 x Read Output data from RAM
1 1 x x Inhibit High-impedence

Typical ROM chip

Chip select 1 CS1
Chip select 2 CS2
512 x 8 8-bit data bus
ROM
9-bit address AD 9
 Main Memory

Address space assignment to each memory chip

Example: 512 bytes RAM and 512 bytes ROM

Hexa Address bus

Component address 10 9 8 7 6 5 4 3 2 1
RAM 1 0000 - 007F 0 0 0 x x x x x x x
RAM 2 0080 - 00FF 0 0 1 x x x x x x x
RAM 3 0100 - 017F 0 1 0 x x x x x x x
RAM 4 0180 - 01FF 0 1 1 x x x x x x x
ROM 0200 - 03FF 1 x x x x x x x x x

Memory Connection to CPU

- RAM and ROM chips are connected to a CPU
through the data and address buses

- The low-order lines in the address bus select

the byte within the chips and other lines in the
address bus select a particular chip through
its chip select inputs
 Main Memory

Address bus CPU

16-11 10 9 8 7-1 RD WR Data bus

Decoder
3 2 1 0
CS1
CS2

Data
RD 128 x 8
RAM 1
WR
AD7

CS1
CS2

Data
RD 128 x 8
RAM 2
WR
AD7

CS1
CS2

Data
RD 128 x 8
RAM 3
WR
AD7

CS1
CS2
RD 128 x 8 Data
RAM 4
WR
AD7

CS1
CS2
Data

1- 7 512 x 8
8
9 } AD9 ROM
 Auxiliary Memory

Information Organization on Magnetic Tapes

file i
block 1 block 2
block 3 EOF
R1
R2 R3 R4
R5
R6
block 3 IRG
R1
EOF R3 R2
R5 R4 block 1
block 2

Organization of Disk Hardware

Moving Head Disk Fixed Head Disk

Track
 Associative Memory

- Accessed by the content of the data rather than by an address

- Also called Content Addressable Memory (CAM)
Hardware Organization Argument register(A)

Key register (K)

Match
register

Input Associative memory

array and logic
M
Read m words
Write n bits per word

- Compare each word in CAM in parallel with the

content of A(Argument Register)
- If CAM Word[i] = A, M(i) = 1
- Read sequentially accessing CAM for CAM Word(i) for M(i) = 1
- K(Key Register) provides a mask for choosing a
particular field or key in the argument in A
(only those bits in the argument that have 1’s in
their corresponding position of K are compared)
 Associative Memory

A1 Aj An

K1 Kj Kn

Word 1 C11 C1j C1n M1

Word i Ci1 Cij Cin Mi

Word m Cm1 Cmj Cmn Mm

Bit 1 Bit j Bit n

Internal organization of a typical cell Cij

Aj Kj
Input

Write

R S
F ij Match To M i
Read logic

Output
 Associative Memory

K1 A1 K2 A2 Kn An

F'i1 F i1 F'i2 F i2 .... F'in F in

Mi
 Cache Memory

Locality of Reference
- The references to memory at any given time
interval tend to be confined within a localized areas
- This area contains a set of information and
the membership changes gradually as time goes by
- Temporal Locality
The information which will be used in near future
is likely to be in use already( e.g. Reuse of information in loops)
- Spatial Locality
If a word is accessed, adjacent(near) words are likely accessed soon
(e.g. Related data items (arrays) are usually stored together;
instructions are executed sequentially)

Cache
- The property of Locality of Reference makes the
Cache memory systems work
- Cache is a fast small capacity memory that should hold those information
which are most likely to be accessed

Main memory
CPU
Cache memory
 Cache Memory

Memory Access
All the memory accesses are directed first to Cache
If the word is in Cache; Access cache to provide it to CPU
If the word is not in Cache; Bring a block (or a line) including
that word to replace a block now in Cache

- How can we know if the word that is required

is there ?
- If a new block is to replace one of the old blocks,
which one should we choose ?
Performance of Cache Memory System

Hit Ratio - % of memory accesses satisfied by Cache memory system

Te: Effective memory access time in Cache memory system
Tc: Cache access time
Tm: Main memory access time

Te = Tc + (1 - h) Tm

Example: Tc = 0.4 s, Tm = 1.2s, h = 0.85%

Te = 0.4 + (1 - 0.85) * 1.2 = 0.58s
 Cache Memory

Mapping Function
Specification of correspondence between main
memory blocks and cache blocks
Associative mapping
Direct mapping
Set-associative mapping
Associative Mapping
- Any block location in Cache can store any block in memory
-> Most flexible
- Mapping Table is implemented in an associative memory
-> Fast, very Expensive
- Mapping Table
Stores both address and the content of the memory word
address (15 bits)

Argument register

Address Data
01000 3450
CAM 02777 6710
22235 1234
 Cache Memory

- Each memory block has only one place to load in Cache

- Mapping Table is made of RAM instead of CAM
- n-bit memory address consists of 2 parts; k bits of Index field and
n-k bits of Tag field
- n-bit addresses are used to access main memory
and k-bit Index is used to access the Cache
Addressing Relationships Tag(6) Index(9)

00 000 32K x 12
000
512 x 12
Main memory Cache memory
Address = 15 bits Address = 9 bits
Data = 12 bits Data = 12 bits
77 777 777

Direct Mapping Cache Organization

Memory
address Memory data
00000 1220 Cache memory
Index
address Tag Data
00777 2340 000 00 1220
01000 3450

01777 4560
02000 5670

777 02 6710
02777 6710
 Cache Memory

Operation

- CPU generates a memory request with (TAG;INDEX)

- Access Cache using INDEX ; (tag; data)
Compare TAG and tag
- If matches -> Hit
Provide Cache[INDEX](data) to CPU
- If not match -> Miss
M[tag;INDEX] <- Cache[INDEX](data)
Cache[INDEX] <- (TAG;M[TAG; INDEX])
CPU <- Cache[INDEX](data)

Direct Mapping with block size of 8 words

Index tag data 6 6 3
000 01 3450 Tag Block Word
Block 0
007 01 6578
010 INDEX
Block 1
017

Block 63 770 02
777 02 6710
 Cache Memory

- Each memory block has a set of locations in the Cache to load

Set Associative Mapping Cache with set size of two

Index Tag Data Tag Data
000 01 3450 02 5670

777 02 6710 00 2340

Operation
- CPU generates a memory address(TAG; INDEX)
- Access Cache with INDEX, (Cache word = (tag 0, data 0); (tag 1, data 1))
- Compare TAG and tag 0 and then tag 1
- If tag i = TAG -> Hit, CPU <- data i
- If tag i  TAG -> Miss,
Replace either (tag 0, data 0) or (tag 1, data 1),
Assume (tag 0, data 0) is selected for replacement,
(Why (tag 0, data 0) instead of (tag 1, data 1) ?)
M[tag 0, INDEX] <- Cache[INDEX](data 0)
Cache[INDEX](tag 0, data 0) <- (TAG, M[TAG,INDEX]),
CPU <- Cache[INDEX](data 0)
 Cache Memory

Many different block replacement policies are available

LRU(Least Recently Used) is most easy to implement

Cache word = (tag 0, data 0, U0);(tag 1, data 1, U1), Ui = 0 or 1(binary)

Implementation of LRU in the Set Associative Mapping with set size = 2

Modifications

Initially all U0 = U1 = 1
When Hit to (tag 0, data 0, U0), U1 <- 1(least recently used)
(When Hit to (tag 1, data 1, U1), U0 <- 1(least recently used))
When Miss, find the least recently used one(Ui=1)
If U0 = 1, and U1 = 0, then replace (tag 0, data 0)
M[tag 0, INDEX] <- Cache[INDEX](data 0)
Cache[INDEX](tag 0, data 0, U0) <- (TAG,M[TAG,INDEX], 0); U1 <- 1
If U0 = 0, and U1 = 1, then replace (tag 1, data 1)
Similar to above; U0 <- 1
If U0 = U1 = 0, this condition does not exist
If U0 = U1 = 1, Both of them are candidates,
Take arbitrary selection
 Cache Memory

Write Through
When writing into memory
If Hit, both Cache and memory is written in parallel
If Miss, Memory is written
For a read miss, missing block may be
overloaded onto a cache block
Memory is always updated
-> Important when CPU and DMA I/O are both executing
Slow, due to the memory access time
Write-Back (Copy-Back)
When writing into memory
If Hit, only Cache is written
If Miss, missing block is brought to Cache and write into Cache
For a read miss, candidate block must be
written back to the memory
Memory is not up-to-date, i.e., the same item in
Cache and memory may have different value
 Virtual Memory

Give the programmer the illusion that the system has a very large memory,
even though the computer actually has a relatively small main memory

Address Space(Logical) and Memory Space(Physical)

address space memory space

virtual address Mapping

physical address
(logical address)

address generated by programs actual main memory address

Address Mapping
Memory Mapping Table for Virtual Address -> Physical Address
Virtual address

Virtual Memory Main memory

address address Main
mapping memory
register table register

Physical
Address
Memory table Main memory
buffer register buffer register
 Virtual Memory

Address Space and Memory Space are each divided

into fixed size group of words called blocks or pages
1K words group Page 0
Page 1
Page 2
Address space Memory space Block 0
Page 3
N = 8K = 213 M = 4K = 212 Block 1
Page 4
Block 2
Page 5
Block 3
Page 6
Page 7
Organization of memory Mapping Table in a paged system
Page no. Line number
1 0 1 0 1 0 1 0 1 0 0 1 1 Virtual address

Table Presence
address bit
000 0 Main memory
001 11 1 Block 0
010 00 1 Block 1
011 0 01 0101010011 Block 2
100 0 Block 3
Main memory
101 01 1 address register
Memory page table 110 10 1
111 0 MBR

01 1
 Virtual Memory

Assume that
Number of Blocks in memory = m
Number of Pages in Virtual Address Space = n
Page Table
- Straight forward design -> n entry table in memory
Inefficient storage space utilization
<- n-m entries of the table is empty

- More efficient method is m-entry Page Table

Page Table made of an Associative Memory
m words; (Page Number:Block Number)
Virtual address
Page no.
1 0 1 Line number Argument register

1 0 1 0 0 Key register

0 0 1 1 1
0 1 0 0 0 Associative memory
1 0 1 0 1
1 1 0 1 0
Page no.Block no.
Page Fault
Page number cannot be found in the Page Table
 Virtual Memory

3 Page is on backing store

1. Trap to the OS OS
2. Save the user registers and program state
3. Determine that the interrupt was a page fault 2 trap
4. Check that the page reference was legal and
determine the location of the page on the
backing store(disk) 1 Reference
5. Issue a read from the backing store to a free LOAD M
0
frame 6
restart
a. Wait in a queue for this device until serviced instruction 4
b. Wait for the device seek and/or latency time bring in
free frame
missing
c. Begin the transfer of the page to a free frame 5 page
reset
6. While waiting, the CPU may be allocated to page
some other process table
7. Interrupt from the backing store (I/O completed)
8. Save the registers and program state for the other user main memory
9. Determine that the interrupt was from the backing store
10. Correct the page tables (the desired page is now in memory)
11. Wait for the CPU to be allocated to this process again
12. Restore the user registers, program state, and new page table, then
resume the interrupted instruction.

Processor architecture should provide the ability

to restart any instruction after a page fault.
 Virtual Memory

Decision on which page to displace to make room for

an incoming page when no free frame is available
Modified page fault service routine
1. Find the location of the desired page on the backing store
2. Find a free frame
- If there is a free frame, use it
- Otherwise, use a page-replacement algorithm to select a victim frame
- Write the victim page to the backing store
3. Read the desired page into the (newly) free frame
4. Restart the user process
valid/
frame invalid bit swap
out
1 victim
page
2
change to victim
f 0 v i invalid
3
4 swap
f v reset page
table for desired
new page page in
backing store
page table

physical memory
 Virtual Memory

FIFO Reference string

7 0 1 2 0 3 0 4 2 3 0 3 2 1 2 0 1 7 0 1
7 7 7 2 2 2 4 4 4 0 0 0 7 7 7
0 0 0 3 3 3 2 2 2 1 1 1 0 0
1 1 1 0 0 0 3 3 3 2 2 2 1
Page frames

FIFO algorithm selects the page that has been in memory the longest time
Using a queue - every time a page is loaded, its
- identification is inserted in the queue
Easy to implement
May result in a frequent page fault

Optimal Replacement (OPT) - Lowest page fault rate of all algorithms

Replace that page which will not be used for the longest period of time
Reference string
7 0 1 2 0 3 0 4 2 3 0 3 2 1 2 0 1 7 0 1
7 7 7 2 2 2 2 2 7
0 0 0 0 4 0 0 0
1 1 3 3 3 1 1
Page frames
 Virtual Memory

LRU
- OPT is difficult to implement since it requires future knowledge
- LRU uses the recent past as an approximation of near future.

Replace that page which has not been

used for the longest period of time

Reference string
7 0 1 2 0 3 0 4 2 3 0 3 2 1 2 0 1 7 0 1
7 7 7 2 2 4 4 4 0 1 1 1
0 0 0 0 0 0 3 3 3 0 0
1 1 3 3 2 2 2 2 2 7
Page frames

- LRU may require substantial hardware assistance

- The problem is to determine an order for the frames
defined by the time of last use
 Virtual Memory

LRU Implementation Methods

• Counters
- For each page table entry - time-of-use register
- Incremented for every memory reference
- Page with the smallest value in time-of-use register is replaced
• Stack
- Stack of page numbers
- Whenever a page is referenced its page number is
removed from the stack and pushed on top
- Least recently used page number is at the bottom
Reference string
4 7 0 7 1 0 1 2 1 2 7 1 2

2 7
1 2
0 1
7 0
LRU Approximation 4 4

- Reference (or use) bit is used to approximate the LRU

- Turned on when the corresponding page is
referenced after its initial loading
- Additional reference bits may be used
 Memory Management Hardware

Basic Functions of MM
- Dynamic Storage Relocation - mapping logical
memory references to physical memory references
- Provision for Sharing common information stored
in memory by different users
- Protection of information against unauthorized access
Segmentation
- A segment is a set of logically related instructions
or data elements associated with a given name
- Variable size
User's view of memory
The user does not think of
Stack memory as a linear array
Subroutine of words. Rather the user
Symbol prefers to view memory as
Table
a collection of variable
SQRT
Main sized segments, with no
Program
necessary ordering among
segments.
User's view of a program
 Memory Management Hardware

- A memory management scheme which supports

user's view of memory
- A logical address space is a collection of segments
- Each segment has a name and a length
- Address specify both the segment name and the
offset within the segment.
- For simplicity of implementations, segments are numbered.

Segmentation Hardware Segment Table

limit base

CPU (s,d)

Memory
y
< +
n

error
 Memory Management Hardware

1400
Subroutine Stack Segment 0
Segment 3
Segment 0 2400

Symbol
Table 3200

SQRT Segment 4 Segment 3

Segment 1 Main
Program 4300
Segment 2
Segment 2 4700

Segment 4
Logical Address Space
5700
6300
Segment 1
Segment Table 6700
limit base
0 1000 1400
1 400 6300
2 400 4300
3 1100 3200
4 1000 4700
 Memory Management Hardware

Editor
limit base 43062
0 25286 43062
Segment 0 1 4425 68348 Editor
Data 1 Segment Table
(User 1) 68348
Segment 1 Data 1
72773

Logical Memory
(User 1)

90003
Data 2
98556

limit base
Editor
0 25286 43062
1 8550 90003 Physical Memory
Segment 0
Segment Table
Data 2 (User 2)

Segment 1

Logical Memory
(User 2)
 Memory Management Hardware

Logical address

Segment Page Word

Segment table Page table

Block Word
Physical address
 Memory Management Hardware

Implementation of the Page Table

- Hardware registers (if the page table is reasonably small)
- Main memory
- Page Table Base Register(PTBR) points to PT
- Two memory accesses are needed to access
a word; one for the page table, one for the word

- Cache memory (TLB: Translation Lookaside Buffer)

- To speedup the effective memory access time,

a special small memory called associative
memory, or cache is used

Implementation of the Segment Table

Similar to the case of the page table
 Memory Management Hardware

Logical and Physical Addresses

Logical address format: 16 segments of 256 pages
each, each page has 256words
4 8 8
Segment Page Word

20
2 x 32
Physical Physical address format: 4096 blocks of 256 words
memory each, each word has 32bits
12 8
Block Word

Logical and Physical Memory Address Assignment

Hexa
address Page number
60000 Page 0 Segment Page Block
60100 Page 1 6 00 012
6 01 000
60200 Page 2 6 02 019
6 03 053
60300 Page 3 6 04 A61
60400 Page 4
604FF
(a) Logical address assignment (b) Segment-page versus
memory block assignment
 Memory Management Hardware

Segment and page table mapping Logical address (in hexadecimal)

6 02 7E

Segment table Page table Physical memory

0 00 00000
Block 0
000FF
6 35 35 012
36 000
37 019 01200
Block 12
38 053 012FF
F A3 39 A61

01900 32-bit word

0197E
A3 012 019FF

Associative memory mapping

Segment Page Block
6 02 019
6 04 A61
 Memory Management Hardware

Protection information can be included in the

segment table or segment register of the memory
management hardware

- Format of a typical segment descriptor

Base address Length Protection

- The protection field in a segment descriptor specifies

the Access Rights to the particular segment

- In a segmented-page organization, each entry in the

page table may have its own protection field to
describe the Access Rights of each page
Full read and write privileges.
- Access Rights:
Read only (write protection)
Execute only (program protection)
System only (O.S. Protection)
 From translator CPU
Virtual Real
Address Address Virtual Address
Page Line Word in To translator
Number Number Line
A
CPU Memory

Hash
Function Real Address Data
TLB
Cache

S S

Compare Virtual Real Compare Addresses

Addresses Address & Select Data
Data

Word Select & Align

To Main Memory
S = Select
Data Out
 Real Address Tag Data Valid

Cache Entry

Entry 1 Entry 2  Entry E Replacement status

Cache Set
 Receive Virtual Address

Hash Page Number Use Line Number

to Select Set
Search TLB
Read Out Address Tags
A
yes
In TLB ? Compare Addresses
no yes
Match ?
Send Virtual Address Update Replacement
to Translator Status in TLB no
Send Real Address
Update Replacement
to Main Memory
Status
Use Page & Segment tables
to Translate Address Select Receive Line from
Correct Main Memory
Line
Put in TLB Store Line
in Cache
Select Correct
Word from Line
A

Read Out
 Page number Byte within page
Byte within line

31 21 20 17 12 11 10 4 3 2 1 0

Byte within
Select set Select set word
in TLB in cache
Map through Word within
page directory Map through
page table Line number line

Virtual Address of Fairchild Clipper