CACHE

MEMORY
 CACHE
MEMORY
• Cache memory is based on Locality of
reference(principle of locality)
It is the phenomenon of the same value or
related storage locations being frequently
accessed.
Types of reference locality:
Temporal locality
Spatial locality
• Temporal Locality:
• Definition: Temporal locality refers to the tendency of a program to
access the same memory locations repeatedly over a short period of
time.
• Example: If a program uses a variable in a loop, there is temporal
locality because the same memory location is accessed in each
iteration of the loop.
• Implication: Caching mechanisms, like CPU caches, take advantage
of temporal locality by keeping recently accessed data in a cache.
This helps in reducing the time it takes to access the data again, as it
is likely to be needed in the near future.
• Spatial Locality:
• Definition: Spatial locality refers to the tendency of a program to
access memory locations that are near each other.
• Example: When a program accesses a memory location, it is likely
to access nearby locations as well. This can occur when iterating
through an array, for instance.
• Implication: Spatial locality is exploited by caching systems that
fetch not only the requested data but also a block of contiguous
memory around it. This is based on the assumption that if a program
accesses one memory location, it is likely to access nearby locations
soon.
• temporal locality is concerned with repeated access to the same
memory locations over time.

• spatial locality is concerned with accessing nearby memory

locations together.

• Both concepts are crucial for designing efficient memory

systems and caching strategies to improve overall system
performance.
 CACHE
MEMORY
• Definition of Cache memory
The cache is a smaller, faster memory which stores copies of
the data from the most frequently used main memory
locations.
- made up of SRAM.
- Effectiveness of cache system depends on hit ratio.
Terminologies:
1. Cache hit – item found in cache
2. Cache miss – item not in cache
3. Hit ratio: ratio between number of hits to total number
of memory references
4. Miss penalty: additional number of cycles required to serve
the
Miss(need to goto MM and fetch the data)
5. Time required for the cache miss depends on both the latency
and bandwidth
• Latency – time to retrieve the first word of the block
• Bandwidth – time to retrieve the rest of this block

Blocks: group of words in main memory.

Block frame or words: term used to refer blocks in cache memory.
[group of words in cache memory]
Number of words in blocks and block frames are same.
Dr. V. Saritha, Associate Professor, SCOPE, VIT University
 STA
RT

Receive Address from CPU

Is No Access Main Memory for

Block Containing Item the block containing the item
in the cache

Yes
Select the cache line to receive
the block from Main
Deliver Block To CPU Memory

Load main Memory Deliver block

Block into cache To CPU

Done
 CACHE MEMORY
MANAGEMENT
TECHNIQUES
• Block Placement
Direct Mapping
Set Associative
Fully Associative
• Block
Identification
• Block Replacement
• Update Policies
 Direct Mapping

Block Placement Set

Associative
Fully Associative

Tag

Block Identification Index

Offset
Cache Memory Management
Techniques
FCFS

Block Replacement LRU

Random

Write

Through
Update Policies
Write
Writearound
back

Write allocate
 DIRECT
MAPPING
A structure in which each memory location is mapped
to exactly one location in the cache.
No replacement policies are used.
• simpler, but much larger than an associative one
to give comparable performance
Cache block= (MM Block address)mod(number of
block in cache)
15
14 DIRECT MAPPING
14
13
7
12
6
11
5
10
4
9
3
8
2
7
1
6
0
5
4
Cache
3
2
1 (MM Block address) mod (Number of lines in a cache)
0
(12) mod (8) =4
Main Memory
 DIRECT MAPPING -
EXAMPLE
• Consider the following configuration of system:
Main memory – 32 blocks
cache memory – 8 block
frames
cache is empty and the block
referred by CPU is 12.
Find the block frame of cache
where the requested
block is placed.
• Cache block frame = (Block address)mod(number of
block
in cache)
= 12 mod 8
=4
Now if the block referred by CPU is 20.
= 20 mod 8
=4
ie – both will be mapping to the same memory location.

Advantage- simplicity in determining where to place

an incoming main memory block in the cache.
Disadvantage : inefficient use of the cache.
Leads to low cache hit ratio.

Consider, for example, the case of a main memory consisting

of 4K blocks, a cache memory consisting of 128 blocks, and
a block size of 16 words.. For example, main memory blocks
0, 128, 256, 384, . . . , 3968 map to cache block 0. We
therefore call the direct-mapping technique a many-to-one
mapping technique.
 SET ASSOCIATIVE CACHE

• Blocks are placed in a restricted set of places in

the cache.
• Set : group of blocks.
• Set is chosen: (block address)mod(number of sets
in the cache)
• N way set Associative: if there are n blocks in the
set
15
14
14
SET ASSOCIATIVE MAPPING
13
7
12 3
6
11
5
10 2
4
9
3
8 1
2
7
1
6 0
0
5
4
Cache
3
2
1 (MM Block address) mod (Number of sets in a cache)
0
(12) mod (4) =0
Main Memory
N WAY SET ASSOCIATIVE
 EXAMPLE
• Consider the following configuration of system:
• Main memory – 32 blocks

cache memory – 8 block

frames
• Number of sets = 4 each
with 2 blocks
• cache is empty and the block
referred by CPU is 12.
• Find the block frame of cache where the requested
block is placed.
 EXAMP
• LE
Set number= (Block address)mod(number of sets in
cache)
= 12 mod 4
=0
12th block can be placed in either in 0th or 1st block frame of
set 0.
• First map is to map the set in the cache and then to
the corresponding block frame.
 FULLY
ASSOCIATIVE
• Incoming main memory block can be placed in
any available cache block.
• The block 12 (for previous example) can be placed
in any one of the available block frames.
• Replacement algorithm is used.
15
FULLY ASSOCIATIVE
14 MAPPING
14
13
7
12
6
11
5
10
4
9
3
8
2
7
1
6
0
5
4
Cache
3
2
1 Random
0

Main Memory
 FULLY ASSOCIATIVE
• MAPPING
Fastest, most flexible but very expensive
• Any block location in cache can store any block in memory
• Stores both the address and the content of the memory word
• CPU address of 15 bits is placed in the argument register and
the associative memory is searched for a matching address
• If found data is read and sent to the CPU else main memory
is
accessed.
• CAM – content addressable memory
• 1.Assume there are three small caches, each consisting of four one-
word blocks. One cache is fully associative, a second is two-way set-
associative, and the third is direct-mapped. Find the number of
misses for each cache organization given the following sequence of
block addresses: 0, 8, 0, 6, and 8.
• Solu:
• The direct-mapped case is easiest. First, let’s determine to which
cache blockeach block address maps:
The direct-mapped cache generates five misses for the five accesses.
• The set-associative cache has two sets (with indices 0 and 1) with
two elements per set.

• Set-associative caches usually replace the least recently

• used block within a set; that is, the block that was used furthest in
the past is replaced.
• The fully associative cache has four cache blocks (in a single set);
any memory block can be stored in any cache block.

The fully associative cache has the best performance, with only
three misses
BLOCK IDENTIFICATION
• Block Identification: The block identification is the part
of the memory address used to determine which block of
data in the main memory corresponds to the requested
data. By using block identification, the cache controller
can quickly check whether the required data is present in
the cache.

• With block identification, the cache controller can

quickly determine if the requested data is already present
in the cache. If the data is found (cache hit), the processor
can access the data much more quickly than if it had to
retrieve it from the slower main memory (cache miss).
• The memory addresses are divided into three main components: the
tag, the index, and the offset.

• The offset is used to identify the specific byte within a cache line.

• The index is used to determine which set (or way) in the cache the
block belongs to.

• The tag is used to uniquely identify the block of data in main

memory.
 ADDING TAGS FOR BLOCK IDENTIFICATION
• We need to add tags to the cache, which supply the rest of
the address bits to let us distinguish between different
memory locations that map to the same cache block.
0000
0001 Here the size of MM is 16 bytes and
0010 cache memory is 4 bytes and block size
0011
0100
is 8 bytes
0101 Index Tag Data
0110 00 00
0111 01 11
1000 10 01
1001 11 01
1010
1011 Cache
1100 Here 0000 address of MM in which
1101
00 is considered as Tag (2 bit)
1110
1111 00 is considered as index(block) bit
of cache ie(2 bit)
Main Memory
 IN DIRECT MAPPING (BLOCK
IDENTIFICATION)

• According to DM technique= (MM Block address) mod (Number of

lines in a cache)
• Here the MM address – 0000,0100,1000,1100 are mapped to cache block
00 index
• Similarly MM address-0001,0101,1001,1101 are mapped to Cache block 01
index
• Here multiple blocks of MM are mapped to a same cache block, So to
identify particularly which block in the MM is mapped currently with the
cache block can be identified using block identification technique.
• DM(Block Identification)- Tag Index(Block) Offset(Word)
• Tag bit can be calculated using the formula
• TAG= = 24/22 = 24-2 = 22 [The power value is Tag size]

• Therefore the size of Tag bit is 2

• Index (Block)-can be calculated using the no of blocks in Cache memory
• Therefore the size of index(block)= 22 ie 2 bit is required to represent it.
• Offset(word)- can be calculated from the length of a block size is-8 ie (23)
• Therefore the size of offset(word) is 3 bit.
• In DM the size of block identification can be represented by
Tag(2) Block(2) Word(3)
 2 WAY SET ASSOCIATIVE IDENTIFIED
BY (BLOCK IDENTIFICATION)

Set Associative=(MM Block address) mod (Number of sets in CM)

• Tag bit can be calculated using the formula

• TAG= = 24/21 = 24-1 = 23 [The power value is Tag size]

• Therefore the size of Tag bit is 3

• Index (Block)-can be calculated using the no of sets in Cache memory

• Therefore the size of index(block)=

21 ie 1 bit is required to represent it.
• Offset(word)- can be calculated from the length of a block size is-8 ie ( 3)
2
• Therefore the size of offset(word) is 3 bit.

• In 2-Set Associative the size of block identification can be represented by

Tag(3) Block(1) Word(3)
 FULLY ASSOCIATIVE IDENTIFIED
BY BLOCK IDENTIFICATION
• Tag bit can be calculated using the formula
• Tag=No of blocks in Main Memory ie-24 [The power value is Tag size]

• Therefore the size of Tag bit is 4

• Offset(word)- can be calculated from the length of a block size is-8 ie (23)
• Therefore the size of offset(word) is 3 bit.
• In Fully Associative the size of block identification can be represented by
Tag(4) Word(3)
 MAPPING FUNCTION

Example 1

No. of lines(blocks) in a cache: 128

No. of blocks in main memory: 4096 blocks/4k words
No. of Words in a block: 16
No. address bits: 12 (4k=210*22)
Direct Mapping
2-Way Set Associative Mapping
Fully Associative Mapping
• 1. A set associative cache consists of 64 lines or slots, divided
into four line sets. Main memory consists 4k blocks of 128
words each. Show the format of main memory addresses.

• Tag = 212/24
PROBLEM 2

• A two-way set associative cache has lines of 16 bytes

and a total size of 8k bytes. The 64-Mbyte main
memory is byte addressable. Show the format of main
memory addresses.
 BLOCK REPLACEMENT

• Least Recently Used: (LRU)

Replace that block in the set that
has been in the cache longest with no
reference to it.
• First Come First Out: (FIFO)
Replace that block in the set that
has been in the cache longest.
• Least Frequently Used: (LFU)
Replace that block in the set that
has experienced the fewest references
 UPDATE POLICIES - WRITE THROUGH
• Update main memory with every memory
write operation
• Cache memory is updated in parallel if it
contains the word at specified address.
• Advantage:
• main memory always contains the same data as the cache
• easy to implement
Disadvantage: -
• write is slower
• every write needs a main memory access
 WRITE BACK
• Only cache is updated during write operation and marked by
flag. When the word is removed from the cache (at the time of
replacement), it is copied into main memory

• Advantage:
- writes occur at the speed of the cache
memory
-multiple writes within a block require only one
write to main memory

Disadvantage:
- harder to implement
-main memory is not always consistent with
cache
 UPDATE POLICIES –
CONTD..

• Write-Allocate
• update the item in main memory and bring the block
containing the updated item into the cache.
• Write-Around or Write-no-allocate
• correspond to items not currently in the cache (i.e. write
misses) the item is updated in main memory only without
affecting the cache.
 UPDATE POLICIES – CONTD..
• Write back:-Write only in cache, updating main
memory only at the time of replacement.
• Write through:-Both are updating for each write operation.
• Write-Allocate:- first in main memory, then copy the block
into cache.
• Write-Around or Write-no- when write miss
allocate:- occurred, updated in main without affecting the
memory cache.
 REFERENCES

• J. L. Hennessy & D.A. Patterson, Computer architecture: A

quantitative approach, Fourth Edition, Morgan Kaufman,
2004.