ECE 463/521

Fall `20
Basic Cache Operation

Prof. Eric Rotenberg

Fall 2020 ECE 463/563, Microprocessor Architecture, 1

Prof. Rotenberg
MAIN
MEMORY

32-bit address
byte #0 00…00000000 1 byte of data
byte #1 00…00000001
byte #2 00…00000010
… 00…00000011
00…00000100
00…00000101
00…00000110
00…00000111
232 bytes 00…00001000
00…00001001
00…00001010
00…00001011
00…00001100
…
…
byte #(232-1) 11…11111111

Fall 2020 ECE 463/563, Microprocessor Architecture, 2

Prof. Rotenberg
 MAIN 1 block = 8 bytes
32-bit address
MEMORY block block
address offset

01 1
01 0
10 1
10 0
11 1
11 0
1
000
00
mem. block #0 00…00000xxx  Block offset
mem. block #1 00…00001xxx  Low-order bits of
address that specify a
mem. block #2 00…00010xxx byte within the block
… 00…00011xxx  Since cache is
managed at
00…00100xxx granularity of blocks,
the block offset bits
00…00101xxx are irrelevant for
determining hit or
00…00110xxx miss
00…00111xxx
232 bytes 00…01000xxx
…
(229 memory blocks)
…
…
…
…
…
11…11111xxx

Fall 2020 ECE 463/563, Microprocessor Architecture, 3

Prof. Rotenberg
MAIN 1 block = 8 bytes CACHE
32-bit address
MEMORY block block
address offset
Example:
• SIZE = 32 bytes

01 1
01 0
10 1
10 0
11 1
11 0
1
000
• BLOCKSIZE = 8 bytes

00
mem. block #0 00…00000xxx • 1 block per cache set
mem. block #1 00…00001xxx
mem. block #2 00…00010xxx
… 00…00011xxx
00…00100xxx cache set #0

00…00101xxx cache set #1

00…00110xxx cache set #2
00…00111xxx cache set #3
00…01000xxx
…
…
…
…
…
…
11…11111xxx

Fall 2020 ECE 463/563, Microprocessor Architecture, 4

Prof. Rotenberg
MAIN 1 block = 8 bytes CACHE
32-bit address
MEMORY block block
address offset

index

01 1
01 0
10 1
10 0
11 1
11 0
1
000
00
mem. block #0 00…00000xxx
mem. block #1 00…00001xxx
mem. block #2 00…00010xxx
… 00…00011xxx
00…00100xxx cache set #0

00…00101xxx cache set #1

00…00110xxx cache set #2
00…00111xxx cache set #3
00…01000xxx
…  Index
 Low-order bits of
… block address indicate
… which cache set the
block will be placed in
… and where it will be
searched for later
…
…
11…11111xxx

Fall 2020 ECE 463/563, Microprocessor Architecture, 5

Prof. Rotenberg
MAIN 1 block = 8 bytes CACHE
32-bit address
MEMORY block block
address offset

index
tag

01 1
01 0
10 1
10 0
11 1
11 0
1
000
00
mem. block #0 00…00000xxx
mem. block #1 00…00001xxx
mem. block #2 00…00010xxx
… 00…00011xxx
00…00100xxx cache set #0

00…00101xxx cache set #1

00…00110xxx cache set #2
00…00111xxx cache set #3 v tag data
00…01000xxx
…  Tag
 Many memory blocks have the same
… index, i.e., they map to the same cache
… set
 High-order bits of block address
… differentiate memory blocks that map to
the same cache set
…
 Record the tag alongside the cached
… memory block, to identify which memory
block is cached in the set
11…11111xxx

Fall 2020 ECE 463/563, Microprocessor Architecture, 6

Prof. Rotenberg
 A direct-mapped cache
31 6 5 3 2 0
block MAR
tag (26) index (3) (3)
offset

set (holds 1 block)

tag (26 bits) blocks (8 bytes)

row
dec

=? word select (mux)

hit/miss MDR

Fall 2020 ECE 463/563, Microprocessor Architecture, 7

Prof. Rotenberg
 A (2-way) set-associative cache
31 5 4 3 2 0
(2) block (3) MAR
tag (27) index
offset

set (holds 2 blocks)

tag (27 bits)
blocks (8 bytes)

row
dec

=? =? block select (mux)

Notes: this cache is the same size as word select (mux)

the (previous) direct-mapped cache, but
the index field is 1 bit shorter (2 bits total)
MDR
A direct-mapped cache is really a 1-way set-associative cache

Fall 2020 ECE 463/563, Microprocessor Architecture, 8

Prof. Rotenberg
 A fully-associative cache
(also called a content addressable memory or CAM)
31 3 2 0
block (3) MAR
tag (29) offset
tag (29 bits)
=?
tag
=?
tag
=?
tag
=?
tag
=?
tag
=?
tag
=?
tag
=?
tag

Notes: Same size as previous

caches, but no row decoder and
word select (mux)
no index at all! Comparators (one
per block) take the place of the row MDR
decoder
Fall 2020 ECE 463/563, Microprocessor Architecture, 9
Prof. Rotenberg
 Summary: decoding the address
• Cache is normally specified as follows:
– {SIZE, ASSOC, BLOCKSIZE}
• SIZE: total bytes of data storage
– Does NOT include “metadata”, aka control bits: valid bit, dirty bit, tag,
replacement counters, coherence bits, etc.
• ASSOC: associativity (# of blocks in a set)
• BLOCKSIZE: size of cache block in bytes
– Question: how do we decode the address?
tag index block offset

? ? ?

– REMEMBER the following

# sets  equation
SIZE
ASSOC BLOCKSIZE
(derivation next slide)

– Then compute sizetagof each address index

field block offset
(# addressbit s )  log 2 (# sets )  log 2 ( BLOCKSIZE) log2 (# sets) log 2 ( BLOCKSIZE)

Fall 2020 ECE 463/563, Microprocessor Architecture, 10

Prof. Rotenberg
 Derivation: # sets
way 0 way 1 way 2 … way (ASSOC-1)
set 0 block block block block
set 1 block block block block
set 2 block block block ... block
…
set (#sets-1) block block block block

total bytes of data storage = (# sets) x (# blocks/set) x (# bytes/block)

total bytes of data storage = (# sets) x (# ways) x (# bytes/block)

SIZE = (# sets) x (ASSOC) x (BLOCKSIZE)

SIZE
# sets 
ASSOC BLOCKSIZE

Fall 2020 ECE 463/563, Microprocessor Architecture, 11

Prof. Rotenberg
 Cache types revisited
• Every cache can be classed as a particular “N-way set-
associative cache”
• Direct-mapped and fully-associative caches are just extremes
• Direct-mapped
– Any cache with only 1 way is a direct-mapped cache.
– A direct-mapped cache is the same as a 1-way set-associative
cache.
• Fully-associative
– Any cache with only 1 set is a fully-associative cache.
– A fully-associative cache is the same as an N-way set-associative
cache with only 1 set containing N blocks.
 Generic picture of
N-way set-associative cache

X
Associatively search for the block among N ways of the set.

way 0 way 1 way 2 … way (N-1)

set 0 block block block block
Index a set using the set 1 block block block block
index bits of the set 2 block block block block
...
block address. …
set (#sets- block block block block
1)

Direct-mapped cache is one extreme where there is only 1 way. Once a set is indexed
using the index bits from the block address, there is no need to search among multiple
blocks within the set because there is only 1 block in the set.
 Generic picture of
N-way set-associative cache

Associatively search for the block among N ways of the set.

way 0 way 1 way 2 … way (N-1)

set 0 block block block block
Index a set using the set 1 block block block block

X
index bits of the
block address.
set 2 block
…
block block
...
block

set (#sets- block block block block

1)

Fully-associative cache is one extreme where there is only 1 set.

There are no index bits within the block address because there is only 1 set:
log2(1 set) = 0 index bits. Must associatively search the entire set, i.e.,
must associatively search the entire cache, for the block.