SUMMARY

MEMORY SYSTEM ORGANIZATION

Memory technology Hierarchical memory systems Characteristics of the storage-device Main memory organization SRAM DRAM Cache memory

COMPUTER ARCHITECTURE

MEMORY TECHNOLOGY
Every computer system contains a variety of devices to store the instructions and data The storage devices + the algorithms (HW or SW implemented) needed to control or manage the stored information constitute the memory system of the computer Desirable: processors should have immediate and uninterrupted access to memory
maximum speed to transfer information between a processor and memory

DESIGNING MEMORY SYSTEM

To provide adequate storage capacity with an acceptable level of performance at a reasonable cost
The use of a number of different memory devices with different cost performance ratios organized to provide a high average performance at a low average cost per bit
The individual memories form a hierarchy of storage devices

Memories that operate at speeds comparable to processor speeds are relatively costly
It is not feasible to employ a single memory using just one type of technology The stored information is distributed over a variety of different memory units with very different physical characteristics
COMPUTER ARCHITECTURE 3

The development of automatic storage-allocation methods to make more efficient use of the available memory space. The development of virtual-memory concepts to free the ordinary user from memory management, and make programs largely independent of the physical memory configurations used. The design of communication links to the memory system so that all processors connected to it can operate at or near their maximum rates
increasing the effective memory processor bandwidth providing protection mechanisms
COMPUTER ARCHITECTURE 4

MAIN COMPONENTS OF THE MEMORY

1.

BLOCK DIAGRAM OF A MEMORY HIERARCHY (main components)

Direct Access Indirect Access

Internal processor memory

A small set of high-speed registers used for temporary storage of instructions and data.

2.

Main memory (or primary memory)

A relatively large & fast memory used for program and data storage during computer operation Locations in main memory can be accessed directly and rapidly by the CPU instruction set Semiconductor technology CPU Main Memory Auxiliary IO Memory (external)

3.

Secondary memory (or auxiliary memory)

Large memory capacity & much slower than main memory Store system programs, large data files, and the like which are not continually required by the CPU Information in secondary storage is accessed indirectly via input-output programs Representative technologies used for secondary memory are magnetic and optic disks
COMPUTER ARCHITECTURE 5

Registers

Cache Memory

S,p,tA c
COMPUTER ARCHITECTURE 6

LOCALITY OF REFERENCES PRINCIPLES

Programs commonly localize their memory references
Only a small portion of the memory is likely to be accessed at a given time This characteristic is not a natural law; it may be violated
But it is a behavior pattern exhibited by many programs at most times Exploiting this behavior is key to designing an effective memory hierarchy

TWO TYPES OF LOCALITY

Temporal Locality (Locality in Time)
The location of a memory reference is likely to be the same as another recent reference Programs tend to reference the same memory locations at a future point in time Due to loops and iteration, programs spending a lot of time in one section of code

Spatial Locality (Locality in Space)

The location of a memory reference is likely to be near another recent reference Programs tend to reference memory locations that are near other recently-referenced memory locations
7 COMPUTER ARCHITECTURE 8

COMPUTER ARCHITECTURE

TWO TYPES OF LOCALITY

Exploit Temporal Locality
Keep recently accessed data close to processor

MEMORY HIERARCHY PRINCIPLES

Increasing distance from the CPU represents also an increasing for the access time
Memory Level 1 CPU
Block A Block B

Exploit Spatial Locality

Move contiguous blocks of data between levels
Memory Level i Memory Level i+1

At any given time, data is copied only between two adjacent levels
Upper Level (the one closer to the processor)
Smaller, faster, and uses more expensive technology

Lower Level (the one further from the processor)

Bigger, slower, and uses less expensive technology
COMPUTER ARCHITECTURE 9 COMPUTER ARCHITECTURE 10

MEMORY HIERARCHY PRINCIPLES

The multi-level read-write memory system must satisfy two properties : Inclusion property: All information located on a upper memory level it is also stored on a lower memory level (ILi represents the information stored in memory level i):
IL1 IL2 ... ILn word miss / word hit

HIERARCHY TERMINOLOGY
Hit: Accessed data is found in upper level (for example Block A in level i)
Hit Rate = fraction of accesses found in upper level Hit Time = Time to access the upper level = Memory access time + Time to determine hit/miss

Miss: Accessed data found only in lower level (for example Block B in level i+1)
Processor waits until data is fetched then restarts the access Miss Rate = 1 (Hit Rate) Miss Penalty
Time to get block from lower level + time to replace in upper level + time to deliver the block the processor

Coherence property: An information stored at a specific address in level i must be the same on all lower memory levels
Maintain coherence:
propagation of the modified value to the lower levels ("write-through") updating information on lower levels when replaced from level i ("write-back")
COMPUTER ARCHITECTURE 11

Hit Time << Miss Penalty

COMPUTER ARCHITECTURE 12

Memory-Device Functional Characteristics

1. Storage capacity (S). Expressed in multiple of bits, or bytes.
210 bits = 1024 bits = 1 Kb; 1024 Mb = 1 Gb; 1024 Kb = 1 Mb 1024 Gb = 1 Tb 8 bits = 1 Byte = 1 B

Memory-Device Characteristics
3. Access time
The average time required to read a fixed amount of information, e.g., one word, from the memory - read access time or, more commonly, the access time of the memory (tA) The write access time is defined similarly - it is typically, but not always, equal to the read access time Access time depends on the physical characteristics of the storage medium, and also on the type of access mechanism used tA is usually calculated from the time a read request is received by the memory unit to the time at which all the requested information has been made available at the memory output terminals The access rate bA of the memory defined as 1/tA and measured in words per second
COMPUTER ARCHITECTURE 14

2. Cost. The price include not only the cost of the information storage cells themselves but also the cost of the peripheral equipment or access circuitry essential to the operation of the memory Let C be the price in dollars of a complete memory system with S bits of storage capacity, and specific cost c of the memory:
c= C dollars / bit S

COMPUTER ARCHITECTURE

13

Memory-Device Characteristics
4. Access modes - the order of sequence in which information can be accessed
Random-access memory (RAM) - if locations may be accessed in any order and access time is independent of the location being accessed
Semiconductor memories Each storage location has a separate access (addressing) mechanism

Memory-Device Characteristics
Serial-access memories - storage locations can be accessed only in certain predetermined sequences
Magnetic-tape, optical memories. The access mechanism is shared among different locations, and must be assigned to different locations at different times moving the stored information, the read-write head, or both Serial access tends to be slower than random access

Magnetic disks contain a large number of independent rotating tracks If each track has its own read-write head, the tracks may be accessed randomly, although access within each track is serial The access mode is sometimes called semi-random or, rather misleadingly, direct access

COMPUTER ARCHITECTURE

15

COMPUTER ARCHITECTURE

16

Memory-Device Characteristics
5. Alterability of information contained
ROMs - memories whose contents cannot be altered on-line
ROMs whose contents can be changed (usually offline and with some difficulty) are called programmable read-only memories (PROMs).

Memory-Device Characteristics
6. Permanence of storage - the stored information may be lost over a period of time unless appropriate action is taken Destructive readout (reading the memory destroys the stored information)
Each read operation must be followed by a write operation that restores the original state of the memory The restoration is usually carried out automatically, by write back into the location originally addressed from a buffer register

Dynamic storage
Over a period of time, a stored charge tends to leak away, causing a loss of information unless the charge is restored The process of restoring is called refreshing (dynamic memories)

RWM - memories in which reading or writing can be done on-line

Volatility
A memory is said to be volatile if the stored information can be destroyed by a power failure

COMPUTER ARCHITECTURE

17

COMPUTER ARCHITECTURE

18

Memory-Device Characteristics
7. Cycle time and data transfer rate Cycle time (tM) is the time needed to complete any read or write operation in the memory This means that the minimum time that must elapse between the initiation of two different accesses by the memory can be greater than tM Data transfer rate or bandwidth (bM) - the maximum amount of information that can be transferred to or from the memory every second (= 1/tM)

Memory-Device Characteristics
Cycle time vs access time
Memory cycle time Access time Seek Transfer time time Latency time Timp

t1

t2

t3

t4

bM =

1 tM

[ words / sec]

bM =

w tM

[bits / sec]

In cases where tA tM both are used to measure memory speed

w = memory bus width

COMPUTER ARCHITECTURE 19 COMPUTER ARCHITECTURE 20

Memory-Device Characteristics
8. Power: specific power
Ptot S

Main memory devices

p=

[W / bit ]

SRAM DRAM Cache memory

9. Geometry: only for semiconductor memories

N lines M columns

COMPUTER ARCHITECTURE

21

COMPUTER ARCHITECTURE

22

MAIN MEMORY
Main memory (primary or internal memory):
Used for program and data storage during computer operation Directly accessed by the CPU (microprocessor) Semiconductor technology Volatile memory (RWM) A random accessed memory Every cell is connected to selection lines (bit line for RD / WR) Every cell is connected to data lines (column bit lines for RD / WR)

Example cell memory

ROM-MOS
VGG

To output amplifier Column select ROM cell

Line select

Q1

COMPUTER ARCHITECTURE

23

COMPUTER ARCHITECTURE

24

Example cell memory

Static RAM (SRAM) cell
Word Line VDD

Example cell memory

Dynamic RAM
Word Line Word Line

Bit Line

pMOS

pMOS

nMOS

nMOS

Transfer Gate
Bit Line Bit Line

Sense Amplifier

Bit Line
COMPUTER ARCHITECTURE

Bit Line
25 COMPUTER ARCHITECTURE 26

Example: RAM selection cells

Bit Line Line Bit Line Bit

Simple example: 4 x 4 bits RAM

Data In Read/Write Register 1 0 1 Input Data pins Register 2
Address input

Line 0
Bit Line Line Bit Line A0 Bit A1 2 to 4 Decoder

b3 A1 A0 b3

b2

b1

b0
R/W

Line 1
Bit Line Line Bit Line Bit

2 Register 3

CS

Control signals

Address Chip Select

44 RAM b2 b1 b0

CS

Line 2

Output Data pins Register 4

Data Out

COMPUTER ARCHITECTURE

27

COMPUTER ARCHITECTURE

28

Static RAM (SRAM)

SRAM
A general approach to reducing the access circuitry cost in randomaccess memories is called matrix (array) organization The storage cells are physically arranged as rectangular arrays of cells
This is primarily to facilitate layout of the connections between the cells and the access circuitry

Bipolar or unipolar technology Operating cycles:

Read cycle Write cycle

Every cell contain a latch (1 bit)

Access circuitry:
Decode the address word Control the cell selection lines Read and write cells Control internal logic
COMPUTER ARCHITECTURE 29

The memory address is partitioned into d components so that the address Ai of cell Ci becomes a d-dimensional vector (Ai,1, Ai,2, ..., Ai,d) = Ai Each of the d parts of an address word goes to a different address decoder and a different set of address drivers A particular cell is selected by simultaneously activating all d of its address lines A memory unit with this kind of addressing is said to be a ddimensional memory
COMPUTER ARCHITECTURE 30

SRAM
The simplest array organizations have d = 1 and are called one-dimensional, or 1-D, memories If the storage capacity of the unit is N w bits, then the access circuitry typically contains a one-out-of-N address decoder and N address drivers For example, for a 8K 8 RAM memory, for this 1-D organization: a=13, N = 213 and w = 8.

Principle of 1-d addressing scheme

Storage locations Address Bus a

Address Register

Address Decoder

Internal Control Signals

Timing and control circuits

Data drivers and registers

Data Bus

CS
COMPUTER ARCHITECTURE 31

R/W

OE
32

COMPUTER ARCHITECTURE

Explanation
N = 2a The CS (chip select) input is used to enable the device
This is an active low input, so it will only respond to its other inputs if the CS line is low.

Principle of 2-D addressing

The address field is divided into two components, called X and Y, which consist of ax and ay bits, respectively
Array X Address Decoder of memory storage locations Nx

The OE (output enable) input is used to cause the memory device to place the contents of the location specified by the address inputs onto the data pins The WR (write enable) input is used to cause the memory device to write the internal location specified by the address inputs with the data currently appearing on the data pins
This is an active low signal, and the actual write occurs on the rising edge when WR goes from low back to the high state
COMPUTER ARCHITECTURE 33

Address Bus

ax

Ny Y Address Decoder
COMPUTER ARCHITECTURE 34

ay

Theoretical example of a 2-d organization

Principle of 2-D addressing

The cells are arranged in a rectangular array of rows and columns The total number of cells is N = NxNy A cell is selected by the coincidence of signals on its X and Y address lines The 2-D organization requires substantially less access circuitry than the 1-D for a fixed amount of storage For example, if Nx = Ny = N the number of address drivers needed is 2 N For N >> 4 the difference between 1-D and 2-D organizations is significant If the 1-bit storage cells are replaced by w-bit registers, then an entire word can be accessed in each read or write cycle but the bits within a word are not individually addressable For the example of 8K8 memory the 2-D organization, for 8-bits on a word can look theoretically like in next slide
COMPUTER ARCHITECTURE 35

for a 8K8 bits memory

8 8K cells

DCD X ADRESS 7

128 128

3 8K celule 2 8K celule 1 8K cells

64 64

DCD Y 6

COMPUTER ARCHITECTURE

36

Real structure example of a 2-d organization for a 8K8 bits memory

8 ADRESS DCD X 256 Array of cells 256 256 cells

Read Cycle Timing Diagram

tCR ADR

CS R/W
HiZ
MDX 0 d0 MDX 0 d1 Read / Write amplifiers 8 MDX 0 d7

32 5

32

32

tAC
HiZ VALID DATA

DATA OUT

tAA

Data Bus

37

tCR = read cycle time tAA = access time (from address change) tAC = access time (from CS active)
COMPUTER ARCHITECTURE 38

COMPUTER ARCHITECTURE

Write Cycle Timing Diagram

tCW ADR

DYNAMIC MEMORIES
DRAMs are currently the primary memory device of choice because they provide the lowest cost per bit and greatest density among solidstate memory technologies. Every cell contain MOS capacitor(s) DRAM - dynamic since needs to be refreshed periodically (8 ms, 1% time) Addresses divided (address pins multiplexed) into 2 halves (Memory as a 2D matrix):
RAS or Row Access Strobe CAS or Column Access Strobe

CS tW R/W tAW

Data BUS tDW tCW = write cycle time tDH = data hold time tDS = data set-up time tW = write width tDS tDH

tDW = delay for input data to write tAW = delay between address change and write control

E.g. 222 cells = 4 Mbit, memory array 2048 x 2048 cells, number of address IC pins = 11 Operating cycles:
Read cycle Write cycle Refreshment cycle
COMPUTER ARCHITECTURE 40

COMPUTER ARCHITECTURE

39

The Conventional DRAM (asynchronous interface)

The Conventional DRAM

The multiplexed address bus uses two control signals - the row and column address strobe signals, (RAS and CAS respectively) The row address causes a complete row in the memory array to propagate down the bit lines to the sense amps The column address selects the appropriate data subset from the sense amps and causes it to be driven to the output pins. Access transistors called sense amps are connected to the each column and provide the read and restore operations of the chip Since the cells are capacitors that discharge for each read operation, the sense amp must restore the data before the end of the access cycle. A refresh controller determines the time between refresh cycles, and a refresh counter ensures that the entire array (all rows) is refreshed
COMPUTER ARCHITECTURE 42

COMPUTER ARCHITECTURE

41

The Conventional DRAM

(typical memory access)

Conventional DRAM read cycles

1. 2. 3. 4. 5.

The row address bits are placed onto the address pins After a period of time the RAS\ signal falls, which activates the sense amps and causes the row address to be latched into the row address buffer The column address bits are set up The column address is latched into the column address buffer when CAS\ falls, at which time the output buffer is also turned on When CAS\ stabilizes, the selected sense amp feeds its data onto the output buffer /WE = 0 write, = 1 read. If became active (LOW) before /CAS (advance write cycle) data outputs remains in HiZ state If /WE became active (LOW) after /CAS the cycle is a write-read cycle

COMPUTER ARCHITECTURE

43

COMPUTER ARCHITECTURE

44

Key DRAM Timing Parameters

tRAC: minimum time from RAS line falling to the valid data output (quoted as the speed of a DRAM when buy)
A typical 4Mb DRAM tRAC = 60 ns

DRAM - fast speed op. modes

Modes: change the access to columns, to reduce the average access time Fast page mode
Page = All bits on the same ROW (spatial locality) Row-address is held constant and data from multiple columns is read from the sense amplifiers Dont need to wait for word-line to recharge Toggle CAS with new column address

tRC: minimum time from the start of one row access to the start of the next.
tRC = 110 ns for a 4Mbit DRAM with a tRAC of 60 ns

tCAC: minimum time from CAS line falling to valid data output.
15 ns for a 4Mbit DRAM with a tRAC of 60 ns

tPC: minimum time from the start of one column access to the start of the next.
35 ns for a 4Mbit DRAM with a tRAC of 60 ns

Nibble mode
Four successive column addresses generated internally

A 60 ns (tRAC) DRAM can

perform a row access only every 110 ns (tRC) perform column access (tCAC) in 15 ns, but time between column accesses is at least 35 ns (tPC).
COMPUTER ARCHITECTURE 45

Burst mode
More successive column addresses (in the same page) generated internally
COMPUTER ARCHITECTURE 46

Fast Page Mode DRAM (FPM DRAM)

Extended Data Out DRAM (EDO DRAM)

EDO DRAM adds a latch between the sense-amps and the output pins. This latch holds output pin state and permits the CAS to rapidly de-assert, allowing the memory array to begin precharging sooner The latch in the output path also implies that the data on the outputs of the DRAM circuit remain valid longer into the next clock phase
48

COMPUTER ARCHITECTURE

47

COMPUTER ARCHITECTURE

Extended Data Out DRAM

Nibble mode (serial four)

COMPUTER ARCHITECTURE

49

COMPUTER ARCHITECTURE

50

Burst mode

Refreshment modes
RAS only refresh (conventional refresh) CAS-before-RAS refresh (internal logic for refresh) Hidden refresh (refresh cycle is hidden in an read/write access cycle)

COMPUTER ARCHITECTURE

51

COMPUTER ARCHITECTURE

52

RAS only refresh

CAS-before-RAS refresh

COMPUTER ARCHITECTURE

53

COMPUTER ARCHITECTURE

54

Hidden refresh

Synchronous DRAM (SDRAM)

SDRAM exchanges data with the processor synchronized to an external clock signal SDRAM latches information to and from the controller based on a clock signal SDRAM employs a burst mode In burst mode, a series of data bits can be clocked out rapidly after the first bit has been accessed
This mode is useful when all the bits to be accessed are in sequence and in the same row of the array as the initial access

SDRAM has a dual-bank internal architecture that improves opportunities for on-chip parallelism (interleaved memory banks)
COMPUTER ARCHITECTURE 55 COMPUTER ARCHITECTURE 56

Synchronous DRAM (SDRAM)

SDRAM contains a programmable mode register and associated control logic that provides a mechanism to customize the SDRAM to suit specific system needs
The mode register specifies the burst length, which is the number of separate units of data synchronously fed onto the bus The register also allows the programmer to adjust the latency between receipt of a read request and the beginning of data transfer.

SDRAM Read Operation Clock Diagram

For SDRAM, there are 5 important timings:

The time required to switch internal banks The time required between /RAS and /CAS access The amount of time necessary to "prepare" for the next output in burst mode The column access time The time required to make data ready by the next clock cycle in burst mode (read cycle time)
COMPUTER ARCHITECTURE 57 COMPUTER ARCHITECTURE 58

DRAM vs SDRAM
DRAM No clock RAS control by change level One bank (array) of memory A transfer for every column address (or CAS pulse) Read delay (latency) no programmable
SDRAM Operate at the clock frequency RAS control at the clock impulse Two interleaved memory banks Sometimes static cache at the interface Burst transfer programmable (1, 2, 4, 8, or 256 transfers for a single provided column address, in the same row Read latency programmable

CACHE MEMORY

COMPUTER ARCHITECTURE

59

COMPUTER ARCHITECTURE

60

Cache principles
Cache memory is an intermediate temporary memory unit positioned between the processor registers and main memory Large and slow main memory smaller and faster cache memory The cache contains a copy of portions of main memory When the processor attempts to read a word of memory, a check is made to determine if the word is in the cache
If so, the word is delivered to the processor If not, a block of main memory, consisting of some fixed number of words, is read into the cache and then the word is delivered to the processor

Cache and Main Memory

Because of the phenomenon of locality of reference, when a block of data is fetched into the cache to satisfy a single memory reference, it is likely that future references will be to other words in the block.
COMPUTER ARCHITECTURE 61 COMPUTER ARCHITECTURE 62

STRUCTURE OF A CACHE/MAINMEMORY SYSTEM CACHE READ OPERATION

M = 2a/K blocks m<<M

COMPUTER ARCHITECTURE

63

COMPUTER ARCHITECTURE

64

HIT RATIO
The performance of cache memory can be given by a synthetic parameter, called hit ratio (HR) Hit ratio determined running benchmarks HR represents the ratio between the total numbers of hits in cache and the total number of memory accesses (hit + miss numbers) The value of HR must be greater then 0.9 From HR value we can compute the average memory access time. For example if HR = 0.9, the access time to main memory (for misses) is 100 nsec. and access time to cache memory is 10 ns, the average memory access time is:

Mapping Function
Because there are fewer cache lines than main memory blocks is needed:
An algorithm for mapping main memory blocks into cache lines A means for determining which main memory block currently occupies a cache line

The choice of the mapping function dictates how the cache is organized Three techniques can be used:
Direct mapping Associative mapping Set associative mapping

t acm

9 10 + 100 = = 19ns 10
COMPUTER ARCHITECTURE 65

COMPUTER ARCHITECTURE

66

Direct mapping
Direct mapping is the simplest technique Direct mapping maps each block of main memory into only one possible cache line The mapping is expressed as: i = j modulo m
where i = cache line number; j = main memory block number; m = number of lines in the cache

Direct mapping
Sometime the word and block fields are called index field, because the index is used to address data in cache The use of a portion of the address as a line number provides a unique mapping of each block of main memory into the cache When a block is actually read into its assigned line, it is necessary to tag the data to distinguish it from other blocks that can fit into that line The effect of this mapping is that blocks of main memory are assigned to lines of the cache as follows:

For purposes of cache access, each main memory address can be viewed as consisting of three fields
The least significant w bits identify a unique word or byte within a block of main memory (in most contemporary machines, the address is at the byte level) The remaining a-w bits specify one of the 2a-w blocks of main memory The cache logic interprets these a-w bits as a tag of t bits (most significant portion) plus a cache line field of r bits (a = t+r+w)

COMPUTER ARCHITECTURE

67

COMPUTER ARCHITECTURE

68

DirectMapping Cache Organization

Example for address 1D00002E hex

32 bits memory address 8 bits tag 20 bits block address 4 bits word addr.

COMPUTER ARCHITECTURE

69

COMPUTER ARCHITECTURE

70

Direct mapping
Advantages for direct mapping:
Simple and cheap The tag field is short
Only those bits have to be stored which are not used to address the cache

Associative mapping
Each main memory block can be loaded into any line of the cache The cache control logic interprets a memory address simply as a tag and a word field In the cache is stored also data and the corresponding address The associative mapping is implemented with associative memories (content addressable memories) as cache memories
more expensive than a random access memory each cell must have storage capacity as well comparison logic circuits for matching its content with an external argument
COMPUTER ARCHITECTURE 72

Access is very fast.

Disadvantage:
There is a fixed cache location for any given block If a program happens to reference words repeatedly from two different blocks that map into the same line, then the blocks will be continually swapped in the cache, and the hit ratio will be low.

COMPUTER ARCHITECTURE

71

Fully associative mapping

Fully associative mapping

COMPUTER ARCHITECTURE

73

COMPUTER ARCHITECTURE

74

Fully associative mapping

Set associative mapping

Set associative mapping is a compromise that exhibits the strengths of both the direct and associative approaches while reducing their disadvantages In this case, the cache is divided into v sets, each of which consists of k lines This is referred to as k-way set associative mapping With set associative mapping, block Bj can be mapped into any of the lines of set i In this case, the cache control logic interprets a memory address simply as three fields: tag, set, and word
With fully associative mapping, the tag in a memory address is quite large and must be compared to the tag of every line in the cache With k-way set associative mapping, the tag in a memory address is much smaller and is only compared to the k tags within a single set

Advantages:
provides the highest flexibility concerning the line to be replaced when a new block is read into the cache i = j modulo 1 where i = cache line number; j = main memory block number;

Disadvantages:
complex the tag field is long fast access can be achieved only using high performance associative memories for the cache, which is difficult and expansive.
COMPUTER ARCHITECTURE 75

COMPUTER ARCHITECTURE

76

Set associative mapping

The mapping relationships are: i = j modulo v m =vk where i = cache set number; j = main memory block number; m = number of lines in the cache; v = number of sets in cache; k = number of cache lines per set In the extreme case of v = m, k = 1, the set associative technique reduces to direct mapping In the extreme case of v = 1, k = m, it reduces to associative mapping The use of two lines per set (v = m/2, k = 2) is the most common set associative organization (two-way set associative mapping) Four-way set associative (v = m/4, k = 4) makes a modest additional improvement for a relatively small additional cost
Further increases in the number of lines per set have little effect.

Example two-way set associative

COMPUTER ARCHITECTURE

77

COMPUTER ARCHITECTURE

78

EXAMPLE
Assume that:
Main memory is divided in 32 blocks Cache cache has 8 block frames (lines) The set-associative organization has 4 sets with 2 blocks per set, called two-way set associative

EXAMPLE

Where in cache main memory block 12 can be placed, for the three categories of cache organization?
Fully associative: block 12 can go into any of the eight block frames of the cache Direct mapped: block 12 can only be placed into block frame 4 (12 modulo 8) Two-way set associative: allows block 12 to be placed anywhere in set 0 (12 modulo 4). Line 0 or line 1
COMPUTER ARCHITECTURE 79 COMPUTER ARCHITECTURE 80

Replacement Algorithms
When a new block is brought into the cache, one of the existing blocks must be replaced For direct mapping, there is only one possible line for any particular block, and no choice is possible For the associative and set associative techniques, a replacement algorithm is needed To achieve high speed, such an algorithm must be implemented in hardware

Replacement Algorithms
LRU (least recently used): Replace that block in the set that has been in the cache longest with no reference to it
For two-way set associative each line includes a USE bit When a line is referenced, its USE bit is set to 1 and the USE bit of the other line in that set is set to 0 When a block is to be read into the set, the line whose USE bit is 0 is used Because we are assuming that more recently used memory locations are more likely to be referenced, LRU should give the best hit ratio.

FIFO (first-in-first-out): Replace that block in the set that has been in the cache longest
FIFO is easily implemented as a round-robin or circular buffer technique.

LFU (least frequently used: Replace that block in the set that has experienced the fewest references
LFU could be implemented by associating a counter with each line

COMPUTER ARCHITECTURE 81

Random replacement: is the simplest to implement and results are surprisingly good.
COMPUTER ARCHITECTURE 82

Write Policy
Write through: all write operations are made to main memory as well as to the cache
Any other processor-cache module can monitor traffic to main memory to maintain consistency within its own cache The main disadvantage of this technique is that it generates substantial memory traffic and may create a bottleneck

Write Policy
Write-through with buffered write
The same as write-through, but instead of slowing the processor down by writing directly to main memory, the write address and data are stored in a high-speed write buffer; the write buffer transfers data to main memory while the processor continues its task

Dirty bits: this status bit indicates whether the block is dirty (modified while in the cache) or clean (not modified)
If it is clean, the block is not written on a miss, since the lower level has identical information to the cache.

Write back: updates are made only in the cache

Minimizes memory writes When an update occurs, an UPDATE bit associated with the line is set When a block is replaced, it is written back to main memory if and only if the UPDATE bit is set The problem with write back is that portions of main memory are invalid, and hence accesses by I/O modules can be allowed only through the cache
COMPUTER ARCHITECTURE 83

Both write back and write through have their advantages

With write back, writes occur at the speed of the cache memory, and multiple writes within a block require only one write to the lower-level memory
write back uses less memory bandwidth, making write back attractive in multiprocessors

With write through, read misses never result in writes to the lower level, and write through is easier to implement than write back
Write through also has the advantage that the next lower level has the most current copy of the data This is important for I/O and for multiprocessors
COMPUTER ARCHITECTURE 84

Write Policy

Write Policy
Possible approaches to cache coherency include the following:
Bus watching with write through: Each cache controller monitors the address lines to detect write operations to memory by other bus masters
If another master writes to a location in shared memory that also resides in the cache memory, the cache controller invalidates that cache entry

In a bus organization in which more than one device (typically a processor) has a cache and main memory is shared: If data in one cache is altered:
This invalidates the corresponding word in main memory This invalidates the corresponding word in other caches (if any other cache happens to have that same word) Even if a write-through policy is used, the other caches may contain invalid data

Hardware transparency: Additional hardware is used to ensure that all updates to main memory via cache are reflected in all caches
Thus, if one processor modifies a word in its cache, this update is written to main memory. In addition, any matching words in other caches are similarly updated.

Noncachable memory: Only a portion of main memory is shared by more than one processor, and this is designated as noncachable
All accesses to shared memory are cache misses, because the shared memory is never copied into the cache The noncachable memory can be identified using chip-select logic or highaddress bits

A system that prevents this problem is said to maintain cache coherency

COMPUTER ARCHITECTURE

85

COMPUTER ARCHITECTURE

86

Write Policy for write miss

Since the data are not needed on a write, there are two common options on a write miss: Write allocate (also called fetch on write)
The block is loaded on a write miss, followed by the write-hit actions above This is similar to a read miss.

No-write allocate (also called write around)

The block is modified in the lower level and not loaded into the cache.

Write-back caches generally use write allocate (hoping that subsequent writes to that block will be captured by the cache) Write-through caches often use no-write allocate (since subsequent writes to that block will still have to go to memory)

COMPUTER ARCHITECTURE

87