Memory System, Memory

Interleaving
Sudhansu Ranjan Lenka
Basic Concepts
• Memory system stores the program and data from which the
processor fetches the instructions to execute at the processing
subunits.
• The processor could place the processed data into the memory for
future reference.
• A 16 bit computer that generates 16 bit addresses is capable of
addressing upto 216 = 64k memory location.
• The data transfer between memory and processor takes place
through two processor registers:
1. MAR
2. MDR.

20-07-2023 Sudhansu Ranjan Lenka 2

Basic Concepts
• If MAR is k bits and MDR is n bits, then memory unit contains 2k
addressable locations and n bits data are transferred between
memory and processor.
• Memory access time : The time consumed between the initialization
of an operation and completion of that operation is called memory
access time.
• Memory cycle time: It is the minimum time delay required between
two successive memory operations.

20-07-2023 Sudhansu Ranjan Lenka 3

EX:- A certain memory chip is specified as 2K×8:
1. How many words can be stored on this chip?
2. What is the words size?
3. How many total bits can this chip store?
SOL:-
1. 2K =2 × 1024 = 2048 words( bytes)
2. The word size is 8-bits (1 byte).
3. Capacity = 2048 × 8 = 16384 bits. Memory chip

20-07-2023 Sudhansu Ranjan Lenka 4

• How many chips are needed to design a 8Mx32 memory using 512Kx8
memory chips?
8M x 32 228
No. of chips= = 22 = 64 chips.
512K x 8 2

Q. How many separate address lines and data lines are needed for a
memory capacity 8M x 32 .

Address lines= 8M = 23 bits

Data Lines= 5 bits

20-07-2023 Sudhansu Ranjan Lenka 5

Cache Memory
• The cache is a smaller and faster memory that stores copies of the
data from frequently used main memory locations.
• The most important use of cache memory is that it is used to reduce
the average time to access data from the main memory.
• It is present between the between the processor and the main
memory.

20-07-2023 Sudhansu Ranjan Lenka 6

Cache Memory
• The effectiveness of the cache mechanism is based on property of
locality of reference.
• It means many instructions of local areas of the program are executed
repeatedly.
• The active segment of the program are placed in cache memory there
by reducing the total execution time.
• The property of Locality of Reference is mainly shown by loops and
subroutine calls in a program
• Two write operations in cache memory:
In write through technique, the cache location and main memory
location are updated simultaneously.

20-07-2023 Sudhansu Ranjan Lenka 7

Cache Memory basic operation
• When the CPU needs to access memory, the cache is examined.
• If the word is found is in the cache, it is read from the fast memory.
• If the word is not found in the cache, then the main memory is accessed to
read the word.
• If not present, read required block from main memory to cache
• Then deliver from cache to CPU
• A block of words containing the one just accessed is then transferred from
main memory to cache memory.
• In this way some data are transferred to cache so that future references to
memory find the required words in the fast cache memory.
• Cache includes tags to identify which block of main memory is in each
cache slot

20-07-2023 Sudhansu Ranjan Lenka 8

In write back, the data is updated only in the cache and updated into
the memory at a later time. Data is updated in the memory only
when the cache line is ready to be replaced.
During write operation, only the cache location is updated in the
write-back method. Then, the location is marked by a flag so that it is
later copied to the main memory when the word is removed from the
cache
Dirty Bit: Each Block in the cache needs a bit to indicate if the data
present in the cache was modified(Dirty) or not modified(Clean).

20-07-2023 Sudhansu Ranjan Lenka 9

Cache Memory
• There are two types of localities −
1. Temporal locality
2. Spatial locality
Temporal Locality – Temporal locality means current data or instruction
that is being fetched may be needed soon. So we should store that data
or instruction in the cache memory so that we can avoid again
searching in main memory for the same data.
Spatial Locality – Spatial locality means instruction or data near to the
current memory location that is being fetched, may be needed soon in
the near future.

20-07-2023 Sudhansu Ranjan Lenka 10

Cache Performance
• When the processor needs to read or write a location in the main
memory, it first checks for a corresponding entry in the cache.
• If the processor finds that the memory location is in the cache,
a Cache Hit has occurred and data is read from the cache.
• If the processor does not find the memory location in the cache,
a cache miss has occurred.
• The ratio of the number of hits divided by the total CPU references to
memory ( hit + misses) is the hit ratio.
• The performance of cache memory is frequently measured in terms
of a quantity called Hit ratio.

20-07-2023 Sudhansu Ranjan Lenka 11

Cache Performance
Hit Ratio(H) = hit / (hit + miss) = no. of hits/total accesses

Miss Ratio = miss / (hit + miss) = no. of miss/total accesses = 1 - hit ratio(H)

• Average access time of any memory system consists of two levels: Cache and Main Memory.
• If Tc is time to access cache memory and Tm is the time to access main memory then we can write:
Average time to access memory (Tavg)=
For simultaneous access Tavg = H * Tc + (1-H)*Tm
For hierarchial access Tavg = H * Tc + (1-H)*(Tm + Tc)

• Miss penalty: The extra time needed to bring the desired information into cache is called miss penalty. This
refers to main memory access time if miss occurs.

20-07-2023 Sudhansu Ranjan Lenka 12

Example
• A computer with cache access time is 100ns, a main memory access
time of 1000ns, and a hit ratio of 0.9. Find the average access time?
• Soln: Average access time= 0.9 * 100 + 0.1(100+1000)= 90 + 110= 200
ns

20-07-2023 Sudhansu Ranjan Lenka 13

20-07-2023 Sudhansu Ranjan Lenka 14
• Soln:
a) Avg read access time= read hit ratio x cache + (1- read hit ratio)x miss penalty = 0.9x 100 +
0.1(100+1000)= 200ns.
Write access time= max(100, 1000)= 1000ns
b) Avg access time for read and write= 80% are read operation + 20% write operation
= 0.8 x 200 + 0.2x 1000= 160 + 200= 360ns
c) Hit ratio= 0.8 x 0.9 + 0.2x 0 ( Hit ratio for write access is always 0)= 0.72
20-07-2023 Sudhansu Ranjan Lenka 15
Cache Mapping
The transformation of data from main memory to cache memory is
called mapping. There are 3 main types of mapping:
• Direct Mapping
• Associative Mapping
• Set-Associative Mapping

20-07-2023 Sudhansu Ranjan Lenka 16

 • Consider a main memory of size 32K words of 12 bits each . The cache
is capable of storing 512 of these words at any given time.
• For every word stored in cache, there is a duplicate copy of main
memory.
• CPU to communicate with both memory, it first sends 15 bit address
to cache.
• If there is a hit, CPU accepts 12 bit data from cache.
• If there is a miss, CPU reads from main memory and the word is then
transferred to cache.

20-07-2023 Sudhansu Ranjan Lenka 17

Associative Mapping
• The fastest and most flexible cache organization uses an associative
memory.
• In associative mapping, cache are made up of associative memory
• The associative memory stores both data and content of the memory
word.
• It permits any location in cache to store any word from main memory.

20-07-2023 Sudhansu Ranjan Lenka 18

• CPU address of 15 bits are stored in
argument register and the associative Associative mapping cache ( all numbers are in octal)
memory is searched for matching address..
• The address of 15 bits is shown as five digit
octal number and its corresponding 12 bits Argument Register
word as four digit octal number.
• If the address is found , the corresponding 12 Address Data
digit data is read and sent to the CPU. 01000 3450
• If no match occur then main memory is 02777 6710
accessed for the word. 22345 1234
• The address-data is then transferred to the
associative memory.
• If the cache is full, FIFO replacement policy is
used to create space in cache memory.

20-07-2023 Sudhansu Ranjan Lenka 19

 Cache

Tag Block 0
• In AM a main memory block can be map to any cache
location.
Block 1
Tag • Suppose total no. blocks in main memory is 4096=2 12 .
... • So a 12 bit tag is used to identify the location of the
Tag
main memory block in cache.
Block 127
Tag • The higher order 12bits of memory address of the
memory address is compared to the tag bits of each
15 bits memory address block of the cache to verify the desired block is present
or not.
12 bits (Tag) 3 bits (word)
• If each block contain 8 words, then the remaining 3
bits comprises word.
• If the desired block is present, the 3 bits word is used to
identify the necessary word in the cache.

20-07-2023 Sudhansu Ranjan Lenka 20

 Direct Mapping
• Direct mapping, maps each block of main memory into only one
possible cache line. Or
• In Direct mapping, assign each memory block to a specific line in the
cache.
• If a line is previously taken up by a memory block when a new block
needs to be loaded, the old block is trashed.
• Line number is calculated using the following function
i = j modulo m
where
i = cache line number
j = main memory block number
m = number of lines in the cache

20-07-2023 Sudhansu Ranjan Lenka 21

Example-
Consider cache memory is divided into ‘n’ number of lines.

Then, block ‘j’ of main memory can map to line number (j mod n) only of the cache.

• In direct mapping, there is no need of any replacement algorithm.

• This is because a main memory block can map only to a particular line of the cache.
• Thus, the new incoming block will always replace the existing block (if any) in that particular line.

20-07-2023 Sudhansu Ranjan Lenka 22

Direct Mapping

• The CPU address of 15 bits is divided into two fields:

• The nine LSBs constitutes index field and the remaining 6 bits form tag field
• The tag and index bits together act as address of Main memory and the index field is equal to the address
bits of cache memory.
• When the CPU generates a memory request, the index field is used to access the address of cache.

20-07-2023 Sudhansu Ranjan Lenka 23

• The tag field of the CPU address is compared with the tag of the cache. Memory Memory
If the two tags matches, then there is a hit. If no match then miss occurs. address data
Suppose the word at address zero is presently Stored in the
00000 1220
cache (index=000, tag=00, data=1220) . Main
Memory
If CPU now wants to access the word at address 02000. The index address
00777 2340
Is 000. Next the two tags are compared. The cache tag is 00 but address tag
01000 3450
Is 02. So, miss occurs.
02000 5670
Therefore, main memory is accessed and the data word 5670 is transferred
02777 6710
To the cache.
The direct mapping in the given slide uses a block size of one word.

Index Address Tag Data

000 00 1220

777 02 6710
Cache Memory
20-07-2023 Sudhansu Ranjan Lenka 24
 Direct Mapping with multiple words in a block
• The same organization but the block size is 8 words. In this case, the index field is now divided into 2 parts: block field and word field.
• Consider a cache consisting of 512 words , 512/ 8 = 64 blocks.
• 64 blocks is defined by 6 bits. So the block field is 6 bits and the word field = 3 bits.
• The tag field stored within the cache is common for all the 8 words of the same block.
• When a miss occurs entire block of 8 words are transferred from main memory to cache.

Index
Index Tag Data
000 01 3450 Tag (6) Block (6) Word (3)
Block 0
007 01 6578

Block 1 010

017
770 02
Block 63 777 02 6710

20-07-2023 Sudhansu Ranjan Lenka 25

Set Associative Mapping
• In set associative mapping a blocks of cache are grouped into sets and mapping allows a block of main
memory to reside in any block of a set.
• Each word of cache can store two or more words of memory under the same index address.
• Each index address refers to two data words and their associated tags.
• Each tag requires 6 bits and 12 bits data, so the word length is 2(6+12)= 36 bits.
• An index address of 9 bits can accommodate 512 (29) words. Thus, cache memory size is 512 x 36. It can
accommodate 512 x 2= 1024 words of main memory.
• The tag field of the processor address is then compared with the tags of all K lines within that set.
• If these two tags do not match the tag of any cache line, a cache miss occurs.
• If these two tags match, a cache hit occurs.
• In case of a cache miss, the required word has to be brought from the main memory.

Index Tag Data Tag Data

000 01 3450 02 5670 2 way set associative mapping cache

Main Memory address

Tag Set Word
20-07-2023 777 02 6710 00 2340 Ranjan Lenka
Sudhansu 26
• In general, a cache that has k blocks per set is known as k-way set associative.
1. Consider a cache of 128 blocks and there are 4 blocks per set. Find the set field size.
No. of sets = 128 / 4 = 32.
Set field = 5 bits .
2. Consider a computer system that has 4k words cache organized in block-set associative with 4 blocks/ set
and 64 words per block. What is the no. of bits in SET and WORD fields of the main memory address format.
Soln: Total no of words= 4k= 212.
No. of Blocks= 212/ 64= 212/ 26= 26.
No. of set= 26/ 4= 24.

64 words/block= 26= 6 bits word.

Set size=4 bits.
3. A set associative map consists of a total no. of 64 blocks which are divided into 4 blocks/set. The MM
contains 4096 blocks and each consisting of 128 words.
a) How many bits are there in main memory address?
b) How many bits are there in each of TAG,SET and WORD fields?

20-07-2023 Sudhansu Ranjan Lenka 27

Memory Interleaving
• It is an abstraction technique that divides memory into many modules such that successive words
in the address space are placed in different modules.
• Higher order bits represents the module address

Figure-1: Consecutive Word in a Module different modules.

20-07-2023 Sudhansu Ranjan Lenka 28

• Let us assume 16 Data’s to be Transferred to the Four Module. Where
Module 00 be Module 1, Module 01 be Module 2, Module 10 be
Module 3 & Module 11 be Module 4.
• Also, 10, 20, 30….130 are the data to be transferred.
• From the figure above in Module 1, 10 [Data] is transferred then 20,
30 & finally, 40 which are the Data. That means the data are added
consecutively in the Module till its max capacity.
• Most significant bit (MSB) provides the Address of the Module & the
least significant bit (LSB) provides the address of the data in the
module.

20-07-2023 Sudhansu Ranjan Lenka 29

Advantages of interleaving
• Whenever Processor requests Data from the main memory. A block
(chunk) of Data is Transferred to the cache and then to Processor.
• So whenever a cache miss occurs the Data is to be fetched from the
main memory.
• But main memory is relatively slower than the cache.
• So to improve the access time of the main memory interleaving is
used.
• We can access all four Modules at the same time thus achieving
Parallelism(Consecutive Word in Consecutive Module).
• From Figure 2 (next slide) the data can be acquired from the Module
using the Higher bits. This method Uses memory effectively.
20-07-2023 Sudhansu Ranjan Lenka 30
Consecutive Word in Consecutive Module:
• Now again we assume 16 Data’s to be transferred to the Four Module. But Now the consecutive Data
are added in Consecutive Module.
• For Example, to get 90 (Data) 1000 will be
provided by the processor.
This 00 will indicate that the data is in
module 00 (module 1) & 10 is the address of 90
in Module 00 (module 1).

20-07-2023 Sudhansu Ranjan Lenka 31

Types of Interleaved Memory
1. High order interleaving: In high order memory interleaving, the
most significant bits of the memory address decides memory banks
(module). The least significant bits are sent as addresses to each
chip. One problem is that consecutive addresses tend to be in the
same chip. The maximum rate of data transfer is limited by the
memory cycle time. It is also known as Memory Banking.

20-07-2023 Sudhansu Ranjan Lenka 32

2. Low order interleaving: The least significant bits select the memory
bank (module) in low-order interleaving. In this, consecutive memory
addresses are in different memory modules, allowing memory access
faster than the cycle time.

20-07-2023 Sudhansu Ranjan Lenka 33

Benefits of Interleaved Memory
• An instruction pipeline may require instruction and operands both at
the same time from main memory, which is not possible in the
traditional method of memory access.
• Similarly, an arithmetic pipeline requires two operands to be fetched
simultaneously from the main memory.
• So, to overcome this problem, memory interleaving comes to resolve
this.

20-07-2023 Sudhansu Ranjan Lenka 34