Contents

• Memory Interleaving
• Cache Memory
 Memory Interleaving
• It is a Technique that divides memory into a
number of modules such that Successive words
in the address space are placed in the different
modules.
• Interleaved memory is designed to compensate
for the relatively slow speed of dynamic random-
access memory (DRAM)
• Contiguous memory reads and writes use each
memory bank, resulting in higher memory
throughput.
 Example of Interleaved Memory
• Suppose we have 4 memory banks, each containing 256 bytes, in
Interleaved memory, virtual address 0 will be with the first bank, 1 with
the second memory bank, 2 with the third bank and 3 with the fourth,
and then 4 with the first memory bank again.
• The CPU can access alternate sections immediately without waiting for
memory to be cached.
• Data with virtual addresses 0, 1, 2 and 3 can be accessed simultaneously
as they reside in separate memory banks.
• An interleaved memory with n banks
is said to be n-way interleaved.
• Example: 2-way interleaved
 Why do we use Memory Interleaving?
• When the processor requests data from the main memory, a block (chunk) of
data is transferred to the cache and then to processor.
• So whenever a cache miss occurs, the data is to be fetched from the main
memory.
• But main memory is relatively slower than the cache.
• So to improve the access time of the main memory, interleaving is used.
• For example, we can access all four modules at the same time, thus achieving
parallelism. The data can be acquired from the module using the higher bits.
This method uses memory effectively.
 Types of
Interleave
d Memory
• 1. High order interleaving: In
high order memory
interleaving, the most
significant bits of the memory
address decides memory
banks where a particular
location resides. But, in low
order interleaving the least
significant bits of the memory
address decides the memory
banks.
Types of Interleaved
Memory
• 2. Low order
interleaving: The least
significant bits select the
memory bank (module) in
low-order interleaving. In
this, consecutive memory
addresses are in different
memory modules,
allowing memory access
faster than the cycle time.
•
 Benefits of Interleaved Memory

• It allows simultaneous access to different modules of memory. The modular

memory technique allows the CPU to initiate memory access with one module
while others are busy with the CPU in reading or write operations. So, we can
say interleave memory honors every memory request independent of the state
of the other modules.
• So, for this obvious reason, interleave memory makes a system more responsive
and fast than non-interleaving. Additionally, with simultaneous memory access,
the CPU processing time also decreases and increasing throughput. Interleave
memory is useful in the system with pipelining and vector processing.
• Memory system with a latency of 40 ns that transfers 1 byte per
operation
• Pipelined to allow 4 operations to overlap execution (assume no
pipelining overhead)
• Dividing the latency of 40 ns by the number of overlapped operations
(4 gives a rate of 1 operation per 10 ns as the throughput of the memory
system
• At 1 byte of data per operation, this gives a bandwidth of 10^8 bytes/s
 Example
• Assume─ A memory system has four banks, each of which has a latency of 100 ns and is pipelined to allow
8 operations to overlap execution
• Each bank returns 4 bytes of data in response to a memory request
• Each bank has a latency of 100 ns and can pipeline 8 operations
• Therefore, the throughput of each bank is 1 operation every 100 ns/ 8 = 12.5 ns, or 80,000,000
operations/s
• Since there are 4 banks, the peak throughput of the memory system is 4 * 80,000.000 =
320,000,000 operations/s
• With each memory operation returning 4 bytes of data, this gives a peak bandwidth of 4 * 320,000,000
= 1,280,000,000 bytes/s
• Peak bandwidth computations assumes no idle time between one and other subsequent accesses after
the first
 Cache Memory
• Cache memory is a high-speed memory, which is small in size but faster than the
main memory (RAM).
• The CPU can access it more quickly than the primary memory. So, it is used to
synchronize with high-speed CPU and to improve its performance.
• It holds the data and programs which are frequently used by the CPU.
 Cache
Memory
smaller than
main cache views
memory main
memory as
block
word organized in
transfer
transfer “blocks”

cache
 Types of Cache Memory: L1
• L1: In this type of cache memory, a small amount of memory is present
inside the CPU itself.
• If a CPU has four cores (quad core cpu), then each core will have its own
level 1 cache.
• The L1 cache further has two types of caches: Instruction cache, which
stores instructions required by the CPU, and the data cache that stores
the data required by the CPU.
• The size of this memory ranges from 2KB to 64 KB.
 Types of Cache Memory: L2
• L2: This level 2 cache may be inside the CPU or outside the CPU.
• All the cores of a CPU can have their own separate level 2 cache, or they
can share one L2 cache among themselves.
• In case it is outside the CPU, it is connected with the CPU with a very
high-speed bus.
• The memory size of this cache is in the range of 256 KB to the 512 KB. In
terms of speed, they are slower than the L1 cache.
 Types of Cache Memory: L3
• L3: It is known as Level 3 cache or L3 cache. This cache is not present in all
the processors; some high-end processors may have this type of cache.
• This cache is used to enhance the performance of Level 1 and Level 2
cache.
• It is located outside the CPU and is shared by all the cores of a CPU. Its
memory size ranges from 1 MB to 8 MB.
• Although it is slower than L1 and L2 cache, it is faster than Random
Access Memory (RAM).
 How does cache memory work with CPU?
• When CPU needs the data, first of all, it looks inside the L1 cache.
• If it does not find anything in L1, it looks inside the L2 cache.
• If does not find the data in L2 cache, it looks into the L3 cache.
• If data is found in the cache memory, then it is known as a cache hit.
• If data is not found inside the cache, it is called a cache miss.
• If data is not available in any of the cache memories, it looks inside the Random Access
Memory (RAM).
• If RAM also does not have the data, then it will get that data from the Hard Disk Drive.
 Locality of
Reference
• It refers to a phenomenon in which a computer program tends to access
same set of memory locations for a particular time period.
• In other words, Locality of Reference refers to the tendency of the
computer program to access instructions whose addresses are near one
another.
• The property of locality of reference is mainly shown by loops and
subroutine calls in a program.
 locality of reference
• Based on the principle of locality of reference. There are two ways with which data or
instruction is fetched from main memory and get stored in cache memory.
• Temporal Locality: Temporal locality means current data or instruction that is being
fetched may be needed soon. So we should store that data or instruction in the cache
memory so that we can avoid again searching in main memory for the same data.
• Spatial Locality: Spatial locality means instruction or data near to the current memory
location that is being fetched, may be needed soon in the near future. This is slightly
different from the temporal locality. Here we are talking about nearly located memory
locations while in temporal locality we were talking about the actual memory location
that was being fetched.
 Parameters of Cache
• Cache Hit
memory
• A referenced item is found in the cache by the processor
• Cache Miss
• A referenced item is not present in the cache
• Hit ratio
• Ratio of number of hits to total number of references => number of hits/(number of hits + number of Miss)
• Miss penalty
• Additional cycles required to serve the miss
• Time required for the cache miss depends on both the latency and bandwidth
• Latency – time to retrieve the first word of the block
• Bandwidth – time to retrieve the rest of this block