CACHE

MEMORY

02/20/22
How caching works

1
02/20/22
2
02/20/22
3
 What is a Cache?
The cache is a very high speed, expensive piece of memory, which is used to

02/20/22
speed up the memory retrieval process. Due to it’s higher cost, the CPU comes
with a relatively small amount of cache compared with the main memory.
Without cache memory, every time the CPU requests for data, it would send the
request to the main memory which would then be sent back across the system
bus to the CPU. This is a slow process. The idea of introducing cache is that this
extremely fast memory would store data that is frequently accessed and if
possible, the data that is around it. This is to achieve the quickest possible
response time to the CPU.

4
CACHE ENTRY STRUCTURE
Cache row entries usually have the following structure:

Data blocks Tag Index Displacement Valid bit

The data blocks (cache line) contain the actual data fetched from the main
memory. The memory address is split into the tag, the index and the
displacement (offset), while the valid bit denotes that this particular entry has
valid data.

•The index length is bits and describes which row the data has been put in.

•The displacement length is and specifies which block of the ones we have stored
we need.

•The tag length is address − index − displacement

 COMPARISON OF CACHE SIZES
Year of
Processor Type L1 cache L2 cache L3 cache
Introduction
IBM 360/85 Mainframe 1968 16 to 32 KB — —
PDP-11/70 Minicomputer 1975 1 KB — —
VAX 11/780 Minicomputer 1978 16 KB — —
IBM 3033 Mainframe 1978 64 KB — —
IBM 3090 Mainframe 1985 128 to 256 KB — —
Intel 80486 PC 1989 8 KB — —
Pentium PC 1993 8 KB/8 KB 256 to 512 KB —
PowerPC 601 PC 1993 32 KB — —
PowerPC 620 PC 1996 32 KB/32 KB — —
PowerPC G4 PC/server 1999 32 KB/32 KB 256 KB to 1 MB 2 MB
IBM S/390 G4 Mainframe 1997 32 KB 256 KB 2 MB
IBM S/390 G6 Mainframe 1999 256 KB 8 MB —
Pentium 4 PC/server 2000 8 KB/8 KB 256 KB —
High-end server/
IBM SP 2000 64 KB/32 KB 8 MB —
supercomputer
CRAY MTAb Supercomputer 2000 8 KB 2 MB —
Itanium PC/server 2001 16 KB/16 KB 96 KB 4 MB
SGI Origin 2001 High-end server 2001 32 KB/32 KB 4 MB —
Itanium 2 PC/server 2002 32 KB 256 KB 6 MB
IBM POWER5 High-end server 2003 64 KB 1.9 MB 36 MB
CRAY XD-1 Supercomputer 2004 64 KB/64 KB 1MB —
 Role of Cache in Computers
In early PCs, the various components had one thing in common: they were all really
slow. The processor was running at 8 MHz or less, and taking many clock cycles to get
anything done. In fact, on some machines the memory was faster than the processor.

02/20/22
With the advancement of technology, the speed of every component has increased
drastically. Now processors run much faster than everything else in the computer. This
means that one of the key goals in modern system design is to ensure that to whatever
extent possible, the processor is not slowed down by the storage devices it works with.
Slowdowns mean wasted processor cycles, where the CPU can't do anything because it
is sitting and waiting for information it needs.
The best way to keep the processor from having to wait is to make everything that it
uses as fast as it is. But that would be very expensive.
There is a good compromise to this however. Instead of trying to make the whole 64
MB out of this faster, expensive memory, you make a smaller piece, say 256 KB. Then
you find a smart algorithm (process) that allows you to use this 256 KB in such a way
that you get almost as much benefit from it as you would if the whole 64 MB was
made from the faster memory. How do you do this? The answer is by using this small
cache of 256 KB to hold the information most recently used by the processor.
Computer science shows that in general, a processor is much more likely to need again
7
information it has recently used, compared to a random piece of information in
memory. This is the principle behind caching
 Types of Cache Memory
• Memory Cache: A memory cache, sometimes called a cache store or RAM
cache, is a portion of memory made of high-speed static RAM (SRAM) instead of
the slower and cheaper dynamic RAM (DRAM) used for main memory. Memory

02/20/22
caching is effective because most programs access the same data or instructions
over and over. By keeping as much of this information as possible in SRAM, the
computer avoids accessing the slower DRAM.

• Disk Cache: Disk caching works under the same principle as memory
caching, but instead of using high-speed SRAM, a disk cache uses conventional
main memory. The most recently accessed data from the disk (as well as
adjacent sectors) is stored in a memory buffer. When a program needs to access
data from the disk, it first checks the disk cache to see if the data is there. Disk
caching can dramatically improve the performance of applications, because
accessing a byte of data in RAM can be thousands of times faster than accessing
a byte on a hard disk.
8
Levels of Cache: Cache memory is categorized in levels based on it’s closeness
and accessibility to the microprocessor. There are three levels of a cache.
Level 1(L1) Cache: This cache is inbuilt in the processor and is made of SRAM(Static RAM)
Each time the processor requests information from memory, the cache controller on the chip uses
special circuitry to first check if the memory data is already in the cache. If it is present, then the

02/20/22
system is spared from time consuming access to the main memory. In a typical CPU, primary cache
ranges in size from 8 to 64 KB, with larger amounts on the newer processors. This type of Cache
Memory is very fast because it runs at the speed of the processor since it is integrated into it.
Level 2(L2) Cache: The L2 cache is larger but slower in speed than L1 cache. It is used to see
recent accesses that is not picked by L1 cache and is usually 64 to 2 MB in size. A L2 cache is also
found on the CPU. If L1 and L2 cache are used together, then the missing information that is not
present in L1 cache can be retrieved quickly from the L2 cache. Like L1 caches, L2 caches are
composed of SRAM but they are much larger. L2 is usually a separate static RAM (SRAM) chip and it
is placed between the CPU & DRAM(Main Memory)

Level 3(L3) Cache: L3 Cache memory is an enhanced form of memory present on the
motherboard of the computer. It is an extra cache built into the motherboard between the processor and
main memory to speed up the processing operations. It reduces the time gap between request and
retrieving of the data and instructions much more quickly than a main memory. L3 cache are being used
with processors nowadays, having more than 3 MB of storage in it.
9
Diagram showing different types of cache and their
position in the computer system

02/20/22
10
 Principle behind Cache Memory
Cache is a really amazing technology. A 512 KB level 2 cache,
caching 64 MB of system memory, can supply the information

02/20/22
that the processor requests 90-95% of the time. The level 2 cache
is less than 1% of the size of the memory it is caching, but it is
able to register a hit on over 90% of requests. That's pretty
efficient, and is the reason why caching is so important.

The reason that this happens is due to a computer science

principle called locality of reference. It states basically that even
within very large programs with several megabytes of
instructions, only small portions of this code generally get used at
once. Programs tend to spend large periods of time working in
one small area of the code, often performing the same work many
times over and over with slightly different data, and then move 11 to
another area. This occurs because of "loops", which are what
programs use to do work many times in rapid succession.
Locality of Reference
Let's take a look at the following pseudo-code to see how

02/20/22
locality of reference works
Output to screen « Enter a number between 1 and 100 »
Read input from user
Put value from user in variable X
Put value 100 in variable Y
Put value 1 in variable Z
Loop Y number of time Divide Z by X
If the remainder of the division = 0
then output « Z is a multiple of X »
Add 1 to Z
Return to loop
End 12
This small program asks the user to enter a number between 1 and
100. It reads the value entered by the user. Then, the program divides
every number between 1 and 100 by the number entered by the user. It
checks if the remainder is 0. If so, the program outputs "Z is a
multiple of X", for every number between 1 and 100. Then the

02/20/22
program ends.
Now it is easy to understand that in the 11 lines of this program, the
loop part (lines 7 to 9) are executed 100 times. All of the other lines
are executed only once. Lines 7 to 9 will run significantly faster
because of caching. This program is very small and can easily fit
entirely in the smallest of L1 caches, but let's say this program is huge.
The result remains the same. When you program, a lot of action takes
place inside loops.
This 95%-to-5% ratio (approximately) is what we call the locality of
reference, and it's why a cache works so efficiently. This is also why
such a small cache can efficiently cache such a large memory system.
You can see why it's not worth it to construct a computer with 13 the
fastest memory everywhere. We can deliver 95 percent of this
 Importance of Cache
Cache is responsible for a great deal of the system performance
improvement of today's PCs. The cache is a buffer of sorts

02/20/22
between the very fast processor and the relatively slow memory
that serves it. The presence of the cache allows the processor to
do its work while waiting for memory far less often than it
otherwise would. Without cache the computer will be very slow
and all our works get delay. So cache is a very important part of
our computer system.

14
Resources:
www.pcguide.com

02/20/22
www.howstuffworks.com
www.webopedia.com

Thank You
15