Cache Memory Performance

Last Updated : 24 Feb, 2023

Types of Caches :

L1 Cache : Cache built in the CPU itself is known as L1 or Level 1

cache. This type of cache holds most recent data so when, the data is
required again so the microprocessor inspects this cache first so it
does not need to go through main memory or Level 2 cache. The
main significance behind above concept is “Locality of reference”,
according to which a location just accessed by the CPU has a higher
probability of being required again.
L2 Cache : This type of cache resides on a separate chip next to the
CPU also known as Level 2 Cache. This cache stores recent used
data that cannot be found in the L1 Cache. Some CPU’s has both L1
and L2 Cache built-in and designate the separate cache chip as level
3 (L3) Cache.

Cache that is built into the CPU is faster than separate cache.
Separate cache is faster than RAM. Built-in Cache runs as a speed of a
microprocessor.

Disk Cache : It contains most recent read in data from the hard disk
and this cache is much slower than RAM.
Instruction Cache Vs Data Cache : Instruction or I-cache stores
instructions only while Data or D-cache stores only data.
Distinguishing the stored data by this method recognizes the
different access behavior pattern of instructions and data. For
example : The programs need to involve few write accesses, and they
often exhibit more temporal and spatial locality than the data they
process.
Unified Cache Vs Split Cache : A cache that stores both instructions
and data is referred to as a unified cache. A split cache on other hand,
 consist of two associated but largely independent units – An I-cache
and D-cache. This type of cache can also be designed to deal with
two independent units differently.

The performance of the cache memory is measured in terms of a

quantity called Hit Ratio. When the CPU refers to the memory and
reveals the word in the cache, it’s far stated that a hit has successfully
occurred. If the word is not discovered in the cache, then the CPU refers
to the main memory for the favored word and it is referred to as a miss
to cache.

Hit Ratio (h) :

Hit Ratio (h) = Number of Hits / Total CPU references to

memory = Number of hits / ( Number of Hits + Number of
Misses )

The Hit ratio is nothing but a probability of getting hits out of some
number of memory references made by the CPU. So its range is 0 <= h
<= 1.

Miss Ratio: The miss ratio is the probability of getting miss out of
some number of memory references made by the CPU.
Miss Ratio = Number of misses / Total CPU references to
memory = Number of misses/ (Number of hits + Number of misses)
Miss Ratio = 1 – hit ratio(h)
Average Access Time ( tavg ) :

tavg = h X tc + ( 1- h ) X ( tc + tm ) = tc + ( 1- h ) X tm

Let tc, h and tm denote the cache access time, hit ratio in cache and and
main access time respectively.

Average memory access time = Hit Time + Miss Rate X Miss Penalty
 Miss Rate : It can be defined as he fraction of accesses that are not in
the cache (i.e. (1-h)).

Miss Penalty : It can be defined as the addition clock cycles to service

the miss, the extra time needed to carry the favored information into
cache from main memory in case of miss in cache.

Cache Memory Structure

Types of Cache misses :

Compulsory Miss (Cold start Misses or First reference Misses) :

This type of miss occurs when the first access to a block happens. In
this type of miss, the block must be brought into the cache.
Capacity Miss : This type of miss occurs when a program working set
is much bigger than the cache storage capacity. Blocks need to be
discarded as keeping all blocks is not possible during program
execution.
AptitudeConflict Miss
Engineering (CollisionDiscrete
Mathematics Misses or Interference
Mathematics OperatingMisses)
System :DBMS
This miss is Netw
Computer
found majorly in the case of set associative or direct mapped block
placement strategies, conflict miss occur when several blocks are
mapped to the same set or block frame.
Coherence Miss (Invalidation) : It occurs when other external
processors ( e.g. I/O ) updates memory.

CPU Performance :
CPU time divide into clock cycles that spends for executing
packages/programs, and clock cycles that spend for waiting for memory
system. Cache hits are part of regular CPU cycle.
 CPU time = ( CPU execution clock cycles + memory stall clock
cycles ) X Clock Cycle time

1. Memory Stall Clock cycles ( for write-back cache ) :

Memory Stall Clock-cycles = Read Stall-cycles + Write Stall-cycles

Read-Write Cycle = ( Read/Programs ) X Read miss rate X read miss
penalty
Write-Stall Cycle = ( Write/Programs ) X Write miss rate X Write
miss penalty + Write Buffer Stalls

2. Memory Stall Clock cycles ( for write-through cache ) :

Assume write buffer stalls are negligible. Every access (read/write)

treated similar.
Memory Stall Clock-cycles = ( Memory Access/Program ) X Miss Rate
X Miss Penalties
Memory Stall Clock-cycles = (Instructions/Program ) X (
Misses/Instructions ) X Miss Penalties

Measuring and Improving Cache Performance :

1. Technique used to minimize the average memory access time :

Reducing hit time, miss penalty or miss rate.

Reducing Miss penalty X Miss rate.

2. Techniques for reducing Hit time :

Small and Simple cache.

Trace caches and pipelined cache access
Avoid time loss in address translation.

3. Techniques for reducing Miss Penalty :

Usage of Multi-level cache.

Giving priority to read misses over write.
Victim Caches

4. Techniques for reducing Miss Rate :

 Increased Block size
Higher Associativity.
Compiler optimization
Large Cache.

5. Techniques for reducing ( Miss Rate X Miss Penalty ) :

Non- blocking cache

Hardware pre-fetching
Compiler controlled pre-fetching

Are you a student in Computer Science or an employed professional

looking to take up the GATE 2025 Exam? Of course, you can get a good
score in it but to get the best score our GATE CS/IT 2025 - Self-Paced
Course is available on GeeksforGeeks to help you with its preparation.
Get comprehensive coverage of all topics of GATE, detailed
explanations, and practice questions for study. Study at your pace.
Flexible and easy-to-follow modules. Do well in GATE to enhance the
prospects of your career. Enroll now and let your journey to success
begin!

I infou… Follow 14

Next Article
Factors affecting Cache Memory
Performance

Similar Reads
Locality of Reference and Cache Operation in Cache Memory
This article deals with the concept of locality of reference and cache
operations in cache memory. Locality of reference refers to the process o…
7 min read
Factors affecting Cache Memory Performance
Computers are made of three primary blocs. A CPU, a memory, and an I/O
system. The performance of a computer system is very much dependent…
5 min read

Difference between Virtual memory and Cache memory

Virtual Memory and Cache Memory are important substructures of
contemporary computing systems that perform an important function in…
5 min read

Cache Memory Design

Prerequisite - Cache Memory A detailed discussion of the cache style is
given in this article. The key elements are concisely summarized here. w…
5 min read

Concept of Cache Memory Design

Cache Memory plays a significant role in reducing the processing time of a
program by provide swift access to data/instructions. Cache memory is…
4 min read

Cache Hits in Memory Organization

The user has a memory machine. It has one layer for data storage and
another layer for the cache. The user has stored an array with length N i…
8 min read

Terminologies Cache Memory Organization

Cache Memory is a small, fast memory that holds a fraction of the overall
contents of the memory. Its mathematical model is defined by its size,…
4 min read

Difference between Cache Coherence and Memory Consistency

1. Cache coherence :Cache coherence in computer architecture refers to
the consistency of shared resource data that is stored in multiple local…
2 min read
 Cache Memory in Computer Organization
Cache memory is a small, high-speed storage area in a computer. The
cache is a smaller and faster memory that stores copies of the data from…
9 min read

Difference Between Efficiency and Speedup in Cache Memory

To understand the operation of cache memory, one has to familiarize the
concepts of efficiency as well as speedup in computer architecture. Thes…
6 min read

Article Tags :

Blogathon Computer Organization & Architecture GATE CS Blogathon-2021

 Corporate & Communications Address:-
A-143, 9th Floor, Sovereign Corporate
Tower, Sector- 136, Noida, Uttar Pradesh
(201305) | Registered Address:- K 061,
Tower K, Gulshan Vivante Apartment,
Sector 137, Noida, Gautam Buddh
Nagar, Uttar Pradesh, 201305

Company Explore
About Us Job-A-Thon Hiring Challenge
Legal Hack-A-Thon
Careers GfG Weekly Contest
In Media Offline Classes (Delhi/NCR)
Contact Us DSA in JAVA/C++
Advertise with us Master System Design
GFG Corporate Solution Master CP
Placement Training Program GeeksforGeeks Videos
Geeks Community

Languages DSA
Python Data Structures
Java Algorithms
C++ DSA for Beginners
PHP Basic DSA Problems
GoLang DSA Roadmap
SQL DSA Interview Questions
R Language Competitive Programming
Android Tutorial

Data Science & ML Web Technologies

Data Science With Python HTML
Data Science For Beginner CSS
Machine Learning JavaScript
ML Maths TypeScript
Data Visualisation ReactJS
Pandas NextJS
NumPy NodeJs
NLP Bootstrap
Deep Learning Tailwind CSS

Python Tutorial Computer Science

Python Programming Examples GATE CS Notes
Django Tutorial Operating Systems
Python Projects Computer Network
Python Tkinter Database Management System
Web Scraping Software Engineering
OpenCV Tutorial Digital Logic Design
Python Interview Question Engineering Maths

DevOps System Design

Git High Level Design
AWS Low Level Design
Docker UML Diagrams
Kubernetes Interview Guide
Azure Design Patterns
GCP OOAD
DevOps Roadmap System Design Bootcamp
Interview Questions

School Subjects Commerce

Mathematics Accountancy
Physics Business Studies
Chemistry Economics
Biology Management
Social Science HR Management
English Grammar Finance
Income Tax

Databases Preparation Corner

SQL Company-Wise Recruitment Process
MYSQL Resume Templates
PostgreSQL Aptitude Preparation
PL/SQL Puzzles
MongoDB Company-Wise Preparation
Companies
Colleges

Competitive Exams More Tutorials

JEE Advanced Software Development
UGC NET Software Testing
UPSC Product Management
SSC CGL Project Management
SBI PO Linux
SBI Clerk Excel
IBPS PO All Cheat Sheets
IBPS Clerk Recent Articles

Free Online Tools Write & Earn

Typing Test Write an Article
Image Editor Improve an Article
 Code Formatters Pick Topics to Write
Code Converters Share your Experiences
Currency Converter Internships
Random Number Generator
Random Password Generator

@GeeksforGeeks, Sanchhaya Education Private Limited, All rights reserved