Lecture 10

Introduction to Computer Memory

System Design

Computer Architecture- NUN 2024 Austin Olom Ogar

MODULE OUTLINE

Introduction to Memory Systems

Cache Memory Design

Main Memory

Design Virtual

Memory

Read-Only Memory (ROM)

Advanced Memory Design Techniques

Memory System Design in

Multiprocessors

Trade-offs in Memory System Design

Computer Architecture- NUN 2024 Austin Olom Ogar

MODULE OBJECTIVE

By the end of this course, students will be able to:

1.Understand the principles of memory hierarchy and the roles of different types of
memory.

2.Analyze and design cache memory systems to optimize performance.

3.Comprehend the organization and design of main memory and virtual memory.

4.Evaluate the trade-offs involved in different memory system designs.

5.Apply memory design concepts to solve real-world problems in computer

architecture.

Computer Architecture- NUN 2024 Austin Olom Ogar

 Introduction to Memory Systems
Memory systems in computer architecture refer to the various components and structures
that are used to store and manage data and instructions in a computer.

These systems are designed to provide storage that varies in speed, size, cost, and
accessibility, forming a hierarchy that balances these trade-offs to optimize overall system
performance..

Key Components of Memory Systems:

1.Registers: The smallest and fastest type of memory located directly within the CPU.
2.Cache Memory: A small-sized, high-speed memory located close to the CPU.
3. Main Memory (RAM): Stands for Random Access Memory and is the primary memory used by the
CPU to store data and instructions that are currently in use.
4.Secondary Storage: Non-volatile memory used for long-term storage of data, such as hard drives (HDD),
solid-state
drives (SSD), and optical drives.
5. Virtual Memory: A technique that allows the use of disk storage to extend the apparent size of main
memory beyond its physical limits.

Computer Architecture - NUN 2024 Austin Olom Ogar

 Functions and Characteristics of Memory Systems:
• Storage and Retrieval: Memory systems store data and instructions and retrieve
them as needed by the CPU and other components.

• Hierarchy and Speed: Different levels of the memory hierarchy offer varying speeds
and sizes, with faster, smaller memories (like caches and registers) closer to the CPU
and slower, larger memories (like RAM and secondary storage) further away.

• Volatility: Some memory types (like RAM) are volatile and lose their data when
powered off,
while others (like SSDs and HDDs) are non-volatile and retain data without power.

• Cost Efficiency: Higher-speed memory is more expensive per byte, while slower
memory types
are cheaper, affecting the overall cost and design of a computer system.

• Data Transfer and Access Time: The design of memory systems impacts data
transfer rates and access times, directly influencing the performance of a computer.
Computer Architecture - NUN 2024 Austin Olom Ogar
 Memory Hierarchy Overview
Register Serve as the CPU's
storage,
s: holding internal
data that is
immediately needed for computations

Cache Memory (L1, L2, L3):

Temporarily stores frequently
accessed data and instructions to
reduce the time needed to fetch them
from main memory.

Main Memory (RAM): Stores data

and instructions that are actively being
used by the CPU; offers a larger storage
area than caches.

Secondary Storage (SSD/HDD):

Provides long-term data storage, holding
the operating system, applications, and
user files.
 Trade-offs Between Different Levels of Memory:
Speed:
• Faster memory types (like registers and cache) are closer to the CPU but are smaller in
size and more expensive.
• Slower memory types (like SSDs and HDDs) are further from the CPU but offer
much larger capacities at lower costs..

• Cost:
• Memory cost increases with speed and decreases with size. Registers and caches are
the most
expensive per byte, while secondary storage is the cheapest.
• Not all memory can be made of the fastest type due to cost constraints and
physical space limitations

• Size:
• Larger memory types (like SSDs and HDDs) are slower but cheaper per gigabyte and
provide bulk storage for data that is not frequently accessed.
• Smaller, faster memory types (like registers and cache) are limited in size but provide
rapid
Computer access
Architecture - NUN 2024 Austin Olom Ogar
comparative chart for Memory Hierarchy
Memory Type Speed (Access Cost per Size (Capacity)
Time) Byte
Registers Fastest Highest Smallest
L1 Cache Very Fast High Small
L2 Cache Fast Moderate Moderate
L3 Cache Moderately Fast Lower Larger
Main
Moderate Lower Much Larger
Memory
(RAM)
Seconda
Slow Lowest Largest
ry
Storage
Computer Architecture - NUN 2024 Austin Olom Ogar
 Cache Memory Design
Cache memory is a small-sized, high-speed volatile
memory located close to the CPU to temporarily
store frequently accessed data and instructions.

• To reduce the average time to access data from the main

memory.
• To speed up data retrieval and improve overall
system performance by keeping frequently used data
closer to the CPU.
• Acts as a buffer between the CPUand main memory,
reducing latency and minimizing memory bottlenecks.
 Types of Cache (L1, L2, L3):
• L1 Cache:
• Characteristics: The smallest and fastest cache level, located directly within
the CPU core.
• Purpose: Stores critical data and instructions that are immediately needed by
the CPU, providing extremely fast access.
• L2 Cache:
• Characteristics: Larger than L1, but slightly slower. It may be located inside the
CPU or on a separate chip close to the CPU.
• Purpose: Acts as a secondary cache to store data that is less frequently
accessed than the data in L1 but still needed quickly.
• L3 Cache:
• Characteristics: The largest and slowest cache level, shared among all CPU
cores in many architectures.
• Purpose: Stores data that is accessed less frequently than L1 or L2 data but still
benefits
from being closer than the main memory.
 Basic Cache Organization:
• Cache Lines and Blocks:
• Data is stored in fixed-size units called cache lines or
blocks. Each line/block contains a set of data words
fetched from the main memory.
• Set and Associativity:
• Cache is organized into sets, where each set contains
multiple lines. The placement of a line in a set is
determined by the cache's mapping technique (covered in
Part II).
• Cache Control:
• Includes mechanisms for managing read and write
operations, cache hits, cache misses, and replacement
Computer Architecture - NUN 2024 Austin Olom Ogar
 Cache Mapping Techniques:
• Direct-Mapped Cache:
• Definition: Each block of main memory maps to exactly one cache line.
• Advantages: Simple and fast to implement; requires less hardware.
• Disadvantages: High collision rate if multiple blocks map to the same line.
• Set-Associative Cache:
• Definition: Each block of main memory maps to any line within a set of
cache lines.
• Advantages: Lower collision rate than direct-mapped; flexible.
• Disadvantages: More complex and requires more hardware than direct-
mapped.
• Fully Associative Cache:
• Definition: Any block of main memory can be placed in any cache line.
• Advantages: Lowest collision rate; very flexible.
• Disadvantages: High implementation cost and complexity.

Computer Architecture - NUN 2024 Austin Olom Ogar

 Cache Hit and Miss Concepts:
• Cache hit: When the CPU needs to read data/instruction from
memory, first cache is examined if the word/data is found in cache, it is
called success /cache hit.

• Cache Hit time: In case of a cache hit, the requested word/data is

read from the cache. This time to access the data from Cache is called
Cache time.

• Cache miss: If the requested data is not found in the cache memory,
then the cache miss is said to occur

• Cache miss time penalty: In case of cache miss, the time required to
fetch the required block from main memory (to deliver to CPU) is called
Cache miss- NUN
Computer Architecture time penalty.
2024 Austin Olom Ogar
 Cache Hit and Miss Concepts:
• Hit Ratio : Number of hits
No of cache Hit + No of misses

Miss Ratio: Number of cache misses

No of cache Hits + No of cache misses

Average memory access time

Miss
Ratio

TA = HTc + (1-h) (tc + tm)

Average
Memory Cache
Cache
Access Hit
+ MM
Time Time

Computer Architecture - NUN 2024 Austin Olom Ogar

 Cache cont..
Example 1: A CPU has a cache with a hit rate of 90%. The access time for the cache
is 2 cycles, and the miss penalty (time to access data from main memory) is 40 cycles.
Calculate the Effective Access Time (EAT) for this cache.?
Solution
 Cache cont..
Example 2: Suppose a CPU's cache has an initial hit rate of 85% with a cache access time of 2 cycles and
a miss penalty of 60 cycles. If an optimization improves the hit rate to 95%, calculate the difference in
Effective Access Time (EAT) before and after the optimization.
Solution
 Cache cont..
Example 3: A cache memory system is designed with a hit rate of 97%. If the cache access time is 1
cycle and the miss penalty is 100 cycles, calculate the Effective Access Time (EAT). Additionally,
discuss the importance of maintaining a high hit rate in real-world applications.
Solution
 Example 4
Cache memory access time is 30ns, Main memory access time
is 15ns, Average access time is 42ns. Find hit ratio?
Solution

TA = Htc + (1-h)(tc+tm)
42 = H(30) + (1-h)(30 + 150)
42 = 30H + (1-h)(180)
42 = 30H + 180 – 180H
42 - 180 = 30H – 180H
-132 = -150H
-132/150 +-H
H= 0.92

Computer Architecture - NUN 2024 Austin Olom Ogar

 Example 5
A Hierarchical memory system Cache access is 50ns
Main memory access time is 300ns, 75% of memory request
are read, hit ratio is 0.8 what is the Avg. Access time?

Solution

Computer Architecture - NUN 2024 Austin Olom Ogar

 Example 6
A computer has a memory hierarchy that consists of a cache level I, cache level II,
main memory and a disk used for Virtual memory. If a referenced word is in the
cache level I, 10ns are required to access it. If the word is in the cache level II and
not in the cache level I, 20ns are needed to load it into the cache level I and then the
reference is started again. If it is in main memory but not in the cache, 60ns are
needed to load it into the cache level II, followed by 20ns to copy it to the cache level
I and then the reference is started again. If the word is not in main memory, 100ns
are required to fetch the word from disk, followed by 60ns to copy it to the cache Il,
followed by 20ns to copy it to the cache level I and then the reference is started
again. If the cache level I miss ratio is 0.5, the cache level II miss ratio is 0.4, the
main memory miss ratio is.0.2 and the disk miss ratio is 0, what is the average time
in ns required to access a referenced word on this system?

Solution

Computer Architecture - NUN 2024 Austin Olom Ogar

 Solution

Computer Architecture - NUN 2024 Austin Olom Ogar

 Solution continue

Computer Architecture - NUN 2024 Austin Olom Ogar

 Solution continue

Computer Architecture - NUN 2024 Austin Olom Ogar

 Main Memory Design
Main memory, also known as RAM (Random
Access Memory), is the primary storage used by the
CPU to store data and instructions currently in use.

• Organized as an array of memory cells, each capable of

storing a bit of information (0 or 1).
• Memory cells are grouped into words (e.g., 8 bits for a byte,
16 bits for a word) and are addressed individually or
in blocks.
 Types of RAM
• Dynamic RAM (DRAM):
• Characteristics:
• Stores each bit of data in a separate capacitor within an integrated
circuit.
• Requires periodic refreshing to maintain data, as the capacitors
slowly discharge.
• Advantages: Higher density, lower cost per bit, suitable for main
memory.
• Disadvantages: Slower than SRAM, more power consumption due to
refresh cycles.
• Static RAM (SRAM):
• Characteristics:
• Stores data using flip-flop circuits that do not require refreshing as
long as power is supplied.
• Advantages: Faster access time, lower power consumption when idle,
 Memory Addressing Techniques
• Direct Addressing:
• Each memory address corresponds directly to a specific
memory location.

• Indirect Addressing:
• Uses pointers or registers to point to a memory location,
offering more flexibility.

• Indexed Addressing:
• Combines a base address with an index register to
determine the memory location, allowing for efficient array
processing and looping.
 Memory Access Control and Timing
• Access Control:
• Involves managing read and write operations to ensure
data integrity and system stability.
• Utilizes memory controllers to coordinate data flow between
the CPU and memory.

• Timing:
• Memory timing parameters (e.g., CAS latency, RAS-to-CAS
delay) determine how quickly data can be accessed.
• Lower latency and faster memory access improve overall
system performance.
 Concepts of Memory Interleaving
• Definition:
• A technique used to increase the memory bandwidth by dividing
memory into multiple banks and accessing them simultaneously.

• Types of Interleaving:
• Low-order interleaving: Alternates memory access between
consecutive addresses across multiple banks.
• High-order interleaving: Allocates blocks of addresses to specific
banks, reducing contention.

• Benefits:
• Increases data throughput and reduces access latency.
• Allows parallel access to different memory banks, enhancing
performance in multi-core systems.
 Error Detection and Correction Techniques
• Parity Bit:
• Adds an extra bit to each byte for error detection; can detect single-bit
errors but not correct them.

• Error-Correcting Code (ECC) Memory:

• Uses more sophisticated algorithms (e.g., Hamming Code) to
detect and correct single-bit errors and detect double-bit errors.
• ECC is crucial in servers and mission-critical systems where data
integrity is paramount.

• Checksum and Cyclic Redundancy Check (CRC):

• Methods for detecting errors in data blocks, used extensively in
data transmission and storage.
 Error Detection and Correction Techniques
• Parity Bit:
• Adds an extra bit to each byte for error detection; can detect single-bit
errors but not correct them.

• Error-Correcting Code (ECC) Memory:

• Uses more sophisticated algorithms (e.g., Hamming Code) to
detect and correct single-bit errors and detect double-bit errors.
• ECC is crucial in servers and mission-critical systems where data
integrity is paramount.

• Checksum and Cyclic Redundancy Check (CRC):

• Methods for detecting errors in data blocks, used extensively in
data transmission and storage.
 Virtual Memory
Virtual memory is a memory management technique
that provides an “idealized abstraction” of the storage
resources available to a program, which makes the
system appear to have more memory than is physically
installed.

• Allows for the executionof processes that may not be

completely in the main memory (RAM).
• Provides a mechanism to efficiently use the
available physical memory by swapping parts
of programs in and out of the main memory as needed.
• Simplifiesmemory managementby providing each processwith its
 Paging vs. Segmentation:
• Paging:
• Divides the virtual memory into fixed-size blocks called pages and the physical memory into
frames of the same size.
• When a process is executed, pages are loaded into available frames, and the page table
keeps track of where each page is stored in physical memory.
• Advantages: Simple to implement and manages memory efficiently with minimal
fragmentation.
• Disadvantages: Can cause internal fragmentation if pages are not fully utilized.

• Segmentation:
• Divides the memory into variable-sized segments based on logical divisions of a program (e.g.,
functions, data structures).
• Each segment is stored in a separate location in physical memory and has a segment table that
keeps track of its location and length.
• Advantages: Provides a more logical view of memory, supports protection and sharing of
memory segments.
• Disadvantages: Can lead to external fragmentation if there isn’t enough contiguous space for
a new segment.
 Page Replacement Algorithms:
• FIFO (First-In, First-Out):
• Description: Replaces the oldest page in memory that has been there the
longest.
• Advantages: Simple to implement.
• Disadvantages: Can result in suboptimal performance (Belady's anomaly).
• LRU (Least Recently Used):
• Description: Replaces the page that has not been used for the longest period.
• Advantages: Provides a good approximation of the optimal algorithm in terms of
minimizing page faults.
• Disadvantages: More complex to implement and requires additional hardware
support.
• Optimal (OPT):
• Description: Replaces the page that will not be used for the longest period in the
future.
• Advantages: Provides the lowest possible page fault rate.
• Disadvantages: Requires future knowledge of memory access, which is not
 Performance Considerations in Virtual Memory Systems:
• Page Fault Rate:
• The frequency of page faults significantly impacts system performance. A high
page fault rate can degrade performance due to frequent disk I/O operations.

• Thrashing:
• Occurs when a system spends more time swapping pages in and out of
memory than executing instructions, leading to a severe decline in
performance.

• Effective Memory Access Time:

• A combination of the time needed to access memory on a hit and the time
required to
retrieve data from the disk on a page fault.
 Read-Only Memory (ROM)
• Read-Only Memory (ROM) is a type of non-volatile
memory used in computers and other electronic devices
to store data that does not need to be modified
frequently.

• The term "read-only" indicates that the data stored in

ROM can be read but not easily altered or rewritten.

• ROM retains its data even when the device is

powered off, making it ideal for storing firmware, which
is the permanent software programmed into a device
to control its basic functions
 Key Characteristics of ROM:
• Non-Volatile:
• ROM retains its data without needing a continuous power supply, unlike RAM
(Random Access Memory), which is volatile and loses its data when power is turned
off.
• Permanence of Data:
• Once data is written to ROM, it cannot be easily modified or erased.
• This permanence ensures that critical programs, such as the system firmware or
bootloader, are reliably stored and remain intact.
• Read-Only Nature:
• Data in ROM is intended to be read, not written to.
• Although some types of ROM can be reprogrammed or updated (like EEPROM),
this process is typically slow and requires special procedures or
equipment.
• Used for Firmware Storage:
• ROM is commonly used to store firmware, the fundamental code that initializes
hardware and provides basic control for the device.
• Examples include the BIOS (Basic Input/Output System) in computers and the
firmware in
 Types of ROM
• Masked ROM (MROM):
• The earliest form of ROM where data is written during the manufacturing process.Non-
reprogrammable; any errors in data or software require replacing the chip.

• Programmable ROM (PROM):

• A type of ROM that can be programmed once after manufacturing using a PROM
• programmer. After programming, the data is permanent and cannot be changed.

• Erasable Programmable ROM (EPROM):

• Can be erased and reprogrammed using ultraviolet (UV) light.
• Requires special equipment (UV light source) for erasure and a PROM programmer for
rewriting data.
• Electrically Erasable Programmable ROM (EEPROM):
• Can be electrically erased and reprogrammed, allowing for in-circuit updates without removing the
chip from
• its socket.
More flexible and user-friendly than EPROM, often used for storing small amounts of data that may
• need to be updated.
• Flash Memory:
• A type of EEPROM that can be erased and rewritten in blocks, rather than one byte at a time.
Used extensively in USB drives, SSDs, and memory cards due to its faster erase and write capabilities
compared
 Applications of ROM:
 Firmware Storage:
 Stores firmware programs that control hardware devices at the most basic level.
Examples include BIOS in PCs, firmware in embedded devices, and controllers
in various consumer electronics.
 Bootloaders:
 ROM stores bootloader code, which initializes the hardware and loads
the main operating system or application software when a device is
powered on.
 Embedded Systems:
 ROM is widely used in embedded systems, which require a stable,
unchangeable software environment to control specific hardware
functions, such as washing machines, industrial robots, and automotive
control systems.
 Lookup Tables and Static Data:
 Used to store lookup tables, static data, and other data structures that do not
change throughout the operation of the device, such as character generators in
 Shared Memory vs. Distributed Memory Architectures:
• Shared Memory Architecture:
• Definition: A type of multiprocessor architecture where multiple processors share a single, unified
memory space.
• Characteristics:
• All processors have access to the same global address space.
• Simplifies programming as all processors can easily communicate through shared variables in
memory.
• Memory access time can vary depending on the location of the data (local vs. remote memory
accesses).
• Advantages:
• Easier to program and manage due to a single address space.
• Efficient for tasks that require frequent communication and data sharing among processors.
• Disadvantages:
• Scalability issues due to contention for the shared memory bus as the number of processors
increases.
• Cache coherence problems need to be addressed to maintain data consistency across caches.
 Shared Memory vs. Distributed Memory Architectures:
• Distributed Memory Architecture:
• Definition: A type of multiprocessor architecture where each processor has its own
private memory.
• Characteristics:
• Processors communicate by passing messages through an interconnection
network.
• No global address space; each processor operates in its local memory, requiring
explicit communication for data sharing.
• Advantages:
• Highly scalable as there is no contention for a shared memory bus.
• No cache coherence problems since each processor has its own private memory.
• Disadvantages:
• More complex programming model due to the need for explicit message passing
and data distribution.
• Higher communication overhead as data must be explicitly sent and received
between processors.
 Module Assignment

• Question 1: Explain the role of cache memory in the memory hierarchy and discuss
how different levels of cache (L1, L2, L3) are optimized for both speed and cost.
What are the trade-offs involved in increasing the size of each cache level

• Question 2: Describe the differences between paging and segmentation in virtual

memory management. How does each method address memory allocation and
protection, and what are the benefits and drawbacks of combining these two
techniques in a hybrid system?