UNIT-IV

Memory Hierarchy Design and its Characteristics

In the Computer System Design, Memory Hierarchy is an enhancement to organize
the memory such that it can minimize the access time. The Memory Hierarchy was
developed based on a program behavior known as locality of references.The figure
below clearly demonstrates the different levels of memory hierarchy :

This Memory Hierarchy Design is divided into 2 main types:

1. External Memory or Secondary Memory –Comprising of Magnetic Disk,
Optical Disk, Magnetic Tape i.e. peripheral storage devices which are accessible
by the processor via I/O Module.
2. Internal Memory or Primary Memory –Comprising of Main Memory, Cache
Memory & CPU registers. This is directly accessible by the processor.
We can infer the following characteristics of Memory Hierarchy Design from above
figure:
1. Capacity:
It is the global volume of information the memory can store. As we move from
top to bottom in the Hierarchy, the capacity increases.
2. Access Time:
It is the time interval between the read/write request and the availability of the
data. As we move from top to bottom in the Hierarchy, the access time
increases.
3. Performance:
Earlier when the computer system was designed without Memory Hierarchy
design, the speed gap increases between the CPU registers and Main Memory
due to large difference in access time. This results in lower performance of the
system and thus, enhancement was required. This enhancement was made in
the form of Memory Hierarchy Design because of which the performance of the
system increases. One of the most significant ways to increase system
performance is minimizing how far down the memory hierarchy one has to go
to manipulate data.
4. Cost per bit:
As we move from bottom to top in the Hierarchy, the cost per bit increases i.e.
Internal Memory is costlier than External Memory.
RAM and ROM architecture.

1) Read-only memory, or ROM, is a form of data storage in computers and other

electronic devices that cannot be easily altered or reprogrammed. RAM is referred
to as volatile memory and is lost when the power is turned off whereas ROM in
non-volatile and the contents are retained even after the power is switched off.
Types of ROM: Semiconductor-Based
Classic mask-programmed ROM chips are integrated circuits that physically
encode the data to be stored, and thus it is impossible to change their contents
after fabrication. Other types of non-volatile solid-state memory permit some
degree of modification:
• Programmable read-only memory (PROM), or one-time programmable ROM
(OTP), can be written to or programmed via a special device called a PROM
programmer. Typically, this device uses high voltages to permanently destroy or
create internal links (fuses or antifuses) within the chip. Consequently, a PROM
can only be programmed once.
• Erasable programmable read-only memory (EPROM) can be erased by exposure
to strong ultraviolet light (typically for 10 minutes or longer), then rewritten with a
process that again needs higher than usual voltage applied. Repeated exposure to
UV light will eventually wear out an EPROM, but the endurance of most EPROM
chips exceeds 1000 cycles of erasing and reprogramming. EPROM chip packages
can often be identified by the prominent quartz "window" which allows UV light to
enter. After programming, the window is typically covered with a label to prevent
accidental erasure. Some EPROM chips are factory-erased before they are
packaged, and include no window; these are effectively PROM.
• Electrically erasable programmable read-only memory (EEPROM) is based on
a similar semiconductor structure to EPROM, but allows its entire contents (or
selected banks) to be electrically erased, then rewritten electrically, so that they
need not be removed from the computer (whether general-purpose or an embedded
computer in a camera, MP3 player, etc.). Writing or flashing an EEPROM is much
slower (milliseconds per bit) than reading from a ROM or writing to a RAM
(nanoseconds in both cases).
• Electrically alterable read-only memory (EAROM) is a type of EEPROM that
can be modified one bit at a time. Writing is a very slow process and again needs
higher voltage (usually around 12 V) than is used for read access. EAROMs are
intended for applications that require infrequent and only partial rewriting.
EAROM may be used as non-volatile storage for critical system setup information;
in many applications, EAROM has been supplanted by CMOS RAM supplied by
mains power and backed-up with a lithium battery.
• Flash memory (or simply flash) is a modern type of EEPROM invented in 1984.
Flash memory can be erased and rewritten faster than ordinary EEPROM, and
newer designs feature very high endurance (exceeding 1,000,000 cycles). Modern
NAND flash makes efficient use of silicon chip area, resulting in individual ICs with
a capacity as high as 32 GB as of 2007; this feature, along with its endurance and
physical durability, has allowed NAND flash to replace magnetic in some
applications (such as USB flash drives). Flash memory is sometimes called flash
ROM or flash EEPROM when used as a replacement for older ROM types, but not
in applications that take advantage of its ability to be modified quickly and
frequently.
2) Random-access memory, or RAM, is a form of data storage that can be
accessed randomly at any time, in any order and from any physical location in
contrast to other storage devices, such as hard drives, where the physical location
of the data determines the time taken to retrieve it. RAM is measured in megabytes
and the speed is measured in nanoseconds and RAM chips can read data faster
than ROM.
Types of RAM: The two widely used forms of modern RAM are static RAM (SRAM)
and dynamic RAM (DRAM). In SRAM, a bit of data is stored using the state of a six
transistor memory cell. This form of RAM is more expensive to produce, but is
generally faster and requires less dynamic power than DRAM. In modern
computers, SRAM is often used as cache memory for the CPU. DRAM stores a bit of
data using a transistor and capacitor pair, which together comprise a DRAM cell.
The capacitor holds a high or low charge (1 or 0, respectively), and the transistor
acts as a switch that lets the control circuitry on the chip read the capacitor's state
of charge or change it. As this form of memory is less expensive to produce than
static RAM, it is the predominant form of computer memory used in modern
computers. The figure below shows DRAM & SRAM resp.

Both static and dynamic RAM are considered volatile, as their state is lost or
reset when power is removed from the system. By contrast, read-only memory
(ROM) stores data by permanently enabling or disabling selected transistors, such
that the memory cannot be altered. Writeable variants of ROM (such as EEPROM
and flash memory) share properties of both ROM and RAM, enabling data to persist
without power and to be updated without requiring special equipment. These
persistent forms of semiconductor ROM include USB flash drives, memory cards
for cameras and portable devices, and solid-state drives. ECC memory (which can
be either SRAM or DRAM) includes special circuitry to detect and/or correct
random faults (memory errors) in the stored data, using parity bits or error
correction codes.
In general, the term RAM refers solely to solid-state memory devices (either
DRAM or SRAM), and more specifically the main memory in most computers. In
optical storage, the term DVD-RAM is somewhat of a misnomer since, unlike CD-
RW or DVD-RW it does not need to be erased before reuse. Nevertheless, a DVD-
RAM behaves much like a hard disc drive if somewhat slower.
Difference between Static Ram And Dynamic Ram
 2D and 2.5D Memory organization
The internal structure of Memory either RAM or ROM is made up of memory cells
that contain a memory bit. A group of 8 bits makes a byte. The memory is in the
form of a multidimensional array of rows and columns. In which, each cell stores a
bit and a complete row contains a word. A memory simply can be divided into this
below form.
2n = N
where n is the no. of address lines and N is the total memory in bytes.
There will be 2n words.
2D Memory organization –
In 2D organization, memory is divided in the form of rows and columns(Matrix).
Each row contains a word, now in this memory organization, there is a decoder. A
decoder is a combinational circuit that contains n input lines and 2 n output lines.
One of the output lines selects the row by the address contained in the MAR and
the word which is represented by that row gets selected and is either read or
written through the data lines.

2.5D Memory organization –

In 2.5D Organization the scenario is the same but we have two different decoders
one is a column decoder and another is a row decoder. Column decoder is used to
select the column and a row decoder is used to select the row. The address from
the MAR goes as the decoders’ input. Decoders will select the respective cell
through the bit outline, then the data from that location will be read or through the
bit, inline data will be written at that memory location.
Read and Write Operations –
1. If the select line is in Reading mode then the Word/bit which is represented by
the MAR will be available to the data lines and will get read.
2. If the select line is in write mode then the data from the memory data register
(MDR) will be sent to the respective cell which is addressed by the memory
address register (MAR).
3. With the help of the select line, we can select the desired data and we can
perform read and write operations on it.

Comparison between 2D & 2.5D Organizations –

1. In 2D organization hardware is fixed but in 2.5D hardware changes.
2. 2D Organization requires more gates while 2.5D requires less.
3. 2D is more complex in comparison to the 2.5D organization.
4. Error correction is not possible in the 2D organization but in 2.5D it could be
done easily.
5. 2D is more difficult to fabricate in comparison to the 2.5D organization.
Advantages:

2D Memory Organization:

Simplicity: 2D memory organization is a simple and straightforward approach, with

memory chips arranged in a two-dimensional grid.
Cost-Effective: 2D memory organization is cost-effective, making it a popular
choice for many low-power and low-cost devices.
Low Power: 2D memory organization has low power consumption, making it ideal
for use in mobile devices and other portable electronics.
2.5D Memory Organization:

Higher Bandwidth: 2.5D memory organization has higher bandwidth since it uses
a high-speed interconnect between memory chips, enabling faster data transfer
rates.
Higher Capacity: 2.5D memory organization has higher capacity since it can stack
multiple memory chips on top of each other, enabling more memory to be packed
into a smaller space.
Scalability: 2.5D memory organization is highly scalable, making it easier to
increase memory capacity or performance without adding more memory chips.
Disadvantages:

2D Memory Organization:

Limited Bandwidth: 2D memory organization has limited bandwidth due to the

sequential access pattern of memory chips, which can lead to slower data transfer
rates.
Limited Capacity: 2D memory organization has limited capacity since it requires
memory chips to be arranged in a two-dimensional grid, limiting the number of
memory chips that can be used.
Limited Scalability: 2D memory organization is not scalable, making it difficult to
increase memory capacity or performance without adding more memory chips.

2.5D Memory Organization:

Complexity: 2.5D memory organization is more complex than 2D memory

organization since it requires additional interconnects and packaging technologies.
Higher Cost: 2.5D memory organization is generally more expensive than 2D
memory organization due to the additional interconnects and packaging
technologies required.
Higher Power Consumption: 2.5D memory organization has higher power
consumption due to the additional interconnects and packaging technologies,
making it less ideal for use in mobile devices and other low-power electronics.
Cache Memory in Computer Organization
Cache Memory is a special very high-speed memory. It is used to speed up
and synchronizing with high-speed CPU. Cache memory is costlier than main
memory or disk memory but economical than CPU registers. Cache memory is an
extremely fast memory type that acts as a buffer between RAM and the CPU. It
holds frequently requested data and instructions so that they are immediately
available to the CPU when needed.
Cache memory is used to reduce the average time to access data from the
Main memory. The cache is a smaller and faster memory which stores copies of the
data from frequently used main memory locations. There are various different
independent caches in a CPU, which store instructions and data.

Levels of memory:
 Level 1 or Register – It is a type of memory in which data is stored and
accepted that are immediately stored in CPU. Most commonly used register is
accumulator, Program counter, address register etc.
 Level 2 or Cache memory – It is the fastest memory which has faster access
time where data is temporarily stored for faster access.
 Level 3 or Main Memory – It is memory on which computer works currently.
It is small in size and once power is off data no longer stays in this memory.
 Level 4 or Secondary Memory – It is external memory which is not as fast as
main memory but data stays permanently in this memory.

Cache Performance:
When the processor needs to read or write a location in main memory, it first
checks for a corresponding entry in the cache.
 If the processor finds that the memory location is in the cache, a cache hit has
occurred and data is read from cache
 If the processor does not find the memory location in the cache, a cache
miss has occurred. For a cache miss, the cache allocates a new entry and
copies in data from main memory, then the request is fulfilled from the
contents of the cache.

The performance of cache memory is frequently measured in terms of a quantity

called Hit ratio.
Hit ratio = hit / (hit + miss) = no. of hits/total accesses
We can improve Cache performance using higher cache block size, higher
associativity, reduce miss rate, reduce miss penalty, and reduce Reduce the time to
hit in the cache.
Cache Mapping:
There are three different types of mapping used for the purpose of cache memory
which are as follows: Direct mapping, Associative mapping, and Set-Associative
mapping. These are explained below.
1. Direct Mapping – The simplest technique, known as direct mapping, maps
each block of main memory into only one possible cache line. or
In Direct mapping, assigne each memory block to a specific line in the cache. If
a line is previously taken up by a memory block when a new block needs to be
loaded, the old block is trashed. An address space is split into two parts index
field and a tag field. The cache is used to store the tag field whereas the rest is
stored in the main memory. Direct mapping`s performance is directly
proportional to the Hit ratio.
i = j modulo m
where
i=cache line number
j= main memory block number
m=number of lines in the cache
For purposes of cache access, each main memory address can be viewed as
consisting of three fields. The least significant w bits identify a unique word or
byte within a block of main memory. In most contemporary machines, the
address is at the byte level. The remaining s bits specify one of the 2 s blocks of
main memory. The cache logic interprets these s bits as a tag of s-r bits (most
significant portion) and a line field of r bits. This latter field identifies one of the
m=2r lines of the cache.

2. Associative Mapping – In this type of mapping, the associative memory is

used to store content and addresses of the memory word. Any block can go into
any line of the cache. This means that the word id bits are used to identify
which word in the block is needed, but the tag becomes all of the remaining
bits. This enables the placement of any word at any place in the cache memory.
It is considered to be the fastest and the most flexible mapping form.
3. Set-associative Mapping – This form of mapping is an enhanced form of direct
mapping where the drawbacks of direct mapping are removed. Set associative
addresses the problem of possible thrashing in the direct mapping method. It
does this by saying that instead of having exactly one line that a block can map
to in the cache, we will group a few lines together creating a set. Then a
block in memory can map to any one of the lines of a specific set..Set-
associative mapping allows that each word that is present in the cache can have
two or more words in the main memory for the same index address. Set
associative cache mapping combines the best of direct and associative cache
mapping techniques.
In this case, the cache consists of a number of sets, each of which consists of a
number of lines. The relationships are
m=v*k
i= j mod v
where
i=cache set number
j=main memory block number
v=number of sets
m=number of lines in the cache number of sets
k=number of lines in each set
 Application of Cache Memory –
1. Usually, the cache memory can store a reasonable number of blocks at any
given time, but this number is small compared to the total number of
blocks in the main memory.
2. The correspondence between the main memory blocks and those in the
cache is specified by a mapping function.

Types of Cache –
 Primary Cache – A primary cache is always located on the processor chip.
This cache is small and its access time is comparable to that of processor
registers.
 Secondary Cache – Secondary cache is placed between the primary cache
and the rest of the memory. It is referred to as the level 2 (L2) cache. Often,
the Level 2 cache is also housed on the processor chip.

Locality of reference – Since size of cache memory is less as compared to

main memory. So to check which part of main memory should be given priority
and loaded in cache is decided based on locality of reference.

Types of Locality of reference

Spatial Locality of reference This says that there is a chance that
element will be present in the close proximity to the reference point and
next time if again searched then more close proximity to the point of
reference.
Temporal Locality of reference In this Least recently used algorithm will
be used. Whenever there is page fault occurs within a word will not only
load word in main memory but complete page fault will be loaded because
spatial locality of reference rule says that if you are referring any word next
word will be referred in its register that’s why we load complete page table
so the complete block will be loaded.

Auxiliary memory

Auxiliary memory devices used in computer systems are magnetic disks and tapes, magnetic
drums, magnetic bubble memory, and optical disks. • A magnetic disk is a circular plate
constructed of metal or plastic coated with magnetized material. Often both sides of the disk
are used and several disks may be stacked on one spindle with read/write heads available on
each surface. All disks rotate together at high speed and are not stopped or started for access
purposes. Bits are stored in the magnetized surface in spots along concentric circles called
tracks. The tracks are commonly divided into sections called sectors. In most systems, the
minimum quantity of information which can be transferred is a sector. The subdivision of one
disk surface into tracks and sectors is shown in figure.

Magnetic tape transport consists of the electrical, mechanical, and electronic components to
provide the parts and control mechanism for a magnetic-tape unit. The tape itself is a strip of
plastic coated with a magnetic recording medium. Magnetic tape units can be stopped, started
to move forward or in reverse, or can be rewound.

Optical discs
Another form of largely read-only memory is the optical compact disc, developed
from videodisc technology during the early 1980s. Data are recorded as tiny pits in a single
spiral track on plastic discs that range from 3 to 12 inches (7.6 to 30 cm) in diameter, though a
diameter of 4.8 inches (12 cm) is most common. The pits are produced by a laser or by a
stamping machine and are read by a low-power laser and a photocell that generates an
electrical signal from the varying light reflected from the pattern of pits. Optical discs are
removable and have a far greater memory capacity than diskettes; the largest ones can store
many gigabytes of information.

A common optical disc is the CD-ROM (compact disc read-only memory). It holds about 700
megabytes of data, recorded with an error-correcting code that can correct bursts of errors
caused by dust or imperfections. CD-ROMs are used to distribute software, encyclopaedias,
and multimedia text with audio and images. CD-R (CD-recordable), or WORM (write-once
read-many), is a variation of CD-ROM on which a user may record information but not
subsequently change it. CD-RW (CD-rewritable) disks can be re-recorded. DVDs (digital video,
or versatile, discs), developed for recording movies, store data more densely than does CD-
ROM, with more powerful error correction. Though the same size as CDs, DVDs typically hold 5
to 17 gigabytes—several hours of video or several million text pages.
 Magneto-optical discs

Magneto-optical discs are a hybrid storage medium. In reading, spots with different directions
of magnetization give different polarization in the reflected light of a low-power laser beam. In
writing, every spot on the disk is first heated by a strong laser beam and then cooled under
a magnetic field, magnetizing every spot in one direction, to store all 0s. The writing process
then reverses the direction of the magnetic field to store 1s where desired.

Virtual Memory
What is Virtual Memory?
Virtual memory in computer organization architecture is a technique and not actually a
memory in physical form present in computer system. This is the reason it is known
as virtual memory.
Virtual memory in COA is simply a technique used to provide an illusion of presence of large
main memory to the programmer, when in actual it’s not present physically.

The size of virtual memory is equivalent to the size of secondary memory. Each virtual address
or logical address referenced by the CPU is mapped to a physical address in main memory.

A hardware device called Memory Management Unit (MMU) performs this mapping
during run time. To perform this activity MMU actually takes help of a memory map table,
which is maintained by the operating system.

The set of all logical address generated by CPU or program is known as logical address space.
It is also called as virtual address space.
The set of all physical addresses corresponding to the above logical addresses is known
as physical address space.

The complete procedure is something like this. When a program is required to be executed
then the CPU or program would generate a addresses called logical addresses. And executing
program occupies corresponding addresses in physical memory called as physical address.

Advantages of Virtual memory

 Virtual memory technique helps in efficient utilization of main memory. As larger size
programs are divided into blocks and partially each block is loaded into main memory
as per need. This makes simultaneous execution of multiple program possible.
 Virtual memory helps in efficient CPU utilization
 Virtual memory helps to improve overall throughput.

Some of the key disadvantages and problems associated with virtual memory usage are:

 Performance overhead: Virtual memory introduces a level of indirection between a

program's virtual address space and its physical memory, leading to increased
performance overhead. This translation from virtual to physical addresses requires
additional time and can slow down the execution of programs.
 Page faults and increased latency: When a program requests data that resides in
secondary storage (e.g., hard disk) rather than primary memory (RAM), a page fault
occurs, causing the data to be fetched from secondary storage and loaded into primary
memory. This process leads to increased latency as secondary storage devices are
slower compared to primary memory.
 Thrashing: Thrashing occurs when a system has insufficient primary memory to
accommodate active programs, causing frequent page faults and continuous swapping
of data between primary memory and secondary storage. This can lead to overall
system performance degradation and poor response times for programs and users.
  Increased hardware requirements: Virtual memory requires the support of specific
hardware components, such as Memory Management Units (MMUs), adding complexity
to system design and increasing hardware costs. Additionally, systems with large
virtual memory requirements may need larger secondary storage devices, thus
increasing hardware expenses.
 Complex management: The management of virtual memory can be complex,
particularly in large systems or multi-user environments, requiring sophisticated
algorithms to efficiently allocate and deallocate memory resources. This complexity
increases resource usage and can hinder the performance of memory-intensive
programs.

In conclusion, virtual memory offers numerous advantages, such as simplifying memory

management, increasing program capacity, and enabling efficient use of limited primary
memory resources. However, it also has some drawbacks, including performance overhead,
increased latency, and potential memory management complexities. Therefore, the benefits of
virtual memory must be carefully weighed against these disadvantages in order to determine
the optimal configuration and resource allocation strategy for a given computer system.

All of us are aware of the fact that our program needs to be available in main memory for the
processor to execute it. Assume that your computer has something like 32 or 64 MB RAM
available for the CPU to use. Unfortunately, that amount of RAM is not enough to run all of the
programs that most users expect to run at once. For example, if you load the operating system,
an e-mail program, a Web browser and word processor into RAM simultaneously, 32 MB is not
enough to hold all of them. If there were no such thing as virtual memory, then you will not be
able to run your programs, unless some program is closed. With virtual memory, we do not
view the program as one single piece. We divide it into pieces, and only the one part that is
currently being referenced by the processor need to be available in main memory. The entire
program is available in the hard disk. As the copying between the hard disk and main memory
happens automatically, you don’t even know it is happening, and it makes your computer feel
like is has unlimited RAM space even though it only has 32 MB installed. Because hard disk
space is so much cheaper than RAM chips, it also has a n economic benefit.

Techniques that automatically move program and data blocks into the physical main
memory when they are required for execution are called virtual-memory techniques.
Programs, and hence the processor, reference an instruction and data space that is
independent of the available physical main memory space. The binary addresses that the
processor issues for either instructions or data are called virtual or logical addresses. These
addresses are translated into physical addresses by a combination of hardware and software
components. If a virtual address refers to a part of the program or data space that is currently
in the physical memory, then the contents of the appropriate location in the main memory are
accessed immediately. On the other hand, if the referenced address is not in the main memory,
its contents must be brought into a suitable location in the memory before they can be used.
Therefore, an address used by a programmer will be called a virtual address, and the set of
such addresses the address space. An address in main memory is called a location or physical
address. The set of such locations is called the memory space, which consists of the actual main
memory locations directly addressable for processing. As an example, consider a computer
with a main-memory capacity of 32M words. Twenty-five bits are needed to specify a physical
address in memory since 32 M = 225. Suppose that the computer has available auxiliary
memory for storing 235, that is, 32G words. Thus, the auxiliary memory has a capacity for
storing information equivalent to the capacity of 1024 main memories. Denoting the address
space by N and the memory space by M, we then have for this example N = 32 Giga words and
M = 32 Mega words.

The portion of the program that is shifted between main memory and secondary storage can
be of fixed size (pages) or of variable size (segments). Virtual memory also permits a
program’s memory to be physically noncontiguous , so that every portion can be allocated
wherever space is available. This facilitates process relocation. Virtual memory, apart from
overcoming the main memory size limitation, allows sharing of main memory among
processes. Thus, the virtual memory model provides decoupling of addresses used by the
program (virtual) and the memory addresses (physical). Therefore, the definition of virtual
memory can be stated as, “ The conceptual separation of user logical memory from physical
memory in order to have large virtual memory on a small physical memory”. It gives an
illusion of infinite storage, though the memory size is limited to the size of the virtual address.

Even though the programs generate virtual addresses, these addresses cannot be used to
access the physical memory. Therefore, the virtual to physical address translation has to be
done. This is done by the memory management unit (MMU). The mapping is a dynamic
operation, which means that every address is translated immediately as a word is referenced
by the CPU. This concept is depicted diagrammatically in Figures 1 and 2. Figure 1 gives a
general overview of the mapping between the logical addresses and physical addresses. Figure
2 shows how four different pages A, B, C and D are mapped. Note that, even though they are
contiguous pages in the virtual space, they are not so in the physical space. Pages A, B and C
are available in physical memory at non-contiguous locations, whereas, page D is not available
in physical storage.