B4EC/E

Computer Architecture and organization

MEMORYAND I/O ORGANIZATION

INTRODUCTION
Memory unit enables us to store data inside the computer. The computer memory always
had here’s to principle of locality.

Principleoflocalityorlocalityofreferenceisthetendencyofaprocessortoaccessthesame set of
memory locations repetitively over a short period of time.

Two different types of locality are:

 Temporal locality: The principle stating that if a data location is referenced then it
will tend to be referenced again soon.
 Spatial locality: The locality principle stating that if a data location is referenced,
data locations with nearby addresses will tend to be referenced soon.
The locality of reference is useful in implementing the memory hierarchy.
Memory hierarchy is a structure that uses multiple levels of memories ;as the distance
from the CPU increases, the size of the memories and the access time both increase
A memory hierarchy consists of multiple levels of memory with different speeds and sizes.
The faster memories are more expensive per bit than the slower memories and thus smaller.

 Main memory is implemented from Dynamic Random Access Memory (DRAM).

 The levels closer to the processor (caches) use Static Random Access Memory
(SRAM).

Eng.Mona Ahmed Ali Abdullah Page 1

B4EC/E
 DRAM is less costly per bit than SRAM, although it is substantially slower. For each
k, the faster, smaller device at level k serves as a cache for the larger, slower device at
level k+1.
 The computer programs tend to access the data at level k more often that at level k+1.
 The storage at level at k+1 can be slower

Cache memory (CPU memory) is high-speed SRAM that a computer Microprocessor can
access more quickly than it can access regular RAM. This memory is typically integrated
directly into the CPU chip or placed on a separate chip that has a separate bus
interconnect with the CPU.

Data access by processor

The data transfer between various levels of memory is done through blocks. The minimum
unit of information is called a block. If the data requested by the processor appears in some
block

in the upper level, this is called a hit. If the data is not found in the upper level, the request is
called a miss. The lower level in the hierarchy is then accessed to retrieve the block
containing the requested data.

The fraction of memory accesses found in a cache is termed as hit rate or hit ratio

Miss rate is the fraction of memory accesses not found in a level of the memory hierarchy.
Hit time is the time required to access a level of the memory hierarchy, including the time
needed to determine whether the access is a hit or a miss.

Eng.Mona Ahmed Ali Abdullah Page 2

B4EC/E
Miss penalty is the time required to fetch a block into a level of the memory hierarchy
from the lower level, including the time to access the block, transmit It from one level to
the other, and insert it in the level that experienced the miss.

Because the upper level is smaller and built using faster memory parts, the hit time will be
much smaller than the time to access the next level in the hierarchy, which is the major
component of the miss penalty.

MEMORY HIERARCHY
A memory unit is a collection of semi-conductor storage cells with circuits to access the data
stored in them. The data storage in memory is done in words. The number of bits in a word
depends on the architecture of the computer. Generally a word is always multiple of 8.
Memory is accessed through unique system assigned address. The accessing of data from
memory is based on principle of locality.

Principle of Locality
The locality of reference or the principle of locality is the term applied to situations where
the same value or related storage locations are frequently accessed. There are three basic
types of locality of reference:

Temporal locality: Here a resource that is referenced at one point in time is referenced
again soon afterwards.
Spatial locality: Here the likelihood of referencing a storage location is greater if a storage
location near it has been recently referenced.
Sequential locality: Here storage is accessed sequentially, in descending or ascending
order. The locality or reference leads to memory hierarchy.

Need for memory hierarchy

Memory hierarchy is an approach for organizing memory and storage systems. It consist of
multiple levels of memory with different speeds and sizes. The following are the reasons for
such organization:
Fast storage technologies cost more per byte and have less capacity Gap between CPU and
main memory speed is widening Well-written programs tend to exhibit good locality.
The memory hierarchy is shown in Fig 4.1. The entire memory elements of the computer fall
under the following three categories:
Processor Memory:
This is present inside the CPU for high-speed data access. This consists of small set of
registers that act as temporary storage. This is the costliest memory component.
Primary memory:
This memory is directly accessed by the CPU. All the data must be brought inside main
memory before accessing them. Semiconductor chips acts as main memory.
Secondary memory:

Eng.Mona Ahmed Ali Abdullah Page 3

B4EC/E
This is cheapest, large and relatively slow memory component. The data from the secondary
memory is accessed by the CPU only after it is loaded to main memory.
There is a trade-off among the three key characteristics of memory namely Cost
Capacity Access time
Terminologies in memory access
Block or line: The minimum unit of information that could be either present or totally
absent.
Hit: If the requested data is found in the upper levels of memory hierarchy it is called hit.

Miss: If the requested data is not found in the upper levels of memory hierarchy it is called
miss.
Hit rate or Hit ratio: It is the fraction of memory access found in the upper level .It is a
performance metric. Hit Ratio = Hit/ (Hit + Miss)
Miss rate: It is the fraction of memory access not found in the upper level (1-hit rate).
Hit Time: The time required for accessing a level of memory hierarchy, including the time
needed for finding whether the memory access is a hit or miss.
Miss penalty: The time required for fetching a block into a level of the memory hierarchy
from the lower level, including the time to access, transmit, insert it to new level and pass
the block to the requestor.
Bandwidth: The data transfer rate by the memory.
Latency or access time: Memory latency is the length of time between the memory is
receipt of a read request and its release of data corresponding with the request.

Cycle time: It is the minimum time between requests to memory.

The memory access time increases as the level increases. Since the CPU registers are located
in very close proximity to the CPU they can be accessed very quickly and they are the more
costly. As the level increases, the memory access time also increases thereby decreasing the
costs.

Levels in Memory Hierarchy

The following are the levels in memory hierarchy:
CPU Registers:
They are at the top most level of this hierarchy, they hold the most frequently used data.
They are very limited in number and are the fastest. They are of ten used by the CPU and the
ALU for performing arithmetic and logical operations, for temporary storage of data.
Static Random Access Memory (SRAM):
Static Random Access Memory (Static RAM or SRAM) is a type of RAM that holds data in
a static form, that is, as long as the memory has power. SRAM stores a bit of data on four
transistors using two cross-coupled inverters. The two stable states characterize 0 and 1.
During read and write operations another two access transistors are used to manage the
availability to a memory cell.
Main memory or Dynamic Random Access Memory (DRAM):

Eng.Mona Ahmed Ali Abdullah Page 4

B4EC/E
Dynamic random access memory (DRAM) is a type of memory that is typically used for
data or program code that a computer processor needs to function. In other words it is said to
be the main memory of the computer. Random access allows processor to access any part of
the memory directly rather than having to proceed sequentially from a starting place. The
main advantages of DRAM are its simple design, speed and low cost in comparison to
alternative types of memory. The main disadvantages of DRAM are volatility and high
power consumption relative to other options.
Local Disks (Local Secondary Storage):
A local drive is a computer disk drive that is installed directly within the host or the local
computer. )t is a computer’s native hard disk drive ゅ(DDょ, which is directly accessed by
the computer for storing and retrieving data. It is a cheaper memory with more memory
access time.
Remote Secondary Storage:
This includes Distributed file system (DFS) and online storage like cloud. The storage area
is vast with low cost but larger access time.

Eng.Mona Ahmed Ali Abdullah Page 5

B4EC/E
CLASSIFICATION O MEMORY

The instructions and data are stored in memory unit of the computer system are divided into
following main groups:
Main or Primary memory Secondary memory.

Primary Memory:
Primary memory is the main area in a computer in which data is stored for quick access by
the computer’s processor. it is divided into two parts:
i) Random Access Memory(RAM):
RAM is a type of computer primary memory. It accessed any piece of data at any time.
RAM stores data for as long as the computer is switched on or is in use. This type of
memory is volatile. The two types of RAM are:
Static RAM: This type of RAM is static in nature, as it does not have to be refreshed at
regular intervals. Static RAM is made of large number of flip-flops on IC. It is being costlier
and having packing density.
Dynamic RAM: This type of RAM holds each bit of data in an individual capacitor in an
integrated circuit. It is dynamic in the sense that the capacitor charge is repeatedly
refreshed to ensure the data remains intact.
ii) Read Only Memory(ROM):
The ROM is nonvolatile memory. It retains stored data and information if the power is
turned off. )n ROM, data are stored permanently and can╆t alter by the programmer. There
are four types of ROM:
MROM (mask ROM): MROM (mask ROM) is manufacturer-Programmed ROM in which
data is burnt in by the manufacturer of the electronic equipment in which it is used and it is
not possible for a user to modify programs or data stored inside the ROM chip.
PROM (programmable ROM): PROM is one in which the user can load and store ╉
readonly╊ programs and data. )n PROM the programs or data are stored only fast time and
the stored data cannot modify the user.
EPROM (erasable programmable ROM): EPROM is one in which is possible to erase

Eng.Mona Ahmed Ali Abdullah Page 6

B4EC/E
information stored in an EPROM chip and the chip can be reprogrammed to store new
information. When an EPROM is in use, information stored in it can only be ╉ read╊ and the
information remains in the chip until it is erased.
EEPROM (electronically erasable and programmable ROM): EEPROM is one type of
EPROM in which the stored information is erased by using high voltage electric pulse. It is
easier to alter information stored in an EEPROM chip.

Secondary Memory:
Secondary memory is where programs and data are kept on a long time basis. It is cheaper
from of memory and slower than main or primary memory. It is non-volatile and cannot
access data directly by the computer processor. It is the external memory of the computer
system.
Example: hard disk drive, floppy disk, optical disk/ CD-ROM.

MEMORY CHIP ORGANISATION

A memory consists of cells in the form of an array. The basic element of the semiconductor
memory is the cell. Each cell is capable of storing one bit of information. Each
row of the cells constitutes a memory words and all cells of a row are connected to a
common line referred to as a word line. AW×b memory has w words, each word having ╅b╆
number of bits.

The basic memory element called cell can be in two states (0 or 1). The data can be written
into the cell and can be read from it.

In the above diagram there are 16 memory locations named as w0, w1, w3…w15. Each
location
can store at most 8 bits of data (b0, b1, b3… b7). Each location (wn) is the word line. The
word
line of Fig 4.4 is 8.

Eng.Mona Ahmed Ali Abdullah Page 7

B4EC/E
Each row of the cell is a memory word. The memory words are connected to a common line
termed as word line. The word line is activated based on the address it receives from the
address bus.
An address decoder is used to activate a word line.
The cells in the memory are connected by two bit lines (column wise).These are connected
to data input and data output lines through sense/ write circuitry.
Read Operation: During read operation the sense/ write circuit reads the information by
selecting the cell through word line and bit lines. The data from this cell is transferred
through the output data line.
Write Operation: During write operation, the sense/ write circuitry gets the data and
writes into the selected cell.
The data input and output line of sense / write circuit is connected to a bidirectional data
line. It is essential to have n bus lines to read 2nwords.
CACHE MEMORY
The cache memory exploits the locality of reference to enhance the speed of the processor.
Cache memory or CPU memory, is high-speed SRAM that a processor can access more
quickly than a regular RAM. This memory is integrated directly into the CPU chip or
placed on a separate chip that has a separate bus interconnect with the CPU.

The cache memory stores instructions and data that are more frequently used or data that is
likely to be used next. The processor looks first in the cache memory for the data. If it finds
the instructions or data then it does perform a more time-consuming reading of data from
larger main memory or other data storage devices.
The processor do not need to know the exact location of the cache. It can simply issue
readandwriteinstructions.Thecachecontrolcircuitrydetermineswhethertherequesteddata
resides in the cache.
Cache and temporal reference: When data is requested by the processor, the data should
be loaded in the cache and should be retained till it is needed again.
Cache and spatial reference: Instead of fetching single data, a contiguous block of data is
loaded into the cache.
Terminologies in Cache
Split cache: It has separate data cache and a separate instruction cache. The two caches
work in parallel, one transferring data and the other transferring instructions.
A dual or unified cache: The data and the instructions are stored in the same cache. A
combined cache with a total size equal to the sum of the two split caches will usually have a
better hit rate.
Mapping Function: The correspondence between the main memory blocks and those in the
cache is specified by a mapping function

Cache Replacement: When the cache is full and a memory word that is not in the cache is
referenced, the cache control hardware must decide which block should be removed to
create space for the new block that contains the referenced word. The collection of rules for
making this decision is the replacement algorithm.

Eng.Mona Ahmed Ali Abdullah Page 8

B4EC/E
4.3.1 Cache performance:
When the processor needs to read or write a location in main memory, it first checks for a
corresponding entry in the cache. If the processor finds that the memory location is in the
cache,a cache hit has said to be occurred. If the processor doesn ot find the memory location
in the cache, a cache miss has occurred. When a cache miss occurs ,the cache replacement is
made by allocating a new entry and copies in data from main memory. The performance of
cache memory is frequently measured in terms of a quantity called Hit ratio.

Hit ratio = hit / (hit + miss) = Number of hits/ Total accesses to the cache

Miss penalty or cache penalty is the sum of time to place a bock in the cache and time to
deliver the block to CPU.

Miss Penalty= time for block replacement + time to deliver the block to CPU

Cache performance can be enhanced by using higher cache block size, higher associativity,
reducing miss rate, reducing miss penalty, and reducing the time to hit in the cache. CPU
execution Time of a given task is defined as the time spent by the system executing that task,
including the time spent executing run-time or system services.

CPU execution time=(CPU clock cycles + memory stall cycles (if any)) x Clock cycle time

The memory stall cycles are a measure of count of the memory cycles during which the
CPU is waiting for memory accesses. This is dependent on caches misses and cost per miss
(cache penalty).
Memory stall cycles = number of cache misses x miss penalty
Instruction Count x (misses/ instruction) x miss penalty
Instruction Count (IC) x (memory access/ instruction) x miss penalty
IC x Reads per instruction x Read miss rate X Read miss penalty + IC x Write per
instruction x Write miss rate X Write miss penalty

Misses / instruction = (miss rate x memory access)/ instruction

Issues in Cache memory:
Cache placement: where to place a block in the cache?
Cache identification: how to identify that the requested information is available in the
cache or not?
Cache replacement: which block will be replaced in the cache, making way for an
incoming block?

Eng.Mona Ahmed Ali Abdullah Page 9