CSC315 Computer Memory System
CSC315 Computer Memory System
Outline:
The various characteristics of memory are as presented in Table 1 and describe as follows in
the next section.
3.1 Memory Characteristics
1. Location: Refers to whether memory is internal or external to the computer Internal
memory is often equated with main memory, but there are other forms of internal
memory; Processor requires its own local memory in the form of registers, control unit
portion of the processor may also require its own internal memory. Cache is another form
of internal memory. External memory consists of peripheral storage devices, such as disk
and tape, which are accessible to the processor via I/O controllers.
3
2. Capacity: is the amount of data that can be stored in the memory storage unit. This
storage capacity is expressed in terms of Bytes. For internal memory, this is typically
expressed in terms of bytes (1 byte = 8 bits) or words. Common word lengths are 8, 16,
and 32 bits. External memory capacity is typically expressed in terms of bytes or higher
units of store.
3. Unit of transfer: For internal memory, the unit of transfer is equal to the number of
electrical lines into and out of the memory module. This may be equal to the word length,
but is often larger, such as 64, 128, or256 bits.
The internal memory transfer rate is governed by the data bus width. For external
memory, usually a block is much larger than a word Addressable. To clarify the units of
transfer concept, we hereby define the following related terms:
i. Word: The “natural” unit of organization of memory. The size of a word is typically equal
to the number of bits used to represent an integer and to the instruction length. Unfortunately,
there are many exceptions. For example, the CRAY C90 (an older model CRAY
supercomputer) has a 64-bit word length but uses a 46-bit integer representation. The Intel
x86 architecture has a wide variety of instruction lengths, expressed as multiples of bytes, and
a word size of 32 bits.
ii. Addressable units: In some systems, the addressable unit is the word. However, many
systems allow addressing at the byte level. In any case, the relationship between the length in
bits A of an address and the number N of addressable units is 2A = N.
iii. Unit of transfer: For main memory, this is the number of bits read out of or written into
memory at a time. The unit of transfer need not equal a word or an addressable unit. For
external memory, data are often transferred in much larger units than a word, and these are
referred to as blocks.
i. Sequential access: Memory is organized into units of data, called records. Access
must be made in a specific linear sequence. Stored addressing information is used to
separate records and assist in the retrieval process. A shared read–write mechanism is
used, and this must be moved from its current location to the desired location, passing
and rejecting each intermediate record.
Thus, the time to access an arbitrary record is highly variable. Tape units are an
example sequential access.
ii. Direct access: As with sequential access, direct access involves a shared read–write
mechanism. However, individual blocks or records have a unique address based on
physical location. Access is accomplished by direct access to reach general vicinity
plus sequential searching, counting, or waiting to reach the final location. Again,
access time is variable.
iii. Random access: Each addressable location in memory has a unique, physically
wired-in addressing mechanism. The time to access, a given location is independent
of the sequence of prior accesses and is constant. Thus, any location can be selected
4
at random and directly addressed and accessed. Main memory and some cache
systems are random access.
iv. Associative: This is a random access type of memory that enables one to make a
comparison of desired bit locations within a word for a specified match, and to do this
for all words simultaneously. Thus, a word is retrieved based on a portion of its
contents rather than its address. As with ordinary random-access memory, each
location has its own addressing mechanism, and retrieval time is constant
independent of location or prior access patterns. Cache memories may employ
associative access.
5. Performance. One of the chief memory characteristic from the user’s point of view is its
performance, the other being the capacity. Three performance parameters are used:
i. Access time (latency): For random-access memory, this is the time it takes to
perform a read or write operation, that is, the time from the instant that an address is
presented to the memory to the instant that data have been stored or made available
for use. For non-random-access memory, access time is the time it takes to position
the read–write mechanism at the desired location.
ii. Memory cycle time: This concept is primarily applied to random-access memory and
consists of the access time plus any additional time required before a second access
can commence. This additional time may be required for transients to die out on
signal lines or to regenerate data if they are read destructively.
Note that memory cycle time is concerned with the system bus, not the processor.
iii. Transfer rate: This is the rate at which data can be transferred into or out of a
memory unit. For random-Access memory, it is equal to 1/(cycle time). For non-
random-access memory, the following relationship holds:
n
T n
T A
R
Where
Tn = Average time to read or write n bits
TA = Average access time
n = Number of bits
R = Transfer rate, in bits per second (bps)
6. Physical types: The most common today are semiconductor memory, magnetic surface
memory, used for disk and tape, and optical and magneto-optical.
There is a trade-off among the three key characteristics of memory: capacity, access time,
and cost. A variety of technologies is used to implement memory systems, and across this
spectrum of technologies, the following relationships hold:
Faster access time, greater cost per bit;
Greater capacity, smaller cost per bit;
Greater capacity, slower access time.
The dilemma facing the designer is clear. The designer would like to use memory
technologies that provide for large- capacity memory, both because the capacity is needed
and because the cost per bit is low. However, to meet performance requirements, the
designer needs to use expensive, relatively lower-capacity memories with short access times.
The way out of this dilemma is not to rely on a single memory component or technology, but
to employ a memory hierarchy.
Figure 3.1 illustrates a typical hierarchy. As one goes down the hierarchy, the following
occur:
a. Decreasing cost per bit;
b. Increasing capacity;
c. Increasing access time;
d. Decreasing frequency of access of the memory by the processor.
Thus, smaller, more expensive, faster memories are supplemented by larger, cheaper, slower
memories. The key to the success of this organization is item (d): decreasing frequency of
access which is based on the principle of locality of reference stated above.
6
Figure 3.1: The Memory Hierarchy
This principle can be applied across more than two levels of memory, as suggested by the
hierarchy shown in Figure 3.1. The fastest, smallest, and most expensive type of memory
consists of the registers internal to the processor. Typically, a processor will contain a few
dozen such registers, although some machines contain hundreds of registers. Main memory is
the principal internal memory system of the computer. Each location in main memory has a
unique address. Main memory is usually extended with a higher-speed, smaller cache. The
cache is not usually visible to the programmer or, indeed, to the processor. It is a device for
staging the movement of data between main memory and processor registers to improve
performance. The three forms of memory just described are, typically, volatile and employ
semiconductor technology. The use of three levels exploits the fact that semiconductor
memory comes in a variety of types, which differ in speed and cost. Data are stored more
7
permanently on external mass storage devices, of which the most common are hard disk and
removable media, such as removable magnetic disk, tape, and optical storage. External,
nonvolatile memory is also referred to as secondary memory or auxiliary memory. These are
used to store program and data files and are usually visible to the programmer only in terms
of files and records, as opposed to individual bytes or words. Disk is also used to provide an
extension to main memory known as virtual memory.
Cache memory is designed to combine the memory access time of expensive, high-speed
memory combined with the large memory size of less expensive, lower-speed memory. The
concept is illustrated in Figure 3.3a. There is a relatively large and slow main memory
together with a smaller, faster cache memory. The cache contains a copy of portions of main
memory. When the processor attempts to read a word of memory, a check is made to
determine if the word is in the cache. If so, the word is delivered to the processor. If not, a
block of main memory, consisting of some fixed number of words, is read into the cache and
then the word is delivered to the processor. Because of the phenomenon of locality of
reference, when a block of data is fetched into the cache to satisfy a single memory reference,
it is likely that there will be future references to that same memory location or to other words
in the block.
Figure 3.3b depicts the use of multiple levels of cache. The L2 cache is slower and typically
larger than the L1 cache, and the L3 cache is slower and typically larger than the L2 cache.
Figure 3.4 depicts the structure of a cache/main-memory system. Main memory consists of
up to 2n addressable words, with each word having a unique n-bit address.
8
Figure 3.3: Cache and main Memory
In this organization, the cache connects to the processor via data, control, and address lines.
The data and address lines also attach to data and address buffers, which attach to a system
bus from which main memory is reached. When a cache hit occurs, the data and address
buffers are disabled and communication is only between processor and cache, with no
system bus traffic. When a cache miss occurs, the desired address is loaded onto the system
bus and the data are returned through the data buffer to both the cache and the processor. In
other organizations, the cache is physically interposed between the processor and the main
memory for all data, address, and control lines. In this latter case, for a cache miss, the
desired word is first read into the cache and then transferred from cache to processor.
10
Figure 3.6: Typical Cache organization
11
(4) Replacement Algorithms
Once the cache has been filled, when a new block is brought into the cache, one of
the existing blocks must be replaced. For direct mapping, there is only one possible
line for any particular block, and no choice is possible. For the associative and set-
associative techniques, a replacement algorithm is needed. To achieve high speed,
such an algorithm must be implemented in hardware. A number of algorithms have
been tried. Four of the most common algorithms are stated here.
Probably the most effective is Least Recently Used (LRU), others are: FIFO) First-In
First Out), LFU (Least Frequently Used), and RR (Random replacement).
12
Figure 3.7 : Elements of Cache Designs
13