Memory Unit

1
Outline
• Introduction
• Computer Memory System Overview
• Characteristics of Memory Systems
• The Memory Hierarchy
• Cache Memory
• Cache Memory Principles
• Elements of Cache Design

2
Introduction
• Computer Memory
• exhibits the widest range of
• Type
• Technology
• Organization
• Performance
• Cost
• As a result a typical computer system
• Equipped with a hierarchy of memory subsystem

3
Characteristics of Memory Systems

• Classifying and studying memory systems using

their key characteristics
• Location
• Capacity
• Unit of transfer
• Access method
• Performance
• Physical type
• Physical characteristics
• Organisation

4
 Characteristics of Memory …
• Location
• Refers to whether memory is internal or external to
computer system
• Internal memory
• directly accessible by the processor
• Types of internal memory
• Main memory – often equated with internal memory
• Cache
• CPU registers
• External memory
• Accessible via the I/O module/controller
• Also called secondary memory
• Disks and tapes

5
 Characteristics of Memory …
• Capacity
• the amount of information that can be contained in a memory unit
• For internal memory
• expressed in terms of bytes or words
• For external memory
• expressed in terms of bytes

• Terminologies
• Word
• natural unit of organisation of memory
• Typical size
• instruction length
• Bits used to represent number
• Addressable unit
• Smallest location which can be uniquely addressed
• It could be 6
• A word
• Byte level
 Memory Addressing
 Byte Ordering
• How should bytes within multi-byte word be ordered in
memory ?
• What order do we read numbers that occupy more than one
byte?
• The problem is called Endian
• Big endian machines
• Most significant byte has lowest address
• Example: IBM 370, Motorola 680x0 (Mac) and most RISC …
• Little endian machines
• Least significant byte has lowest address
• Example: Pentium (80x86), …

7
11110100 10011000 10110111 00001111
Characteristics of Memory …
• Unit of Transfer
• For internal memory
• For internal memory, the unit of transfer is equal to the
number of electrical lines into and out of the memory module
• Usually governed by data bus width
• May be equal to a word
• Often larger : 64 ,128, or 256 bits
• For external memory
• Data transferred in much larger units than a word
• Usually a block which is much larger than a word

8
 Characteristics of Memory …
• Access Methods
• methods of accessing units of data
• different methods exist

• Sequential Access
• Start at the beginning and read through in order
• Access time depends on location of data and previous location
• e.g. Tape drive units

• Direct Access
• Individual blocks have unique address
• Access is by jumping to vicinity or to nearest plus sequential
search
• Access time depends on location of data and previous location
9
• e.g. Disk drive units
 Characteristics of Memory …
• Random Access
• The time to access a given location is independent of the
sequence of prior accesses
• Thus, any location can be selected at random and directly
addressed and accessed
• individual addresses identify locations exactly
• access time is independent of location or previous access
• e.g. RAM
• Associative Access
• This is a random access type of memory that enables one to
make a comparison of desired bit locations within a
word , for a specified match
• a word is retrieved based on a portion of its contents
rather than its address. 10
• access time is independent of location or previous access
• e.g. Some cache memory units
 Characteristics of Memory …
• Performance
• Access time (latency)
• Time between presenting an address and getting access to a valid
data
• For Random Access Memory
• It is the time it takes to perform a read or write operation
• the time from the instant that an address is presented to the memory to
the instant that data have been stored or made available for use
• For Non Random Access Memory
time it takes to position the read –write mechanism at the
• It is the
desired location
• Memory Cycle time
• applicable to RAM
• consists of : access time + recovery time
• This additional time(recovery time) may be required for transients to 11
die out on signal lines or to regenerate data
• is concerned with the system bus, not the processor.
 Characteristics of Memory …
• Transfer rate
• the rate at which data can be transferred into or out of a
memory unit

• For non-random access memory

• Tn --- average time to read or write N bits
• Tn = Ta + N/R
• Where
• Tn : average time to read or write N bits
• Ta : average access time
• N : number of bits
• R : transfer rate (bps)
12
Characteristics of Memory …
• Physical Types
• Semiconductor
• RAM
• Magnetic
• Disk & Tape
• Optical
• CD & DVD
• Magneto-optical
•A magneto-optical disk is a rewritable disk
that makes use of both magnetic disk and
optical technologies. Sony Minidisc.

13
Characteristics of Memory …
• Physical Characteristics
• Volatility
• Volatile memory
• Information is lost when electrical power is switched off
• Some semiconductor memories
• E.g RAM
• Non Volatile memory
• Information once recorded remains without deterioration
• Electrical power not needed to retain information
• E.g magnetic storage unit
• Non erasable memory
• cannot be altered
• ROM

14
Characteristics of Memory …
• Organisation
• Physical arrangement of bits into words

15
Memory Hierarchy
• Design constraint on computer’s memory system
• How much ?
• capacity
• How fast ?
• To achieve performance, memory must keep up with
processor
• How expensive ?
• Cost must be reasonable in relationship to other
components

16
Memory Hierarchy
• Memory
hierarchy
diagram

17
Why hierarchy?
Consider hierarchy in business organization
 Purpose: right quantity of right quality human
resource , to achieve the required performance
Realities of storage: size speed tradeoff
 Disks: large storage, slow speed,
• A variety of technologies are used to implement
memory systems, and across this spectrum of
technologies, the following relationships hold:
• Faster access time, greater cost per bit
• Greater capacity, smaller cost per bit
• Greater capacity, slower access speed

18
Memory Hierarchy...
• Memory hierarchy characteristics, going down the
diagram

• Decreasing cost per bit, i.e. internal memory is

expensive compared with external
memory
• Increasing capacity
• Increasing access time
• Decreasing frequency of access of the memory by
the processor

19
Memory Hierarchy...
• Basic elements in memory hierarchy
• Registers
• In CPU
• Internal or Main memory
• May include one or more levels of cache
• “RAM”
• External memory
• Backing store

20
Memory Hierarchy...
• Hierarchy List
• Registers
• L1 Cache
• L2 Cache
• L3 Cache
• Main memory
• Disk
• Optical
• Tape

21
Cache Memory Principles
• Cache memory
• Small amount of fast memory
• Sits between main memory and CPU
• May be located on CPU chip or module
• Contains copies of sections of main memory

22
Cache Memory Principles...
• Cache operation – overview
• CPU requests contents of memory location
• Cache checked for this data
• If present, get from cache (fast)
• If not present, read required block from main
memory to cache
• Then deliver from cache to CPU
• Cache includes tags to identify which block of main
memory is in each cache slot

23
Cache Memory Principles...
• Typical Cache Organization

24
Cache terminology
Cache hit: a memory reference where the required
data is found in cache

Cache miss: a memory reference where the

required data is not found in cache

25
 Cache Memory Principles...
When a cache hit occurs, the data and address buffers
are disabled and communication is only between
processor and cache, with no system bus traffic.
When a cache miss occurs, the desired address is
loaded onto the system bus and the data are returned
through the data buffer to both the cache and the
processor
for a cache miss, the desired word is first read into the
cache and then transferred from cache to processor.

26
Cache Memory Principles...
• Cache/Main Memory Structure

27
Elements of Cache
Design!
• Basic design elements used to classify cache
architectures:
• Cache address
• Size
• Mapping Function
• Replacement Algorithm
• Write Policy
• Line/Block Size
• Number of Caches

28
 Elements of Cache Design!
Cache address
• Almost all processors support virtual memory
• virtual memory is that when the available RAM
memory is not sufficient for the system to run the
current applications it will take some memory from
hard disk.
• virtual memory (also virtual storage) is a memory
management technique that provides an "idealized
abstraction of the storage resources “ which "creates
the illusion to users of a very large (main) memory."
• When virtual memory is used, the address fields of
machine instructions contain virtual addresses.

29
Elements of Cache Design!

30
 Elements of Cache Design!
• For reads to and writes from main memory, a
hardware memory management unit (MMU)
translates each virtual address into a physical
address in main memory.
• When virtual addresses are used, the cache may
place between the processor and the MMU or
between the MMU and main memory
• A logical cache, also known as a virtual cache,
stores data using virtual addresses.
• The processor accesses the cache directly,
without going through the MMU.
• But A physical cache stores data using main
31
memory physical addresses
 Elements of Cache Design!
• advantage : cache access speed is faster than for a
physical cache, because the cache can respond
before the MMU an address translation performs
• disadvantage : most virtual memory systems supply
each application with the same virtual memory
address space. That is, each application sees a virtual
memory that starts at address 0.
• Thus, the same virtual address in two different
applications refers to two different physical
addresses

32
 Elements of Cache Design...
Cache Size
• The larger the cache, the larger the number of gates
involved in addressing the cache.
• The result is that the larger caches tends to be slightly
slower than smaller ones , even when built with the
same integrated circuit
• size of the cache to be small enough so that the overall
average cost per bit is close to that of main memory
and
• large enough so that the overall average access time is
close to that of the cache .
• Small enough ---not to be costly or expensive
• Large enough so overall average access time is small
33
 Elements of Cache Design...
• Mapping Function

• No of cache lines <<< No of blocks in main memory

• Mapping function needed
• A method to map main memory blocks into cache lines
• Three mapping techniques used
• Direct
• Associative
• Set Associative

34
Direct Mapping
• Each block of main memory maps to only one cache
line
• i=j modulo m
• i=cache line number,
• j=main memory block number and
• m=number of lines in the cache
• i.e. if a block is in cache, it must be in one specific
place
• Mapping function implemented using main memory
address

35
 Direct Mapping...
• Address viewed as having three fields
• Word, line and tag identifier
• Least Significant w bits identify unique word in a block
• Most Significant s bits specify one of 2s memory block
• The MSBs are split into
• a tag of s-r bits (most significant)
• Stored in the cache along with the data words of the line
• a cache line field of r bits
• Identifies one of m=2r lines of the cache

36
Direct Mapping...
• • Word : number of bits required to identify a
particular word in the block

• Index : number of bits required to identify the block

or line number in cache memory where a main
memory block will be placed .

Example: let be give a 16W , 64KW main memory & a

128 line cache. Then find the index, tag and word in
direct mapping

37
Elements of Cache Design...
• Direct Mapping
• Address Structure
Tag s-r Line or Slot r Word w

5 7 4

• 16 bit address
• 2 bit word identifier (4 byte block)
• 12 bit block identifier
• 5 bit tag (=12-7)
• 7 bit slot or line
• No two blocks in the same line have the same Tag field
Elements of Cache Design...
• Direct Mapping Cache Organization

39
Elements of Cache Design...
• Direct Mapping Summary
• Address length = (s + w) bits
• Number of addressable units = 2(s+ w) words / bytes
• Block size = line size = 2w words or bytes
• Number of blocks in main memory = 2(s+ w)/2w = 2s
• Number of lines in cache = m = 2r
• Size of tag = (s – r) bits

40
Elements of Cache Design...
• Direct Mapping Advantage and Disadvantage
• Advantage
• Easy to implement
• Inexpensive to implement
• Easy to determine the right cache line

• Disadvantage
• Fixed location for given block i.e. each block in main
memory reserved for a particular cache line

41
 Elements of Cache Design...
• Associative Mapping
• A main memory block can loaded into any line of cache
• Memory address is interpreted as tag and word
• Tag field uniquely identifies a block of memory

Total bit: number of bits required to represent main memory

log2(size of M.M)
Tag bits: number of bits required to represent the block of
main memory
log2(no. of blocks in M.M)
Word bits: number of bits required to represent block
log2(size of 1 Block)
Disadvantage 42
 It needs more comparators , cost of architecture is huge
i.e. cost of hardware will be huge.
Elements of Cache Design...
• Associative Cache Organization

43
Elements of Cache Design...
• Associative Mapping Summary
• Address length = (s + w) bits
• Number of addressable units = 2(s+w) words / bytes
• Block size = line size = 2w words or bytes
• Number of blocks in main memory = 2(s+ w)/2w = 2s
• Number of lines in cache = undetermined
• Size of tag = s bits

44
 Elements of Cache Design...
• Set Associative Mapping
• Compromise between direct and associative
• It is the combination of direct and associative
• Cache is divided into a number of sets
• v sets
• Each set contains a number of lines
• k lines
• The following relationship holds:
• M=v*k
• i = j modulo v
• i=cache set number,
• j=main memory block number and
• M=number of lines in the cache
• V= number of set
• K= number of line in the cache set
• Assume that k=2 --- 2 lines per set 45
• 2 way associative mapping
• A given block can be in one of 2 lines in only one set
 Elements of Cache
Design...
Set Associative Mapping .....
 As with associative mapping, each word maps into
multiple cache lines.
For set-associative mapping, each word maps into all
the cache lines in a specific cache set
Thus, the set-associative cache can be physically
implemented as associative caches.
It is also possible to implement the set-associative
cache as k direct mapping caches,

46
 Elements of Cache Design...
• Set Associative Mapping ...
• Cache block are divided in to sets
• Set size is always in the power of 2
i.e. if cache has 2 block/set, then it is called 2- way set
associative
similarly
4 – way set associative = 4 block/set
2 – way set associative = 2 blocks/set
• memory address viewed as three fields
• Tag, set and word fields
Word: number of bit required to identify a particular word with in the
block =
log2(no. of words/block)
Set : number of bit required to identify the corresponding set no. inside
the cache where a main memory block will be placed =
log2(no. of sets in cache) 47
Tag : number of bit required to compare two blocks which belongs to
the same set
Set Associative Mapping
Address Structure
Tag 6 bit Set 6 Word
4bit
• For 16 bit address

48
Elements of Cache Design...
• Set Associative Mapping Summary
• Address length = (s + w) bits
• Number of addressable units = 2s+w words / bytes
• Block size = line size = 2w words or bytes
• Number of blocks in main memory = 2s
• Number of lines in set = k
• Number of sets = v = 2d
• Number of lines in cache = kv = k * 2d
• Size of tag = (s – d) bits

49
 Elements of Cache Design...
• Replacement Algorithms
• Once the cache has been filled, when a new block is
brought into the cache, one of the existing blocks must be
replaced
• determine which existing block should be replaced
when a new block is brought into the cache
• There are a number of algorithms
• LRU, FIFO, LFU…

• Direct mapping
• No choice, no algorithm needed
• Each block only maps to one line

50
 Elements of Cache Design...
• For the associative and set associative techniques, a
replacement algorithm is needed. To achieve high
speed, such an algorithm must be implemented in
hardware. Replacement Algorithms …
• Least Recently used (LRU)
• Most effective
• Replaces a block not used recently
• First in first out (FIFO)
• replace a block that has been in cache longest
• Least frequently used(LFU)
• replace block which has had fewest hits
• counter associated with each cache line

51
 Elements of Cache Design...
• Write Policy
• When a block that is resident in the cache is to be
replaced, there are two cases to consider.
• If the old block in the cache has not been altered, then it
may be overwritten with a new block without first writing
out the old block.
• If at least one write operation has been performed on a
word in that line of the cache, then main memory must
be updated by writing the line of cache out to the block
of memory before bringing in the new block.

52
Elements of Cache
Design...
• Line Size
• How much data should be transferred from main
memory to the cache in a single memory reference

53
 Elements of Cache
Design...
• Number of Caches
• Multilevel Caches : Most designs include both on-chip and
external caches.
 The simplest such organization is known as a two-level
cache,
 with the internal cache designated as level 1 (L1) and
the external cache designated as level 2 (L2).
There can also be 3 or more levels of cache. This helps in
reducing main memory accesses.
• Single versus 2 level
• L1 provides best performance gains
• L2 off-chip cache
• Unified versus Split
• Unified cache
• Stores data and instruction in one cache : on-chip cache designs
• Split cache
54
• Uses 2 caches
• 1 for instruction and 1 for data : These two caches both exist at the same
level
55