Chapter 10: Virtual Memory

Questions?

! What is virtual memory and when is it useful?

CSCI 4730 ! What is demand paging?
Operating Systems ! When should pages in memory be replaced?
! When should pages be put on disk?
Virtual Memory ! What is the working set model?

2
Maria Hybinette, UGA Maria Hybinette, UGA

Operating System’s Goals Idea: Virtual Memory

! Support processes when there is not enough ! OS provides an illusion of

more memory than is
physical memory
physically available:
» Single process with very large address space » Large logical space but
» Multiple processes with combined address spaces really
! User code should be independent of amount » small physical memory
of physical memory ! Why does this work?
» Correctness, if not performance » Only part of the program
needs to be in memory (at a
particular time) for execution
» Relies on key properties of
user processes (workload)
and machine architecture
(hardware)
3 4
Maria Hybinette, UGA Maria Hybinette, UGA

Observations

! Sequential memory accesses of a process are

predictable and tend to have locality of reference:
» Spatial: reference memory addresses near previously
referenced addresses (in memory)
» Temporal: reference memory addresses that have
referenced in the past
! Processes spend majority of time in small portion
of code
» Estimate: 90% of time in 10% of code
! Implication:
» Process only uses small amount of address space at any
moment
» Only small amount of address space must be resident in
physical memory
5
Maria Hybinette, UGA
 Approach: Demand Paging Virtual Memory Intuition

! Bring in pages into ! Idea: OS keeps unreferenced pages on disk

memory only when needed » Slower, cheaper backing store than memory
» Less memory ! Process can run when not all pages are loaded into
» Less I/O main memory
» Faster response time ! OS and hardware cooperate to provide illusion of large
disk as fast as main memory
! Process viewed as a
» Same behavior as if all of address space in main memory
sequence of pages rather
» Hopefully have similar performance
than contiguous address
space ! Requirements:
» OS must have mechanism to identify location of each
» Pager (whole process) page in address space in memory or on disk
» Swapper (portion of a » OS must have policy for determining which pages live in
process) memory and which on disk

7 8
Maria Hybinette, UGA Maria Hybinette, UGA

Virtual Address Space Virtual Address Space

Mechanisms Mechanisms
Frame # valid-invalid bit

! Each page in virtual address space maps to one of three ! Extend page tables with an extra bit to 1
indicate whether it is in memory 1
locations: 1
(resident bit): valid (invalid) or on 1
» Physical main memory: Small, fast, expensive disk. 0
M
» Disk (backing store): Large, slow, cheap » permissions (r/w), valid 0
» Nothing (error): Free » Page in memory: valid bit set in page page table
0
table entry (PTE)
» Page out to disk: valid bit cleared
Smaller, faster (invalid)
and more
Leverage memory registers expensive – PTE points to block on disk
hierarchy of machine – Causes trap into OS when page is
architecture cache referenced
Each layer acts as – Trap: page fault
“backing store” for the main memory Bigger, smaller
and cheaper
layer above

Disk Storage
9 10
Maria Hybinette, UGA Maria Hybinette, UGA

Paging Operations
Virtual Memory Mechanisms
PTE in TLB?

CPU checks TLB

! Hardware and OS cooperate to translate addresses
! First, hardware checks TLB for virtual address PTE in TLB?
Yes

» TLB hit: Address translation is done; page in physical memory No

» TLB miss: Page fault routine

Access page table

– Hardware or OS walk page tables OS Instructs CPU No

to read the page Page in MM?
– If PTE designates page is valid, then page in physical memory from disk
Yes
! Not in main memory: Page fault (i.e., invalid) CPU activates
I/O hardware Update TLB

» Trap into OS (not handled by hardware) Page transferred

from disk to CPU generates
» OS selects victim page in memory to replace main memory physical address

– Write victim page out to disk if modified (add dirty bit to PTE) Yes
Memory
» OS reads referenced page from disk into memory Full?

No
» Page table is updated, valid bit is set Page replacement

Page tables
» Process continues execution updated

11
Maria Hybinette, UGA
 Virtual Memory Policies Page Selection

! OS needs to decide on policies on page faults

concerning: ! When should a page be brought from disk into memory?
» Page selection ! Request paging: User specifies which pages are needed
for process
– When should a page (or pages) on disk be brought into memory?
» Problems:
– Two cases
– Manage memory by hand
! When process starts, code pages begin on disk
– Users do not always know future references
! As process runs, code and data pages may be moved to disk
– Users are not impartial
» Page replacement ! Demand paging: Load page only when page fault occurs
– Which resident page (or pages) in memory should be thrown out
to disk? » Intuition: Wait until page must absolutely be in memory
» When process starts: No pages are loaded in memory
! Goal: Minimize number of page faults » Advantage: Less work for user
» Page faults require milliseconds to handle (reading from disk) » Disadvantage: Pay cost of page fault for every newly
» Implication: Plenty of time for OS to make good decision accessed page

13 14
Maria Hybinette, UGA Maria Hybinette, UGA

Page Selection Continued Virtual Page Optimizations

! Prepaging (anticipatory, prefetching): OS loads page ! Copy-on-Write: on process creation allow

into memory before page is referenced
parent and child to share the same page in
» OS predicts future accesses (oracle) and brings pages
into memory ahead of time memory until one modifies the page.
– How?
– Works well for some access patterns (e.g., sequential)
» Advantages: May avoid page faults
» Problems? :
! Hints: Combine demand or prepaging with user-
supplied hints about page references
» User specifies: may need page in future, don’t need this
page anymore, or sequential access pattern, ... copy page C

» Example: madvise() in Unix

15 16
Maria Hybinette, UGA Maria Hybinette, UGA

What happens if there is no free

frame? Page Replacement

! Page replacement
! Which page in main memory should selected as
» find some page in memory, but not really in use, victim?
swap it out » Write out victim page to disk if modified (dirty bit set)
! Observation: Same page may be brought into » If victim page is not modified (clean), just discard
memory several times
! OPT: Replace page not used for longest time in future
» Advantage: Guaranteed to minimize number of page
faults
» Disadvantage: Requires that OS predict the future
– Not practical, but good for comparison
! Random: Replace any page at random
» Advantage: Easy to implement
» Works okay when memory is not severely over-committed

17 18
Maria Hybinette, UGA Maria Hybinette, UGA
 Page Replacement Continued Evaluate Page Replacement Algorithms

! FIFO: Replace page that has been in memory the ! Want lowest page-fault rate.
longest
» Intuition: First referenced long time ago, done with it now
! Evaluate algorithm by running it on a particular
» Advantages:
string of memory references (reference string)
– Fair: All pages receive equal residency and computing the number of page faults on
– Easy to implement (circular buffer) that string.
» Disadvantage: Some pages may always be needed ! Convert address to page
! LRU: Replace page not used for longest time in past » Example: Assume 100 bytes per page and
» Intuition: Use past to predict the future – Assume the address sequence:
» Advantages: ! 0100, 0210, 0250, 0300, 0350, 0380, 0400, 0160, 0250, 0505,
– With locality, LRU approximates OPT 0100, 0110, 0230, 0350, 0450, 0450, 0500, 0500
» Disadvantages: – Convert address to a reference string:
– Harder to implement, must track which pages have been ! 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5.
accessed
– Does not handle all workloads well 19 20
Maria Hybinette, UGA Maria Hybinette, UGA

Example: FIFO Page

Replacement Page Replacement Example
Page reference string: ABCABDADBCB
Three pages of physical memory
OPT FIFO LRU
ABC
A

B
D
A

! FIFO: Replace page that has been in memory the D

longest
B

21 B 22
Maria Hybinette, UGA Maria Hybinette, UGA

First-In-First-Out (FIFO)
Page Replacement: Add memory Algorithm: Add Memory
! Add more physical memory, what happens to ! Reference string: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5
performance? ! 3 frames (3 pages can be in memory at a time per process)
» Ideally the numbers of page faults should should
decrease as number of available frames increases 1 2 3

» 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5. 4 1 2
5 3 4
» If 1 page frame : Number of page faults?
– 12 page faults, one fault for every page (11 faults) ! 4 frames
» If 12 frames : Number of page faults?
1 2 3 4
– 5 page faults
5 1 2 3
4 5

! FIFO Replacement – Belady’s Anomaly

» more frames ! less page faults

23 24
Maria Hybinette, UGA Maria Hybinette, UGA
 Page Replacement Comparison Implementing LRU
! Software Perfect LRU
! Add more physical memory, what happens to
» OS maintains ordered list of physical pages by reference time
performance? » When page is referenced: Move page to front of list (top)
» LRU, OPT: Add more memory, guaranteed to have » When need victim: Pick page at back of list (bottom)
fewer (or same number of) page faults » Trade-off: Slow on memory reference, fast on replacement
– Smaller memory sizes are guaranteed to ! Hardware Perfect LRU
contain a subset of larger memory sizes » Associate register with each page
» FIFO: Add more memory, usually have fewer page » When page is referenced: Store system clock in register
faults » When need victim: Scan through registers to find oldest clock
– Belady’s anomaly: May actually have more page » Trade-off: Fast on memory reference, slow on replacement
faults! (especially as size of memory grows)
! In practice, do not implement Perfect LRU
» LRU is an approximation anyway, so approximate more
» Goal: Find an old page, but not necessarily the very oldest
25 26
Maria Hybinette, UGA Maria Hybinette, UGA

Clock or Second Chance

Algorithm Clock Algorithm Example

! Hardware
» Keep use (or reference) bit for each page frame
initialized to 0.
» When page is referenced: set use bit (1)
! Operating System
» Page replacement: Look for page with use bit cleared (0)
(has not been referenced for awhile)
» Implementation:
– Keep pointer to last examined page frame
– Traverse pages in circular buffer
– Clear use bits while searching for replacement
– Stop when find page with already cleared use bit, replace
this page
! Worst Case:
» All bits are set -> FIFO (slow)
27 28
Maria Hybinette, UGA Maria Hybinette, UGA

Clock Extensions More Clock Extensions

! Replace multiple pages at once ! Add software byte

» Intuition: Expensive to run replacement algorithm and to
write single block to disk » Intuition: Keep track of history when last used
» Find multiple victims each time ! Implementation: Reference bit
! Two-handed clock » With each page associate a bit, initially = 0
» Intuition:
» When page is referenced bit set to 1.
– If takes long time for clock hand to sweep through pages,
then all use bits might be set » Keep a history of reference bit in an 8 bit byte:
– Traditional clock cannot differentiate between usage of – Shift reference bit for each page to high order bit,
different pages
and other bits right one bit.
» Allow smaller time between clearing use bit and testing
– 11000100 (more recently used than below)
– First hand: Clears use bit
– Second hand: Looks for victim page with use bit still – 01110111
cleared

29 30
Maria Hybinette, UGA Maria Hybinette, UGA
 More Clock Extensions More Clock Extensions

! Add software counter (“chance”) ! Use dirty bit to give preference to dirty pages
» Intuition: Better ability to differentiate across pages (how » Intuition: More expensive to replace dirty pages
much they are being accessed) – Dirty pages must be written to disk, clean pages do not
» Increment software counter chance if use bit is 0 » Replace pages that have use bit and dirty bit cleared
» Replace when chance exceeds some specified limit

0, 0 Not recently used, not modified Best to replace

0, 1 Not recently used, but modified Needs to be written out
1, 0 Recently used, not modified Probably used again soon
1, 1 Recently used and modified Probably used again soon and
need to be written out

31 32
Maria Hybinette, UGA Maria Hybinette, UGA

Problems with Problems with

LRU-based Replacement LRU-based Replacement

! Locality of reference: ! Example: 2, 1, 3, 2, 4, 2, 4, 1, 5, 6, 2, …

» Same pages referred frequently (warm pages) ! Problem:
» Dislodges warm pages if a long sequence of one time
» Example: 2, 1, 3, 2, 4, 2, 4, 1, 5, 6, 2, … page references occur.
! LRU takes advantage of this! – In the above ex, page 2 may get dislodged by the access
pattern …, 4, 1, 5, 6,
! Leading question: » LRU does not consider frequency of accesses
» Is a page that has been accessed once in the past ! Solution: Track frequency of accesses to page
as likely to be accessed in the future as one that » Pure LFU (Least-frequently-used) replacement
has been accessed N times? ! Problem: but LFU can never forget pages from the far
past…

33 34
Maria Hybinette, UGA Maria Hybinette, UGA

Combine LRU and LFU Combine LRU and LFU

! LRU-K: Combines recency and frequency

attributes ! 2Q: More efficient than LRU-2, similar performance
» Intuition: Instead of removing cold pages, only admit hot
» Idea: Take into account history: last K references pages to main buffer
(classic LRU K=1). Replace page with oldest (LRU) » A1in for short-term accesses, managed with FIFO (not
K-th reference (or does not have K-th reference) yet proven hot)
» Expensive to implement, LRU-2 used in databases » Am for long-term accesses, managed with LRU
» On first page fault, page enters A1in queue
» On subsequent page faults, page enters Am queue
» A1out: pages not in buffer cache, but remembered

35 36
Maria Hybinette, UGA Maria Hybinette, UGA
 Allocating Memory across
Questions Processes

! How to allocate memory across competing ! Problem:

processes? » 2 processes and 25 free frames how are they divided up?
! What is thrashing? What is a working set? ! Three General Approaches:
! How to ensure working set of all processes » Global Replacement
fit? » Per-Process Replacement
» Per-User Replacement

37 38
Maria Hybinette, UGA Maria Hybinette, UGA

Global Replacement Per-process replacement

! Global replacement ! Per-process free pool of pages

» Pages from all processes lumped into single replacement » Equal, Fixed Allocation: Fixed number of pages per
pool process
– Example: Run clock over all page frames – 100 frames and 5 processes, give each 20 pages.
» Each process competes with other processes for frames – Fixed fraction of physical memory
» Advantages: » Proportional Allocation:
– Flexibility of allocation – Proportional to size of address space
– Minimize total number of page faults – Adjust size allocated if a process have higher priority
» Disadvantages ! Page fault in one process only replaces frame of that
– One memory-intensive process can hog memory, hurt all process
processes
! Advantage: Relieves interference from other processes
– Paging behavior of one process depends on the behavior of
other processes ! Disadvantage: Potentially inefficient allocation of
resources

39 40
Maria Hybinette, UGA Maria Hybinette, UGA

Per-User Replacement Over Committing Memory

! Advantages: Users running more processes ! When does the Virtual Memory illusion break?
cannot hog memory ! Example:
» Set of processes frequently referencing 33 important
! Disadvantage: Inefficient allocation pages
» Physical memory can fit 32 pages
! What happens?
» System Repeat Cycle:
– Reference page not in memory
– Replace a page in memory with newly referenced page
– Replace another page right away again, since all its pages
are in active use…

41 42
Maria Hybinette, UGA Maria Hybinette, UGA
 System does not know it is
Thrashing thrashing
! If a process does not have “enough” pages, the page-fault rate is very
high. This leads to:
! Thrashing:
» low CPU utilization.
» Spends more time paging than execution, i.e. system » operating system thinks that it needs to increase the degree of
reading and writing pages instead of executing useful multiprogramming.
instructions » another process added to the system
» Average memory access time equals to disk access time ! Why the CPU utilization decreases:
» Illusion breaks: Memory appears slow as disk rather than » Suppose a process need more frames, starts faulting, removing frames
disk appearing fast as memory from others, in turn making the other processes fault
! Add more processes, thrashing gets worse » Processes queue up for the paging device, CPU decreases
! System is reading and writing pages instead of » OS add processes that immediately need new frames further taking away
pages from running processes
executing useful instructions
» Implication: Average memory access time = disk access
time
» Memory appears as slow as disk, instead of disk
appearing as fast as memory

43 44
Maria Hybinette, UGA Maria Hybinette, UGA

Thrashing: Solutions Motivation for Solution

! Limit thrashing by using a local replacement
» Process does not steal frames from other and cause ! Thrashing cannot be fixed with better replacement
latter to thrash policies
» Average service time for a page fault will still increase… » Page replacement policies do not indicate that a page
! Admission Control: must be kept in memory
» Determine of much memory each process needs » Only show which pages are better than others to replace
» Long-term scheduling policy: ! Student’s analogy to thrashing: Too many courses
– Run only processes whose memory requirement can be » Solution: Drop a course
satisfied
! OS solution: Admission control
! What if memory requirement of one process is too » Determine how much memory each process needs
high?
» Long-term scheduling policy
» Observation: a process moves through different
– Run only those processes whose memory requirements can
``localities’’ through out is lifetime be satisfied
– Locality: Set of pages that are actively used together
» What if memory needs of one process are too large?
» Solution: Amortize page allocated so that a process get
enough page for its current locality
45 46
Maria Hybinette, UGA Maria Hybinette, UGA

Working Set Balance Set

! Informal definition
» Collection of pages the process is referencing frequently ! Motivation: Process should not be scheduled unless
» Collection of pages that must be resident to avoid working set is resident in main memory
thrashing ! Divide runnable processes into two groups:
! Formal definition » Active: Working set is loaded
» Assume locality; use recent past to predict future » Inactive: Working set is swapped to disk

» Pages referenced by process in last T seconds of ! Balance set: Sum of working sets of all active
execution processes
» Working set changes slowly over time ! Interaction with scheduler
» If balance set exceeds size of memory, move some
! Example: process to inactive set
– Which process???
"=8 Time
» If balance set is less than size of memory, move some
AABCBBBCDCDEBBEEDFB FDBBEDB process to active set
– Which process?
ABC BDEF
47 » Any other decisions? 48
Maria Hybinette, UGA Maria Hybinette, UGA
 Working Set Implementation Unresolved Questions

! Leverage use bits (as in clock algorithm) ! How should value of T be configured?
» What if T is too large?
! OS maintains idle time for each page
! How should working set be defined when pages are
» Amount of CPU received by process since last shared?
access to page » Put jobs sharing pages in same balance set
» Periodically scan all resident pages of a process ! What processes should compose balance set?
– If use bit is set, clear page’s idle time ! How much memory needed for a “balanced system”?
– If use bit is clear, add process CPU time (since last » Balanced system: Each resource (e.g., CPU, memory,
scan) to idle time disk) becomes bottleneck at nearly same time
» If idle time < T, page is in working set » How much memory is needed to keep the CPU busy?
» With working set approach, CPU may be idle even with
runnable processes

49 50
Maria Hybinette, UGA Maria Hybinette, UGA

Page-Fault Frequency Scheme Current Trends

! Observation: Thrashing has a high page-fault rate
! Idea: Control page fault-rate by controlling # frames that are allocated ! VM code is not as critical
to a process
» Reason #1: Personal vs. time-shared machine
» Too high page fault rate : process need more frames
– Why does this matter?
» Too low : process have to many frames
» Reason #2: Memory is more affordable, more memory
! Approach: Establish “acceptable” page-fault rate (upper and lower
bound) ! Less hardware support for replacement policies
» If actual rate falls below lower limit, process loses frame. » Software emulation of use and dirty bits
» If actual rate exceeds upper limit, process gains frame. ! Larger page sizes
» Better TLB coverage
» Smaller page tables
» Disadvantage: More internal fragmentation
– Multiple page sizes

51 52
Maria Hybinette, UGA Maria Hybinette, UGA

53 54
Maria Hybinette, UGA Maria Hybinette, UGA
 Memory Hierarchy Average Access Time calculation

Leverage memory hierarchy of machine architecture ! Average access time calculation:

Each layer acts as “backing store” for the layer above » H: Percentage of references that hit page in physical
memory
Smaller, faster » CAccessMemory: Cost of referencing page in memory (e.g.,
and more
registers expensive
100 ns)
» CPageFault: Cost of page fault (e.g., 20 ms or 20,000,000ns)
cache
» H * CAccessMemory + (1-H) * CPageFault

main memory Bigger, smaller ! Example: 1 out of every 33 references misses, H = 97%
and cheaper
» 0.97 * (100 ns) + (0.03) * (20000000ns) = 750000 ns = 750
us
Disk Storage » More than 1000 times slower than physical memory
access
! Need very high hit rate for acceptable performance

55 56
Maria Hybinette, UGA Maria Hybinette, UGA