CSSE2310/7231 – Lecture – Week 12

Virtual Memory

Peter Sutton
School of Electrical Engineering and Computer Science
The University of Queensland
 Outline

I Admin
I Virtual Memory
I Page Tables
I User space memory management

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 2
 Admin

I SECaTs are open – please take the time to do them

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 3
 Abstraction of memory

We have assumed that there is a way for processes to have their own view of memory.

We’d like:
I Protection
I Other processes should not be able to interfere with another process.
I Sharing
I For interprocess communication (IPC)
I Avoiding redundancy, e.g.
I Unmodified parts of fork()ed children
I every program loads/links libc.so - how many copies?!
I Optimisation
I Demand paging allows us to load things only as they are required

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 4
 Abstraction of memory

We’d like (continued)

I Varying overall allocation
I Don’t want to fix memory allocation at start of runtime.
I Want to be flexible as to how much mem is needed at different times.
I Exceeding physical memory
I Use secondary storage to store “idle” memory.

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 5
 Virtual memory

Let’s have a look at /proc/pid/maps

I Smallest address?
I Largest address?
I What’s the range?
I Does this machine have that much memory?!
I What’s going on?

(Recall that we looked at this in detail in week 4)

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 6
 Virtual memory

Two types of addresses:

I virtual addresses — Used by the CPU when running user processes, e.g.
I pointers in your code
I instruction fetches
I also called logical addresses
I physical addresses — Locations in physical RAM
Hardware support to allow dynamic translation between them without the program
needing to be aware of it.

Ideally, the kernel doesn’t have to get involved very often either.

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 7
 Virtual memory - hardware view

Memory Management Unit

[zilog.com]
The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 8
 Page table

I The virtual address space is divided into equal sized pieces called “pages”.
I The physical address space is divided into “frames”
I Frames and pages are the same size.
I Page/frame sizes being a power of two means addresses can be easily split, e.g.
4k pages in a 32-bit address space
(4096 = 212 )

31 12 11 0
PAGE/FRAME Number Offset

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 9
 Virtual address translation - simplified view

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 10
 Page table

A page table is a map from (virtual) pages to (physical) frames.

I Kernel maintains a page table for each process.
I There does not need to be any relationship between virtual and physical layout.
Process 1 Process 2
Virtual Physical Virtual Physical
... ...
0x0A 0x87 0x0A 0x88
0x0B 0x89 0x0B 0x8B
0x0C 0x73 0x0C 0xFA
0x0D 0x74 0x0D 0xFB
Note: contiguous virtual addresses1 doesn’t require contiguous physical pages.
Offsets/addresses within pages/frames are always contiguous.
1
critical for arrays
The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 11
 Address translation

Split a virtual address into a page number and an offset.

Hypothetical example (page size=4096 = 212 ):
Virt Addr. Page Offset
0x00000000→0x00000|000 0x00000 0x000
0x00000001→0x00000|001 0x00000 0x001
0x00C00234→0x00C00|234 0x00C00 0x234
0xFFFFF64B→0xFFFFF|64B 0xFFFFF 0x64B
0x81430FFF→0x81430|FFF 0x81430 0xFFF

I Page# = VA / pageSize
I Offset = VA mod pageSize

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 12
 Address translation

page#|offset → frame#|offset

ie:
page#|offset → PT(page#)|offset

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 13
 Implementation

Page Table Entries (PTE) describe a single virtual page:

I the physical frame number
I Misc. status/settings bits, e.g. permissions, page state
Note the (virtual) page number is not part of the PTE - that is implicit from the
location of the PTE in the page table
I Suppose 32bit virtual address with 4kiB page size.
I A single process’ page table would need: 232 /212 = 220 page table entries
I to map the entire address space
I For a 4 Byte PTE, this potentially means a 4MiB table, per process.
I In reality
I the kernel only maps the pages that are required, as they are required
I clever data structures are used to minimise overhead (see later)
The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 14
 TLB

I Needing to lookup a frame in the page table means that each memory access
from a program would need two memory accesses2 :
I One into the page table
I One for the actual memory access
I Reduce this burden using a Translation Lookaside Buffer (TLB).
I Hardware cache for (page→frame) mappings
I Associative (content addressable) memory
I Fast
I Hardware only goes to actual memory-based table if there is a TLB “miss”
I hardware-dependent
I some CPU architectures can ”walk” memory-based page tables, others can’t so the
kernel does it for the CPU

2
In the general case
The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 15
 TLB

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 16
 Page faults

When there is no frame corresponding to a given page, a “page fault” occurs.

Causes:
I The page is legal for that process.
I It hasn’t been loaded yet
I It has been “paged” out to disk3
The kernel needs to suspend the process until it can get that page back into RAM
and find a frame to put it in.
If we are under memory pressure, this usually means somebody else’s pages get
swapped out → “thrashing”
I The page or page access is not legal for that process, e.g.
I followed the null or some other bogus pointer
I writing to a read-only page.
I instruction fetch from a non-executable page
Kernel needs (probably) to inform the process (and possibly kill it).
3
called swapping sometimes.
The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 17
 What about null pointers?

I Core idea: legal vs illegal decisions are made at the level of pages not individual
addresses.
I If page 0 (and possibly other very low numbered pages) are always marked as
invalid, then hardware makes sure the null pointer will seg fault.

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 18
 Page replacement algorithms?

Beyond the scope of this course - do COMP3301!

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 19
 Separate address spaces

I This comes as a consequence of each process having its own page table.
I Shared pages happen when the kernel maps the same frame into multiple page
tables.
I Memory protection:
I Pages are only valid if the kernel maps them to a frame.
I A process can not construct a pointer to another process’ frames
*((char *)(rand())= 0x0; // :)
I Processes can ask the kernel to do it... shared mem, mmap() and friends

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 20
 Multi-level tables

Remember how we said a 4kiB pages in a 32 bit address space require 4MiB per
process? We can be a lot smarter than that.

I Split the page numbers into sub fields.

I Entries in the top level of the table can be “empty”. That is there are no valid
pages in the range for that top-level entry.
So, the table doesn’t need to explictly store every entry.

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 21
 Multi-level tables — gaps

[stackoverflow.com]
The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 22
 Multi-level tables

This concept can be extended to an arbitrary number of levels, e.g.

I 4 level page table
I 48 bit virtual addresses (VA), 8 byte page table entries (PTE)
I 4KiB pages/frames (12 bits of offset)
working backwards:
I each page (4KiB) can hold 512 = 29 PTEs
→ each PTE field in the VA is 9 bits
I VA structure will be
Lvl1 Lvl2 Lvl3 Lvl4 offset
9 bits 9 bits 9 bits 9 bits 12 bits

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 23
 Multi-level tables

Note that there are relationships among field sizes and the other parameters of the
system:
I PTEs must be large enough to uniquely identify a page or frame:
I at least (PA size - offset size) bits wide
I additional bits in PTE are used for storing flag such as permissions (RW/RO/EXEC)
etc
I in CSSE2310 we will specify PTE sizes in bytes so you don’t need to work
this out
I Page size and PTE size define the PTE field size, e.g.
I 4KiB pages and 4 byte PTEs
→ 1024 PTEs per page
→ 10 bits per PTE field in the virtual address
I Each PTE table is exactly one page in size (first level table can be an exception, see
next slides)
The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 24
 Multi-level tables

A trickier example:
I 4 level page table
I 64 bit virtual addresses (VA), 8 byte page table entries (PTE)
I 8KiB pages/frames (13 bits)
working backwards:
I each page (8KiB) can hold 1024 = 210 PTEs
→ each PTE field in the VA is 10 bits
I VA structure will be
?? Lvl1 Lvl2 Lvl3 Lvl4 offset
11 bits 10 bits 10 bits 10 bits 10 bits 13 bits

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 25
 Multi-level tables continued

?? Lvl1 Lvl2 Lvl3 Lvl4 offset

11 bits 10 bits 10 bits 10 bits 10 bits 13 bits
That’s only 53 “useful” bits - what about the 11 most significant bits?
I this is architecture dependent, we could
I expand the first level table to capture all of the remaining bits (like a single level
page table) e.g.
Lvl1 Lvl2 Lvl3 Lvl4 offset
21 bits 10 bits 10 bits 10 bits 13 bits
I ignore these bits
I force them to zero or some other value
I for CSSE2310, you can assume that the first level table is sized to account
for all of these bits unless we specify otherwise

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 26
 User Space Memory Management

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 27
 Memory layout

top of mem
kernel memory
bottom of stack
top of stack
...
...
top of heap
bottom of heap
other data
text “segment”
forbidden
0 forbidden
Memory mapped content goes somewhere between heap and stack.
The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 28
 Lots of room in 64bit?

It’s possibly quite crowded in a 32bit address space.

In a 64bit address space, there is more room. The precise location of heap and stack
could vary if ASLR4 is enabled.

4
Address Space Layout Randomisation
The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 29
 Kernel heap interaction?

The kernel only cares about where the top of the heap is
I If the heap needs more space, then a system call (sbrk()) will allocate more
valid pages to the process.
I malloc() is a userspace function (which will ask the kernel for more pages if
needed).
Note that modern malloc() implementations use a variety of methods to request
more memory from the kernel - e.g. anonymous memory maps using mmap().

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 30
 Some fun with /proc/pid/mem

/proc/pid/mem is a virtual file representing a process’s entire virtual memory space

/proc/self/mem always maps to the current process
$ ls -al /proc/self/mem
-rw------- 1 uqpsutto uusers 0 May 14 11:20 /proc/self/mem

Hmmm, it’s writeable....

Let’s have a look at target.c and writeat.c

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 31
 Coming Up

I Friday – virtual memory exercises

I Next week
I File systems
I A4 due

The University of Queensland – School of EECS CSSE2310/7231 – Semester 1, 2024 – Lecture – Week 12 32