Lecture Outline

• Memory Hierarchy Management

• Register Allocation
Register Allocation
– Register interference graph
– Graph coloring heuristics
Lecture 16
– Spilling

• Cache Management

Prof. Aiken CS 143 Lecture 16 1 Prof. Aiken CS 143 Lecture 16 2

The Memory Hierarchy Managing the Memory Hierarchy

• Programs are written as if there are only two

Registers 1 cycle 256-8000 bytes
kinds of memory: main memory and disk
– Programmer is responsible for moving data from
disk to memory (e.g., file I/O)
Cache 3 cycles 256k-1M – Hardware is responsible for moving data between
memory and caches
Main memory 20-100 cycles 32M-1G – Compiler is responsible for moving data between
memory and registers

Disk 0.5-5M cycles 4G-1T

Prof. Aiken CS 143 Lecture 16 3 Prof. Aiken CS 143 Lecture 16 4

Current Trends The Register Allocation Problem

• Power usage limits • Intermediate code uses unlimited temporaries

– Size and speed of registers/caches
– Speed of processors
– Simplifies code generation and optimization
– Complicates final translation to assembly
• But
– The cost of a cache miss is very high • Typical intermediate code uses too many
– Typically requires 2 caches to bridge fast processor with
large main memory temporaries
• It is very important to:
– Manage registers properly
– Manage caches properly

• Compilers are good at managing registers

Prof. Aiken CS 143 Lecture 16 5 Prof. Aiken CS 143 Lecture 16 6

1
The Register Allocation Problem (Cont.) History

• The problem: • Register allocation is as old as compilers

Rewrite the intermediate code to use no more – Register allocation was used in the original
temporaries than there are machine registers FORTRAN compiler in the ‘50s
– Very crude algorithms
• Method:
– Assign multiple temporaries to each register • A breakthrough came in 1980
– But without changing the program behavior – Register allocation scheme based on graph coloring
– Relatively simple, global and works well in practice

Prof. Aiken CS 143 Lecture 16 7 Prof. Aiken CS 143 Lecture 16 8

An Example The Idea

• Consider the program • Can allocate a, e, and f

all to one register (r1):
a := c + d Temporaries t1 and t2 can share the same
e := a + b r1 := r2 + r3 register if at any point in the program at
r1 := r1 + r4
f := e - 1
r1 := r1 - 1 most one of t1 or t2 is live .

• Assume a and e dead • A dead temporary is not Or

after use needed
– Temporary a can be
– A dead temporary can be
“reused” after e := a + b
reused If t1 and t2 are live at the same time, they
– So can temporary e
cannot share a register
Prof. Aiken CS 143 Lecture 16 9 Prof. Aiken CS 143 Lecture 16 10

Algorithm: Part I The Register Interference Graph

• Compute live variables for each point: • Construct an undirected graph

a := b + c {b,c,f} – A node for each temporary
{a,c,f}
d := -a – An edge between t1 and t2 if they are live
{c,d,f} e := d + f simultaneously at some point in the program
{c,d,e,f}
{c,e}

f := 2 * e
b := d + e
{b,c,e,f} • This is the register interference graph (RIG)
e := e - 1
– Two temporaries can be allocated to the same
{c,f} {c,f}
register if there is no edge connecting them
b := f + c {b}
{b}
Prof. Aiken CS 143 Lecture 16 11 Prof. Aiken CS 143 Lecture 16 12

2
Example Notes on Register Interference Graphs

• For our example: • Extracts exactly the information needed to

a characterize legal register assignments
b
f • Gives a global (i.e., over the entire flow graph)
picture of the register requirements
e c
• After RIG construction the register allocation
d
algorithm is architecture independent
• E.g., b and c cannot be in the same register
• E.g., b and d could be in the same register
Prof. Aiken CS 143 Lecture 16 13 Prof. Aiken CS 143 Lecture 16 14

Definitions Register Allocation Through Graph Coloring

• A coloring of a graph is an assignment of • In our problem, colors = registers

colors to nodes, such that nodes connected by – We need to assign colors (registers) to graph nodes
an edge have different colors (temporaries)

• A graph is k-colorable if it has a coloring with • Let k = number of machine registers

k colors
• If the RIG is k-colorable then there is a
register assignment that uses no more than k
registers

Prof. Aiken CS 143 Lecture 16 15 Prof. Aiken CS 143 Lecture 16 16

Graph Coloring Example Example Review

• Consider the example RIG

a r2 a := b + c
d := -a
b r3 e := d + f
r1 f

r2 e c r4 b := d + e
f := 2 * e
e := e - 1
d r3
• There is no coloring with less than 4 colors b := f + c
• There are 4-colorings of this graph
Prof. Aiken CS 143 Lecture 16 17 Prof. Aiken CS 143 Lecture 16 18

3
Example After Register Allocation Computing Graph Colorings

• Under this coloring the code becomes: • How do we compute graph colorings?
r2 := r3 + r4
r3 := -r2
r2 := r3 + r1
• It isn’t easy:
1. This problem is very hard (NP-hard). No efficient
algorithms are known.
r3 := r3 + r2 – Solution: use heuristics
r1 := 2 * r2
r2 := r2 - 1 2. A coloring might not exist for a given number of
registers
– Solution: later
r3 := r1 + r4

Prof. Aiken CS 143 Lecture 16 19 Prof. Aiken CS 143 Lecture 16 20

Graph Coloring Heuristic Graph Coloring Heuristic

• Observation: • The following works well in practice:

– Pick a node t with fewer than k neighbors in RIG – Pick a node t with fewer than k neighbors
– Eliminate t and its edges from RIG – Put t on a stack and remove it from the RIG
– If resulting graph is k-colorable, then so is the – Repeat until the graph has one node
original graph
• Assign colors to nodes on the stack
• Why? – Start with the last node added
– Let c1,…,cn be the colors assigned to the neighbors – At each step pick a color different from those
of t in the reduced graph assigned to already colored neighbors
– Since n < k we can pick some color for t that is
different fromProf.
those of its neighbors
Aiken CS 143 Lecture 16 21 Prof. Aiken CS 143 Lecture 16 22

Graph Coloring Example (1) Graph Coloring Example (2)

• Start with the RIG and with k = 4:

a
b b
f f
Stack: {} Stack: {a}

e c e c

d d
• Remove a • Remove d

Prof. Aiken CS 143 Lecture 16 23 Prof. Aiken CS 143 Lecture 16 24

4
Graph Coloring Example (3) Graph Coloring Example (4)

• Note: all nodes now have fewer than 4

neighbors
b b
f f
Stack: {d, a} Stack: {c, d, a}

e c e

• Remove c • Remove b

Prof. Aiken CS 143 Lecture 16 25 Prof. Aiken CS 143 Lecture 16 26

Graph Coloring Example (5) Graph Coloring Example (6)

f f
Stack: {b, c, d, a} Stack: {e, b, c, d, a}

• Remove e • Remove f

Prof. Aiken CS 143 Lecture 16 27 Prof. Aiken CS 143 Lecture 16 28

Graph Coloring Example (7) Graph Coloring Example (8)

• Now start assigning colors to nodes, starting

with the top of the stack

r1 f
Stack: {f, e, b, c, d, a} Stack: {e, b, c, d, a}

Prof. Aiken CS 143 Lecture 16 29 Prof. Aiken CS 143 Lecture 16 30

5
Graph Coloring Example (9) Graph Coloring Example (10)

b r3
r1 f r1 f
Stack: {b, c, d, a} Stack: {c, d, a}

r2 e r2 e

• e must be in a different register from f

Prof. Aiken CS 143 Lecture 16 31 Prof. Aiken CS 143 Lecture 16 32

Graph Coloring Example (11) Graph Coloring Example (12)

b r3 b r3
r1 f r1 f
Stack: {d, a} Stack: {a}

r2 e c r4 r2 e c r4

d r3
• d can be in the same register as b

Prof. Aiken CS 143 Lecture 16 33 Prof. Aiken CS 143 Lecture 16 34

Graph Coloring Example (13) What if the Heuristic Fails?

• What if all nodes have k or more neighbors ?

a r2
b r3 • Example: Try to find a 3-coloring of the RIG:
r1 f
a
r2 e c r4
b
f
d r3

e c

d
Prof. Aiken CS 143 Lecture 16 35 Prof. Aiken CS 143 Lecture 16 36

6
What if the Heuristic Fails? What if the Heuristic Fails?

• Remove a and get stuck (as shown below) • Remove f and continue the simplification
– Simplification now succeeds: b, d, e, c
• Pick a node as a candidate for spilling
– A spilled temporary “lives” in memory
– Assume that f is picked as a candidate
b b
f

e c e c

d d
Prof. Aiken CS 143 Lecture 16 37 Prof. Aiken CS 143 Lecture 16 38

What if the Heuristic Fails? Spilling

• Eventually we must assign a color to f • If optimistic coloring fails, we spill f

– Allocate a memory location for f
• Typically in the current stack frame
• We hope that among the 4 neighbors of f we • Call this address fa
use less than 3 colors  optimistic coloring
• Before each operation that reads f, insert
? f b r3 f := load fa

c r1 • After each operation that writes f, insert

r2 e store f, fa
d r3
Prof. Aiken CS 143 Lecture 16 39 Prof. Aiken CS 143 Lecture 16 40

Spilling Example Recomputing Liveness Information

• This is the new code after spilling f • The new liveness information after spilling:
a := b + c
{a,c,f} a := b + c {b,c,f}
d := -a d := -a
f := load fa {c,d,f} f := load fa
e := d + f e := d + f {c,d,e,f}
{c,d,f} {c,e}
f := 2 * e b := d + e f := 2 * e b := d + e
{c,f} {b,c,e,f}
store f, fa e := e - 1 store f, fa e := e - 1
{c,f}
{c,f}
f := load fa f := load fa
b := f + c {c,f} b := f + c {b}
{b}
Prof. Aiken CS 143 Lecture 16 41 Prof. Aiken CS 143 Lecture 16 42

7
Recomputing Liveness Information Recompute RIG After Spilling

• New liveness information is almost as before • Some edges of the spilled node are removed
• In our case f still interferes only with c and d
• f is live only • And the resulting RIG is 3-colorable
– Between a f := load fa and the next instruction
– Between a store f, fa and the preceding instr. a
b
f
• Spilling reduces the live range of f
– And thus reduces its interferences e c
– Which results in fewer RIG neighbors for f
d
Prof. Aiken CS 143 Lecture 16 43 Prof. Aiken CS 143 Lecture 16 44

Spilling Notes Caches

• Additional spills might be required before a • Compilers are very good at managing registers
coloring is found – Much better than a programmer could be

• Compilers are not good at managing caches

• The tricky part is deciding what to spill
– This problem is still left to programmers
– But any choice is correct – It is still an open question how much a compiler can
do to improve cache performance
• Possible heuristics:
– Spill temporaries with most conflicts • Compilers can, and a few do, perform some
– Spill temporaries with few definitions and uses cache optimizations
– Avoid spilling in inner loops
Prof. Aiken CS 143 Lecture 16 45 Prof. Aiken CS 143 Lecture 16 46

Cache Optimization Cache Optimization (Cont.)

• Consider the loop • Consider the program:

for(j := 1; j < 10; j++) for(i=1; i<1000; i++)
for(j := 1; j < 10; j++)
for(i=1; i<1000; i++)
a[i] *= b[i]
a[i] *= b[i]
– Computes the same thing
– But with much better cache behavior
• This program has terrible cache – Might actually be more than 10x faster
performance
• A compiler can perform this optimization
• Why?
– called loop interchange

Prof. Aiken CS 143 Lecture 16 47 Prof. Aiken CS 143 Lecture 16 48

8
Conclusions

• Register allocation is a “must have” in

compilers:
– Because intermediate code uses too many
temporaries
– Because it makes a big difference in performance

• Register allocation is more complicated for

CISC machines

Prof. Aiken CS 143 Lecture 16 49