Computer Science 146

Computer Architecture
Spring 2004
Harvard University
Instructor: Prof. David Brooks
[email protected]
Lecture 8: Multiple Issue and Speculation

Computer Science 146

David Brooks

Lecture Outline
Dynamic Branch Predictor Review
Superscalar/Multiple-Issue Designs
Speculative Execution
Tomasulo with ROB example

Computer Science 146

David Brooks

Dynamic Branch Prediction

Branch History Table: 2-bits for good loop prediction
Correlation: Recently executed branch give insight
into the next branch
Different Branches or different executions of the same
branch

History can be global or per-branch PC (or per-set)

Tournament Predictors Combine many approaches
Branch Target Buffer Predicts target of branch
Computer Science 146
David Brooks

Return Address Stack

Say foo() is called from many different locations in a
program
It will then return to many different locations!
RAS can predict which location to return to because it
stores the caller PC
This is faster than having to load up indirect jumps
(jump r31)
If the call-depth doesnt exceed the size of the RAS,
this prediction will always be correct
Computer Science 146
David Brooks

Multiple Issue
Goal: Sustain a CPI of less than 1 by issuing and
processing multiple instructions per cycle
SuperScalar
Issue varying number of instructions per clock
Statically Scheduled
Dynamically Scheduled

VLIW (EPIC)
Issue a fixed number of instructions formatted as one
large instruction or instruction packet
Similar to static-scheduled superscalar
Computer Science 146
David Brooks

Multiple Issue Choices

Common
Name

Issue
Structure

Hazard
Detection

Scheduling

Examples

Superscalar
(static)

Dynamic

Hardware

Static

Sun UltraSPARC II/III

Superscalar
(dynamic)

Dynamic

Hardware

Dynamic

IBM POWER2

Superscalar Dynamic
(speculative)

Hardware

Dynamic with
speculation

Pentium III/4, MIPS

R10K, Alpha 21264, IBM
POWER4, HP PA8500

VLIW

Static

Software

Static

Trimedia, i860

EPIC

mostly
static

mostly
software

mostly static

Itanium (IA64)

Computer Science 146

David Brooks

Multiple Issue Example

Single Issue Clock Cycle
i
i+1
i+2
i+3
i+4
i+5

IF

ID

EX

WB

IF

ID

EX

WB

IF

ID

EX

IF

ID
IF

EX

Multiple Issue Clock Cycle

9

10

IF

ID

EX

WB

IF

ID

EX

WB

WB

IF

ID

EX

WB

IF

ID

EX

WB

IF

ID

EX

WB

IF

ID

EX

WB

WB

ID

EX

WB

IF

ID

EX

WB

Maybe 1 ALU + 1 FP
2 ALU + 2 LD/ST + 2 FP
Many combinations possible restriction ease implementation
Computer Science 146
David Brooks

Multiple Issue: Hazards

As usual, we have to deal with the big three
hazards:
Structural Hazards
Data Hazards
Control Hazards

Multiple issue gives:

More opportunity for hazards (why?)
Larger performance hit from hazards (why?)

Computer Science 146

David Brooks

Structural Hazards
If both instructions per cycle are int/float we may
need two int ALUs and two FP ALUs
What about register files?
This may lead to issue restrictions
Compiler/hardware has to manage these restrictions

2-issue machines typically do 1 INT/1 FP per cycle

Good performance for many apps (+)
Hazard Detection is easy (+)
No performance boost for non-FP apps (-)
Computer Science 146
David Brooks

Data Hazards
ADD R1, R2, R3
ADD R4, R5, R6
ADD R8, R1, R7
ADD R10, R9, R1

Assume full-bypassing
How many stalls for single issue?
How many stalls for dual issue?

Full bypassing?
Not easy

Computer Science 146

David Brooks

Control Hazards
Multiple Issue Clock Cycle

Branch
3 Branch
Delay Slots

IF

ID

EX

WB

i+1

IF

ID

EX

WB

i+2

IF

ID

EX

WB

i+3

IF

ID

EX

WB

i+4

IF

ID

EX

WB

i+5

IF

ID

EX

WB

10

Branch stalls bubbles are compounded in n-way

machines
Computer Science 146
David Brooks

Example: Pipeline Problem

IF1
IF2
RF
EX

M1
M2
WB

First part of instruction fetch (TLB access)

Instruction fetch completes (I-cache accessed)
Instruction decoded and register file read
Perform Operation; compute memory address
(base+displacement); compute branch target
address; compute branch condition
First part of memory access (TLB access)
Memory access completes (D-cache accessed)
Write back results into register file

How many read/write ports needed?

Computer Science 146

David Brooks

Pipeline Problem Cont.

Single Issue Clock Cycle
i

IF1

IF2

ID

EX

M1

M2

WB

IF1

IF2

ID

EX

M1

M2

i+1

IF1

i+2
i+3

10

WB

IF2

ID

EX

M1

M2

WB

IF1

IF2

ID

EX

M1

M2

WB

IF1

IF2

ID

EX

M1

M2

IF1

IF2

ID

EX

M1

i+4
i+5

What is the branch delay?

What is the load delay?
How many adders are needed to prevent structural hazards?
How many destination RegIDs and comparators are needed for forwarding?
Computer Science 146
David Brooks

Pipeline Problem Cont.

Multiple Issue Clock Cycle
1

IF1

IF2

ID

EX

M1

M2

WB

i+1

IF1

10

IF2

ID

EX

M1

M2

WB

i+2

IF1

IF2

ID

EX

M1

M2

WB

i+3

IF1

IF2

ID

EX

M1

M2

WB

i+4

IF1

IF2

ID

EX

M1

M2

WB

i+5

IF1

IF2

ID

EX

M1

M2

WB

i+6

IF1

IF2

ID

EX

M1

M2

WB

i+7

IF1

IF2

ID

EX

M1

M2

WB

i+8

IF1

IF2

ID

EX

M1

M2

i+9

IF1

IF2

ID

EX

M1

M2

Branch Delay? Load Delay? Forwarding IDs? Read/Write ports?

Computer Science 146
David Brooks

Putting things together

Talked about these things independently
Instruction Fetch
Branch Prediction (fill scheduler with instruction + multiple
instruction per cycle)

Scheduling/Hazard elimination
Dynamic Scheduling with Tomasulo (RAW Hazards)
Register Renaming (WAR and WAW Hazards)

Multiple functional units, register file ports

Potentially can reduce CPI < 1

Speculative Execution
Precise Interrupts
Memory systems (later this semester)
Computer Science 146
David Brooks

Focus on Speculation/Interrupts
Precise Interrupts
All instructions before interrupt must complete
All instructions after interrupt must seem to never start

Speculation (similar problem!)

If branch prediction is wrong, could update state
incorrectly leading to wrong program behavior

Out-of-Order completion
Post-interrupt/mispredict writebacks change state
Does Out-of-Order scheduling require this?
Computer Science 146
David Brooks

Solving both problems with one

solution
Need the ability to squash/restart any instruction
Gives us precise state
Need for memory ops (page faults, etc)
Need for FP ops (divide by 0)

Gives us ability to recover mis-speculations

Need for branches

Providing precise state solves both these problems

Computer Science 146

David Brooks

How to get precise state?

Imprecise state
As weve said this is a bad idea
For speculation it is unacceptable

Force in-order completion at WB (stall when

necessary)
Precise state in software: save recovery info for traps
Traps on all faulting memory, FP, and mis-predicted
branch ops?

Precise state in hardware: save recovery info online

Computer Science 146
David Brooks

Solution: Writeback and Commit

Allow out of order issue/writeback
Require in-order commit when instruction is no longer
speculative
Prevent speculative changes from changing state
e.g. memory write or register write

Collect pre-commit instructions

in a reorder buffer
holds completed but not committed instruction

Effectively contains a set of virtual registers

similar to a reservation station
and becomes a bypass (forwarding) source
Computer Science 146
David Brooks

Reorder Buffer: HW buffer for

results of uncommitted instructions
3 fields: instr, destination, value
Reorder buffer can be operand source
=> more registers like RS
Use reorder buffer number instead of
FP
reservation station when execution
Op
completes
Queue
Supplies operands between execution
complete & commit
Once operand commits,
result is put into register
Res Stations
Instructions commit
FP Adder
As a result, its easy to undo speculated
instructions
on mispredicted branches
or on exceptions

Reorder
Buffer

FP Regs

Res Stations
FP Adder

Computer Science 146

David Brooks

10

Four Steps of Speculative

Tomasulo Algorithm
1. Issueget instruction from FP Op Queue
If reservation station and reorder buffer slot free, issue instr & send operands
& reorder buffer no. for destination (this stage sometimes called dispatch)

2. Executionoperate on operands (EX)

When both operands ready then execute; if not ready, watch CDB for result;
when both in reservation station, execute; checks RAW (sometimes called
issue)

3. Write resultfinish execution (WB)

Write on Common Data Bus to all awaiting FUs & reorder buffer; mark
reservation station available. (tags are now ROB #s not RS #s)

4. Commitupdate register with reorder result

When instr. at head of reorder buffer & result present, update register with
result (or store to memory) and remove instr from reorder buffer. Mispredicted
branch flushes reorder buffer (sometimes called graduation)
Computer Science 146
David Brooks

Tomasulo With Reorder Buffer - Cycle 0

Time Name

Busy

Op

Vj

Vk

Qj

Qk

Dest

Add1

No

Reservation

Add2

No

Stations

Add3

No

Mult1

No

Mult2

No
Busy
Entry Busy

Instruction

State

Destination Value

Address

Load1

Load2

Load3

3
4
5
6

Reorder Buffer

7
8
9
10
F0

F2

F4

F6

F8

F10

F12

Busy no

no

no

no

no

no

no

...

F30

Reorder #
no

11

Tomasulo With Reorder Buffer - Cycle 1

Time Name

Busy

Op

Vj

Vk

Qj

Qk

Dest

Add1

No

Reservation

Add2

No

Stations

Add3

No

Mult1

No

Mult2

No
Busy
Entry Busy
1

Yes

Instruction

State

Destination Value

Load1 Yes

LD F6, 34(R2)

Issue

F6

Load2

Address
34+Regs[R2]

Load3

3
4
5
6

Reorder Buffer

7
8
9
10
F0

F2

F4

Reorder #

F6

F8

F10

F12

no

no

no

...

F30

#1

Busy no

no

no

Yes

no

Tomasulo With Reorder Buffer - Cycle 2

Time Name

Busy

Op

Vj

Vk

Qj

Qk

Dest

Add1

No

Reservation

Add2

No

Stations

Add3

No

Mult1

No

Mult2

No
Busy
Entry Busy

head
tail

Address

Instruction

State

Destination Value

Load1 Yes

34+Regs[R2]
45+Regs[R3]

Yes

LD F6, 34(R2)

Ex1

F6

Load2 Yes

Yes

LD F2, 45(R3)

Issue

F2

Load3

3
4
5
6

Reorder Buffer

7
8
9
10
F0
Reorder #
Busy no

F2

F4

#2
Yes

F6

F8

F10

F12

no

no

no

...

F30

#1
no

Yes

no

12

Tomasulo With Reorder Buffer - Cycle 3

Time Name

Busy

Op

Vj

Vk

Qj

Qk

Dest

Add1

No

Reservation

Add2

No

Stations

Add3

No

Mult1

Yes

Mult2

No

Mult

Regs[F4]

#2

#3

Instruction

State

Destination Value

Load1 No
Load2 Yes

Busy
Entry Busy

head
tail

Yes

LD F6, 34(R2)

write

F6

Yes

LD F2, 45(R3)

Ex1

F2

Yes

MULT F0, F2, F4

Issue

F0

Mem[load1]

Address
45+Regs[R3]

Load3

4
5
6

Reorder Buffer

7
8
9
10
F0

F2

Reorder # #3

#2

Busy Yes

Yes

F4

F6

F8

F10

F12

no

no

no

...

F30

#1
no

Yes

no

Tomasulo With Reorder Buffer - Cycle 4

Time Name

Busy

Op

Vj

Vk

Add1

Yes

SUB

Regs[F6]

Mem[45+Regs[R3]]

Qj

Add2

No

Add3

No

Mult1

Yes

Mult2

No

Qk

Dest
#4

Reservation
Stations

Mult

Mem[45+Regs[R3]] Regs[F4]

#3
Busy

Entry Busy

head
tail

Instruction

State

Destination Value

Load1 No

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

Yes

LD F2, 45(R3)

write

F2

Mem[load2]

Load3

Yes

MULT F0, F2, F4

EX1

F0

Yes

SUBD F8, F6, F2

Issue

F8

Address

5
6

Reorder Buffer

7
8
9
10
F0

F2

Reorder # #3

#2

Busy Yes

Yes

F4

F6

F8

F10

F12

no

no

...

F30

#4
no

no

Yes

no

13

Tomasulo With Reorder Buffer - Cycle 5

Time Name

Busy

Op

Vj

Vk

Add1

Yes

SUB

Regs[F6]

Mem[45+Regs[R3]]

Qj

Add2

No

Add3

No

Mult1

Yes

Mult

Mult2

Yes

DIV

Qk

Dest
#4

Reservation
Stations

Mem[45+Regs[R3]] Regs[F4]

#3

Regs[F6]

#3

#5

Instruction

State

Destination Value

Load1 No

Busy
Entry Busy

head
tail

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

No

LD F2, 45(R3)

commit

F2

Mem[load2]

Load3

Yes

MULT F0, F2, F4

Ex2

F0

Yes

SUBD F8, F6, F2

Ex1

F8

Yes

DIVD F10, F0, F6

Issue

F10

Address

Reorder Buffer

7
8
9
10
F0

F2

F4

F6

Reorder # #3
Busy Yes

no

no

no

F8

F10

#4

#5

Yes

Yes

F12

...

no

F30
no

Tomasulo With Reorder Buffer - Cycle 6

Time Name

Busy

Op

Vj

Vk

Add1

Yes

SUB

Regs[F6]

Mem[45+Regs[R3]]

Qj

Qk

Dest
#4

Reservation

Add2

Yes

Add

Regs[F2]

#4

#6

Stations

Add3

No

Mult1

Yes

Mult

Mem[45+Regs[R3]] Regs[F4]

Mult2

Yes

DIV

Regs[F6]

#3

#5

Instruction

State

Destination Value

Load1 No

#3
Busy

Entry Busy

head

tail

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

No

LD F2, 45(R3)

commit

F2

Mem[load2]

Load3

Yes

MULT F0, F2, F4

Ex3

F0

Yes

SUBD F8, F6, F2

Ex2

F8

Yes

DIVD F10, F0, F6

Issue

F10

Yes

ADDD F6, F8, F2

Issue

F6

F2

F4

F6

F8

F10

#6

#4

#5

Yes

Yes

Yes

Address

Reorder Buffer

7
8
9
10
F0
Reorder # #3
Busy Yes

no

no

F12
no

...

F30
no

14

Tomasulo With Reorder Buffer - Cycle 7

Time Name

Busy

Op

Vj

Vk

Add

#4

Regs[F2]

Yes

Mult

Mem[45+Regs[R3]] Regs[F4]

Yes

DIV

Add1

No

Add2

Yes

Add3

No

Mult1

Mult2

Qj

Qk

Dest
Reservation
#6

Stations

#3

Regs[F6]

#3

#5

Instruction

State

Destination Value

Load1 No

Busy
Entry Busy

head

tail

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

No

LD F2, 45(R3)

commit

F2

Mem[load2]

Load3

Yes

MULT F0, F2, F4

Ex4

F0

Yes

SUBD F8, F6, F2

write

F8

Yes

DIVD F10, F0, F6

Issue

F10

Yes

ADDD F6, F8, F2

EX1

F6

F2

F4

F6

F8

F10

#6

#4

#5

Yes

Yes

Yes

Address

F6 - #2
Reorder Buffer

7
8
9
10
F0
Reorder # #3
Busy Yes

no

no

F12

...

no

F30
no

Tomasulo With Reorder Buffer - Cycle 8

Time Name

Busy

Op

Vj

Vk

Add

#4

Regs[F2]

Yes

Mult

Mem[45+Regs[R3]] Regs[F4]

Yes

DIV

Add1

No

Add2

Yes

Add3

No

Mult1

Mult2

Qj

Qk

Dest
Reservation
#6

Stations

#3

Regs[F6]

#3

#5

Instruction

State

Destination Value

Load1 No

Busy
Entry Busy

head

tail

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

No

LD F2, 45(R3)

commit

F2

Mem[load2]

Load3

Yes

MULT F0, F2, F4

Ex5

F0

Yes

SUBD F8, F6, F2

write

F8

Yes

DIVD F10, F0, F6

Issue

F10

Yes

ADDD F6, F8, F2

Ex2

F6

F2

F4

F6

F8

F10

#6

#4

#5

Yes

Yes

Yes

Address

F6 - #2
Reorder Buffer

7
8
9
10
F0
Reorder # #3
Busy Yes

no

no

F12
no

...

F30
no

15

Tomasulo With Reorder Buffer - Cycle 9

Time Name

Busy

Op

Vj

Vk

Add

#4

Regs[F2]

Yes

Mult

Mem[45+Regs[R3]] Regs[F4]

Yes

DIV

Add1

No

Add2

Yes

Add3

No

Mult1

Mult2

Qj

Qk

Dest
Reservation
#6

Stations

#3

Regs[F6]

#3

#5

Instruction

State

Destination Value

Load1 No

Busy
Entry Busy

head

tail

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

No

LD F2, 45(R3)

commit

F2

Mem[load2]

Load3

Yes

MULT F0, F2, F4

Ex6

F0

Yes

SUBD F8, F6, F2

write

F8

Yes

DIVD F10, F0, F6

Issue

F10

Yes

ADDD F6, F8, F2

write

F6

#4 + F2

Reorder Buffer

F2

F4

F6

F8

F10

F12

#6

#4

#5

Yes

Yes

Yes

Address

F6 - #2

7
8
9
10
F0
Reorder # #3
Busy Yes

no

no

...

no

F30
no

Tomasulo With Reorder Buffer - Cycle 10

Time Name

Busy

Op

Vj

Vk

Qj

Qk

Dest

Add1

No

Reservation

Add2

No

Stations

Add3

No

Mult1

Yes

Mult

Mult2

Yes

DIV

Mem[45+Regs[R3]] Regs[F4]

#3

Regs[F6]

#3

#5

Instruction

State

Destination Value

Load1 No

Busy
Entry Busy

head

tail

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

No

LD F2, 45(R3)

commit

F2

Mem[load2]

Load3

Yes

MULT F0, F2, F4

Ex7

F0

Yes

SUBD F8, F6, F2

write

F8

Yes

DIVD F10, F0, F6

Issue

F10

Yes

ADDD F6, F8, F2

write

F6

#4 + F2

Reorder Buffer

F2

F4

F6

F8

F10

F12

#6

#4

#5

Yes

Yes

Yes

Address

F6 - #2

7
8
9
10
F0
Reorder # #3
Busy Yes

no

no

no

...

F30
no

16

Tomasulo With Reorder Buffer - Cycle 11

Time Name

Busy

Op

Vj

Vk

Qj

Qk

Dest

Add1

No

Reservation

Add2

No

Stations

Add3

No

Mult1

Yes

Mult

Mult2

Yes

DIV

Mem[45+Regs[R3]] Regs[F4]

#3

Regs[F6]

#3

#5

Instruction

State

Destination Value

Load1 No

Busy
Entry Busy

head

tail

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

No

LD F2, 45(R3)

commit

F2

Mem[load2]

Load3

Yes

MULT F0, F2, F4

Ex8

F0

Yes

SUBD F8, F6, F2

write

F8

Yes

DIVD F10, F0, F6

Issue

F10

Yes

ADDD F6, F8, F2

write

F6

#4 + F2

Reorder Buffer

F2

F4

F6

F8

F10

F12

#6

#4

#5

Yes

Yes

Yes

Address

F6 - #2

7
8
9
10
F0
Reorder # #3
Busy Yes

no

no

...

no

F30
no

Tomasulo With Reorder Buffer - Cycle 12

Time Name

Busy

Op

Vj

Vk

Qj

Qk

Dest

Add1

No

Reservation

Add2

No

Stations

Add3

No

Mult1

Yes

Mult

Mult2

Yes

DIV

Mem[45+Regs[R3]] Regs[F4]

#3

Regs[F6]

#3

#5

Instruction

State

Destination Value

Load1 No

Busy
Entry Busy

head

tail

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

No

LD F2, 45(R3)

commit

F2

Mem[load2]

Load3

Yes

MULT F0, F2, F4

Ex9

F0

Yes

SUBD F8, F6, F2

write

F8

Yes

DIVD F10, F0, F6

Issue

F10

Yes

ADDD F6, F8, F2

write

F6

#4 + F2

Reorder Buffer

F2

F4

F6

F8

F10

F12

#6

#4

#5

Yes

Yes

Yes

Address

F6 - #2

7
8
9
10
F0
Reorder # #3
Busy Yes

no

no

no

...

F30
no

17

Tomasulo With Reorder Buffer - Cycle 13

Time Name

Busy

Op

Vj

Vk

Qj

Qk

Dest

Add1

No

Reservation

Add2

No

Stations

Add3

No

Mult1

No

Mult2

Yes

DIV

#2xRegs[F4]

Regs[F6]

#5

Instruction

State

Destination Value

Load1 No

Busy
Entry Busy

head

tail

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

No

LD F2, 45(R3)

commit

F2

Mem[load2]

Load3

Yes

MULT F0, F2, F4

write

F0

#2 x Regs[F4]

Yes

SUBD F8, F6, F2

write

F8

F6 - #2

Yes

DIVD F10, F0, F6

Ex1

F10

Yes

ADDD F6, F8, F2

write

F6

#4 + F2

Address

Reorder Buffer

Figure 3.30

P 230

9
10
F0

F2

F4

Reorder # #3
Busy Yes

no

no

F6

F8

F10

#6

#4

#5

Yes

Yes

Yes

F12

...

no

F30
no

Tomasulo With Reorder Buffer - Cycle 14

Time Name

Busy

Op

Vj

Vk

Qj

Qk

Dest

Add1

No

Reservation

Add2

No

Stations

Add3

No

Mult1

No

Mult2

Yes

DIV

#2xRegs[F4]

Regs[F6]

#5

Instruction

State

Destination Value

Load1 No

Busy
Entry Busy

head
tail

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

No

LD F2, 45(R3)

commit

F2

Mem[load2]

Load3

No

MULT F0, F2, F4

commit

F0

#2 x Regs[F4]

Yes

SUBD F8, F6, F2

write

F8

F6 - #2

Yes

DIVD F10, F0, F6

Ex2

F10

Yes

ADDD F6, F8, F2

write

F6

#4 + F2

Reorder Buffer

F2

F4

F6

F8

F10

F12

#6

#4

#5

Yes

Yes

Yes

Address

7
8
9
10
F0
Reorder #
Busy No

no

no

no

...

F30
no

18

Tomasulo With Reorder Buffer - Cycle 15

Time Name

Busy

Op

Vj

Vk

Qj

Qk

Dest

Add1

No

Reservation

Add2

No

Stations

Add3

No

Mult1

No

Mult2

Yes

DIV

#2xRegs[F4]

Regs[F6]

#5

Instruction

State

Destination Value

Load1 No

Busy
Entry Busy

head
tail

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

No

LD F2, 45(R3)

commit

F2

Mem[load2]

Load3

No

MULT F0, F2, F4

commit

F0

#2 x Regs[F4]

No

SUBD F8, F6, F2

commit

F8

F6 - #2

Yes

DIVD F10, F0, F6

Ex3

F10

Yes

ADDD F6, F8, F2

write

F6

#4 + F2

Reorder Buffer

F2

F4

F6

F8

F10

F12

Address

7
8
9
10
F0
Reorder #

#6

Busy no

no

no

Yes

...

F30

#5
no

Yes

no

no

Tomasulo With Reorder Buffer - Cycle 16

Time Name

Busy

Op

Vj

Vk

Qj

Qk

Dest

Add1

No

Reservation

Add2

No

Stations

Add3

No

Mult1

No

Mult2

Yes

DIV

#2xRegs[F4]

Regs[F6]

#5

Instruction

State

Destination Value

Load1 No

Busy
Entry Busy

head
tail

No

LD F6, 34(R2)

commit

F6

Mem[load1]

Load2 No

No

LD F2, 45(R3)

commit

F2

Mem[load2]

Load3

No

MULT F0, F2, F4

commit

F0

#2 x Regs[F4]

No

SUBD F8, F6, F2

commit

F8

F6 - #2

Yes

DIVD F10, F0, F6

Ex4

F10

Yes

ADDD F6, F8, F2

write

F6

#4 + F2

Reorder Buffer

Need 36 more
EX cycles for
DIV to finish

8
9
10
F0

F2

F4

Reorder #
Busy no

Address

F6

F8

#6
no

no

Yes

F10

F12

...

F30

#5
no

Yes

no

no

19

Tomasulo With Reorder Buffer:

Summary
Instruction

Issue

Exec Comp

Writeback

Commit

LD F6, 34(R2)

LD F2, 45(R3)

MULT F0, F2, F4

12

13

14

SUBD F8, F6, F2

15

DIVD F10, F0, F6

52

53

54

ADDD F6, F8, F2

55

In-order Issue/Commit, Out-of-Order Execution/Writeback

Computer Science 146
David Brooks

Precise State with ROB

ROB maintains precise state and allows
speculation
Waits until precise condition reaches retire/commit
stage
(Or until branch is noted mis-predicted)
Clear ROB, RS, and register status table (Flush)
Service exception/Restart from True Branch target

Need to do similar things with memory ops

Called Memory Ordering Buffer (MOB)
Completed stores write to MOB then complete (write to
memory) in-order (when they read head of buffer)
Computer Science 146
David Brooks

20

Example of Speculative State of Reorder Buffer

0

Add1

No

Reservation

Add2

No

Stations

Add3

No

Mult1

No

MULT

Mem[0+Regs[R1]]

Regs[F2]

#2

Mult2

No

MULT

Mem[0+Regs[R1]]

Regs[F2]

#7

Instruction

State

Destination Value

Load1 No

Busy
Entry Busy

First
loop

Second
loop

No

LD F0, 0(R1)

commit

F0

Mem[0+R1]

Load2 No

No

MULT F4, F0, F2

commit

F4

F0 x F2

Load3

Yes

SD 0(R1), F4

write

0+Reg[R1]

#2

Yes

SUBI R1, R1, 8

write

R1

R1 - 8

Yes

BNEZ R1, Loop

write

Yes

LD F0, 0(R1)

write

F0

Mem[#4]

Yes

MULT F4, F0, F2

write

F4

#6 X F2

Yes

SD 0(R1), F4

write

0+Regs[R1] #7

Yes

SUBI R1, R1, 8

write

R1

#4 - 8

10

Yes

BNEZ R1, Loop

write

F2

F4

F6

F8

F10

F12

no

no

no

no

F0
Reorder # 6

Address

Reorder Buffer

...

F30

Busy yes

no

yes

no

Multiply has just reached commit, so other instructions can start committing

Tomasulo + ROB Summary

Many implementations are very similar
Pentium III, PowerPC, etc

Some limitations
Too many value copy operations
Register file => RS => ROB => Register File

Too many muxes/busses (CDB)

Values are coming from everywhere to everywhere else!

Reservation Stations mix values(data) and tags(control)

Slows down the max clock frequency

Computer Science 146

David Brooks

21

For next time

Case Studies (P6, Pentium 4, MIPS R10K)
Limits of ILP

Computer Science 146

David Brooks

22