The Big Picture

SPEC

Requirements
f1 f2 f3 f4 s p q fp

Problem Focus i
j f3

Algorithms
f2() { f3(s2, &j, &i); *s2->p = 10; i = *s2->q + i; }

f5

Prog. Lang./OS ISA uArch

i1: i2: i3: i4: i5:

ld r1, b ld r2, c ld r5, z mul r6, r5, 3 add r3, r1, r2

<p1> <p1> <p3> <p3> <p1>

Circuit
Gate
Source Drain

Performance Focus

BOX

Si fin - Body!

Device
Appendix A - Pipelining 1

Instruction Set Architecture

Application

Instruction Set Architecture

SPARC MIPS ARM x86 HP-PA IA-64

Implementation

Intel Pentium X AMD K6, Athlon, Opteron Transmeta Crusoe TM5x00 Appendix A - Pipelining 2

Instruction Set Architecture

Strong influence on cost/performance
New ISAs are rare, but versions are not
16-bit, 32-bit and 64-bit X86 versions

Longevity is a strong function of marketing prowess

Appendix A - Pipelining 3

Traditional Issues
Strongly constrained by the number of bits available to instruction encoding Opcodes/operands Registers/memory Addressing modes Orthogonality 0, 1, 2, 3 address machines Instruction formats Decoding uniformity
Appendix A - Pipelining 4

Introduction
A.1 What is Pipelining?

A.2 The Major Hurdle of Pipelining-Structural Hazards

Data Hazards Control Hazards

A.3 How is Pipelining Implemented

A.4 What Makes Pipelining Hard to Implement?

A.5 Extending the MIPS Pipeline to Handle Multi-cycle Operations

Appendix A - Pipelining

What Is Pipelining
Laundry Example Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, and fold Washer takes 30 minutes Dryer takes 40 minutes Folder takes 20 minutes
Appendix A - Pipelining 6

What Is Pipelining
6 PM

9
Time

10

11

Midnight

30 40 20 30 40 20 30 40 20 30 40 20
T a s k O r d e r

A B C D

Sequential laundry takes 6 hours for 4 loads If they learned pipelining, how long would laundry take?
Appendix A - Pipelining 7

What Is Pipelining Start work ASAP

6 PM 7 8 9
Time

10

11

Midnight

30 40
T a s k O r d e r

40

40

40 20 Pipelined laundry takes 3.5 hours for 4 loads

B
C D

Appendix A - Pipelining

What Is Pipelining
6 PM
T a s k O r d e r

Pipelining Lessons
8 9
Time Pipelining doesnt help latency of single task, it helps throughput of entire workload Pipeline rate limited by slowest pipeline stage Multiple tasks operating simultaneously Potential speedup = Number pipe stages Unbalanced lengths of pipe stages reduces speedup Time to fill pipeline and time to drain it reduces speedup

30 40
A B C

40

40

40 20

D
Appendix A - Pipelining 9

What Is Pipelining
Instruction Fetch Instr. Decode Reg. Fetch

MIPS Without Pipelining

Execute Addr. Calc Memory Access Write Back

IR

L M D

Appendix A - Pipelining

10

What Is Pipelining
Instruction Fetch Instr. Decode Reg. Fetch Execute Addr. Calc

MIPS Functions
Memory Access Write Back

IR

L M D

Passed To Next Stage IR <- Mem[PC] NPC <- PC + 4

Instruction Fetch (IF):

Send out the PC and fetch the instruction from memory into the instruction register (IR); increment the PC by 4 to address the next sequential instruction. IR holds the instruction that will be used in the next stage. NPC holds the value of the next PC.

Appendix A - Pipelining

11

What Is Pipelining
Instruction Fetch Instr. Decode Reg. Fetch Execute Addr. Calc

MIPS Functions
Memory Access Write Back

IR

L M D

Passed To Next Stage A <- Regs[IR6..IR10]; B <- Regs[IR10..IR15]; Imm <- ((IR16) ##IR16-31

Instruction Decode/Register Fetch Cycle (ID):

Decode the instruction and access the register file to read the registers. The outputs of the general purpose registers are read into two temporary registers (A & B) for use in later clock cycles. We extend the sign of the lower 16 bits of the Instruction Register.

Appendix A - Pipelining

12

What Is Pipelining
Instruction Fetch Instr. Decode Reg. Fetch Execute Addr. Calc

MIPS Functions
Memory Access Write Back

IR

L M D

Passed To Next Stage A <- A func. B cond = 0;

Execute Address Calculation (EX):

We perform an operation (for an ALU) or an address calculation (if its a load or a Branch). If an ALU, actually do the operation. If an address calculation, figure out how to obtain the address and stash away the location of that address for the next cycle. Appendix A - Pipelining 13

What Is Pipelining
Instruction Fetch Instr. Decode Reg. Fetch Execute Addr. Calc

MIPS Functions
Memory Access Write Back

IR

L M D

Passed To Next Stage A = Mem[prev. B] or Mem[prev. B] = A

MEMORY ACCESS (MEM):

If this is an ALU, do nothing. If a load or store, then access memory.

Appendix A - Pipelining

14

What Is Pipelining
Instruction Fetch Instr. Decode Reg. Fetch Execute Addr. Calc

MIPS Functions
Memory Access Write Back

Passed To Next Stage Regs <- A, B; L M D

IR

WRITE BACK (WB):

Update the registers from either the ALU or from the data loaded.

Appendix A - Pipelining

15

The Basic Pipeline For MIPS

Latches between each stage provide pipelining.

Appendix A - Pipelining

16

The Basic Pipeline For MIPS

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7
ALU

I n s t r.
O r d e r

Ifetch

Reg

DMem

Reg

ALU

Ifetch

Reg

DMem

Reg

ALU

Ifetch

Reg

DMem

Reg

ALU

Ifetch

Reg

DMem

Reg

Figure 3.3
Appendix A - Pipelining 17

Pipeline Hurdles
A.1 What is Pipelining? A.2 The Major Hurdle of PipeliningStructural Hazards -- Structural Hazards Data Hazards Control Hazards

A.3 How is Pipelining Implemented A.4 What Makes Pipelining Hard to Implement?

A.5 Extending the MIPS Pipeline to Handle Multi-cycle Operations

Limits to pipelining: Hazards prevent next instruction from executing during its designated clock cycle Structural hazards: HW cannot support this combination of instructions (single person to fold and put clothes away) Data hazards: Instruction depends on result of prior instruction still in the pipeline (missing sock) Control hazards: Pipelining of branches & other instructions that change the PC Common solution is to stall the pipeline until the hazard is resolved, inserting one or more bubbles in the pipeline

Appendix A - Pipelining

18

Pipeline Hurdles
Definition conditions that lead to incorrect behavior if not fixed Structural hazard two different instructions use same h/w in same cycle Data hazard two different instructions use same storage must appear as if the instructions execute in correct order Control hazard one instruction affects which instruction is next Resolution Pipeline interlock logic detects hazards and fixes them simple solution: stall increases CPI, decreases performance better solution: partial stall some instruction stall, others proceed better to stall early than late

Appendix A - Pipelining

19

Structural Hazards
Time (clock cycles)
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7

ALU

I Load Ifetch n s Instr 1 t r.

Reg

DMem

Reg

When two or more different instructions want to use same hardware resource in same cycle e.g., MEM uses the same memory port as IF as shown in this slide.
Reg

ALU

ALU

Ifetch

Reg

DMem

Reg

Instr 2

Ifetch

Reg

DMem

Reg

O r d e r

Instr 3
Instr 4
Figure 3.6

Ifetch

Reg

ALU

DMem

ALU

Ifetch

Reg

DMem

Reg

Appendix A - Pipelining

20

Structural Hazards
Time (clock cycles)
Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7
This is another way of looking at the effect of a stall.

ALU

I Load Ifetch n s Instr 1 t r. O r d e r

ALU

Reg

DMem

Reg

ALU

Ifetch

Reg

DMem

Reg

Instr 2
Stall Instr 3
Figure 3.7

Ifetch

Reg

DMem

Reg

Bubble

Bubble Bubble

Bubble
ALU

Bubble
Reg

Ifetch

Reg

DMem

Appendix A - Pipelining

21

Structural Hazards

This is another way to represent the stall we saw on the last few pages.

Appendix A - Pipelining

22

Structural Hazards
Dealing with Structural Hazards
Stall low cost, simple Increases CPI use for rare case since stalling has performance effect Pipeline hardware resource useful for multi-cycle resources good performance sometimes complex e.g., RAM Replicate resource good performance increases cost (+ maybe interconnect delay) useful for cheap or divisible resources

Appendix A - Pipelining

23

Structural Hazards
Structural hazards are reduced with these rules: Each instruction uses a resource at most once Always use the resource in the same pipeline stage Use the resource for one cycle only Many RISC ISAa designed with this in mind Sometimes very complex to do this. For example, memory of necessity is used in the IF and MEM stages.

Some common Structural Hazards: Memory - weve already mentioned this one. Floating point - Since many floating point instructions require many cycles, its easy for them to interfere with each other. Starting up more of one type of instruction than there are resources. For instance, the PA-8600 can support two ALU + two load/store instructions per cycle - thats how much hardware it has available.

Appendix A - Pipelining

24

Structural Hazards
This is the example on Page 144.
We want to compare the performance of two machines. Which machine is faster? Machine A: Dual ported memory - so there are no memory stalls Machine B: Single ported memory, but its pipelined implementation has a 1.05 times faster clock rate Assume: Ideal CPI = 1 for both Loads are 40% of instructions executed

SpeedUpA = Pipeline Depth/(1 + 0) x (clockunpipe/clockpipe) = Pipeline Depth SpeedUpB = Pipeline Depth/(1 + 0.4 x 1) x (clockunpipe/(clockunpipe / 1.05) = (Pipeline Depth/1.4) x 1.05 = 0.75 x Pipeline Depth SpeedUpA / SpeedUpB = Pipeline Depth / (0.75 x Pipeline Depth) = 1.33
Machine A is 1.33 times faster

Appendix A - Pipelining

25

Data Hazards
A.1 What is Pipelining? A.2 The Major Hurdle of PipeliningStructural Hazards -- Structural Hazards Data Hazards Control Hazards

These occur when at any time, there are instructions active that need to access the same data (memory or register) locations. Where theres real trouble is when we have:

A.3 How is Pipelining Implemented A.4 What Makes Pipelining Hard to Implement?

A.5 Extending the MIPS Pipeline to Handle Multi-cycle Operations

instruction A instruction B
and B manipulates (reads or writes) data before A does. This violates the order of the instructions, since the architecture implies that A completes entirely before B is executed.

Appendix A - Pipelining

26

Data Hazards
Execution Order is: InstrI InstrJ

Read After Write (RAW) InstrJ tries to read operand before InstrI writes it

I: add r1,r2,r3 J: sub r4,r1,r3

Caused by a Dependence (in compiler nomenclature). This hazard results from an actual need for communication.

Appendix A - Pipelining

27

Data Hazards
Execution Order is: InstrI InstrJ

Write After Read (WAR) InstrJ tries to write operand before InstrI reads i Gets wrong operand

I: sub r4,r1,r3 J: add r1,r2,r3 K: mul r6,r1,r7

Called an anti-dependence by compiler writers. This results from reuse of the name r1. Cant happen in MIPS 5 stage pipeline because: All instructions take 5 stages, and Reads are always in stage 2, and Writes are always in stage 5

Appendix A - Pipelining

28

Data Hazards
Execution Order is: InstrI InstrJ

Write After Write (WAW) InstrJ tries to write operand before InstrI writes it Leaves wrong result ( InstrI not InstrJ )

I: sub r1,r4,r3 J: add r1,r2,r3 K: mul r6,r1,r7

Called an output dependence by compiler writers This also results from the reuse of name r1.
Cant happen in MIPS 5 stage pipeline because: All instructions take 5 stages, and Writes are always in stage 5 Will see WAR and WAW in later more complicated pipes

Appendix A - Pipelining

29

Data Hazards
Simple Solution to RAW Hardware detects RAW and stalls Assumes register written then read each cycle + low cost to implement, simple -- reduces IPC Try to minimize stalls

Minimizing RAW stalls

Bypass/forward/shortcircuit (We will use the word forward) Use data before it is in the register + reduces/avoids stalls -- complex Crucial for common RAW hazards

Appendix A - Pipelining

30

Data Hazards
Time (clock cycles)
IF ID/RF EX
MEM WB
DMem Reg

ALU

I n s t r. O r d e r

add r1,r2,r3Ifetch sub r4,r1,r3

Reg

ALU

ALU

Ifetch

Reg

DMem

Reg

and r6,r1,r7
or r8,r1,r9

Ifetch

Reg

DMem

Reg

ALU

Ifetch

Reg

DMem

Reg

ALU

xor r10,r1,r11

Ifetch

Reg

DMem

Reg

The use of the result of the ADD instruction in the next three instructions causes a hazard, since the register is not written until after those instructions read it.

Figure 3.9

Appendix A - Pipelining

31

Data Hazards
Forwarding To Avoid Data Hazard
I n s t r. O r d e r

Forwarding is the concept of making data available to the input of the ALU for subsequent instructions, even though the generating instruction hasnt gotten to WB in order to write the memory or registers.

Time (clock cycles)

ALU Reg DMem Reg

add r1,r2,r3 Ifetch

sub r4,r1,r3 and r6,r1,r7 or r8,r1,r9

ALU

Ifetch

Reg

DMem

Reg

ALU

Ifetch

Reg

DMem

Reg

ALU

Ifetch

Reg

DMem

Reg

ALU

xor r10,r1,r11
Figure 3.10
Appendix A - Pipelining

Ifetch

Reg

DMem

Reg

32

Data Hazards
Time (clock cycles)

The data isnt loaded until after the MEM stage.

I n s t r.
O r d e r

lw r1, 0(r2) Ifetch

sub r4,r1,r6 and r6,r1,r7 or r8,r1,r9

ALU

Reg

DMem

Reg

ALU

Ifetch

Reg

DMem

Reg

ALU

Ifetch

Reg

DMem

Reg

ALU

Ifetch

Reg

DMem

Reg

There are some instances where hazards occur, even with forwarding. Figure 3.12
Appendix A - Pipelining 33

Data Hazards
Time (clock cycles) I n s t r. O r d e r

The stall is necessary as shown here.

ALU

lw r1, 0(r2) sub r4,r1,r6 and r6,r1,r7

Ifetch

Reg

DMem

Reg

Ifetch

Reg

Bubble

ALU

DMem

Reg

ALU

Ifetch

Bubble

Reg

DMem

Reg

Ifetch

or r8,r1,r9

Reg

ALU

Bubble

DMem

There are some instances where hazards occur, even with forwarding. Figure 3.13
Appendix A - Pipelining 34

Data Hazards
LW R1, 0(R2) IF ID IF EX ID IF MEM EX ID SUB R4, R1, R5 AND R6, R1, R7

This is another representation of the stall.

WB MEM EX WB MEM WB

OR

R8, R1, R9

IF

ID

EX

MEM

WB

LW

R1, 0(R2)

IF

ID IF

EX ID IF

MEM stall stall stall

WB EX ID IF MEM EX ID WB MEM EX WB MEM WB

SUB R4, R1, R5 AND R6, R1, R7 OR R8, R1, R9

Appendix A - Pipelining

35

Data Hazards
lw Rb, b lw Rc, c Add Ra, Rb, Rc sw a, Ra lw Re, e lw Rf, f sub Rd, Re, Rf sw d, Rd

Pipeline Scheduling

Instruction scheduled by compiler - move instruction in order to reduce stall. code sequence for a = b+c before scheduling

stall
code sequence for d = e+f before scheduling stall

Arrangement of code after scheduling. lw Rb, b lw Rc, c lw Re, e Add Ra, Rb, Rc lw Rf, f sw a, Ra sub Rd, Re, Rf sw d, Rd

Appendix A - Pipelining

36

Data Hazards
scheduled gcc spice tex 0% 20% 14%

Pipeline Scheduling

unscheduled 54%

31% 42% 65% 25% 40% 60% 80%

% loads stalling pipeline

Appendix A - Pipelining

37

Control Hazards
A.1 What is Pipelining? A.2 The Major Hurdle of PipeliningStructural Hazards -- Structural Hazards Data Hazards Control Hazards

A.3 How is Pipelining Implemented A.4 What Makes Pipelining Hard to Implement?

A control hazard is when we need to find the destination of a branch, and cant fetch any new instructions until we know that destination.

A.5 Extending the MIPS Pipeline to Handle Multi-cycle Operations

Appendix A - Pipelining

38

Control Hazards
10: beq r1,r3,36 14: and r2,r3,r5 18: or r6,r1,r7
ALU Ifetch Reg

Control Hazard on Branches Three Stage Stall

DMem Reg

ALU

Ifetch

Reg

DMem

Reg

ALU

Ifetch

Reg

DMem

Reg

ALU

22: add r8,r1,r9

36: xor r10,r1,r11

Ifetch

Reg

DMem

Reg

ALU

Ifetch

Reg

DMem

Reg

Appendix A - Pipelining

39

Control Hazards

Branch Stall Impact

If CPI = 1, 30% branch, Stall 3 cycles => new CPI = 1.9! (Whoa! How did we get that 1.9???) Two part solution to this dramatic increase: Determine branch taken or not sooner, AND Compute taken branch address earlier MIPS branch tests if register = 0 or ^ 0 MIPS Solution: Move Zero test to ID/RF stage Adder to calculate new PC in ID/RF stage must be fast can't afford to subtract compares with 0 are simple Greater-than, Less-than test signbit, but not-equal must OR all bits more general compares need ALU 1 clock cycle penalty for branch versus 3

In the next chapter, well look at ways to avoid the branch all together.

Appendix A - Pipelining

40

Control Hazards
#1: Stall until branch direction is clear

Five Branch Hazard Alternatives

#2: Predict Branch Not Taken Execute successor instructions in sequence Squash instructions in pipeline if branch actually taken Advantage of late pipeline state update 47% MIPS branches not taken on average PC+4 already calculated, so use it to get next instruction #3: Predict Branch Taken 53% MIPS branches taken on average But havent calculated branch target address in MIPS MIPS still incurs 1 cycle branch penalty Other machines: branch target known before outcome

Appendix A - Pipelining

41

Control Hazards
#4: Execute Both Paths

Five Branch Hazard Alternatives

#5: Delayed Branch Define branch to take place AFTER a following instruction branch instruction sequential successor1 sequential successor2 ........ sequential successorn branch target if taken

Branch delay of length n

1 slot delay allows proper decision and branch target address in 5 stage pipeline MIPS uses this

Appendix A - Pipelining

42

Control Hazards

Delayed Branch

Where to get instructions to fill branch delay slot? Before branch instruction From the target address: only valuable when branch taken From fall through: only valuable when branch not taken Cancelling branches allow more slots to be filled Compiler effectiveness for single branch delay slot: Fills about 60% of branch delay slots About 80% of instructions executed in branch delay slots useful in computation About 50% (60% x 80%) of slots usefully filled Delayed Branch downside: 7-8 stage pipelines, multiple instructions issued per clock (superscalar)

Appendix A - Pipelining

43

Control Hazards
Pipeline speedup =

Evaluating Branch Alternatives

Pipeline depth 1 +Branch frequency Branch penalty

Scheduling Branch scheme penalty Stall pipeline 3 Predict taken 1 Predict not taken 1 Delayed branch 0.5

CPI 1.42 1.14 1.09 1.07

speedup v. unpipelined 3.5 4.4 4.5 4.6

Speedup v. stall 1.0 1.26 1.29 1.31 65% change PC

Conditional & Unconditional = 14%,

Appendix A - Pipelining

44

Control Hazards

Pipelining Introduction Summary

Just overlap tasks, and easy if tasks are independent Speed Up Pipeline Depth; if ideal CPI is 1, then:
Pipeline Depth X 1 + Pipeline stall CPI Clock Cycle Unpipelined Clock Cycle Pipelined

Speedup =

Hazards limit performance on computers: Structural: need more HW resources Data (RAW,WAR,WAW): need forwarding, compiler scheduling Control: delayed branch, prediction

Appendix A - Pipelining

45

Control Hazards
The compiler can program what it thinks the branch direction will be. Here are the results when it does so.
70%
Frequency of Misprediction

Compiler Static Prediction of Taken/Untaken Branches

14% 12%
Misprediction Rate
doduc gcc

60% 50% 40% 30% 20% 10% 0%

compress alvinn

10% 8% 6% 4% 2% 0%
doduc gcc

espresso

hydro2d

ora

tomcatv

compress

espresso

mdl jsp2

swm256

hydro2d

ora

Always taken

Appendix A - Pipelining

Taken backwards Not Taken Forwards

46

swm256

mdl jsp2

tomcatv

alvinn

Control Hazards

Compiler Static Prediction of Taken/Untaken Branches

Improves strategy for placing instructions in delay slot Two strategies Backward branch predict taken, forward branch not taken Profile-based prediction: record branch behavior, predict branch based on prior run

Appendix A - Pipelining

47

Control Hazards
100000
Instructions per mispredicted branch

Evaluating Static Branch Prediction Strategies

Misprediction ignores frequency of branch Instructions between mispredicted branches is a better metric

10000 1000 100 10 1

compress alvinn

doduc

espresso

gcc

hydro2d

ora

mdljsp2

Profile-based

Direction-based

Appendix A - Pipelining

swm256

48

tomcatv

What Makes Pipelining Hard?

A.1 What is Pipelining? A.2 The Major Hurdle of PipeliningStructural Hazards Data Hazards Control Hazards

A.3 How is Pipelining Implemented A.4 What Makes Pipelining Hard to Implement? A.5 Extending the MIPS Pipeline to Handle Multi-cycle Operations

Appendix A - Pipelining

49

What Makes Pipelining Hard?

Examples of interrupts:
Power failing, Arithmetic overflow, I/O device request, OS call, Page fault

Interrupts cause great havoc!

There are 5 instructions executing in 5 stage pipeline when an interrupt occurs:

How to stop the pipeline? How to restart the pipeline? Who caused the interrupt?

Interrupts (also known as: faults, exceptions, traps) often require

surprise jump (to vectored address) linking return address saving of PSW (including CCs)

state change (e.g., to kernel mode)

Appendix A - Pipelining

50

What Makes Pipelining Hard?

Interrupts cause great havoc!

What happens on interrupt while in delay slot ? Next instruction is not sequential solution #1: save multiple PCs Save current and next PC Special return sequence, more complex hardware solution #2: single PC plus Branch delay bit PC points to branch instruction Stage IF ID EX MEM Problem that causes the interrupt Page fault on instruction fetch; misaligned memory access; memory-protection violation Undefined or illegal opcode Arithmetic interrupt Page fault on data fetch; misaligned memory access; memory-protection violation

Appendix A - Pipelining

51

What Makes Pipelining Hard?

Interrupts cause great havoc!

Simultaneous exceptions in more than one pipeline stage, e.g., Load with data page fault in MEM stage Add with instruction page fault in IF stage Add fault will happen BEFORE load fault Solution #1 Interrupt status vector per instruction Defer check until last stage, kill state update if exception Solution #2 Interrupt ASAP Restart everything that is incomplete Another advantage for state update late in pipeline!

Appendix A - Pipelining

52

What Makes Pipelining Hard?

Interrupts cause great havoc!

Heres what happens on a data page fault. 1 2 3 4 5 6 7 8 9 i F D X M W i+1 F D X M W < page fault i+2 F D X M W < squash i+3 F D X M W < squash i+4 F D X M W < squash i+5 trap > F D X M W i+6 trap handler > F D X M W

Appendix A - Pipelining

53

What Makes Pipelining Hard?

Complex Instructions

Complex Addressing Modes and Instructions Address modes: Autoincrement causes register change during instruction execution Interrupts? Need to restore register state Adds WAR and WAW hazards since writes are no longer the last stage. Memory-Memory Move Instructions Must be able to handle multiple page faults Long-lived instructions: partial state save on interrupt Condition Codes

Appendix A - Pipelining

54

Handling Multi-cycle Operations

A.1 What is Pipelining? A.2 The Major Hurdle of PipeliningStructural Hazards Data Hazards Control Hazards

Multi-cycle instructions also lead to pipeline complexity. A very lengthy instruction causes everything else in the pipeline to wait for it.

A.3 How is Pipelining Implemented A.4 What Makes Pipelining Hard to Implement? A.5 Extending the MIPS Pipeline to Handle Multi-cycle Operations

Appendix A - Pipelining

55

Multi-Cycle Operations

Floating Point

Floating point gives long execution time. This causes a stall of the pipeline. Its possible to pipeline the FP execution unit so it can initiate new instructions without waiting full latency. Can also have multiple FP units. FP Instruction Add, Subtract Multiply Divide Square root Negate Absolute value FP compare Latency 4 8 36 112 2 2 3 Initiation Rate 3 4 35 111 1 1 2

Appendix A - Pipelining

56

Multi-Cycle Operations

Floating Point

Divide, Square Root take 10X to 30X longer than Add Interrupts? Adds WAR and WAW hazards since pipelines are no longer same length
i I+1 I+2 I+3 I+4 I+5 I+6 1 IF 2 ID IF 3 EX ID IF 4 MEM EX ID IF 5 WB EX EX ID IF 6 EX MEM EX ID IF 7 EX WB EX EX ID IF 8 9 10 11

MEM WB EX MEM --EX WB --MEM WB EX ID EX EX

Notes: I + 2: no WAW, but this complicates an interrupt I + 4: no WB conflict I + 5: stall forced by structural hazard I + 6: stall forced by in-order issue Appendix A - Pipelining

57

Summary of Pipelining Basics

Hazards limit performance Structural: need more HW resources Data: need forwarding, compiler scheduling Control: early evaluation & PC, delayed branch, prediction Increasing length of pipe increases impact of hazards; pipelining helps instruction bandwidth, not latency Interrupts, Instruction Set, FP makes pipelining harder Compilers reduce cost of data and control hazards Load delay slots Branch delay slots Branch prediction

Appendix A - Pipelining

58

Credits
I have not written these notes by myself. Theres a great deal of fancy artwork here that takes considerable time to prepare.
I have borrowed from:
Wen-mei & Patel: http://courses.ece.uiuc.edu/ece511/lectures/lecture3.ppt Patterson: http://www.cs.berkeley.edu/~pattrsn/252S98/index.html Rabaey: (He used lots of Patterson material):

http://bwrc.eecs.berkeley.edu/Classes/CS252/index.htm
Katz: (Again, he borrowed heavily from Patterson): http://http.cs.berkeley.edu/~randy/Courses/CS252.F95/CS252.Intro.html

Mark Hill: (Follows text fairly well): http://www.cs.wisc.edu/~markhill/cs752/

Appendix A - Pipelining

59

Summary
A.1 What is Pipelining? A.2 The Major Hurdle of Pipelining-Structural Hazards
Data Hazards

Control Hazards

A.3 How is Pipelining Implemented A.4 What Makes Pipelining Hard to Implement? A.5 Extending the MIPS Pipeline to Handle Multi-cycle Operations

Appendix A - Pipelining

60