Single-cycle datapath, slightly rearranged

PCSrc

4
Add
P Add
C Shift
RegWrite left 2

Read Read
register 1 data 1 MemWrite
ALU
Read Instruction Zero
Read Read
address [31-0] 0
register 2 data 2 Result Address
Write
1 Data
Instruction register MemToReg
memory
memory Registers ALUOp
Write
data ALUSrc Write Read
1
data data
Instr [15 - 0] Sign
RegDst
extend MemRead
0
Instr [20 - 16]
0
Instr [15 - 11]
1

1
 Pipeline registers
 In pipelining, we divide instruction execution into multiple cycles
— IF ID EX MEM WB
 Information computed during one cycle may be needed in a later
cycle:
— Instruction read in IF stage determines which registers are
fetched in ID stage, what immediate is used for EX stage, and
what destination register is for WB
— Register values read in ID are used in EX and/or MEM stages
— ALU output produced in EX is an effective address for MEM or
a result for WB

 A lot of information to save!

— Saved in intermediate registers called pipeline registers

 The registers are named for the stages they connect:

IF/ID ID/EX EX/MEM MEM/WB

 No register is needed after the WB stage, because after WB the
instruction is done
2
 Pipelined datapath
1

PCSrc

IF/ID ID/EX EX/MEM MEM/WB

4
Add
P Add
C Shift
RegWrite left 2

Read Read
register 1 data 1 MemWrite
ALU
Read Instruction Zero
Read Read
address [31-0] 0
register 2 data 2 Result Address
Write
1 Data
Instruction register MemToReg
memory
memory Registers ALUOp
Write
data ALUSrc Write Read
1
data data
Instr [15 - 0] Sign
RegDst
extend MemRead
0
Instr [20 - 16]
0
Instr [15 - 11]
1

3
 Propagating values forward

 Data values required later propagated through the pipeline

registers

 The most extreme example is the destination register (rd or rt)

— It is retrieved in IF, but isn’t updated until the WB
— Thus, it must be passed through all pipeline stages, as shown
in red on the next slide

 Notice that we can’t keep a single “instruction register,” because

the pipelined machine needs to fetch a new instruction every
clock cycle

4
 The destination register
1

PCSrc

IF/ID ID/EX EX/MEM MEM/WB

4
Add
P Add
C Shift
RegWrite left 2

Read Read
register 1 data 1 MemWrite
ALU
Read Instruction Zero
Read Read
address [31-0] 0
register 2 data 2 Result Address
Write
1 Data
Instruction register MemToReg
memory
memory Registers ALUOp
Write
data ALUSrc Write Read
1
data data
Instr [15 - 0] Sign
RegDst
extend MemRead
0
Instr [20 - 16]
0
Instr [15 - 11]
1

5
 What about control signals?

 Control signals generated similar to the single-cycle processor

— in the ID stage, the processor decodes the instruction fetched
in IF and produces the appropriate control values

 Some of the control signals will not be needed until later stages
— These signals must be propagated through the pipeline until
they reach the appropriate stage
— We just pass them in the pipeline registers, along with the
data

 Control signals can be categorized by the pipeline stage that uses

them Stag Control signals needed
e
EX ALUSrc ALUOp RegDst
MEM MemRead MemWrite PCSrc
WB RegWrite MemToRe
g
6
 Pipelined datapath and control
1

0
ID/EX
WB EX/MEM
PCSrc
Control M WB MEM/WB
IF/ID EX M WB
4
Add
P Add
C Shift
RegWrite left 2

Read Read
register 1 data 1 MemWrite
ALU
Read Instruction Zero
Read Read
address [31-0] 0
register 2 data 2 Result Address
Write
1 Data
Instruction register MemToReg
memory
memory Registers ALUOp
Write
data ALUSrc Write Read
1
data data
Instr [15 - 0] Sign
RegDst
extend MemRead
0
Instr [20 - 16]
0
Instr [15 - 11]
1

7
 An example execution sequence
 Here’s a sample sequence of instructions to execute

addresse 1000: lw $8, 4($29)

1004: sub $2, $4, $5
s in
1008: and $9, $10, $11
decimal 1012: or $16, $17, $18
1016: add $13, $14, $0

 We’ll make some assumptions, just so we can show actual data

values:
— Each register contains its number plus 100. For instance,
register $8 contains 108, register $29 contains 129, etc.
— Every data memory location contains 99
 Our pipeline diagrams will follow some conventions:
— An X indicates values that aren’t important, like the constant
field of an R-type instruction
— Question marks ??? indicate values we don’t know, usually
resulting from instructions coming before and after the ones
in our example

8
 Cycle 1 (filling)
IF: lw $8, 4($29) ID: ??? EX: ??? MEM: ??? WB: ???

0 ID/EX
WB EX/MEM
PCSrc Control M WB MEM/WB
IF/ID EX M WB
4
Add
P 1004
Add
C Shift
RegWrite (?) left 2

??? ??? ???

1000 Read Read
register 1 data 1 MemWrite (?)
ALU
Read Instruction ??? ??? Zero
Read Read ??? ???
address [31-0] 0
register 2 data 2 Result Address
??? Write ??? MemToReg
1 Data
Instruction register (?)
memory
memory ??? Registers ALUOp (???)
Write
???
data ALUSrc (?) ??? Write Read
1
data data
??? Sign ???
RegDst (?)
extend MemRead (?) ???
0
??? ???
0 ??? ??? ???
??? ???
1

???

9
 Cycle 2
IF: sub $2, $4, $5 ID: lw $8, 4($29) EX: ??? MEM: ??? WB: ???

0 ID/EX
WB EX/MEM
PCSrc Control M WB MEM/WB
IF/ID EX M WB
4
Add
P 1008
Add
C Shift
RegWrite (?) left 2

29 129 ???
1004 Read Read
register 1 data 1 MemWrite (?)
ALU
Read Instruction X X ??? Zero
Read Read ???
address [31-0] 0
register 2 data 2 Result Address
??? Write ??? MemToReg
1 Data
Instruction register (?)
memory
memory ??? Registers ALUOp (???)
Write ???
data ALUSrc (?) ??? Write Read
1
data data
4 Sign ???
RegDst (?) ???
extend MemRead (?)
0
8 ???
0 ??? ??? ???
X ???
1

???

10
 Cycle 3
IF: and $9, $10, $11 ID: sub $2, $4, $5 EX: lw $8, 4($29) MEM: ??? WB: ???

0 ID/EX
WB EX/MEM
PCSrc Control M WB MEM/WB
IF/ID EX M WB
4
Add
P 1012
Add
C Shift
RegWrite (?) left 2

4 104 129
1008 Read Read
register 1 data 1 MemWrite (?)
ALU
Read Instruction 5 X Zero
Read Read 105 ???
address [31-0] 0
register 2 data 2 Result Address
4
??? Write 133 MemToReg
1 Data
Instruction register (?)
memory
memory ??? Registers ALUOp (add)
Write
??? Write ???
data ALUSrc (1) Read
1
data data
X Sign 4
RegDst (0)
extend MemRead (?) ???
0
X 8
0 8 ??? ???
2 X
1

???

11
 Cycle 4
IF: or $16, $17, $18 ID: and $9, $10, $11 EX: sub $2, $4, $5 MEM: lw $8, 4($29) WB: ???

0 ID/EX
WB EX/MEM
PCSrc Control M WB MEM/WB
IF/ID EX M WB
4
Add
P 1016
Add
C Shift
RegWrite (?) left 2

10 110 104
1012 Read Read
register 1 data 1 MemWrite (0)
ALU
Read Instruction 11 105 Zero
Read Read 111 133
address [31-0] 0
register 2 data 2 Result Address
–1
??? Write MemToReg
1 Data
Instruction register (?)
memory
memory ??? Registers ALUOp (sub)
Write
99 ???
data ALUSrc (0) X Write Read
1
data data
X Sign X
RegDst (1)
extend MemRead (1) ???
0
X X
0 2 8 ???
9 2
1

???

12
 Cycle 5 (full)
IF: add $13, $14, $0 ID: or $16, $17, $18 EX: and $9, $10, $11 MEM: sub $2, $4, $5 WB:
lw $8, 4($29)
1

0 ID/EX
WB EX/MEM
PCSrc Control M WB MEM/WB
IF/ID EX M WB
4
Add
P 1020
Add
C Shift
RegWrite (1) left 2

17 117 110
1016 Read Read
register 1 data 1 MemWrite (0)
ALU
Read Instruction 18 111 Zero
Read Read 118 -1
address [31-0] 0
register 2 data 2 Result Address
8 Write 110 MemToReg
1 Data
Instruction register (1)
memory
memory 99 Registers ALUOp (and)
Write
X 99
data ALUSrc (0) 105 Write Read
1
data data
X Sign X
RegDst (1)
extend MemRead (0) 133
0
X X
0 9 2 8
16 9
1

99

13
 Cycle 6 (emptying)
IF: ??? ID: add $13, $14, $0 EX: or $16, $17, $18 MEM: and $9, $10, $11 WB: sub
$2, $4, $5
1

0 ID/EX
WB EX/MEM
PCSrc Control M WB MEM/WB
IF/ID EX M WB
4
Add
P ???
Add
C Shift
RegWrite (1) left 2

14 114 117
1020 Read Read
register 1 data 1 MemWrite (0)
ALU
Read Instruction 0 0 118 Zero
Read Read 110
address [31-0] 0
register 2 data 2 Result Address
2 Write 119 MemToReg
1 Data
Instruction register (0)
memory
memory -1 Registers ALUOp (or)
Write
X X
data ALUSrc (0) 111 Write Read
1
data data
X Sign X
RegDst (1)
extend MemRead (0) -1
0
X X
0 16 9 2
13 16
1

-1

14
 Cycle 7
IF: ??? ID: ??? EX: add $13, $14, $0 MEM: or $16, $17, $18 WB: and
$9, $10, $11
1

0 ID/EX
WB EX/MEM
PCSrc Control M WB MEM/WB
IF/ID EX M WB
4
Add
P ???
Add
C Shift
RegWrite (1) left 2

??? ??? 114

??? Read Read
register 1 data 1 MemWrite (0)
ALU
Read Instruction ??? 0 Zero
Read Read ??? 119
address [31-0] 0
register 2 data 2 Result Address
9 Write 114 MemToReg
1 Data
Instruction register (0)
memory
memory 110 Registers ALUOp (add)
Write
X X
data ALUSrc (0) 118 Write Read
1
data data
??? Sign X
RegDst (1)
extend MemRead (0) 110
0
??? X
0 13 16 9
??? 13
1

110

15
 Cycle 8
IF: ??? ID: ??? EX: ??? MEM: add $13, $14, $0 WB: or $16,
$17, $18
1

0 ID/EX
WB EX/MEM
PCSrc Control M WB MEM/WB
IF/ID EX M WB
4
Add
P ???
Add
C Shift
RegWrite (1) left 2

??? ??? ???

??? Read Read
register 1 data 1 MemWrite (0)
ALU
Read Instruction ??? ??? Zero
Read Read ??? 114
address [31-0] 0
register 2 data 2 Result Address
16 Write ??? MemToReg
1 Data
Instruction register (0)
memory
memory 119 Registers ALUOp (???)
Write
X X
data ALUSrc (?) 0 Write Read
1
data data
??? Sign ???
RegDst (?)
extend MemRead (0) 119
0
??? ???
0 ??? 13 16
??? ???
1

119

16
 Cycle 9
IF: ??? ID: ??? EX: ??? MEM: ??? WB: add
$13, $14, $0
1

0 ID/EX
WB EX/MEM
PCSrc Control M WB MEM/WB
IF/ID EX M WB
4
Add
P ???
Add
C Shift
RegWrite (1) left 2

??? ??? ???

??? Read Read
register 1 data 1 MemWrite (?)
ALU
Read Instruction ??? ??? Zero
Read Read ??? ???
address [31-0] 0
register 2 data 2 Result Address
13 Write ??? MemToReg
1 Data
Instruction register (0)
memory
memory 114 Registers ALUOp (???)
Write
X X
data ALUSrc (?) ? Write Read
1
data data
??? Sign ???
RegDst (?)
extend MemRead (?) 114
0
??? ???
0 ??? ??? 13
??? ???
1

114

17
 That’s a lot of diagrams there
Clock cycle
1 2 3 4 5 6 7 8 9
lw $t0, 4($sp) IF ID EX MEM WB
sub $v0, $a0, IF ID EX ME WB
$a1 M
and $t1, $t2, $t3 IF ID EX ME WB
M
or $s0, $s1, IF ID EX MEM WB
$s2Compare the last few slides with the pipeline diagram above
add $t5, $t6, $0 IF ID EX MEM WB
— You can see how instruction executions are overlapped
— Each functional unit is used by a different instruction in each
cycle
— The pipeline registers save control and data values generated
in previous clock cycles for later use
— When the pipeline is full in clock cycle 5, all of the hardware
units are utilized. This is the ideal situation, and what makes
pipelined processors so fast

 See the textbook for more examples

18
 Instruction set architectures and pipelining
 The MIPS instruction set was designed especially for easy
pipelining:
— All instructions are 32-bits long, so the instruction fetch stage
just needs to read one word on every clock cycle
— Fields are in the same position in different instruction formats
—the opcode is always the first six bits, rs is the next five bits,
etc. This makes things easy for the ID stage
— MIPS is a register-to-register architecture, so arithmetic
operations cannot contain memory references. This keeps the
pipeline shorter and simpler
 Pipelining is harder for older/more complex instruction sets:
— If different instructions had different lengths or formats, the
fetch and decode stages would need extra time to determine
the actual length of each instruction and the position of the
fields
— With memory-to-memory instructions, additional pipeline
stages may be needed to compute effective addresses and
read memory before the EX stage

19
 Note how everything goes left to right,
except …
1

PCSrc

IF/ID ID/EX EX/MEM MEM/WB

4
Add
P Add
C Shift
RegWrite left 2

Read Read
register 1 data 1 MemWrite
ALU
Read Instruction Zero
Read Read
address [31-0] 0
register 2 data 2 Result Address
Write
1 Data
Instruction register MemToReg
memory
memory Registers ALUOp
Write
data ALUSrc Write Read
1
data data
Instr [15 - 0] Sign
RegDst
extend MemRead
0
Instr [20 - 16]
0
Instr [15 - 11]
1

20
 An example with dependencies

sub $2, $1, $3

and $12, $2, $5
or $13, $6, $2
add $14, $2, $2
sw $15, 100($2)

 There are several dependencies in this new code fragment

— the first instruction, SUB, stores a value into $2
— that register is used as a source in the rest of the instructions

 This is not a problem for the single-cycle datapath

— each instruction is executed completely before the next one
begins, so instructions 2 through 5 above use the new value of
$2

 How would this code sequence fare in our 5-stage MIPS pipeline?
21
 Data hazards in the pipeline diagram

Clock cycle
1 2 3 4 5 6 7 8 9

sub $2, $1, $3 IF ID EX ME WB

M

and $12, $2, $5 IF ID EX ME WB

M

or $13, $6, $2 IF ID EX ME WB
M

add $14, $2, $2 IF ID EX ME WB

M
 The sub instruction does not write to register $2 until clock cycle 5. This
causes two data hazards in our current pipelined datapath:
sw $15, IF ID EX ME WB
— the and reads register $2 in cycle 3, and since sub hasn’t modified
100($2) M
the register yet, this will be the old value of $2, not the new one
— the or instruction uses register $2 in cycle 4, again before it’s
actually updated by sub

22
 Things that are okay

Clock cycle
1 2 3 4 5 6 7 8 9

sub $2, $1, $3 IF ID EX ME WB

M

and $12, $2, $5 IF ID EX ME WB

M

or $13, $6, $2 IF ID EX ME WB
M

add $14, $2, $2 IF ID EX ME WB

M file design
 The add instruction is okay, because of the register
— registers
sw $15, are written at the beginning
IF of aIDclock
EXcycle
ME WB
— the new value will be available by the end of that cycle
100($2) M

 The sw is no problem at all, since it reads $2 after the sub finishes

23
 Dependency arrows

Clock cycle
1 2 3 4 5 6 7 8 9

sub $2, $1, $3 IF ID EX ME WB

M

and $12, $2, $5 IF ID EX ME WB

M

or $13, $6, $2 IF ID EX ME WB
M

add $14, $2, $2 IF ID EX ME WB

M
 Arrows indicate the flow of data between instructions
— The
sw $15,tails of the arrows show when register
IF ID $2 is
EXwritten
ME WB
— The heads of the arrows show when $2 is read
100($2) M

 Any arrow that points backwards in time represents a data hazard

in our basic pipelined datapath

24
 Bypassing the register file
 The actual result $1 - $3 is computed in clock cycle 3, before it
is needed in cycles 4 and 5

 If we could somehow bypass the writeback and register read

stages when needed, then we can eliminate these data hazards

 Essentially, we need to pass the ALU output from sub directly to

the and and or instructions, without going through the register
file
Clock cycle
1 2 3 4 5 6 7

sub $2, $1, $3 IF ID EX ME WB

M

and $12, $2, $5 IF ID EX ME WB

M

or $13, $6, $2 IF ID EX ME WB
M
25
 Pipleline Registers to the rescue!
 Pipeline stages communicate through pipeline registers:
IF/ID ID/EX EX/MEM MEM/WB

 We “forward” data from pipeline registers to later instructions

IF/ID ID/EX EX/MEM MEM/WB

PC ALU output
available here

ALU
Registers
Instruction
memory
Data
memory

Rt 0
0
Rd
1

26