The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

The Pipelined MIPS Processor

We complete our study of ALU architecture by investigating an
approach providing even higher performance for the MIPS CPU.
We first saw how the MIPS CPU performance could be improved
by converting the so-called single-cycle CPU to a multi-cycle design.
In the multi-cycle approach, instead of using a single clock cycle for the
whole instruction, the clock is accelerated, and instructions execute in
phases over several clock cycles.
Each instruction phase takes one clock cycle.
This means that as each instruction executes, only one section of the
CPU will be active per clock cycle -- the one executing that phase of the
instruction.

This suggests that perhaps we might redesign the CPU slightly so

that every CPU section can operate independently on an instruction
at the same time.
1

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

The Laundry Example

As an introduction to the concept of pipelining, Patterson and
Hennessy use the example of doing ones laundry.
Most people have or have access to a washer and dryer.
Assume that you need to wash several washer loads of clothing.
Would anyone divide the clothing into washer loads and then
wash, dry, fold and put away the first load before starting the
second?
No, if you were washing clothes, you would finish washing the first
load, put it in the dryer, and start the second load washing.
If there were more loads to wash, you would begin to fold and put
away finished clothing while the later loads were washing and
drying.
We can see this schematically on the next slide.
2

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

Graphical Example of the Laundry Cycle

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

The Pipeline Processor

Patterson and Hennessy applied this simultaneous wash-dry-foldput away concept to the single-cycle computer model.
The idea was to wash, dry, fold, and put away instructions
simultaneously so that the instruction throughput the number of
clock cycles per instructions could be dramatically decreased.
In the case of the single cycle model, one instruction is done per clock
cycle, but the clock must be as slow as the slowest instruction.
In the multi-cycle implementation, the clock runs faster, instructions
takes 3-5 cycles, but only one instruction is processed at a time.
What if, each time the clock ticked, we could process an instruction
in each section of the multicycle processor? Then we could process
several instructions simultaneously, approaching the goal of
completing an instruction every clock cycle.
4

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

Pipeline Architecture
A pipelined computer executes instructions concurrently.
Hardware units are organized into stages:
Execution in each stage takes exactly 1 clock period.
Stages are separated by pipeline registers that preserve and pass
partial results to the next stage.

As noted earlier, performance = complexity + cost. The

pipeline approach brings additional expense plus its own
set of problems and complications, called hazards, which
we will also study.
5

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Sequential Versus Pipelined Execution

Timeline
(clock
cycles)

1
Instruc.
Fetch

lw $t0, 16($a3)

Reg.
Fetch

lw $t1, 32($a3)

ALU
Process

10

Mem. R/W Reg.

or ALU Out Write
Instruc.
Fetch

4 clock cycles

Reg.
Fetch

lw $t2, 48($a3)

Timeline
(clock
cycles)

ALU
Process

Mem. R/W Reg.

or ALU Out Write
Instruc.
Fetch

4 clock cycles

Reg. AL
Fetch Proc

etc.

10

5 clock cycles

lw $t0, 16($a3)
lw $t1, 32($a3)
lw $t2, 48($a3)

Instruc.
Fetch

Reg.
Fetch
Instruc.
Fetch

ALU
Process
Reg.
Fetch
Instruc.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process
Reg.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process

Mem. R/W Reg.

or ALU Out Write

etc.

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

Speed Advantage of the Pipeline

The multicycle, serial processor that we studied last lecture can
execute n instructions in ns clock periods, or

ETS = ns, where

ET is the execution time and s is the number of stages.

A pipelined processor with s stages can execute n instructions in
ETP = s + (n 1) clock periods.

The ideal pipeline speedup depends on the number of stages, and

can be greater for more stages (hence Intels choice of a 20-stage
pipeline for the current P-IV).
Thus the speed advantage of pipeline over multicycle can be
defined as:

=
SP

ETs
=
ETP

ns
s + (n 1)

n>> s

Lecture #20: The Pipeline MIPS Processor

s
N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Pipeline Stages
Clock cycles
0

IF

ID/
RF

ALU

MEM

WB

The MIPS R2000 pipeline processor is divided into five processing

stages:
1. Instruction fetch (IF)
2. Instruction decode (ID) and register fetch (RF)
3. ALU instruction execution (ALU) ALU processing, branch
condition evaluation, memory address computation, etc. This is also
referred to as execution (EX)
4. Memory access (MEM)
5. Write back (WB) to register file

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Overlapped Pipeline Execution

Clock cycles
0

IF

ID/
RF

ALU

MEM

WB

IF

ID/
RF

ALU

MEM

WB

Instruction 2

IF

ID/
RF

ALU

MEM

WB

Instruction 3

Instruction 1

Instruction execution order

9

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Single-Cycle Datapath
Reg. Dest.

32

32

ADD
+4

Instruction
Address

Instruction bits 0-31

P
C

6 (Bits 26-31)

5
5

Inst.
0-31

Instruction
Memory

Lines indicate need for

storage between stages if
processor is converted to
pipeline
10

Control

Branch
Mem. Read
Mem. To Reg.
ALU Op.
Mem. Write
ALU Srce.
Reg. Write

M
32
U
X

32

ADD
Left
shift
2

32

Rs
Read
Data 1

Rt
M 5
U
Rd
X
Write
Data

Read
Data 2

32

ALU

32

Reg. Block

16 (Bits 0-15)

Write

Sign 32
Extend

M
32
U
X

32

Data
Address
Write
Data

Read
Mem./Reg.
Select

Read 32
Data
32

M 32
U
X

Data
Memory

ALU
6 (Bits 0-5) Control

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Single-Cycle Datapath with Pipeline Registers

Inter-stage registers are master-slave D flip-flops; the master can
be receiving new data from the previous stage of the instruction
while the slave flip-flop is providing data to the next stage

M
U
X

ADD
+4

Memory

P
C

Instruction
Address

Reg. Block

Compare
result

Rs
Inst.
0-31

Read
Data 1

Rd

Read
Data 2

Memory
ALU

IF/ID

Data
Address

M
U
X

Read
Data

M
U
X

Write
Data
16

Slave side
of register
Note: Control lines and
logic not shown for clarity

Rt

Write
Data

Master side
of register

11

ADD

Left
shift
2

Sign 32
Extend
ID/EX

EX/MEM

After David A. Patterson and John L. Hennessy, Computer Organization and Design, 2nd Edition

Lecture #20: The Pipeline MIPS Processor

MEM/WB
N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Instruction Process Through Pipeline (1)

M
U
X

ADD
+4

Memory

P
C

Instruction
Address

ADD

Left
shift
2

Reg. Block

Compare
result

Rs
Inst.
0-31

Rt

Read
Data 1

Rd

Read
Data 2

Memory
ALU

Write
Data

Stage 1: Instruction
loaded into IF/ID
register, PCPC+4
IF/ID

Data
Address

M
U
X

M
U
X

Write
Data
16

Sign 32
Extend
ID/EX

EX/MEM

After David A. Patterson and John L. Hennessy, Computer Organization and Design, 2nd Edition

12

Read
Data

Lecture #20: The Pipeline MIPS Processor

MEM/WB
N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Instruction Process Through Pipeline (2)

M
U
X

Stage 2: Instruction
decoded, register data
accessed, immediates
sign-extended

ADD
+4

Memory

P
C

Instruction
Address

ADD

Left
shift
2

Reg. Block

Compare
result

Rs
Inst.
0-31

Rt

Read
Data 1

Rd

Read
Data 2

Memory
ALU

Write
Data

M
U
X

Sign 32
Extend
ID/EX

EX/MEM

After David A. Patterson and John L. Hennessy, Computer Organization and Design, 2nd Edition

13

Read
Data

Write
Data
16

IF/ID

Data
Address

M
U
X

Lecture #20: The Pipeline MIPS Processor

MEM/WB
N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Instruction Process Through Pipeline (3)

Stage 3: Instruction
executed / branch
address computed

M
U
X

ADD
+4

Memory

P
C

Instruction
Address

ADD

Left
shift
2

Reg. Block

Compare
result

Rs
Inst.
0-31

Rt

Read
Data 1

Rd

Read
Data 2

Memory
ALU

Write
Data

M
U
X

Sign 32
Extend
ID/EX

EX/MEM

After David A. Patterson and John L. Hennessy, Computer Organization and Design, 2nd Edition

14

Read
Data

Write
Data
16

IF/ID

Data
Address

M
U
X

Lecture #20: The Pipeline MIPS Processor

MEM/WB
N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Instruction Process Through Pipeline (4)

Stage 4: Memory load or
store, branch taken/not taken
ALU results bypass taken to
MEM/WB register

M
U
X

ADD
+4

Memory

P
C

Instruction
Address

ADD

Left
shift
2

Reg. Block

Compare
result

Rs
Inst.
0-31

Rt

Read
Data 1

Rd

Read
Data 2

Memory
ALU

Write
Data

M
U
X

Sign 32
Extend
ID/EX

EX/MEM

After David A. Patterson and John L. Hennessy, Computer Organization and Design, 2nd Edition

15

Read
Data

Write
Data
16

IF/ID

Data
Address

M
U
X

Lecture #20: The Pipeline MIPS Processor

MEM/WB
N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Instruction Process Through Pipeline (5)

M
U
X

ADD
+4

Memory

P
C

Instruction
Address

Reg. Block

Compare
result

Rs
Inst.
0-31

Rt

Read
Data 1

Rd

Read
Data 2

Memory
ALU

Read
Data

M
U
X

Write
Data
16

IF/ID

Data
Address

M
U
X

Write
Data

16

ADD

Left
shift
2

Sign 32
Extend
ID/EX

Stage 5: Result
write-back to
dest. register
EX/MEM

MEM/WB

After David A. Patterson and John L. Hennessy, Computer Organization and Design, 2nd Edition
N. B. Dodge 9/15
Lecture #20: The Pipeline MIPS Processor

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

Adding Control
Control information must be carried along as a part of
the instruction, since this information is required at
different stages of the pipeline.
This can be done by adding more inter-stage storage
register bits to forward control data yet to be used.
The result is very large inter-stage registers. For
example, the storage capacity required between the
instruction decode and ALU execution stages (ID/EX
register) is more than 120 bits.
The resulting processor with full control functionality
is shown on the next slide
17

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Memory

P
C

Instruction
Address

Inst.
0-31

Control
Decode

Reg. Block
Rs

18

ADD

Left
shift
2

Branch
ALU Srce

Rt

Read
Data 1

Rd

Read
Data 2

Write
Data

Full Pipeline
Design with
Control Lines

MEM/WB

Bits 0-15

ALU
M
U
X

Read
Data

M
U
X

Write
Data

Sign 32
Extend

ALU
Cont.

Bits 16-20
Bits 11-15

Data
Address

Memory/ALU Result

ADD
+4

Instruction bits 0-31

IF/ID

EX/MEM

Memory Write
Memory Read

M
U
X

Register Write

ID/EX

M
U
X

Memory

ALU Op
Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

After David A. Patterson

John L. Hennessy,
N.and
B. Dodge
9/15
Computer Organization and Design, 2nd Edition

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

The Pipeline in Action

The following instruction sequence from the P&H text
illustrates the pipeline in action.
lw $t0, 20($t1)
sub $t4, $t2, $t3
and $t7, $t5, $t6
or $s2, $s1, $s0
add $s5, $s3, $s4
Note that registers are identified by letter ids, not
numbers. This is counter to P&H, which uses the
number ids. In reading, P&H, remember the numberletter id conversions, e.g., $8-15=$t0-t7, $16-21 = $s0-s5.
19

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

ID/RF: Idle

EX: Idle

ADD
+4

Memory

P
C

Instruction
Address

Inst.
0-31

Instruction bits 0-31

IF/ID

EX/MEM

MEM/WB

Control
Decode

ADD

Left
shift
2

Reg. Block
Rs

Branch
ALU Srce

Rt

Read
Data 1

Rd

Read
Data 2

ALU
M
U
X

Write
Data
Bits 0-15

Sign
Extend

Bits 11-15

Data
Address

Read
Data

M
U
X

Write
Data

32
ALU
Cont.

Bits 16-20

20

WB:
Idle

Memory/ALU Result

M
U
X

Register Write

ID/EX

MEM: Idle

Memory Write
Memory Read

IF: Idle

M
U
X

Memory

ALU Op
Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

After David A. Patterson and John L. Hennessy,

Computer Organization and Design, 2nd Edition

ID/RF: Idle

EX: Idle

ADD
+4

Memory

P
C

Instruction
Address

Inst.
0-31

Instruction bits 0-31

IF/ID

EX/MEM

MEM/WB

Control
Decode

Reg. Block
Rs

ADD

Left
shift
2

Branch
ALU Srce

Rt

Read
Data 1

Rd

Read
Data 2

Write
Data
Bits 0-15

ALU
M
U
X

Bits 11-15

Data
Address

Read
Data

M
U
X

Write
Data

Sign 32
Extend

ALU
Cont.

Bits 16-20

21

WB:
Idle

Memory/ALU Result

M
U
X

Register Write

ID/EX

MEM: Idle

Memory Write
Memory Read

IF: lw $t0, 20($t1)

M
U
X

Memory

ALU Op
Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

After David A. Patterson and John L. Hennessy,

Computer Organization and Design, 2nd Edition

ID/RF: lw $t0, 20($t1)

EX: Idle

Register Write

ID/EX

ADD
+4

Memory

P
C

Instruction
Address

Inst.
0-31

Instruction bits 0-31

IF/ID

ADD

Left
shift
2

Rs

Branch
ALU Srce

$ t0

Rt

Read
[ $t1 ]
Data 1

Rd

Read
Data 2

Bits 0-15

Bits 16-20
Bits 11-15

ALU

X
M
U
X

Write
Data

22

MEM/WB

Control
Decode

Reg. Block
$t 1

EX/MEM

WB:
Idle

Read
Data

M
U
X

Write
Data

Sign
Extend 0x14
$ t0

Data
Address

Memory/ALU Result

M
U
X

MEM: Idle

Memory Write
Memory Read

IF: sub $t4, $t2, $t3

ALU
Cont.
M
U
X

Memory

ALU Op
Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

After David A. Patterson and John L. Hennessy,

Computer Organization and Design, 2nd Edition

ID/RF: sub $t4, $t2, $t3

EX: lw $t0, 20($t1)

Register Write

ID/EX

ADD
+4

Memory

P
C

Instruction
Address

Inst.
0-31

Instruction bits 0-31

IF/ID

$t3

ADD

Left
shift
2

Rs

Branch
ALU Srce

Rt

Read [ $t2 ]
Data 1

Rd

Read [ $t3 ]
Data 2

[ $t1 ]

ALU
M
U
X

Write
Data
Bits 0-15

Bits 16-20
Bits 11-15

23

MEM/WB

Control
Decode

Reg. Block
$t2

EX/MEM

WB:
Idle

Sign X
Extend

0x14

$ t0

$ t4

0x14

Data
Address

Read
Data

add

Memory/ALU Result

M
U
X

MEM: Idle

Memory Write
Memory Read

IF: and $t7, $t5, $t6

M
U
X

Write
Data

Memory

ALU
Cont.
M
U
X

ALU Op $ t0
Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

After David A. Patterson and John L. Hennessy,

Computer Organization and Design, 2nd Edition

ID/RF: and $t7, $t5, $t6

EX: sub $t4, $t2, $t3 MEM: lw $t0, 20($t1)

Register Write

ID/EX

ADD
+4

Memory

P
C

Instruction
Address

Inst.
0-31

Instruction bits 0-31

IF/ID

$t6

ADD

Left
shift
2

Rs

Branch
ALU Srce

Rt

Read [ $t5 ]
Data 1

Rd

Read [ $t6]
Data 2

[ $t2 ]

Bits 0-15

Bits 16-20
Bits 11-15

ALU

[$t3]
M
U
X

Write
Data

24

MEM/WB

Control
Decode

Reg. Block
$t5

WB:
Idle

[ $t3 ]
sub

Read
Data

M
U
X

Write
Data

Sign X
Extend

Memory

ALU
Cont.

X
$ t7

Data
Address

Memory/ALU Result

M
U
X

EX/MEM

Memory Write
Memory Read

IF: or $s2, $s1, $s0

$ t4

M
U
X

ALU Op $ t4

$ t0

Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

After David A. Patterson and John L. Hennessy,

Computer Organization and Design, 2nd Edition

ID/RF: or $s2, $s1, $s0

EX: and $t7, $t5, $t6 MEM: sub $t4,$t2,$t3 WB: lw $t0,
20($t1)

Register Write

ID/EX

ADD
+4

Memory

P
C

Instruction
Address

Inst.
0-31

Instruction bits 0-31

IF/ID

Control
Decode

$s0

Rs

Branch
ALU Srce

Rt

Read [$s1]
Data 1

$t0 Rd

Read [$s0]
Data 2

[ $t5 ]

Bits 0-15

Bits 16-20
Bits 11-15

ALU

[$t6]
M
U
X

Write
Data

25

ADD

Left
shift
2

Reg. Block
$s1

MEM/WB

[ $t6 ]
and

Read
Data

M
U
X

Write
Data

Sign X
Extend

Memory

ALU
Cont.

X
$s2

Data
Address

Memory/ALU Result

M
U
X

EX/MEM

Memory Write
Memory Read

IF: add $s5, $s3, $s4

$ t7

M
U
X

ALU Op $ t7

$ t4

$ t0

Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

After David A. Patterson and John L. Hennessy,

Computer Organization and Design, 2nd Edition

ID/RF: add $s5, $s3, $s4

EX: or $s2, $s1, $s0 MEM: and $t7,$t5,$t6 WB: sub

$t4, $t2, $t3

Register Write

ID/EX

ADD
+4

Memory

P
C

Instruction
Address

Inst.
0-31

Instruction bits 0-31

IF/ID

Control
Decode

$s4

Rs

Branch
ALU Srce

Rt

Read
[$s3]
Data 1

$t4 Rd

Read [$s4]
Data 2

[$s1]

Bits 0-15

Bits 16-20
Bits 11-15

ALU

[$s0]
M
U
X

Write
Data

26

ADD

Left
shift
2

Reg. Block
$s3

MEM/WB

Read
Data

M
U
X

Write
Data

Sign X
Extend

ALU
Cont.

X
$s5

[$s0] or

Data
Address

Memory/ALU Result

M
U
X

EX/MEM

Memory Write
Memory Read

IF: Idle

$s2

M
U
X

ALU Op $s2

Memory
$ t7

$ t4

Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

After David A. Patterson and John L. Hennessy,

Computer Organization and Design, 2nd Edition

ID/RF: Idle (WB)

EX: add $s5,$s3,$s4

ADD
+4

Memory

P
C

Instruction
Address

Inst.
0-31

Instruction bits 0-31

IF/ID

EX/MEM

Control
Decode

Reg. Block
Rs

ADD

Left
shift
2

Branch
ALU Srce

Rt

Read
Data 1

$t7 Rd

Read
Data 2

[$s3]

Bits 0-15

ALU

[$s4]
M
U
X

Write
Data

Bits 11-15

[$s4]

Data
Address

Read
Data

add

M
U
X

Write
Data

Sign 32
Extend

ALU
Cont.

Bits 16-20

27

MEM/WB

Memory/ALU Result

M
U
X

Register Write

ID/EX

MEM: or $s2,$s1,$s0 WB: and

$t7, $t5, $t6

Memory Write
Memory Read

IF: Idle

$s5

M
U
X

ALU Op $s5

Memory
$s2

$ t7

Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

After David A. Patterson and John L. Hennessy,

Computer Organization and Design, 2nd Edition

ID/RF: Idle (WB)

EX: Idle

Register Write

ID/EX

ADD
+4

Memory

P
C

Instruction
Address

Inst.
0-31

Instruction bits 0-31

IF/ID

EX/MEM

Control
Decode

Reg. Block
Rs

ADD

Left
shift
2

Branch
ALU Srce

$s2

Rt

Read
Data 1

Rd

Read
Data 2

Write
Data
Bits 0-15

ALU
M
U
X

Bits 11-15

Data
Address

Read
Data

M
U
X

Write
Data

Sign 32
Extend

ALU
Cont.

Bits 16-20

28

MEM/WB

Memory/ALU Result

M
U
X

MEM: add $s5,$s3,$s4 WB: or $s2,

$s1, $s0

Memory Write
Memory Read

IF: Idle

M
U
X

ALU Op

Memory
$s5

$s2

Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

After David A. Patterson and John L. Hennessy,

Computer Organization and Design, 2nd Edition

ID/RF: Idle (WB)

EX: Idle

ADD
+4

Memory

P
C

Instruction
Address

Inst.
0-31

Instruction bits 0-31

IF/ID

EX/MEM

MEM/WB

Control
Decode

Reg. Block
Rs

ADD

Left
shift
2

Branch
ALU Srce

Rt

Read
Data 1

$s5 Rd

Read
Data 2

Write
Data
Bits 0-15

ALU
M
U
X

Bits 11-15

Data
Address

Read
Data

M
U
X

Write
Data

Sign 32
Extend

ALU
Cont.

Bits 16-20

29

WB: add
$s5, $s3, $s4

Memory/ALU Result

M
U
X

Register Write

ID/EX

MEM: Idle

Memory Write
Memory Read

IF: Idle

M
U
X

ALU Op

Memory
$s5

Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

After David A. Patterson and John L. Hennessy,

Computer Organization and Design, 2nd Edition

ID/RF: Idle

EX: Idle

ADD
+4

Memory

P
C

Instruction
Address

Inst.
0-31

Instruction bits 0-31

IF/ID

EX/MEM

MEM/WB

Control
Decode

Reg. Block
Rs

ADD

Left
shift
2

Branch
ALU Srce

Rt

Read
Data 1

Rd

Read
Data 2

Write
Data
Bits 0-15

ALU
M
U
X

Bits 11-15

Data
Address

Read
Data

Write
Data

Sign 32
Extend

ALU
Cont.

Bits 16-20

30

WB:
Idle

M
U
X

ALU Op
Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

Memory

Memory/ALU Result

M
U
X

Register Write

ID/EX

MEM: Idle

Memory Write
Memory Read

IF: Idle

M
U
X

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

Pipeline Processor Operation Summary

Pipelining replaces the single-cycle processor with a
row of five mini-processors, each capable of
completing one part of each instruction.
A new instruction is started every clock cycle.
Inter-process registers store instruction information
(data, write register, branch conditions) between cycles
so that the entire instruction envelope is passed
between the pipeline stages.
When the pipeline is filled with instructions, an
instruction completes every clock cycle.
31

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

Exercise 1
On the diagram on the next page, identify the
following:
1. Highlight all the control lines that must be active during a load
word instruction.
2. As in our exercise in Lecture 20, identify the decoder
locations.
3. The ID/EX Register interface stores the most bits of any of the
pipeline section interfaces. Approximately how many bits is
that, according to the diagram?

32

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Print out a copy of this diagram and bring to class.

Memory

P
C

Instruction
Address

Inst.
0-31

MEM/WB

Control
Decode

Reg. Block
Rs

ADD

Left
shift
2

Branch
ALU Srce

Rt

Read
Data 1

Rd

Read
Data 2

Write
Data
Bits 0-15

ALU
M
U
X

Read
Data

Write
Data

Sign 32
Extend

ALU
Cont.

Bits 16-20
Bits 11-15

Data
Address

M
U
X

ALU Op
Reg. Dst.

Lecture #20: The Pipeline MIPS Processor

Memory

Memory/ALU Result

ADD
+4

Instruction bits 0-31

IF/ID

EX/MEM

Memory Write
Memory Read

M
U
X

Register Write

ID/EX

M
U
X

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Hazards
Hazards occur because data required for executing the
current instruction may not be available.
An instruction in the register fetch cycle may need
data from a register whose value will be changed by an
instruction downstream but still in process in the
pipeline (in the ALU, memory/memory bypass or
writeback cycle).
Thus an upstream instruction could access a register
and get incorrect data because the register data has not
yet been updated by a downstream instruction.
35

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

Hazards (2)
There are two types of hazards, data hazards, and
control hazards.
Both occur because an instruction in the ID/RF stage of
the MIPS pipeline needs register data that will be
shortly updated by instructions in the EX or
MEM/Bypass, or WB stage.
Data hazards occur when an instruction needs register
contents for an arithmetic/ logical/memory instruction.
Control hazards occur when a branch instruction is
pending and the data necessary to initiate/bypass the
branch is not yet available in the same sort of scenario.
36

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Timeline
(clock
cycles)

Data Hazard in the Pipeline

0

10

5 clock cycles

sub $2, $1, $3

and $12, $2, $5
or $13, $6, $2
add $14, $2, $2
sw $15, 100($2)

Instruc.
Fetch

Reg.
Fetch
Instruc.
Fetch

ALU
Process
Reg.
Fetch
Instruc.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process
Reg.
Fetch
Instruc.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process
Reg.
Fetch
Instruc.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process
Reg.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process

Mem. R/W Reg.

or ALU Out Write

In the instruction sequence above, the last four instructions

require data from $2, which is changed in the first instruction.
The $2 data will not be rewritten until cycle 4, so the AND and OR
(2nd and 3rd instructions) will fetch incorrect data from $2.
Even the add may not get the correct information (sw is okay).
37

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Control Hazards in the Pipeline

Timeline
(clock
cycles)

10

5 clock cycles

sub $2, $1, $3

blt $2, $8, wait
bgt $2, $7, go
add $14, $2, $2
sw $15, 100($2)

Instruc.
Fetch

Reg.
Fetch
Instruc.
Fetch

ALU
Process
Reg.
Fetch
Instruc.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process
Reg.
Fetch
Instruc.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process
Reg.
Fetch
Instruc.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process
Reg.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process

Mem. R/W Reg.

or ALU Out Write

Here the problem is changed, with two branch instructions added.

Neither branch instruction may be executed correctly, once again
because the new $2 data will not be ready.
This wrong data could cause an incorrect branch.
38

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Forwarding as a Solution to Data Hazards

Clock cycles
0

IF
2

ID/
RF

ALU

MEM

WB

IF

ID/
RF

ALU

MEM

WB

One solution to the problem of data hazards is forwarding.

Forwarding uses the fact that although instruction 2 needs register
data two clock cycles before instruction 1 enters the WB stage, that
data is already available as the output of the ALU.
If a mechanism were available, instruction 1 could forward required
register data after its ALU cycle to the ID/RF cycle of instruction 2.
39

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Forwarding Unit in the Pipeline

ID/EX
Read
Data 1

Rs

EX/MEM
M
U
X

Rt

Forward A

Rd
Write
Data

MEM/WB

Data
Address

ALU

Read
Data 2
M
U
X

Reg. Block

Read
Data

M
U
X

Write
Data
Forward B

Memory

Rs
Rt
Rd

M
U
X

EX/MEM Register Rd

Forwarding
Unit

MEM/WB Register Rd

After David A. Patterson and John L. Hennessy, Computer Organization and Design, 2nd Edition

40

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Forwarding Unit Operation

Reg. Block
ALU
Memory

Forwarding
Unit

The forwarding unit samples register ids in the EX/MEM and

MEM/WB registers to determine if source registers in the ID/RF
cycle are the same.
If so, source register data is replaced by pipeline (as yet unwritten)
data by the forwarding unit.
The correct information is thus processed and the instruction can
proceed to correct execution.
41

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Stalls
Forwarding will not always solve the problems of data hazards.
For example, suppose an add instruction follows a load word (lw),
and the add involves the register that receives the memory data.
In this case, forwarding will not work.
The reason is that the data must be read from memory, and so it
will not be available until the end of the MEM cycle. Thus the
required data is not available for a forward, and the add
instruction. if it proceeds, will process the wrong data.
A solution to this problem is the stall.
A stall halts the instruction awaiting data, while the key
instruction (a lw in this case) proceeds to the end of the MEM
cycle, after which the desired data is available to the add.
42

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Timeline
(clock
cycles)

Result of Stall Approach

0

10

5 clock cycles

lw $2, 32($3)
add $14, $6, $2
sw $15, 80($2)

Instruc.
Fetch

Reg.
Fetch
Instruc.
Fetch

ALU
Process
Reg.
Fetch
Instruc.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process
Reg.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process

Mem. R/W Reg.

or ALU Out Write

Consider the 3 instructions above, the last two

depending on the lw.
$2 contents will be available at the beginning of the WB
stage in the first instruction, but not before.
A solution is to let the lw proceed down the pipe, while
the add and sw instructions hold place for one cycle.
N. B. Dodge 9/15
43

Lecture #20: The Pipeline MIPS Processor

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Timeline
(clock
cycles)

Result of Stall Approach (2)

0

10

5 clock cycles

lw $2, 32($3)

Instruc.
Fetch

Reg.
Fetch

add $14, $6, $2 (delayed 1 count)

sw $15, 80($2) (delayed 1 count)

ALU
Process

Mem. R/W Reg.

or ALU Out Write

Instruc.
Fetch

Reg.
Fetch
Instruc.
Fetch

ALU
Process
Reg.
Fetch

Mem. R/W Reg.

or ALU Out Write
ALU
Process

Mem. R/W Reg.

or ALU Out Write

With the delay, the lw result feeds the ALU input stage
of the add instruction, and the fetch stage of the sw.
Note that forwarding in still required (this time from
the MEM/WB interface, not the ALU output).
However, in addition to forwarding, instructions
following a lw must also be delayed for one clock N.cycle.
B. Dodge 9/15
44

Lecture #20: The Pipeline MIPS Processor

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

Other Problems With Branches

A remaining problem is what to do about instructions following a
branch. Even assuming forwarding and stalls, the branch/no
branch decision is not made until the third stage. This means that
in the MIPS pipeline, two following instructions will enter the pipe
before the branch/no branch decision is made. What if:
The following instructions were for the case of branch taken and
the branch was not taken.
The following instructions were for branch not taken and it was
taken.

In either case, the wrong instructions are in the pipe and they must
be eliminated (flushed). How can this problem be prevented?
A few approaches to the problem are shown in the following slides.
45

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Control Hazard Approaches (1)

MIPS R-2000 Pipeline Processor
IF

ID/RF

ALU/EX

(Branch)

MEM/
Bypass

WB

Direction of pipeline flow

One approach is to always assume the branch is (or is not) taken:

Say we assume the branch is never taken. Then if the instruction in ALU/EX
is a branch, the instructions in IF and ID/RF will be those in the not taken
program line (branch determination is made in ALU/EX).
It this assumption is correct, the pipeline will continue to flow without delay.
When the branch is taken, instructions in IF and ID/RF must be flushed,
usually by changing the op code of those instructions to a nop and letting
them proceed to the end of the pipe.
Clearly, a 2-clock time delay is involved here, and it would be worse for longer
pipelines (P-IV pipeline ~ 20 stages).

46

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Control Hazard Approaches (2)

MIPS R-2000 Pipeline Processor
IF

ID/RF

Branch

ALU/EX

MEM/
Bypass

WB

Branch
Comparator

Reducing the cost of taking the branch:

In this case, a branch assumption is still made (taken or not taken).
The difference is that since register contents (and/or immediates) are
identified in the ID/RF stage, a comparator can be added there to do the
branch/no-branch determination.
With the branch determination made in this early stage, only one
instruction must be flushed, in the IF stage (only a 1-instruction delay).
47

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

Erik Jonsson School of Engineering and

Computer Science

The University of Texas at Dallas

Control Hazard Approaches (3)

MIPS R-2000 Pipeline Processor
IF

ID/RF

Branch

Branch feedback based on History

ALU/EX

MEM/
Bypass

WB

Branch
History

Dynamic branch prediction based on recent branch history:

In this approach, an indicator bit (0/1) gives the last branch condition.
The next branch can be made according to the bit setting.
This is useful in highly repetitive loops, which may continue for a long
time until a substantial number of calculations are complete.
Some schemes use 2 bits and do not change the prediction until the
predictor is wrong twice, after which the alternate behavior is chosen.
In either case, incorrect predictions will still be made, but hopefully not
as often.
48

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

Exercise 2
1. Explain forwarding in your own words.
2. Why doesnt forwarding always work? How can this
problem be solved?
3. Why could 2-bit dynamic branch prediction work to
ensure about a 1% error rate in branch prediction in
a subroutine that loops about 100 times before
completion? Hint: Assume that the subroutine is
called frequently, and that it always executes 100 or
more loop traversals before returning to the calling
program.
49

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15

The University of Texas at Dallas

Erik Jonsson School of Engineering and

Computer Science

Summary
The pipeline approach to CPU design provides a significant speed
increase over a single-cycle design, up to several hundred percent.
The improvement is so dramatic that today, all general-purpose
processor units (now usually clustered in groups of 2, 4, 6, 8 or
more CPUs) are designed using a pipeline approach.
However, this performance improvement must be paid for with
increased (1) price, and (2) complexity. Pipelines introduce
processing problems, including:
Hazards
Incorrect branch prediction

The increased price and complexity come about in the hardware

approaches to solving these and other similar problems.
52

Lecture #20: The Pipeline MIPS Processor

N. B. Dodge 9/15