CS/CoE 1541
Single and Multi-cycle
Implementations
Introduction to Computer Architecture
Typical Instruction Execution
Single Cycle Recap
1) Fetch instruction from memory
2) Decode instruction
3) If necessary, perform an ALU operation
4) If memory access, perform load/store
5) Write results back to register file and increment the PC
Introduction to Computer Architecture
Fetching Instruction
Memory to hold instructions
Program Counter (PC) to
generate address of instruction
Adder to increment the PC for
the next instructions address
Program Counter
(Register)
Next
Instruction
Memory (RAM)
Instruction
address
Instruction
Adder
Current PC
Next PC
Current PC
Introduction to Computer Architecture
Instruction Fetch Unit
Current PC
Adder
4
Program Counter
(Register)
Next
Current PC
Instruction
instruction
address
Write
Introduction to Computer Architecture
Instruction
Memory (RAM)
4
ALU Operation
Consider a basic ALU operation
add
R1,R2,R3
Requires a Register File and an ALU
Register
Numbers
Read Reg 1
Read
Read Reg 2 Data 1
Write Reg
Data
ALU Operation
Read
Data 2
Write Data
Data
ALU
Register File
Introduction to Computer Architecture
ADD R2, R3, R4
000000 00011
op
rs
00100
rt
00010
rd
00000
shamt
100000
funct
Register File
Instruction
Register
Fields
ALU Operation
Read Reg 1
Read
Read Reg 2 Data 1
Write Reg
Data
Introduction to Computer Architecture
Read
Data 2
Write Data
Write Enable
ALU
Accessing Memory (loads and stores)
LW R4,0x10(R2)
Instruction
Register File
Read Reg 1
Read
Read Reg 2 Data 1
Write Reg
Write Data
Introduction to Computer Architecture
Read
Data 2
ALU
Data Memory
(RAM)
Load: From Memory to Register File
LW R4,0x10(R2)
Instruction
Register File
Data Memory
(RAM)
Read Reg 1
Read
Read Reg 2 Data 1
Write Reg
Read
Data 2
Write Data
ALU
Read
data
Write Enable
Introduction to Computer Architecture
Computing the Address (part 1)
LW R4,0x10(R2)
Instruction
Data Memory
(RAM)
Register File
Read Reg 1
Read
Read Reg 2 Data 1
Write Reg
Read
Data 2
Write Data
Read/Write
address
ALU
Read
data
Write Enable
Introduction to Computer Architecture
Computing the Address (part 2)
LW R4,0x10(R2)
Instruction
ALU Add
Data Memory
(RAM)
Register File
Read Reg 1
Read
Read Reg 2 Data 1
Write Reg
Data
Read/Write
address
Read
Data 2
Write Data
ALU
Read
data
Write Enable
16
Introduction to Computer Architecture
Sign
extend
32
10
Sign Extender
000000 00011
op
rs
00100
rt
00010000 00100000
immediate
Register File
Read Reg 1
Read
Read Reg 2 Data 1
Write Reg
Read
Data 2
Write Data
16 bits
Write Enable
Introduction to Computer Architecture
16 bits
15
...
ALU
16 bits
Sign extend
0
11
Store: From Register File to Memory
SW R4,0x10(R2)
ALU Add
Instruction
Data Memory
(RAM)
Register File
Read Reg 1
Read
Read Reg 2 Data 1
Write Reg
Data
Read/Write
address
Read
Data 2
Write Data
ALU
Write Data
Read
data
Write Enable
16
Introduction to Computer Architecture
Sign
extend
32
12
Branch
BEQ
R1, R2, label
Instruction
Register File
Read Reg 1
Data
Read
Read Reg 2 Data 1
Write Reg
Write Data
Introduction to Computer Architecture
Zero
Read
Data 2
To Branch
Control
Logic
ALU
13
Need Adder to Compute Branch Target
BEQ
R1, R2, label
ADDER
PC + 4 from
instruction datapath
Instruction
Branch
Target
<< 2
Register File
Read Reg 1
Data
Read
Read Reg 2 Data 1
Write Reg
Zero
Read
Data 2
Write Data
16
Introduction to Computer Architecture
To Branch
Control
Logic
ALU
Sign
extend
32
14
Data Path for Memory and R-type Instructions
Register File
Read Reg 1
Read
Read Reg 2Data 1
Instruction
Write Reg Read
Write Data Data 2
Zero
Data Memory
(RAM)
M
U
X
M
U
X
ALU
Write
Data
Sign
extend
16
Introduction to Computer Architecture
Read
data
32
15
Complete Single-cycle Datapath
Current PC
4
PC
M
U
X
Adder
Instruction
Memory (RAM)
<< 2
ADDER
Register File
Instruction
Read Reg 1
Read
Read Reg 2Data 1
Write Reg Read
Write Data Data 2
Zero
Data Memory
(RAM)
M
U
X
M
U
X
ALU
16
Introduction to Computer Architecture
Sign
exten
d
Write
Data
Read
data
32
16
Whats Wrong with a Single-Cycle
Implementation
That was a single cycle machine?
Yep! It was assumed that data flows through all parts of the datapath
in ONE clock cycle
How long is a cycle
ALU
10 ns
Register File
5 ns
Memory
10 ns
Assume everything else takes zero time
Introduction to Computer Architecture
17
Instruction Timings
Instr Type
R-format
Load
Store
Branch
Jump
InstrMem
10
10
10
10
10
Reg Read
5
5
5
5
-
ALU
10
10
10
10
-
DataMem
10
10
-
Register File
PC
Read Reg 1
Read
Read Reg 2Data 1
Write Reg Read
Write Data Data 2
Zero
Reg Write
5
5
-
Data Memory
(RAM)
M
U
X
M
U
X
ALU
16
Introduction to Computer Architecture
Sign
exten
d
Total
30 ns
40 ns
35 ns
25 ns
10 ns
Write
Data
Read
data
32
18
Whats Wrong with a Single-Cycle
Implementation
Difficult to implement variable cycle clock
Usually run the clock at the SLOWEST speed
This is called the critical path
The critical path is the path through the system which limits
performance
What if we add a floating point unit?
FP (floating point) math can take a very long time
100s of ns for multiply and divide
Lots of techniques to reduce time - will cover later on
How about breaking the machine into parts
Introduction to Computer Architecture
19
Multiple Cycle Implementation Datapath
Targ
et
30
PC[31:28]
Shift
left 2
Jump
Address
M
U
X
32
Memory
Read Reg1
Read
Read Reg2Data 1
PC
M
U
X
M
U
X
Instruction
Register
Write Reg Read
Data 2
Write Data
A
B
4
M
U
X
Zero
ALUOut
M
U
X
ALU
Write Data
MDR
M
U
X
16
Introduction to Computer Architecture
Sign
Exten
d
Shift
left 2
32
20
Full Diagram of Multi-cycle Machine
Figure 5.33
Introduction to Computer Architecture
21
Execution Steps (1)
Instruction Fetch
IR = Memory[PC];
PC = PC + 4;
Introduction to Computer Architecture
22
Execution Steps (2)
Instruction Decode and Register Fetch
A = Reg[IR[25..21]];
B = Reg[IR[20..16]];
ALUOut = PC + (signExtend(IR[15..0]) << 2);
Introduction to Computer Architecture
23
Execution Step (3)
Execution, memory address computation or branch completion
Memory Reference
ALUOut = A + signExtend(IR[15..0]);
Arithmetic/Logical Operation
ALUOut = A + B
Branch
If (A == B) PC = ALUOut;
Jump
PC = PC[31 ..28] || (IR[25..0) << 2);
Introduction to Computer Architecture
24
Execution Step (4)
Memory access or R-type instruction completion
Memory Reference
MDR = Memory[ALUOut];
or
Memory[ALUOut] = B;
Arithmetic/Logical Instructions (R-type)
Reg[IR[15..11]] = ALUOut;
Introduction to Computer Architecture
25
Execution Step (5)
Memory Read completion
Reg[IR[20..16]] = MDR;
Introduction to Computer Architecture
26
Multicycle Control
MemReadMemWrite
RegWrite
IRWrite
IorD
RegDest
PC
M
U
X
ALU SelA
M
U
X
Instruction
Register
Read Reg1
Read
Read Reg2Data 1
Write Reg Read
Data 2
Write Data
A
B
4
M
U
X
Zero
ALUOut
ALU
M
U
X
Write Data
MDR
M
U
X
16
MemToReg
ALU SelB
Shift
Sign
left 2
Exten
32
d
Instruction [5:0]
ALU
Contr
ol
ALU Op
Introduction to Computer Architecture
27
Performance of Multicycle Implementation
Each type of instruction can take a variable # of cycles
Example
Assume the following instruction distributions:
loads
stores
R-type
branches
jump
5 cycles
4 cycles
4 cycles
3 cycles
3 cycles
22%
11%
49%
16%
2%
Whats the average Cycles Per Instruction (CPI)
CPI = (CPU clock cycles/Instruction Count)
CPI = (5 cycles * 0.22) + (4 cycles * 0.11) + (4 cycles * 0.49)
+ (3 cycles * 0.16) + (3 cycles * 0.02)
CPI = 4.04 cycles per instruction
What was the CPI for the single-cycle machine?
Single cycle implies 1 clock cycle per instruction --> CPI = 1.0
So isnt the single-cycle machine faster?
Introduction to Computer Architecture
28
CS/CoE 1541
Pipelining
Introduction to Computer Architecture
29
Looks a Lot Like a Multicycle Processor
What are the steps
Fetch an instruction
Decode the instruction
ALU OP
Memory Access
Write-back
Memory
M
U
X
M
U
X
Instruction
Register
Read Reg1
Read
Read Reg2Data 1
M
U
X
Write Reg Read
Data 2
Write Data
M
U
X
Zero
ALU
Write Data
M
U
X
Introduction to Computer Architecture
16
Sign
Exten
d
Shift
left 2
32
30
Performance of Pipelined Systems
time
Unpipelined
instructions
Pipelined
time
latency
instructions
Ideally, Speeduppipeline =
Introduction to Computer Architecture
Timesequential
Pipeline Depth
31
MIPS Pipeline Stages
Stage 1: Instruction Fetch
Stage 2: Instruction Decode
Stage 3: Execute
Stage 4: Memory Access
Stage 5: Write Back (to register file)
Introduction to Computer Architecture
32
How Do We Partition the Datapath into Stages
STAGE 1
Instruction Fetch
STAGE 3
ALU
STAGE2
Decode
STAGE 4
MemAcc
STAGE 5
Writeback
Current PC
4
Adder
PC
Register File
Read Reg 1
Read
Data
Read Reg
2 1
Instruction
Write Reg
Read
Data 2
Write Data
Instruction
Memory (RAM)
16
Introduction to Computer Architecture
Sign
exte
nd
ALU
M
U
X
Data Memory
(RAM)
Zero
M
U
X
Write
Data
Read
data
32
33
But How to We Separate the Different Stages
STAGE 1 -Instruction Fetch
STAGE2
Decode
STAGE 3
ALU
STAGE 4 STAGE 5
MemAcc Writeback
Current PC
4
PC
Adder
R
E
G
I
S
Instruction
Memory (RAM)T
E
R
S
Introduction to Computer Architecture
R
E
Register File
Read Reg 1 G
Read
Data
Read Reg
2 1I
S
Write Reg
Read
Data 2T
Write Data
E
Sign
R
exte
nd
32 S
16
M
U
X
R
E
ALU
G
I
S
T
Write E
Data R
S
R
E
Data Memory
(RAM) G
I
S
T
E
R
S
M
U
X
Read
data
34
Complete 5 Stage Pipeline
M
U
X
Current PC
4
IF/ID
ID/EX
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
16
Introduction to Computer Architecture
Sign
exten
d
M
U
X
M
U
X
ALU
Read
data
32
35
Flow of Instructions Through Pipeline
Program
Execution
Time
Clock
Clock Clock Clock
Clock
Clock Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7
LW R1, 100(R0)
IM
LW R2,200(R0)
LW R3, 300(R0)
Introduction to Computer Architecture
REG
IM
ALU
REG
IM
DM
ALU
REG
Reg
DM
ALU
Reg
DM
Reg
36
Stage 1 - IF (Instruction Fetch)
Instruction Fetch
LW
M
U
X
Current PC
4
IF/ID
ID/EX
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
16
Introduction to Computer Architecture
Sign
exte
nd
M
U
X
ALU
M
U
X
Read
data
32
37
Stage 2 - ID (Instruction Decode)
Instruction Decode
LW
M
U
X
Current PC
4
IF/ID
ID/EX
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
16
Introduction to Computer Architecture
Sign
exte
nd
M
U
X
ALU
M
U
X
Read
data
32
38
Stage 3 - EX (Execution)
Execution
LW
M
U
X
Current PC
4
IF/ID
ID/EX
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
16
Introduction to Computer Architecture
Sign
exte
nd
M
U
X
ALU
M
U
X
Read
data
32
39
Stage 4 - MEM (Memory)
Memory
LW
M
U
X
Current PC
4
IF/ID
ID/EX
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
16
Introduction to Computer Architecture
Sign
exte
nd
M
U
X
ALU
M
U
X
Read
data
32
40
Stage 5 - WB (Write Back)
WriteBack
LW
M
U
X
Current PC
4
IF/ID
ID/EX
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
M
U
X
ALU
M
U
X
Read
data
Sign
extend
16
Introduction to Computer Architecture
32
41
Clock Speed
If a single-cycle machine is broken into 2 pipeline stages,
how much faster can the clock run?
Latency is time from start to completion of instruction
100 nsecs
Instructions
Result
Instructions
Result
Introduction to Computer Architecture
42
How Far Can We Go?
Latency is time from start to completion of instruction
100 nsecs
Instructions
Result
Instructions
Result
Instructions
Introduction to Computer Architecture
Result
43
5 Stage Pipeline
M
U
X
Current PC
4
IF/ID
ID/EX
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
M
U
X
ALU
M
U
X
Read
data
Sign
extend
16
Introduction to Computer Architecture
32
44
Pipeline Control
M
U
X
Current PC
4
W
B
M
C
O
N
T
R
O
L
IF/ID
W
B
M
EX
EX/MEM
ID/EX
Adder
<< 2
ADDER
PC
ALU Control
Register File
Write RegRead
Write DataData 2
16
Introduction to Computer Architecture
Sign
exte
nd
Data Memory
(RAM)
Zero
Read Reg 1
Read
Read RegData
2 1
Instruction
Memory (RAM)
W
B
MEM/WB
M
U
X
M
U
X
ALU
Read
data
32
45
Flow of Instructions Through Pipeline
Program
Execution
Time
Clock
Clock Clock Clock
Clock
Clock Clock
Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8
ADD R2,R3,R1
IM
SUB R5,R6,R7
ADD R10,R11,R12
Introduction to Computer Architecture
REG
IM
ALU
REG
IM
DM
ALU
REG
Reg
DM
ALU
Reg
DM
Reg
46
Contention at the Register File
Program
Execution
Time
Clock
Clock Clock Clock
Clock
Clock Clock
Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8
ADD R10, R11, R12
IM
ADD R17, R0, R0
ADD R16, R0, R0
SUB R20, R21, R22
ADD R30, R17, R18
Introduction to Computer Architecture
REG
IM
ALU
REG
IM
DM
REG
ALU
DM
REG
IM
ALU
REG
IM
REG
DM
ALU
REG
REG
DM
REG
ALU
DM
47
Oops - Sometimes Results Are Not Ready
Program
Execution
Time
Clock
Clock Clock Clock
Clock
Clock Clock
Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8
ADD R2,R3,R1
IM
SUB R5,R6,R7
REG
IM
ALU
REG
Reg
DM
ALU
DM
Reg
Writeback
Result into R10
ADD R10,R11,R12
ADD R12,R10,R11
IM
REG
IM
ALU
REG
DM
ALU
Reg
DM
Reg
Read value out of R10
Introduction to Computer Architecture
48
Data Hazards
Programs assume instructions are executed sequentially with one
instruction completing before the next one begins
Usually the compiler assumes the single machine model
Pipelining violates this assumption
Dependencies can occur between instructions executing concurrently
within the pipeline - if the dependencies are based on data
requirements, we call them Data Hazards
Types of data hazards
Read-after-write (RAW)
A true dependency
Write-after-read (WAR)
Artificial dependency due to register assignment
Write-after-write (WAW)
Artificial dependency due to register assignment
Introduction to Computer Architecture
49
Coping with Data Hazards
Program
Execution
Time
Clock
Clock Clock Clock
Clock
Clock Clock
Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8
ADD R10, R11, R12
IM
REG
ADD R12, R10, R11
IM
ADD R11, R10, R12
Introduction to Computer Architecture
ALU
REG
IM
DM
ALU
REG
Reg
DM
Reg
ALU
DM
Reg
50
Solution 1 : Stall
Program
Execution
Time
Clock
Clock Clock Clock
Clock
Clock Clock
Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8
ADD R10, R11, R12
IM
REG
ADD R12, R10, R11
IM
ADD R11, R10, R12
Introduction to Computer Architecture
ALU
DM
bubble bubble
Reg
REG
IM
ALU
REG
DM
ALU
51
Recall the Registers Between Pipeline Stages
M
U
X
Current PC
4
IF/ID
ID/EX
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
M
U
X
ALU
M
U
X
Read
data
Sign
extend
16
Introduction to Computer Architecture
32
52
Stall Conditions
Need to detect data hazard
Occurs when one instruction tries to read result from previous
instruction that hasnt completed yet.
Specifically,
When Instruction in Execute stage tries to read a register that an
instruction in the MemAcc or WB stages will write back to the
Register File
H&P Notation
ID/EX.RegisterRs refers to the number of the first source register
found in the pipeline register ID/EX.
ID/EX. RegisterRt refers to the number of the second source register
found in the pipeline register ID/EX.
Introduction to Computer Architecture
53
Recall What an Instruction Looks Like
add R8, R17, R18
is stored in binary format as
00000010
00110010
01000000 00100000
MIPS lays out instructions into fields
31 26 25
21 20 16 15 11 10 6
000000 10001 10010
01000 00000
op
rs
rt
rd
shamt
5
0
100000
funct
op
operation of the instruction
rs
first register source operand
rt second register source operand
rd
register destination operand
shamt
shift amount
funct
function (select type of operation)
Introduction to Computer Architecture
54
Remember the Registers In Between Each
Stage
M
U
X
Current PC
4
IF/ID
ID/EX
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
M
U
X
ALU
M
U
X
Read
data
Sign
extend
16
Introduction to Computer Architecture
32
55
Data Hazard Stall Conditions
1a EX/MEM. RegisterRd
1b EX/MEM.RegisterRd
IF/ID
==
==
ID/EX
Rs
Rt
Rd
Current PC
4
ID/EX.RegisterRs
ID/EX. RegisterRt
EX/MEM
Rs =? Rd
Rt =? Rd Rd
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Data 1
Read Reg 2
Write Reg Read
Write Data Data 2
Instruction
Memory (RAM)
M
U
X
ALU
M
U
X
Read
data
Sign
extend
Introduction to Computer Architecture
16
32
56
Data Hazard Stall Conditions (cont)
1a
1b
2a
2b
EX/MEM. RegisterRd
EX/MEM.RegisterRd
MEM/WB. RegisterRd
MEM/WB. RegisterRd
IF/ID
==
==
==
==
ID/EX
EX/MEM
Rs =? Rd
Rt =? RdRd
Rs
Rt
Rd
Current PC
4
ID/EX.RegisterRs
ID/EX. RegisterRt
ID/EX. RegisterRs
ID/EX. RegisterRt
MEM/WB
Rd
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Data 1
Read Reg 2
Write Reg Read
Write Data Data 2
Instruction
Memory (RAM)
M
U
X
ALU
M
U
X
Read
data
Sign
extend
Introduction to Computer Architecture
16
32
57
Data Hazard Logic
Data Hazard Logic
Rs =? Rd
Rt =? Rd
between ID/EX, EX/MEM, and MEM/WB Stages
IF/ID
ID/EX
Rs
Rt
Rd
Current PC
4
EX/MEM
MEM/WB
Rd
Rd
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Data 1
Read Reg 2
Write Reg Read
Write Data Data 2
Instruction
Memory (RAM)
M
U
X
ALU
M
U
X
Read
data
Sign
extend
Introduction to Computer Architecture
16
32
58
Example
sub
and
or
add
sw
R2, R1, R3
R12, R2, R5
R13, R6, R2
R14, R2, R2
R15, 100(R2)
Rd = R2
Rd = R12
Rd = R13
Rd = R14
Rd = R15
Rs = R1
Rs = R2
Rs = R6
Rs = R2
Rs = R2
Rt = R3
Rt = R5
Rt = R2
Rt = R2
Rt = XX
SUB-AND Hazard
EX/MEM.RegisterRd
== ID/EX. RegisterRs
== R2
== ID/EX. RegisterRt
== R2
SUB-OR Hazard
MEM/WB.RegisterRd
Do we care about the interaction between sub (instruction 1) and add
(instruction 4)?
Introduction to Computer Architecture
59
Example (cont)
Data Hazard Logic
Current PC
4
Adder
ID/EX
EX/MEM
MEM/WB
Rs =
Rt =
Rd =
Rd =
Rd =
SUB R2, R1, R3
IF/ID
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Data 1
Read Reg 2
Write Reg Read
Write Data Data 2
Instruction
Memory (RAM)
M
U
X
ALU
M
U
X
Read
data
Sign
extend
Introduction to Computer Architecture
16
32
60
Example (cont)
Data Hazard Logic
PC
Adder
SUB R2, R1, R3
Current PC
ID/EX
AND R12, R2, R5
IF/ID
Rs = R3
Rt = R1
Rd = R2
<< 2
Rd =
Rd =
Data Memory
(RAM)
Zero
Read Reg 1
Read
Data 1
Read Reg 2
Instruction
Memory (RAM)
MEM/WB
ADDER
Register File
Write Reg Read
Write Data Data 2
EX/MEM
M
U
X
ALU
M
U
X
Read
data
Sign
extend
Introduction to Computer Architecture
16
32
61
Example (cont)
Data Hazard Logic
EX/MEM.RegisterRD = R2 != ID/EX.RegisterRs = R5
EX/MEM.RegisterRD = R2 == ID/EX.RegisterRt = R2
Adder
PC
EX/MEM
SUB R2, R1, R3
OR R13, R6, R2
Current PC
ID/EX
AND R12, R2, R5
IF/ID
Rs = R5
Rt = R2
Rd = R12
<< 2
ADDER
Rd =
Zero
Read Reg 1
Read
Data 1
Read Reg 2
Instruction
Memory (RAM)
Rd = R2
Data Memory
(RAM)
Register File
Write Reg Read
Write Data Data 2
MEM/WB
M
U
X
ALU
M
U
X
Read
data
Sign
extend
Introduction to Computer Architecture
16
32
62
Example (cont)
Data Hazard Logic
PC
EX/MEM
AND R12, R2, R5
Adder
ADD
OR R13, R6, R2
Current PC
ID/EX
R14, R2, R2
IF/ID
ID/EX.RegisterRs = R6
ID/EX.RegisterRt = R2
ID/EX.RegisterRs = R6
ID/EX.RegisterRt = R2
Rs = R6
Rt = R2
Rd = R13
<< 2
ADDER
Zero
Read Reg 1
Read
Data 1
Read Reg 2
Instruction
Memory (RAM)
Rd = R2
Data Memory
(RAM)
Register File
Write Reg Read
Write Data Data 2
Rd = R12
MEM/WB
SUB R2, R1, R3
EX/MEM.RegisterRD = R12 !=
EX/MEM.RegisterRD = R12 !=
MEM/WB.RegisterRD = R2 !=
MEM/WB.RegisterRD = R2 ==
M
U
X
ALU
M
U
X
Read
data
Sign
extend
Introduction to Computer Architecture
16
32
63
No Dependence Between Instruction 1 and 4
Program
Execution
Clock
Clock Clock Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4
SUB R2, R1, R3 IM
AND R12, R2, R5
OR R13, R6, R2
ADD R14, R2, R2
Introduction to Computer Architecture
REG
IM
ALU
REG
IM
Clock
Clock Clock
Clock
Cycle 5 Cycle 6Cycle 7 Cycle 8
DM
REG
ALU
DM
REG
ALU
DM
REG
IM
REG
ALU
REG
DM
REG
64
How Do We Stall the Pipeline?
Compiler can insert nops
Program
Execution
Time
Clock
Clock Clock Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4
ADD R10, R11, R12
IM
nop
nop
ADD R12, R10, R11
Introduction to Computer Architecture
REG
IM
ALU
REG
IM
DM
ALU
REG
IM
Clock
Clock Clock
Clock
Cycle 5 Cycle 6Cycle 7 Cycle 8
Reg
DM
Reg
ALU
DM
REG
ALU
Reg
DM
Reg
65
Hardware Can Simulate NOPS
Program
Execution
Time
Clock
Clock Clock Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4
ADD R10, R11, R12
IM
stall
stall
ADD R12, R10, R11
Introduction to Computer Architecture
REG
IM
ALU
DM
Clock
Clock Clock
Clock
Cycle 5 Cycle 6Cycle 7 Cycle 8
Reg
bubble bubble bubble
IM
bubble bubble
IM
REG
bubble
bubble bubble
ALU
DM
Reg
66
Reducing Data Hazards: Forwarding
Data may be already computed - just not in the Register File
Program
Execution
Time
Clock
Clock Clock Clock
Clock
Clock Clock
Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8
ADD R10, R11, R12
IM
REG
ADD R12, R10, R11
IM
Introduction to Computer Architecture
ALU
REG
DM
Reg
ALU
DM
Reg
67
Additions to the Datapath for Forwarding
M
U
X
Current PC
4
ADD R12, R11, R10 ADD R10,R11, R12
IF/ID
ID/EX
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
M
U
X
M
U
X
ALU
Read
data
Sign
extend
16
Introduction to Computer Architecture
32
68
Forwarding Continued
Program
Execution
Time
Clock
Clock Clock Clock
Clock
Clock Clock
Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle
ADD R10, R11, R12
IM
REG
ADD R12, R10, R11
IM
ADD R4, R5, R10
Introduction to Computer Architecture
ALU
REG
IM
DM
Reg
ALU
DM
REG
ALU
Reg
DM
Reg
69
More Additions to the Datapath
M
U
X
Current PC
4
ADD R4, R5, R10
IF/ID
ID/EX
ADD R12, R11, R10 ADD R10,R11, R12
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
M
U
X
M
U
X
ALU
Read
data
Sign
extend
16
Introduction to Computer Architecture
32
70
Forwarding Doesnt Always Work
Program
Execution
Time
Clock
Clock Clock Clock
Clock
Clock Clock
Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8
LW R10, 0x00(R4)IM
REG
ADD R12, R10, R11
IM
Introduction to Computer Architecture
ALU
REG
DM
Reg
ALU
DM
Reg
71
Loads and Stores Require a Load Delay Slot
Program
Execution
Time
Clock
Clock Clock Clock
Clock
Clock Clock
Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8
LW R10, 0x00(R4)IM
nop
ADD R12, R10, R11
Introduction to Computer Architecture
REG
IM
ALU
REG
IM
DM
Reg
ALU
DM
REG
ALU
Reg
DM
Reg
72
MIPS Load-Delay Slot
MIPS exposed the load-delay slot to the compiler
This makes it part of the architecture, not just an implementation
detail
Therefore, its up to the compiler (or assembly code writer) to
make sure that the instruction after a load does not depend on
the result of the load
An alternative would have been to force the hardware to
detect the data hazard and stall the pipeline
Most of todays architectures detect the hazard and stall
Introduction to Computer Architecture
73
3 Types of Data Hazards
Read-after-write (RAW)
a true dependency
Example
ADD R1, R2, R3
SUB R6, R7,R1
Write-after-read (WAR)
artificial dependency due to register assignment
Example
LW R1,0(R2)
ADD R2, R6, R3
Write-after-write (WAW)
artificial dependency due to register assignment
Example
LW R1, 0(R2)
ADD R1, R3, R4
Introduction to Computer Architecture
74
Taxonomy of Hazards
Data Hazards are just one type of hazard that can occur
in a machine. There are actually 3 basic types of hazards
Hazard Taxonomy
Data hazards
Instruction depends on result of prior computation which is not ready
yet
Structural hazards
HW cannot support a combination of instructions
Control hazards
pipelining of branches and other instructions which change the PC
Introduction to Computer Architecture
75
Structural Hazards
Structural hazards
HW cannot support a combination of instructions
Occurs when two or more instructions want to use the same
hardware resource in the same cycle
Causes bubble (stall) in pipelined machines
Overcome by replicating hardware resources
Examples
Multiple accesses to the register file
Branch adder and ALU
Multiple accesses to memory
Introduction to Computer Architecture
76
Structural Hazard Example 1
M
U
X
Current PC
4
W/out adder, both the address computation and the
arithmetic computation would require access to the ALU in
the same cycle
beq
r1,r2, offset
IF/ID
; if r1 == r2, then PC <-- PC + offset
ID/EX
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
M
U
X
ALU
M
U
X
Read
data
Sign
extend
16
Introduction to Computer Architecture
32
77
Structural Hazard Example 2
Two instructions need access to memory in Clock
Cycle 4.
If there is only one memory port, then only one
instruction can read/write memory at a time
Program
Execution
Time
Clock
Clock Clock Clock
Clock
Clock Clock
Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8
LW R2, 0x10(R4)
IM
SUB R5,R6,R7
ADD R10,R11,R12
ADD R12, R10, R11
Introduction to Computer Architecture
REG
IM
ALU
REG
IM
DM
ALU
Reg
DM
REG
ALU
IM
REG
Reg
DM
ALU
Reg
DM
Reg
78
Structural Example 2 (cont)
Two instructions need access to memory in Clock Cycle 4.
If there is only one memory port, then only one instruction can
read/write memory at a time
Program
Execution
Time
Clock
Clock Clock Clock
Clock
Clock Clock
Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8
LW R2, 0x10(R4)
IM
SUB R5,R6,R7
ADD R10,R11,R12
Stall
ADD R12, R10, R11
Introduction to Computer Architecture
REG
IM
ALU
REG
IM
DM
ALU
REG
Reg
DM
ALU
Reg
DM
Reg
bubble bubble bubble bubble bubble
IM
REG
ALU
DM
79
Control Hazards - Branches
Example code
Address
36
40
44
48
52
56
60
64
68
72
76
Instruction
NOP
ADD R30,R30,R30
BEQ R1, R3, 24
<- this branchs to address 72
AND R12, R2, R5
OR R13, R6, R2
ADD R14, R2, R2
...
...
...
LW R4, 50(R7)
...
Flow of instructions if branch is taken: 36, 40, 44, 72, ...
Flow of instructions if branch is not taken: 36, 40, 44, 48, ...
Introduction to Computer Architecture
80
Branch Hazards
Flow of instructions if branch is taken: 36, 40, 44, 72, ...
Flow of instructions if branch is not taken: 36, 40, 44, 48, ...
Clock
Clock Clock Clock
Clock
Clock Clock Clock Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8 Cycle 9
44 BEQ R1, R3,IM
24
48 AND R12, R2, R5
REG
ALU
IM
52 OR R13, R6, R2
56 ADD R14, R2, R2
60 or 72 (depending on branch)
Introduction to Computer Architecture
REG
IM
DM
Reg
ALU
DM
REG
IM
ALU
REG
IM
Reg
DM
ALU
REG
Reg
DM
ALU
Reg
Reg
DM
81
Always Stalling hurts the No-branch case
Flow of instructions if branch is not taken: 36, 40, 44, 48, ...
Clock
Clock Clock Clock
Clock
Clock Clock Clock Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8 Cycle 9
44 BEQ R1, R3,IM
24
stall
REG
IM
stall
stall
48 AND R12, R2, R5
Introduction to Computer Architecture
ALU
DM
Reg
bubble bubble bubble
IM
bubble bubble
IM
bubble
bubble bubble
bubble bubble bubble bubble
IM
REG
ALU
DM
Reg
82
Solution: Assume Branch Not Taken
Flow of instructions if branch is taken: 36, 40, 44, 72, ...
Flow of instructions if branch is not taken: 36, 40, 44, 48, ...
Clock
Clock Clock Clock
Clock
Clock Clock Clock Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4 Cycle 5 Cycle 6Cycle 7 Cycle 8 Cycle 9
44 BEQ R1, R3,IM
24
48 AND R12, R2, R5
REG
IM
52 OR R13, R6, R2
56 ADD R14, R2, R2
ALU
REG
IM
DM
Reg
ALU
DM
REG
IM
60 or 72 (depending on outcome of branch)
Introduction to Computer Architecture
ALU
REG
IM
Reg
DM
ALU
REG
Reg
DM
ALU
Reg
DM
Reg
83
What Happens When the Branch IS Taken
Flow of instructions if branch is taken: 36, 40, 44, 72, ...
Clock
Clock Clock Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4
44 BEQ R1, R3,IM
24
48 AND R12, R2, R5
REG
IM
52 OR R13, R6, R2
56 ADD R14, R2, R2
72 LW R4, 50(R7)
Introduction to Computer Architecture
ALU
REG
IM
Clock
Clock Clock Clock Clock
Cycle 5 Cycle 6Cycle 7 Cycle 8 Cycle 9
DM
Reg
ALU
DM
REG
IM
ALU
REG
IM
Reg
DM
ALU
REG
Reg
DM
ALU
Reg
Reg
DM
84
Move the Branch Computation Forward
M
U
X
Current PC
4
IF/ID
ID/EX
EX/MEM
MEM/WB
Adder
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
M
U
X
ALU
M
U
X
Read
data
Sign
extend
16
Introduction to Computer Architecture
32
85
Branch with New Datapath
Reducing penalty 1 cycle
Clock
Clock Clock Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4
44 BEQ R1, R3,IM
24
48 AND R12, R2, R5
REG
IM
52 OR R13, R6, R2
72 LW R4, 50(R7)
Introduction to Computer Architecture
ALU
REG
IM
Clock
Clock Clock Clock Clock
Cycle 5 Cycle 6Cycle 7 Cycle 8 Cycle 9
DM
Reg
ALU
DM
REG
IM
ALU
REG
Reg
DM
ALU
Reg
DM
Reg
86
Move the Branch Computation Further Forward
M
U
X
Current PC
4
Compare Controls MUX Selext
IF/ID
ADDER
ID/EX
EX/MEM
MEM/WB
Adder
Compare
<< 2
ADDER
PC
Data Memory
(RAM)
Register File
Zero
Read Reg 1
Read
Read RegData
2 1
Write RegRead
Instruction
Memory (RAM)
Write DataData 2
M
U
X
ALU
M
U
X
Read
data
Sign
extend
16
Introduction to Computer Architecture
32
87
Another New and Improved Datapath
Voila - the branch delay slot
Clock
Clock Clock Clock
Cycle 1 Cycle 2 Cycle 3Cycle 4
44 BEQ R1, R3,IM
24
48 AND R12, R2, R5
REG
IM
72 LW R4, 50(R7)
Introduction to Computer Architecture
ALU
REG
IM
Clock
Clock Clock Clock Clock
Cycle 5 Cycle 6Cycle 7 Cycle 8 Cycle 9
DM
Reg
ALU
DM
REG
ALU
Reg
DM
Reg
88
Rewriting the Code for a Branch Delay Slot
Without Branch Delay Slot
Address
36
40
44
48
52
56
60
64
68
72
76
Instruction
NOP
ADD R30,R30,R30
BEQ R1, R3, 24
AND R12, R2, R5
OR R13, R6, R2
ADD R14, R2, R2
...
...
...
LW R4, 50(R7)
...
With Branch Delay Slot
Address
36
40
44
48
52
56
60
64
68
72
76
Instruction
NOP
BEQ R1, R3, 28
ADD R30, R30, R30
AND R12, R2, R5
OR R13, R6, R2
ADD R14, R2, R2
...
...
...
LW R4, 50(R7)
...
Flow of instructions if branch is taken: 36, 40, 44, 72, ...
Flow of instructions if branch is not taken: 36, 40, 44, 48, ...
Introduction to Computer Architecture
89
Performance of Pipelined Systems
Stalls due to data and branch hazards make performance
less than one instruction per cycle
Compiler is critical in determining overall performance
Compiler generates code that avoids stalls
Example
lw R15, 0x00(R2)
add R14, R15, R15
lw R16, 0x04(R2)
Might become:
lw R15, 0x00(R2)
lw R16, 0x04(R2)
add R14, R15, R15
Introduction to Computer Architecture
90
Performance of Pipelined Systems
time
Unpipelined
instructions
time
Pipelined
latency
instructions
Ideally, Throughputpipeline =
Introduction to Computer Architecture
Timesequential
Pipeline Depth
91
Pipeline Speedup and Throughput
Assume instruction execution takes N stages
s1, s2, ... sn take time t1, t2, ... tn
Without pipelining
Throughput = 1/ ti (for i = 1 to n)
Latency = 1/throughput
With pipelining
Throughput = 1/max ti <= n/ ti
Latency = n/throughput
Speedup = ti / max ti <= n
Introduction to Computer Architecture
(for i = 1 to n)
92
What Makes Pipelines Hard to Implement?
Detecting and resolving hazards
Exceptions and Interrupts
Instruction Set Architecture
CISC instructions are difficult to pipeline
Example:
stringMov from 0x1234, to 0x4000, 0x1000 bytes
Introduction to Computer Architecture
93
