UNIT 4 Pipelining: Basic and

Intermediate Concepts
 What Is A Pipeline?
• Pipelining is used by virtually all modern
microprocessors to enhance performance by
overlapping the execution of instructions.
• A common analogue for a pipeline is a factory
assembly line. Assume that there are three stages:
1. Welding
2. Painting
3. Polishing
• For simplicity, assume that each task takes one hour.
 What Is A Pipeline?
• If a single person were to work on the product it
would take three hours to produce one product.
• If we had three people, one person could work on
each stage, upon completing their stage they could
pass their product on to the next person (since each
stage takes one hour there will be no waiting).
• We could then produce one product per hour
assuming the assembly line has been filled.
 Pipelining
• Pipelining is the process of accumulating instruction from the
processor through a pipeline. It allows storing and executing
instructions in an orderly process. It is also known as pipeline
processing.
• Pipelining is a technique where multiple instructions are
overlapped during execution. Pipeline is divided into stages
and these stages are connected with one another to form a
pipe like structure. Instructions enter from one end and exit
from another end.
• Pipelining increases the overall instruction throughput.
In pipeline system, each segment consists of an input register followed by
a combinational circuit. The register is used to hold data and
combinational circuit performs operations on it. The output of
combinational circuit is applied to the input register of the next segment.
 Design of a basic pipeline

• In a pipelined processor, a pipeline has two ends, the

input end and the output end. Between these ends,
there are multiple stages/segments such that the output
of one stage is connected to the input of the next stage
and each stage performs a specific operation.
• Interface registers are used to hold the intermediate
output between two stages. These interface registers
are also called latch or buffer.
• All the stages in the pipeline along with the interface
registers are controlled by a common clock.
 Non-overlapped execution:
Stage /
Cycle 1 2 3 4 5 6 7 8

S1 I1 I2

S2 I1 I2

S3 I1 I2

S4 I1 I2

Total time = 8 Cycle

 Overlapped execution:
Stage /
Cycle 1 2 3 4 5

S1 I1 I2

S2 I1 I2

S3 I1 I2

S4 I1 I2

Total time = 5 Cycle

Throughput = Number of instructions /
Total time to complete the instructions
 Characteristics Of Pipelining
• If the stages of a pipeline are not balanced and one
stage is slower than another, the entire throughput of
the pipeline is affected.
• In terms of a pipeline within a CPU, each instruction
is broken up into different stages. Ideally if each stage
is balanced (all stages are ready to start at the same
time and take an equal amount of time to execute.) the
time taken per instruction (pipelined) is defined as:

Time per instruction (Unpipelined) / Number of stages

 Characteristics Of Pipelining
• The previous expression is ideal. We will see later that
there are many ways in which a pipeline cannot
function in a perfectly balanced fashion.
• In terms of a CPU, the implementation of pipelining
has the effect of reducing the average instruction time,
therefore reducing the average CPI(Cycle per
Instruction).
• EX: If each instruction in a microprocessor takes 5
clock cycles (unpipelined) and we have a 4 stage
pipeline, the ideal average CPI with the pipeline will
be 1.25 .
 Pipeline Stages
• Stage 1 (Instruction Fetch) In this stage the CPU reads instructions
from the address in the memory whose value is present in the
program counter.
• Stage 2 (Instruction Decode) In this stage, instruction is decoded
and the register file is accessed to get the values from the registers
used in the instruction.
• Stage 3 (Instruction Execute) In this stage, ALU operations are
performed.
• Stage 4 (Memory Access) In this stage, memory operands are read
and written from/to the memory that is present in the instruction.
• Stage 5 (Write Back) In this stage, computed/fetched value is
written back to the register present in the instructions.

• .
 Types of Pipeline

It is divided into 2 categories:

1. Arithmetic Pipeline
2. Instruction Pipeline
 Arithmetic Pipeline
Arithmetic pipelines are usually found in most of the computers.
They are used for floating point operations, multiplication of fixed
point numbers etc. For example: The input to the Floating Point
Adder pipeline is:
•X = A*2^a
•Y = B*2^b
Here A and B are mantissas (significant digit of floating point
numbers), while a and b are exponents.
•The floating point addition and subtraction is done in 4 parts:
1. Compare the exponents.
2. Align the mantissas.
3. Add or subtract mantissas
4. Produce the result.
 Instruction Pipeline
• In this a stream of instructions can be executed by
overlapping fetch, decode and execute phases of an
instruction cycle. This type of technique is used to increase
the throughput of the computer system.
• An instruction pipeline reads instruction from the memory
while previous instructions are being executed in other
segments of the pipeline. Thus we can execute multiple
instructions simultaneously. The pipeline will be more
efficient if the instruction cycle is divided into segments of
equal duration.
 Pipeline Hazards
• The performance gain from using pipelining occurs
because we can start the execution of a new
instruction each clock cycle. In a real implementation
this is not always possible.
• Another important note is that in a pipelined
processor, a particular instruction still takes at least as
long to execute as non-pipelined.
• Pipeline hazards prevent the execution of the next
instruction during the appropriate clock cycle.
 Types Of Hazards
• There are three types of hazards in a pipeline, they are
as follows:
 Data Hazards: When there is an instruction in the
pipeline that affects the result of another instruction in
the pipeline.
 Structural Hazards: are created when the data path

hardware in the pipeline cannot support all of the

overlapped instructions in the pipeline.(multiple
instruction same recourse use)
 Control Hazards: The PC causes these due to the

pipelining of branches and other instructions that change

the PC.
 Data Hazards (A Programming Problem?)

• Consider the following operations:

R2<- R2 + R3
R5<- R2 + R4
 Structural Hazards

• Structural hazards result from the CPU data path not

having resources to service all the required
overlapping resources.
• Suppose a processor can only read and write from the
registers in one clock cycle. This would cause a
problem during the ID and WB stages.
• Assume that there are not separate instruction and data
caches, and only one memory access can occur during
one clock cycle. A hazard would be caused during the
IF and MEM cycles.
 Control Hazards
• Control hazards are caused by branches in the code.
• During the IF stage remember that the PC is
incremented by 4 in preparation for the next IF cycle
of the next instruction.
• What happens if there is a branch performed and we
aren’t simply incrementing the PC by 4.
• The easiest way to deal with the occurrence of a
branch is to perform the IF stage again once the
branch occurs.
 Performing IF Twice
• We take a big performance hit by performing the
instruction fetch whenever a branch occurs. Note, this
happens even if the branch is taken or not. This
guarantees that the PC will get the correct value.

IF ID EX MEM WB
branch IF ID EX MEM WB
IF IF ID EX MEM WB
 Control Hazards (other solutions)

• These following solutions assume that we are dealing

with static branches. Meaning that the actions taken
during a branch do not change.
• We already saw the first example, we stall the pipeline
until the branch is resolved (in our case we repeated
the IF stage until the branch resolved and modified the
PC)
 Control Hazards (other solutions)

• If the branch is actually taken we need to clear the

pipeline of any code loaded in from the “not-taken”
path.
• Likewise we can assume that the branch is always
taken. Does this work in our “5-stage” pipeline?
• No, the branch target is computed during the ID cycle.
Some processors will have the target address
computed in time for the IF stage of the next
instruction so there is no delay.
Thank You