Pipelining and Parallel

Processing
 What is Pipelining?
 A way of speeding up execution of
instructions

 Key idea:
overlap execution of multiple instructions
 What is Pipelining
 A technique used in advanced microprocessors
where the microprocessor begins executing a
second instruction before the first has been
completed.

- A Pipeline is a series of stages, where some work

is done at each stage. The work is not finished
until it has passed through all stages.
 With pipelining, the computer architecture allows
the next instructions to be fetched while the
processor is performing arithmetic operations,
holding them in a buffer close to the processor
until each instruction operation can performed.
 How Pipelines Works

 The pipeline is divided into segments

and each segment can execute it
operation concurrently with the other
segments. Once a segment
completes an operations, it passes
the result to the next segment in the
pipeline and fetches the next
operations from the preceding
segment.
 Ann, Brian, Cathy, Dave
each have one load of clothes
to wash, dry, and fold
A B C D
 Washer takes 30 minutes

 Dryer takes 30 minutes

 “Folder” takes 30 minutes

 “Stasher” takes 30 minutes

to put clothes into drawers
 If we do laundry
6 PM 7
sequentially...
8 9 10 11 12 1 2 AM

3030 30 30 3030 30 30 3030 30 30 3030 30 30

T Time
a A
s
k
B
O
r C
d
e D
r
 Time Required: 8 hours for 4 loads
 To Pipeline, We Overlap
6 PM 7 8 9
Tasks
10 11 12 1 2 AM

3030 30 30 30 3030 Time

T
a A
s
k
B
O
r C
d
e D
r
 Time Required: 3.5 Hours for 4 Loads
 To Pipeline, We Overlap
6 PM 7 8 9
Tasks
10 11 12 1 2 AM

3030 30 30 30 3030 Time

T
a A
s
k
B • Pipelining doesn’t help latency of
O
r C single task, it helps throughput of
d entire workload
e D
r
Computer performance in terms of
Latency and throughput

•Latency—the amount of time that a

single operation takes to execute

•Throughput —the rate at which

operations get executed (generally
expressed as operations/second or
operations/cycle)

•Pipelining increases throughput, but

 Parallel Processing

Execution of Concurrent Events in the computing

process to achieve faster Computational Speed

The purpose of parallel processing is to speed up the

computer processing capability and increase its
throughput.

The amount of hardware increases with parallel

processing, and with it, the cost of the system
increases.

However, technological developments have reduced

hardware costs to the point where parallel processing
techniques are economically feasible.
 Basic Ideas
 Parallel processing  Pipelined processing
time time
P1 a1 a2 a3 a4 P1 a1 b1 c1 d1

P2 b1 b2 b3 b4 P2 a2 b2 c2 d2

P3 c1 c2 c3 c4 P3 a3 b3 c3 d3

P4 d1 d2 d3 d4 P4 a4 b4 c4 d4

Less inter-processor communication

More inter-processor communication
Complicated processor hardware Simpler processor hardware

Colors: different types of operations performed

a, b, c, d: different data streams processed
 Data Dependence
 Parallel processing  Pipelined processing will
requires NO data involve inter-processor
dependence between communication
processors
P1 P1

P2 P2

P3 P3

P4 P4

time time
 Pipelining a Processor
 Recall the 5 steps in instruction
execution:
1. Fetch instruction (FI)
2. Decode instruction (DI)
3. Calculate operands (CO)
4. Fetch operands (FO)
5. Execute instructions (EI)
6. Write result (WR)

 Review: Single-Cycle Processor

 All 6 steps done in a single clock cycle
 Dedicated hardware required for each step
 Instructions Fetch

 The instruction Fetch (IF) stage is responsible for

obtaining the requested instruction from memory.
The instruction address are stored in the register
as temporary storage.
 Instruction Decode

 The Instruction Decode (ID) stage is responsible

for decoding the instruction and sending out the
various control lines to the other parts of the
processor. The instruction is sent to the control
unit where it is decoded
 Calculate Operands

• The CO stage is where any calculations are

performed. The main component in this stage
is the ALU. The ALU is made up of arithmetic,
logic and capabilities.
 Fetch Operands and
Execute Instruction

 The FO and EI stages are responsible

for storing and loading values to and
from memory. They also responsible
for input and output from the
processor respectively.
 Write Operands

 The WO stage is responsible for writing the result

of a calculation into main memory.

•
Timing diagram for instruction
pipeline operation
 Advantages/Disadvantages
Advantages:
 More efficient use of processor
 Quicker time of execution of large number of
instructions

Disadvantages:
 Pipelining involves adding hardware to the
chip
 Inability to continuously run the pipeline
at full speed because of pipeline hazards
which disrupt the smooth execution of the
pipeline.
Comments about Pipelining
 The good news
 Multiple instructions are being processed at same time
 This works because stages are isolated by registers

 The bad news

 Instructions interfere with each other - hazards
 Example: different instructions may need the same

piece of hardware (e.g., memory) in same clock cycle

 Example: instruction may require a result produced by

an earlier instruction that is not yet complete

 Pipeline Hazards
 Data Hazards – an instruction uses the result of
the previous instruction. A hazard occurs exactly
when an instruction tries to read a register in its
ID stage that an earlier instruction intends to
write in its WB stage.

 Structural Hazards – two instructions need to

access the same resource.
 two different instructions use same h/w in same
cycle
 Data Hazards
Select R2 and R3 for ADD R2 and R3 STORE SUM IN
ALU Operations R1

ADD R1, R2, R3 IF ID EX M WB

SUB R4, R1, R5 IF ID EX M WB

Select R1 and R5 for

ALU Operations
 Stalling
 Stalling involves halting the flow of instructions
until the required result is ready to be used.
However stalling wastes processor time by
doing nothing while waiting for the result.
ADD R1, R2, R3 IF ID EX M WB

STALL IF ID EX M WB

STALL IF ID EX M WB

STALL IF ID EX M WB

SUB R4, R1, R5 IF ID EX M WB

 Summary - Pipelining
Overview
 Pipelining increase throughput (but
not latency)
 Hazards limit performance
 Structural hazards
 Data hazards