EA 2004

Computer Architecture - II

Pipelining
 Parallel processing

A parallel processing system is able to perform

simultaneous data processing to achieve faster
execution time

The system may have two or more ALUs and be able

to execute two or more instructions at the same time

Goal is to increase the throughput the amount of

processing that can be accomplished during a given
interval of time
 Parallel processing

Parallel processing can happen in two basic streams:

instruction stream - consists of the sequence of

instructions read from memory

data stream - encapsulates the operations performed

on the memory

computers can be classified into 4 different categories more

focused on the behavioral aspects of Parallel Processing
 Parallel processing classification

Single instruction stream, single data stream SISD

Single instruction stream, multiple data stream SIMD

Multiple instruction stream, single data stream MISD

Multiple instruction stream, multiple data stream MIMD

 Single instruction stream, single
data stream SISD

A single control unit

A Processor Unit

A memory unit

Instructions are executed sequentially. Parallel processing

may be achieved by means of multiple functional
units or by pipeline processing
 Single instruction stream,
multiple data stream SIMD
A single control unit
Many Processor Units
A memory unit

Includes multiple processing units with a single control

unit. All processors receive the same instruction, but
operate on different data.
 Multiple instruction stream,
single data stream MISD
Many Processor Units
Which on its own contains
A control unit
A local memory
Theoretical only

processors receive different instructions, but operate

on same data.

i.e. Space shuttle flight control systems

 Multiple instruction stream,
multiple data stream MIMD
Many Processor Units
Many Control Units
A computer system capable of processing several
programs at the same time.

Most multiprocessor and supercomputer systems can

be classified in this category

Parallel processing also be classified via pipelining which

concerns operational and structural interconnections
 What is a Pipeline
Pipelining is used by all modern microprocessors to
enhance performance by overlapping the execution
of instructions.
A common analogue for a pipeline is a factory
assembly line. Assume that there are three stages:
o Welding
o Painting
o Polishing

For simplicity, assume that each task takes one hour.

 What is a Pipeline
A single person would take three hours to produce one
product.

Three people, one person could work on each stage,

upon completing their stage they could pass their
product on to the next person (since each stage takes
one hour there will be no waiting).

Then produce one product per hour assuming the

assembly line has been filled.
Pipelining: Laundry Example

Small laundry has one

washer, one dryer and one
operator, it takes 90 A B C D
minutes to finish one load:

Washer takes 30 minutes

Dryer takes 40 minutes
operator folding takes 20
minutes
 Sequential Laundry
6 PM 7 8 9 10 11 Midnight
Time

30 40 20 30 40 20 30 40 20 30 40 20
T
a A
s
k
B
O
r
d C
e 90 min
r
D
This operator scheduled his loads to be delivered to the laundry every 90
minutes which is the time required to finish one load. In other words he
will not start a new task unless he is already done with the previous task
The process is sequential. Sequential laundry takes 6 hours for 4 loads
Efficiently scheduled laundry: Pipelined Laundry

6 PM 7 8 9 10 11 Midnight
Time

30 40 40 40 40 20
40 40 40
T
a A
s
k
B
O
r
d C
e
r
D
Another operator asks for the delivery of loads to the laundry every 40
minutes!?.
Pipelined laundry takes 3.5 hours for 4 loads
 Pipelining Facts Multiple tasks operating
simultaneously
Pipelining doesnt help
6 PM latency of single task, it
7 8 9
helps throughput of entire
Time workload
T
a 30 40 40 40 40 20 Pipeline rate limited by
s slowest pipeline stage
k A
Potential speedup =
O Number of pipe stages
r B
d Unbalanced lengths of pipe
e The washer stages reduces speedup
r C waits for the
dryer for 10
minutes Time to fill pipeline and
D time to drain it reduces
speedup
 Building a Car
Unpipelined Start and finish a job before moving to the next

Parallelism = 1 car

24 hrs.
Latency= 24 hrs.
Throughput = 1/24 hrs.
24 hrs.

Jobs
24 hrs.

Time
Latency the amount of time that a single operation takes to execute
Throughput the rate at which operations get executed (generally
expressed as operations/second or operations/cycle)
 The Assembly Line
Pipelined Break the job into smaller stages
Eng. Body Paint 8h
A B C Parallelism = 3 cars
8h Eng. Body Paint Latency= 24 hrs.
A B C
Throughput = 1/8 hrs.
Eng. Body Paint
A B C
Jobs
Eng. Body Paint
3X
A B C

Time
 In computer..
Unpipelined Start and finish a job before moving to the next
FET DEC EXE

FET DEC EXE

Jobs

Time
 In computer..
Pipelined Break the job into smaller stages
FET DEC EXE
A B C
I1 I1 I1
Cycle 1 FET DEC EXE
A B C
I2 I2
Cycle 2 FET DEC EXE
A B C
Jobs I3
Cycle 3
A B C

Time
 In computer..
Unpipelined Start and finish a job before moving to the next
FET DEC EXE

3 ns FET DEC EXE

Jobs

Clock Speed = 1/3ns = 333 MHz

Time
 In computer..
Pipelined Break the job into smaller stages
FET DEC EXE
A B C
I1 I1 I1 Clock Speed = 1/1ns = 1 GHz
Cycle 1 FET DEC EXE
A B C
I2 I2
Cycle 2 FET DEC EXE
A B C
Jobs I3
Cycle 3
A B C
1ns
3 ns
Time
 Pipelining

Latency the amount of time that a single operation takes to execute

Throughput the rate at which operations get executed (generally
expressed as operations/second or operations/cycle)
 Clocks and Latches

Stage 1 Stage 2
 Clocks and Latches

Stage 1 L Stage 2 L

Clk
 Clocks and Latches

Stage 1 L Stage 2 L

Clk

Four segment pipeline:

Clock

Input S1 R1 S2 R2 S3 R3 S4 R4
 Example
Assume a 2 ns flip-flop delay
 Characteristics Of Pipelining
Decomposes a sequential process into segments.

Divide the processor into segment processors each one

is dedicated to a particular segment.

Each segment is executed in a dedicated segment-

processor operates concurrently with all other segments.

Information flows through these multiple hardware

segments.

If the stages of a pipeline are not balanced and one

stage is slower than another, the entire throughput of
the pipeline is affected
 Pipelining
Instruction execution is divided into k segments or
stages
Instruction exits pipe stage k-1 and proceeds into pipe

stage k
All pipe stages take the same amount of time; called

one processor cycle

Length of the processor cycle is determined by the

slowest pipe stage

k segments
 Pipeline Performance
n:instructions n is equivalent to number of loads in
the laundry example
k: stages in
k is the stages (washing, drying and
pipeline folding.
: clock cycle Clock cycle is the slowest task time
Tk: total time

Tk (k (n 1))
n
T1 nk
Speedup
Tk k (n 1) k
Efficiently scheduled laundry: Pipelined Laundry

6 PM 7 8 9 10 11 Midnight
Time

30 40 40 40 40 20
40 40 40
T
a A
s
k
B
O
r
d C
e
r
D
 Speedup
Consider a k-segment pipeline operating on n data
sets. (In the above example, k = 3 and n = 4.)

It takes k clock cycles to fill the pipeline and get the

first result from the output of the pipeline.

After that the remaining (n - 1) results will come out

at each clock cycle.

It therefore takes (k + n - 1) clock cycles to complete

the task.
 Speedup
If we execute the same task sequentially in a
single processing unit, it takes (k * n) clock
cycles.
The speedup gained by using the pipeline is:

S = k * n / (k + n - 1 )
 Speedup
S = k * n / (k + n - 1 )

For n >> k (such as 1 million data sets on a 3-stage

pipeline),

S~k

So we can gain the speedup which is equal to the

number of functional units for a large data sets. This
is because the multiple functional units can work in
parallel except for the filling and cleaning-up cycles.
 Speedup
Example
- 4-stage pipeline
- subopertion in each stage; tp = 20nS
- 100 tasks to be executed
- 1 task in non-pipelined system; 20*4 = 80nS

Pipelined System
(k + n - 1)*tp = (4 + 99) * 20 = 2060nS

Non-Pipelined System
n*k*tp = 100 * 80 = 8000nS

Speedup
Sk = 8000 / 2060 = 3.88

4-Stage Pipeline is basically identical to the system with 4

identical function units
 Example of Pipelining
Suppose we want to perform the combined
multiply and add operations with a stream
of numbers:

Ai * Bi + Ci for i =1,2,3,,7
 Example of Pipelining
The sub-operations performed in each
segment of the pipeline are as follows:

R1 Ai, R2 Bi
R3 R1 * R2 R4 Ci
R5 R3 + R4
 Example of Pipelining
Ai Bi Ci

R1 Ai , R2 Bi R1 R2
Input Ai and Bi

R3 R1 * R2, R4 Ci
Multiplier
Multiply and input Ci

R5 R3 + R4 R3 R4
Add Ci to product
Adder

R5
Content of registers in pipeline example

Clock
Pulse
Segment1 Segment2 Segment3
number R1 R2 R3 R4 R5

1 A1 B1 ---- ---- ----

2 A2 B2 A1*B1 C1 ----
3 A3 B3 A2*B2 C2 A1*B1+C1
4 A4 B4 A3*B3 C3 A2*B2+C2
5 A5 B5 A4*B4 C4 A3*B3+C3
6 A6 B6 A5*B5 C5 A4*B4+C4
7 A7 B7 A6*B6 C6 A5*B5+C5
8 ---- ---- A7*B7 C7 A6*B6+C6
9 ---- ---- ---- ---- A7*B7+C7

Exercise: Looking at the above example define how the operation of

Ai*Bi + Ci*Di+ Ei
is executed using a pipeline
 Arithmetic Pipeline
From the early times of computing arithmetics withheld an
important aspect, yet arithmetic operations happen to
consume much of the time with in the arithmetic and logic
unit.

Thus pipelining is used to boost the performance of ALUs

and has opened up to many means of High performance of
computing.

Arithmetic pipelines are generally used for fixed point

operations and floating point operations.
Arithmetic Pipeline: Floating Point Adder

A generic floating point number can be stated as

X = A * 2a

Where X happens to be a binary value.

A is defined to be the mantissa and a is called the
exponent.
Arithmetic Pipeline: Floating Point Adder

X = A * 2a
Y = B * 2b
A floating point adder can be executed via 4 simple
sub operations

Compare the exponents.

Align the mantissas.
Add or subtract the mantissas.
Normalize the result.
Arithmetic Pipeline: Floating Point Adder

Given below is a simple demonstration of how two

decimal floats are added.

Consider the two input floats of X and Y

X = 0.9832* 103
Y = 0.8929* 102

Note: Decimal numbers are used for simplicity of explanation

Arithmetic Pipeline: Floating Point Adder

X = 0.9832* 103
Y = 0.8929* 102

In the initial segment the two exponents are compared.

The larger exponent is 3 and thus it is chosen as the
exponent for the result.

difference between the two exponents is 1 (3-2).

Arithmetic Pipeline: Floating Point Adder
X = 0.9832* 103
Y = 0.8929* 102
Since Y is with the lesser exponent its mantissa is
shifted to the right and the two gained values are,
X = 0.9832* 103
Y = 0.08929* 103

Afterwards the two mantissas are simply added and the

value Z is gained
Z = 1.07249* 103
Finally the gained result is normalized in manner which
staples a mantissa with a fraction with a none zero
value for the first decimal point.
Arithmetic Pipeline for Floating Point Adder

Exponents
Mantissas
a b
A B

R
R
Compare
Difference
Segment 1 Exponent
By subtraction
Align mantissas
R
R
Segment 2 Choose exponent

Add or subtract
Segment 3 mantissas

R R

Normalize
Segment 4 Adjust
Exponent result

R R
Arithmetic Pipeline for Floating Point Adder
 Instruction Pipeline

An Instruction pipeline works in a similar manner

to the Arithmetic Pipeline even though it works
with an instruction field as suppose to a data
stream.
 Instruction Pipeline
process of an instruction requires the following sequence
of steps.

Fetch the instruction from memory.

Decode the instruction.
Calculate the effective address.
Fetch the operands from memory.
Execute the instruction.
Store the result in the proper place.
 Instruction Pipeline
Consider the following specification of a pipeline mean to
have 4 separate segments

In such a system up to 4 different instructions can be

processed at the same time.
 Pipeline Conflicts
Difficulties in general can be caused due to the reasons
specified below.
Resource conflicts
when two segments access memory at the same

time.
Data dependency conflicts
occur when an instruction is dependent of a result of a
previous instruction which is not available yet
Branch difficulties conflicts
when branching and other instructions that change the
value of the PC.
Four-segment CPU pipeline for overcome
Pipeline Conflicts
Fetch instruction
Segment 1 from memory

Decode instruction
And calculate
Segment 2
Effective address

yes
Branch?

no
Fetch operand
Segment 3 From memory

Execute
Segment 4
instruction

Interrupt yes
handling Interrupt?

no
Update PC

Empty pipe
 Four-segment CPU pipeline for overcome
Pipeline Conflicts
Timing of Instruction Pipeline

Step: 1 2 3 4 5 6 7 8 9 10 11 12 13

Instruction: 1 FI DA FO EX

2 FI DA FO EX

(Branch) 3 FI DA FO EX

4 FI -- -- FI DA FO EX

5 -- -- -- FI DA FO EX

6 FI DA FO EX

7 FI DA FO EX
 Four-segment CPU pipeline for overcome
Pipeline Conflicts
The four segments illustrated in above table have the following
meanings:

FI is the segment that fetches an instruction.

DA is the segment that decodes the instruction and

calculate the effective address.

FO is the segment that fetches the operand.

EX is the segment that executes the instruction.

Thank You