Pipelining I

Prepared by: Noshaba Nasir

What Is Pipelining?
• Pipelining is an implementation technique whereby multiple
instructions are overlapped in execution.
• It takes advantage of parallelism that exists among the actions
needed to execute an instruction.
• Today, pipelining is the key implementation technique used to make
fast CPUs.
Real World Analogy
• A pipeline is like an assembly line.
• In an automobile manufacturing line, there are many steps, each
contributing something to the construction of the car.
• Each step operates in parallel with the other steps, although on a
different car.
Real World Analogy
• Automobile manufacturing line without pipelining, in 3 stages
• Assemble (A), Paint (P), Fix tires (T)
• Assuming A+P+T take 3 hours

Time 0-3 hours 3-6 Hours 6 -9Hours

line
Car 1 A+P+T

Car 2 A+P+T

Car 3 A+P+T
Real World Analogy
• Automobile manufacturing line with pipelining, in 3 stages
• Assemble (A), Paint (P), Fix tires (T)
• Assuming each stage takes 1 hour
• It takes 5 hours with pipelining and 9hours without pipelining
• Speedup = 9/5

Time 1st 2nd 3rd 4th 5th 6th 7th 8th 9th
line hour hour hour hour hour hour hour hour hour
Car 1 A P T
Car 2 A P T
Car 3 A P T
Real World Analogy
• Here our assumptions are
• There is no time consumed to transition car from one working station on other
• All the stages consume equal time
• However these assumption are not valid
• Each stage can take different amount of time
• For example painting a car takes 2 hours and Assembly and installing types take 0.5 hours
each
• To transfer car from one stage to another will take time.
• For example, it takes 10 minutes to transfer a car from Assembly room to paint rooms and
10 minute to transfer from paint room to tire fixing room.
• Note that this will not be consume in un-pipeline factory as everything will be done is
same room
 Real World Analogy
• Automobile manufacturing line with pipelining, in 3 stages
• If Assemble (A), Paint (P), Fix tires (T) take different time
• If Assembly take 0.5 hour, Paint 2 hours and Tires 0.5 hours
• Paint of Car2 cannot start before 2.5th hour because Paint unit is busy with Car1. So after Assembly Car 2
has to wait for 1.5 Hours
• Total time take to complete 3 instructions =7h
• Speed Up = 9/7

Time line 0-0.5h 0.5-2.5h 2.5-4.5h 4.5-6.5 h 6.5-7 h

Car 1 A (0.5 h) P (2h) T (0.5h)

Car 2 A (0.5 h) +wait for 1.5 h P (2h) T (0.5h)

Car 3 A (0.5h)+ wait for 1.5 h P (2h) T (0.5h)

 Real World Analogy
• Automobile manufacturing line with pipelining, in 3 stages
• If we add time for transition between stages as well
• Total time taken to complete 3 instructions = 7h+40m=7.67h
• Speed Up = 9/7.67
Time line 0-0.5h Transition 0.5 +10m- Transition 2.5h+20m-4.5h+20m Transition 4.5h+30m Transition 6.5h+40m
time 10 2.5h+10m time 10 min time 10 min -6.5 time 10 min -7 h+40m
min h+30m
Car 1 A (0.5 h) P (2h) T (0.5h)

Car 2 A (0.5 h) +wait for 1.5 P (2h) T (0.5h)

h

Car 3 A (0.5h)+ wait for 1.5 h P (2h) T (0.5h)

Real World Analogy::Throughput and
Latency
• In case of car manufacturing throughput is number of cars create per hour
• From slide 8 throughput with pipelining is 3 Cars/ 7.67 Hours ~0.4 Cars/hour
• From slide 4 throughput without pipelining is 3Cars/9Hours ~ 0.33Cars/hour
• So we can see that the throughput with pipelining is higher.
• Latency is time taken to complete one car
• From slide 8 latency with pipelining is 4.5 hours (4.5 hours consumed to create car2
and 3)
• From slide 4 latency without pipelining is 3 hours (3 hours consumed to create each
car)
• So we can see that latency is worse with pipelining
• Observation: Pipeline does not improve latency, it only improves throughput
Throughput and latency for n=3
Case Latency Throughput Speed up
Non pipeline 3 3/9=0.33 NA

Pipeline ideal case 3 3/5=0.6 9/5

Pipeline with all stages 4.5 3/7=0.42 9/7

not of equal time
Non ideal case with latch 4.833 3/7.67=0.39 9/7.67
time
Real World Analogy:: Process Cycle
• The time required between moving an object one step down the
pipeline is a processor cycle
• Assume that all stages procced at the same time
• There is only one conveyer belt and it only moves cars to next step when
longest stage is complete, i.e. every two hours.*
• Processor cycle will be 2 hours
• Even the car in T stage will be there for 2 hours and will only come out of
pipeline when 2 hours of P(of other instruction are completed)
Real World Analogy:: Process Cycle
• Consider process cycle to be 2 hours, throughput with pipelining is 3
Cars/ 10 Hours ~0.3 Cars/hour
• This is even worse than throughput of un-pipeline
• This issue can be resolve if stages are more or less of equal time
duration (paint job can be further divided into smaller stages)

Time line 2hour 2hours 2hours 2hours 2hours

Car 1 A P T

Car 2 A P T

Car 3 A P T
Exercise
• Divide your paint job in 4 stages, 1st coat, 2nd coat, 3rd coat and drying ,
each phase takes 0.5 hours.
• What will be throughput with these 6 stages for 3 cars?
• Ignore the latch/transition time
Pipeline in Computer:: Frequency
• Computer processors can execute one or more instructions per clock
cycle, depending on the type of processor.
• Frequency of computer is 1/clock cycle
• For example if 1 clock cycle= 10 ms;
• Frequency = 1/10ms= 1MHZ
• i.e. 1 000 000 cycles / second
• Frequency is AKA as clock speed
Pipeline in Computer:: Performance
• IDEAL CASE
• In ideal case i.e. all the stages require same amount of time and
ignoring time to transfer data from one stage to another
• Speedup= n*k/ k+(n-1)
• where n is number of instructions and k is number of stages

Time T T T T T
line
Car 1 A P T
Car 2 A P T
Car 3 A P T
.. … ..
Pipeline in Computer:: Performance
Ideal case (Derivations)
• If
• All stages take equal amount of time T
• Latch time=0
• Stages =k
• Number of instructions = n
• Then
• Clock cycle of pipeline=T, clock cycle of non pipeline= k*T
• Frequency of pipeline=1/T , Frequency of non pipeline=1/k*T
• Time taken to complete n instructions without pipeline= n*k*T= n*clock cycle of non pipeline
• Through put for n instructions without pipeline= n/ n*k*T
• Time taken to complete n instructions with pipeline= (k+n-1)*T= (k+n-1)*(Clock Cycle of pipeline)
• Through put for n instructions with pipeline= n/ (k+n-1)*T
• Speedup for n instructions = n*k*T/ (k+n-1)*T = n*k/ (k+n-1)
• Latency without pipelining = k*T
• Latency with pipelining= k*T
Pipeline in Computer:: Performance
Ideal case
• Numerical:
• It takes 6us to complete one instruction in non-pipeline processor
• We were able to convert the circuit into 6 equal sequential pipeline stages.
• Assume latch time is 0
• Answer the following, assuming that there are no stalls in the pipeline.
• What are the clock cycle in the two processors?
• What are the clock speeds(frequency) in two processors?
• How long does it take to finish one instruction in pipeline and no pipeline (latency )?
• What is the throughput for 100 instructions without pipelining?
• What is the throughput for 100 instructions with pipelining?
• What is the speedup from pipelining for 1 instructions?
• What is the speedup from pipelining for 100 instructions?
Pipeline in Computer:: Performance
• Another example
• It takes 5us to complete one instruction in non-pipeline processor
• We were able to convert the circuit into 5 equal duration sequential pipeline stages.
• Latch time is 0us
• Answer the following, assuming that there are no stalls in the pipeline.
• What are the clock cycle in the two processors?
• What are the clock speeds(frequency) in two processors?
• How long does it take to finish one instr in pipeline and no pipeline (latency )?
• What is the throughput for 100 instructions without pipelining?
• What is the throughput for 100 instructions with pipelining?
• What is the speedup from pipelining for 1 instructions?
• What is the speedup from pipelining for 100 instructions?
Exercise
• Question 1: If throughput of one processor1 for n instruction is 10us
and through put of processor 2 is 15us for same n instruction what
speed up is achieved by processor2 are compare to processor 1?
• Question 2: If Latency with pipelining with 5 stages is 6us and latch
time is 0 what is the clock cycle time?