Digital Design and Computer Organization BCS302

PIPELINING
8.1 Basic Concepts
Pipelining is a particularly effective way of organizing concurrent activity in a
computer system.

Consider how the idea of pipelining can be used in a computer. The processor executes
a program by fetching and executing instructions, one after the other. Let Fi and Ei refer to the
fetch and execute steps for instruction Ii. Execution of a program consists of a sequence of fetch
and execute steps, as shown in Figure 8.1a.

Now consider a computer that has two separate hardware units, one for fetching
instructions and another for executing them, as shown in Figure 8.1b.

The instruction fetched by the fetch unit is deposited in an intermediate storage buffer,
B1. This buffer is needed to enable the execution unit to execute the instruction while the fetch
unit is fetching the next instruction. The results of execution are deposited in the destination
location specified by the instruction.

The computer is controlled by a clock whose period is such that the fetch and execute
steps of any instruction can each be completed in one clock cycle. Operation of the computer
proceeds as in Figure 8.1c.

Prof.Neelakantappa T T,Dept. of CS&E,SJMIT pg. 6

Digital Design and Computer Organization BCS302

Figure 8.1: Basic idea of instruction pipelining.

In the first clock cycle, the fetch unit fetches an instruction I1 (step F1) and stores it in
buffer B1 at the end of the clock cycle. In the second clock cycle, the instruction fetch unit
proceeds with the fetch operation for instruction I2 (step F2). Meanwhile, the execution unit
performs the operation specified by instruction I1, which is available to it in buffer B1 (step
E1). By the end of the Second clock cycle, the execution of instruction I1 is completed and
instruction 12 is available. Instruction 12 is stored in B1, replacing I1, which is no longer
needed. In this manner, both the fetch and execute units are kept busy all the time.
The processing of an instruction need not be divided into only two steps. For example, a
pipelined processor may process each instruction in four steps, as follows:
F -- Fetch: read the instruction from the memory.
D -- Decode: decode the instruction and fetch the source operand(s).
E -- Execute: perform the operation specified by the instruction.
W --Write: store the result in the destination location.
The sequence of events for this case is shown in Figure 8.2a. Four instructions are in
progress at any given time. This means that four distinct hardware units are needed, as shown
in Figure 8.2b.
These units must be capable of performing their tasks simultaneously and without
interfering with one another. Information is passed from one unit to the next through a storage
buffer. As an instruction progresses through the pipeline, all the information needed by the
stages downstream must be passed along. For example, during clock cycle 4, the information
in the buffers is as follows:

Prof.Neelakantappa T T,Dept. of CS&E,SJMIT pg. 7

Digital Design and Computer Organization BCS302

Figure 8.2: Four stage pipeline

 Buffer B1 holds instruction I3, which was fetched in cycle 3 and is being decoded by the
instruction-decoding unit.
 Buffer B2 holds both the source operands for instruction I2 and the specification of the
operation to be performed.
 Buffer B3 holds the results produced by the execution unit and the destination information
for instruction I1.

8.1.1 Role Of Cache Memory

Each stage in a pipeline is expected to complete its operation in one clock cycle. Hence,
the clock period should be sufficiently long to complete the task being performed in any stage.
If different units require different amounts of time, the clock period must allow the longest task
to be completed. A unit that completes its task early is idle for the remainder of the clock period.

Prof.Neelakantappa T T,Dept. of CS&E,SJMIT pg. 8

Digital Design and Computer Organization BCS302

Hence, pipelining is most effective in improving performance if the tasks being performed in
different stages require about the same amount of time.

This consideration is particularly important for the instruction fetch step, which is assigned
one clock period in Figure 8.2a. The clock cycle must be equal to or greater than the time
needed to complete a fetch operation.

The use of cache memories solves the memory access problem. In particular, when a cache
is included on the same chip as the processor, access time to the cache is usually the same as
the time needed to perform other basic operations inside the processor. This makes it possible
to divide instruction fetching and processing into steps that are more or less equal in duration.
Each of these steps is performed by a different pipeline stage, and the clock period is chosen
to correspond to the longest one.

8.1.2 PIPELINE PERLORMANCE

The pipelined processor in Figure 8.2 completes the processing of one instruction in each
clock cycle, which means that the rate of instruction processing is four times that of sequential
operation.

The potential increase in performance resulting from pipelining is proportional to the

number of pipeline stages.

For a variety of reasons, one of the pipeline stages may not be able to complete its
processing task for a given instruction in the time allotted. For example, stage E in the four-
stage pipeline of Figure 8.2b is responsible for arithmetic and logic operations, and one clock
cycle is assigned for this task. Although this may be sufficient for most operations, some
operations, such as divide, may require more time to complete.
Figure 8.3 shows an example in which the operation specified in instruction I2 requires
three cycles to complete, from cycle 4 through cycle 6. Thus, in cycles 5 and 6, the Write stage
must be told to do nothing, because it has no data to work with. Thus, steps D4 and F5 must be
postponed as shown.
Pipelined operation in Figure 8.3 is said to have been stalled for two clock cycles. Normal
pipelined operation resumes in cycle 7. Any condition that causes the pipeline to stall is called
a hazard. We have just seen an example of a data hazard. A data hazard is any condition in
which either the source or the destination operands of an instruction are not available at the

Prof.Neelakantappa T T,Dept. of CS&E,SJMIT pg. 9

Digital Design and Computer Organization BCS302

time expected in the pipeline. As a result, some operation has to be delayed, and the pipeline
stalls.

Figure 8.3:Effect of an execution operation taking more than one clock cycle.
The pipeline may also be stalled because of a delay in the availability of an instruction.
For example, this may be a result of a miss in the cache, requiring the instruction to be fetched
from the main memory. Such hazards are often called control hazards or instruction hazards.
The effect of a cache miss on pipelined operation is illustrated in Figure 8.4.
Instruction I1 is fetched from the cache in cycle 1, and its execution proceeds normally.
However, the fetch operation for instruction I2, which is started in cycle 2,results in a cache
miss. The instruction fetch unit must now suspend any further fetch requests and wait for I2 to
arrive. We assume that instruction I2 is received and loaded into buffer B1 at the end of cycle
5. The pipeline resumes its normal operation at that point.
An alternative representation of the operation of a pipeline in the case of a cache miss is
shown in Figure 8.4b. This figure gives the function performed by each pipeline stage in each
clock cycle.
Note that the Decode unit is idle in cycles 3 through 5, the Execute unit is idle in cycles 4
through 6, and the Write unit is idle in cycles 5 through 7. Such idle periods are called stalls.
They are also often referred to as bubbles in the pipeline. Once created as a result of a delay
in one of the pipeline stages, a bubble moves downstream until it reaches the last unit.

Prof.Neelakantappa T T,Dept. of CS&E,SJMIT pg. 10

Digital Design and Computer Organization BCS302

Figure 8.4: Pipeline stall caused by a cache miss in F2

A third type of hazard that may be encountered in pipelined operation is known as a
structural hazard. This is the situation when two instructions require the use of a given
hardware resource at the same time. The most common case in which this hazard may arise is
in access to memory. One instruction may need to access memory as part of the Execute or
Write stage while another instruction is being fetched. If instructions and data reside in the
same cache unit, only one instruction can proceed and the other instruction is delayed. Many
processors use separate instruction and data caches to avoid this delay.
An example of a structural hazard is shown in Figure 8.5. This figure shows how the load
instruction Load X(R1),R2.can be accommodated in our example 4-stage pipeline.
The memory address, X+[R1],is computed in step E2 in cycle 4, then memory access takes
place in cycle 5. The operand read from memory is written into register R2 in cycle 6. This
means that the execution step of this instruction takes two clock cycles (cycles 4 and 5). It
causes the pipeline to stall for one cycle, because both instructions 12 and I3 require access to

Prof.Neelakantappa T T,Dept. of CS&E,SJMIT pg. 11

Digital Design and Computer Organization BCS302

the register file in cycle 6. Even though the instructions and their data are all available, the
pipeline is stalled because one hardware resource, the register file, cannot handle two
operations at once.
If the register file had two input ports, that is, if it allowed two simultaneous write
operations, the pipeline would not be stalled.
In general, structural hazards are avoided by providing sufficient hardware resources on
the processor chip.

Figure 8.5 Effect of a Load instruction on pipeline timing.

It is important to understand that pipelining does not result in individual instructions
being executed faster; rather, it is the throughput that increases, where throughput is
measured by the rate at which instruction execution is completed.

Prof.Neelakantappa T T,Dept. of CS&E,SJMIT pg. 12