Pipelining

Pipelining is the process of accumulating instruction from the processor through a pipeline. It allows
storing and executing instructions in an orderly process. It is also known as pipeline processing.
Pipelining is a technique where multiple instructions are overlapped during execution. Pipeline is
divided into stages and these stages are connected with one another to form a pipe like structure.
Instructions enter from one end and exit from another end.
Pipelining increases the overall instruction throughput.
In pipeline system, each segment consists of an input register followed by a combinational circuit. The
register is used to hold data and combinational circuit performs operations on it. The output of
combinational circuit is applied to the input register of the next segment.

Pipeline system is like the modern day assembly line setup in factories. For example in a car
manufacturing industry, huge assembly lines are setup and at each point, there are robotic arms to
perform a certain task, and then the car moves on ahead to the next arm.
Types of Pipeline
It is divided into 2 categories:
1. Arithmetic Pipeline
2. Instruction Pipeline
Arithmetic Pipeline
Arithmetic pipelines are usually found in most of the computers. They are used for floating point
operations, multiplication of fixed point numbers etc. For example: The input to the Floating Point
Adder pipeline is:
X = A*2^a
Y = B*2^b
Here A and B are mantissas (significant digit of floating point numbers), while a and b are exponents.
The floating point addition and subtraction is done in 4 parts:
1. Compare the exponents.
2. Align the mantissas.
3. Add or subtract mantissas
4. Produce the result.
Registers are used for storing the intermediate results between the above operations.
Pipeline Conflicts
There are some factors that cause the pipeline to deviate its normal performance. Some of these factors
are given below:
1. Timing Variations
All stages cannot take same amount of time. This problem generally occurs in instruction processing
where different instructions have different operand requirements and thus different processing time.
2. Data Hazards
When several instructions are in partial execution, and if they reference same data then the problem
arises. We must ensure that next instruction does not attempt to access data before the current
instruction, because this will lead to incorrect results.
3. Branching
In order to fetch and execute the next instruction, we must know what that instruction is. If the present
instruction is a conditional branch, and its result will lead us to the next instruction, then the next
instruction may not be known until the current one is processed.
4. Interrupts
Interrupts set unwanted instruction into the instruction stream. Interrupts effect the execution of
instruction.
5. Data Dependency
It arises when an instruction depends upon the result of a previous instruction but this result is not yet
available.
Advantages of Pipelining
1. The cycle time of the processor is reduced.
2. It increases the throughput of the system
3. It makes the system reliable.
Disadvantages of Pipelining
1. The design of pipelined processor is complex and costly to manufacture.
2. The instruction latency is more.
Instruction Pipeline
Pipeline processing can occur not only in the data stream but in the instruction stream as well.
Most of the digital computers with complex instructions require instruction pipeline to carry out
operations like fetch, decode and execute instructions.
In general, the computer needs to process each instruction with the following sequence of steps.
1. Fetch instruction from memory.
2. Decode the instruction.
3. Calculate the effective address.
4. Fetch the operands from memory.
5. Execute the instruction.
6. Store the result in the proper place.
Each step is executed in a particular segment, and there are times when different segments may take
different times to operate on the incoming information. Moreover, there are times when two or more
segments may require memory access at the same time, causing one segment to wait until another is
finished with the memory.

The organization of an instruction pipeline will be more efficient if the instruction cycle is divided into
segments of equal duration. One of the most common examples of this type of organization is a Four-
segment instruction pipeline.
A four-segment instruction pipeline combines two or more different segments and makes it as a
single one. For instance, the decoding of the instruction can be combined with the calculation of the
effective address into one segment.
The following block diagram shows a typical example of a four-segment instruction pipeline. The
instruction cycle is completed in four segments.
Segment 1:
The instruction fetch segment can be implemented using first in, first out (FIFO) buffer.
Segment 2:
The instruction fetched from memory is decoded in the second segment, and eventually, the effective
address is calculated in a separate arithmetic circuit.
Segment 3:
An operand from memory is fetched in the third segment.
Segment 4:
The instructions are finally executed in the last segment of the pipeline organization.
Data hazards
Data hazards occur when instructions that exhibit data dependence modify data in different stages of a
pipeline. Ignoring potential data hazards can result in race conditions (also termed race hazards).
Due to the data dependency, data hazards have occurred. If the data is modified in different stages of a
pipeline with the help of instructions that exhibit data dependency, in this case, the data hazard will
occur. When the instructions are read/write the registers that are used by some other instructions, in
this case, the instruction hazards will occur. Because of the data hazard, there will be a delay in the
pipeline. The data hazards are basically of three types:
1. read after write (RAW), a true dependency
2. write after read (WAR), an anti-dependency
3. write after write (WAW), an output dependency

RAW:
RAW hazard can be referred to as 'Read after Write'. It is also known as Flow/True data dependency.
If the later instruction tries to read on operand before earlier instruction writes it, in this case, the RAW
hazards will occur. The condition to detect the RAW hazard is when On and In+1 both have a
minimum one common operand.
For example:
I1: add R1, R2, R3
I2: sub R5, R1, R4
There is a RAW hazard because subtraction instruction reads output of the addition. The hazard for
instructions 'add R1, R2, R3' and 'sub R5, R1, R4' is described as follows:

The RAW hazard is very common.

WAR
WAR can be referred to as 'Write after Read'. It is also known as Anti-Data dependency. If the later
instruction tries to write an operand before the earlier instruction reads it, in this case, the WAR
hazards will occur. The condition to detect the WAR hazard is when In and On+1 both have a
minimum one common operand.
For example:
The dependency is described as follows:
add R1, R2, R3
sub R2, R5, R4
Here addition instruction creates a WAR hazard because subtraction instruction writes R2, which is
read by addition. In a reasonable (in-order) pipeline, the WAR hazard is very uncommon or
impossible. The hazard for instructions 'add R1, R2, R3' and 'sub R2, R5, R4' are described as follows:

When the instruction tries to enter into the write back stage of the pipeline, at that time, all the previous
instructions contained by the program have already passed through the read stage of register and read
their input values. Now without causing any type of problem, the write instruction can write its
destination register. The WAR instructions contain less problems as compared to the WAW because in
WAR, before the write back stage of a pipeline, the read stage of a register occur.

WAW
WAW can be referred to as 'Write after Write'. It is also known as Output Data dependency. If the later
instruction tries to write on operand before earlier instruction writes it, in this case, the WAW hazards
will occur. The condition to detect the WAW hazard is when On and On+1 both have a minimum one
common operand.
For example:
The dependency is described as follows:
add R1, R2, R3
sub R1, R2, R4
Here addition instruction creates a WAW hazard because subtraction instruction writes on the same
register. The hazard for instructions 'add R1, R2, R3' and 'sub R1, R2, R4' are described as follows:

In the write back stage of a pipeline, the output register of instruction will be written. The order in
which the instruction with WAW hazard appears in the program, in the same order these instructions
will be entered the write back stage of a pipeline. The result of these instructions will be written into
the register in the right order. The processor has improved performance as compared to the original
program because it allows instructions to execute in different orders.

Effects of WAR and WAW

The WAR hazards and WAW hazards occur because the process contains a finite number of registers.
Because of this reason, these hazards are also known as the name dependencies.
The processor will use the different registers to generate the output of each instruction if it contains an
infinite number of registers. There is no chance of occurring the WAR and WAW hazards in this case.
The WAR and WAW hazards will not cause the delay if a processor uses the same pipeline for all the
instructions and executes these instructions in the same order in which they appear in the program.
This is all because of the process of instructions flow through a pipeline.
Dynamic scheduling
Dynamically scheduled processor rearranges instruction execution order. This better handles cases
where dependence is unknown at compile time. It simplifies compiler on the expense of increased
hardware complexity.
Dependability
Dependability in computer architecture is a measure of a system's availability, reliability, and its
maintainability, and maintenance support performance, and, in some cases, other characteristics such
as durability, safety and security.
Dependability is the ability of a system to deliver a specified service.
• System service is classified as proper if it is delivered as specified; otherwise it
is improper.
• System failure is a transition from proper to improper service.
• System restoration is a transition from improper to proper service.
The “properness” of service depends on the user’s viewpoint!
Dependability Concepts
• Measures - properties expected from a dependable system
– Availability
– Reliability
– Safety
– Confidentiality
– Integrity
– Maintainability
– Coverage
• Means - methods to achieve dependability
– Fault Avoidance
– Fault Tolerance
– Fault Removal
– Dependability Assessment
• Impairments - causes of undependable operation
– Faults
– Errors
– Failures
Branch cost
Branch prediction is an approach to computer architecture that attempts to mitigate the costs of
branching. Branch predication speeds up the processing of branch instructions with CPUs using
pipelining. The technique involves only executing certain instructions if certain predicates are true.
Branch prediction is typically implemented in hardware using a branch predictor.
Branch prediction is also known as branch predication or simply as predication.
Branch prediction is a technique used to speed execution of instructions on processors that use
pipelining. CPUs initially executed instructions one by one as they came in, but the introduction of
pipelining meant that branching instructions could slow the processor down significantly as the
processor has to wait for the conditional jump to be executed.
Branch prediction breaks instructions down into predicates, similar to predicate logic. A CPU using
branch prediction only executes statements if a predicate is true. One example is using conditional
logic. Since unnecessary code is not executed, the processor can work much more efficiently. Branch
prediction is implemented in CPU logic with a branch predictor.
Branch Prediction
In computer architecture, a branch predictor is a digital circuit that tries to guess which way a branch
(e.g., an if–then–else structure) will go before this is known definitively. The purpose of the branch
predictor is to improve the flow in the instruction pipeline
Branch prediction is a technique used in CPU design that attempts to guess the outcome of a
conditional operation and prepare for the most likely result. A digital circuit that performs this
operation is known as a branch predictor. It is an important component of modern CPU architectures,
such as the x86.

How does it work?

When a conditional operation such as an if…else statement needs to be processed, the branch predictor
"speculates" which condition most likely will be met. It then executes the operations required by the
most likely result ahead of time. This way, they're already complete if and when the guess was correct.
At runtime, if the guess turns out not to be correct, the CPU executes the other branch of operation,
incurring a slight delay. But if the guess was correct, speed is significantly increased.
The first time a conditional operation is seen, the branch predictor does not have much information to
use as the basis of a guess. But the more frequently the same operation is used, the more accurate its
guess can become.
Influence on instruction set
An instruction is a set of codes that the computer processor can understand. The code is usually in 1s
and 0s, or machine language. It contains instructions or tasks that control the movement of bits and
bytes within the processor.
Example of some instruction sets −
ADD − Add two numbers together.
JUMP − Jump to designated RAM address.
LOAD − Load information from RAM to the CPU.

Types of Instruction Set

Generally, there are two types of instruction set used in computers.
Reduced Instruction set Computer (RISC)
A number of computer designers recommended that computers use fewer instructions with simple
constructs so that they can be executed much faster within the CPU without having to use memory as
often. This type of computer is called a Reduced Instruction Set Computer.
The concept of RISC involves an attempt to reduce execution time by simplifying the instruction set of
computers.
Characteristics of RISC
The characteristics of RISC are as follows −
Relatively few instructions.
Relatively few addressing modes.
Memory access limited to load and store instructions.
All operations done within the register of the CPU.
Single-cycle instruction execution.
Fixed length, easily decoded instruction format.
Hardwired rather than micro programmed control.

A characteristic of RISC processors’ ability is to execute one instruction per clock cycle. This is done
by overlapping the fetch, decode and execute phases of two or three instructions by using a procedure
referred as pipelining.

Complex Instruction Set Computer (CISC)

CISC is a computer where a single instruction can perform numerous low-level operations like a load
from memory and a store from memory, etc. The CISC attempts to minimize the number of
instructions per program but at the cost of an increase in the number of cycles per instruction.
The design of an instruction set for a computer must take into consideration not only machine language
constructs but also the requirements imposed on the use of high level programming languages.
The goal of CISC is to attempt to provide a single machine instruction for each statement that is
written in a high level language.

Characteristics of CISC
The characteristics of CISC are as follows −
A large number of instructions typically from 100 to 250 instructions.
Some instructions that perform specialized tasks and are used infrequently.
A large variety of addressing modes- typically from 5 to 20 different modes.
Variable length instruction formats.
Instructions that manipulate operands in memory.

Example
For performing an ADD operation, CISC will execute a single ADD command which will execute all
the required load and store operations.
RISC will execute each operation for loading data from memory, adding values and storing data back
to memory using different low-level instructions.