Computer Organization and

Architecture (AT70.01)
Comp. Sc. and Inf. Mgmt.
Asian Institute of Technology
Instructor: Dr. Sumanta Guha
Slide Sources: Based on CA:
aQA by Hennessy/Patterson.
Supplemented from various
freely downloadable sources
Advanced Topic:
Compiler Support for ILP
CA:aQA Sec. 4.1
Scheduling Code for the MIPS
Pipeline
 Example:
 for (i=1000; i>0; i=i-1)

x[i] = x[i] + s;

 Notes:
 the loop is parallel – the body of each iteration is independent of
that of other iterations
 conceptually : if we had 1000 CPUs, we could distribute one
iteration to each CPU and compute in parallel (=simultaneously)
 Only the compiler can exploit such instruction-level parallelism
(ILP), not the hardware! Why?
 because only the compiler has a global view of the code
 the hardware sees each line of code only after it is fetched from
memory, not all together – in particular, not the whole loop
 the compiler must schedule the code intelligently to expose and
exploit ILP…
 Scheduling Code for the MIPS
Pipeline
 Assume FP operation latencies as below
 latency indicates number of intervening cycles required between
producing and consuming instruction to avoid stall
 Assume integer ALU operation latency of 0 and integer load
latency of 1
FP Latency table
Instruction producing result Instruction using result Latency in clock cycles
FP ALU op Another FP ALU op 3
FP ALU op Store double 2
Load double FP ALU op 1
Load double Store double 0
Unscheduled Code
Original C loop statement: for (i=1000; i>0; i=i-1) x[i] = x[i] + s;
Unscheduled code for the MIPS pipeline:
Loop: L.D F0,0(R1) ;F0 = array element
ADD.D F4,F0,F2 ;add scalar in F2
S.D F4,0(R1) ;store result
DADDUI R1,R1,#-8 ;decrement pointer
;8 bytes per DW
BNE R1,R2,Loop ;branch R1!=R2
Execution cycles for the unscheduled code:
Clock cycle issued
Loop: L.D F0,0(R1) 1
stall 2
ADD.D F4,F0,F2 3
stall 4
stall 5
S.D F4,0(R1) 6
Why one stall? Think of when the
DADDUI R1,R1,#-8 7 optimized MIPS pipeline resolves
stall 8 branch outcomes…
BNE R1,R2,Loop 9 Delayed branch stall
stall 10
10 clock cycles per iteration
Scheduled Code
Scheduled code for the MIPS pipeline:

Loop: L.D F0,0(R1)

DADDUI R1,R1,#-8
ADD.D F4,F0,F2
BNE R1,R2,Loop ;delayed branch
S.D F4,8(R1) ;altered and interchanged with DADDUI

Execution cycles for the scheduled code:

Clock cycle issued
Loop: L.D F0,0(R1) 1
DADDUI R1,R1,#-8 2
ADD.D F4,F0,F2 3
stall 4
BNE R1,R2,Loop 5
S.D F4,8(R1) 6
6 clock cycles per iteration is optimal because of the dependencies. Only 3 of the operations
(L.D, ADD.D & S.D) actually operate on the array, the other three are loop overhead…
 Compiler has to be “smart” to perform this scheduling
 e.g., interchanging the DADDUI and S.D instructions requires
understanding the dependence between them and accordingly
changing the S.D store address from 0(R1) to 8(R1)!
Unrolling Loops
 The 3 clock cycle per iteration overhead delay in the
scheduled code of the previous example may be reduced…
 …by amortizing the loop overhead over multiple loop iterations
 For this we need to unroll the loop and block multiple
iterations into one
 Loop unrolling also allows improved scheduling by exposing
increased ILP – between instruction from different iterations
 Example…
Unrolling Loops – High-level
 for (i=1000; i>0; i=i-1)
x[i] = x[i] + s;

C equivalent of unrolling to block four iterations into one:

 for (i=250; i>0; i=i-1)

{
x[4*i] = x[4*i] + s;
x[4*i-1] = x[4*i-1] + s;
x[4*i-2] = x[4*i-2] + s;
x[4*i-3] = x[4*i-3] + s;
}
 Unrolled Loop – not Scheduled
Unrolled but unscheduled code for the MIPS pipeline:
Loop: L.D F0,0(R1)
ADD.D F4,F0,F2
S.D F4,0(R1) ;drop DADDUI & BNE
L.D F6,-8(R1)
ADD.D F8,F6,F2
S.D F8,-8(R1) ;drop DADDUI & BNE
L.D F10,-16(R1)
ADD.D F12,F10,F2
S.D F12,-16(R1) ;drop DADDUI & BNE
L.D F14,-24(R1)
ADD.D F16,F14,F2
S.D F16,-24(R1)
DADDUI R1,R1,#-32
BNE R1,R2,Loop
 Notes:
 four copies of the loop body have been unrolled
 different registers are used in each copy – to facilitate future
scheduling
 three branches and three decrements of R1 have been eliminated
Executing the Unrolled
Unscheduled Loop Clock cycle issued
Loop: L.D F0,0(R1) 1
stall 2
ADD.D F4,F0,F2 3
stall 4
stall 5
S.D F4,0(R1) 6
L.D F6,-8(R1) 7
stall 8
ADD.D F8,F6,F2 9
stall 10
stall 11
S.D F8,-8(R1) 12
L.D F10,-16(R1) 13
stall 14
ADD.D F12,F10,F2 15
stall 16
stall 17
S.D F12,-16(R1) 18
L.D F14,-24(R1) 19
stall 20
ADD.D F16,F14,F2 21
stall 22
stall 23
S.D F16,-24(R1) 24
DADDUI R1,R1,#-32 25
stall 26
BNE R1,R2,Loop 27
stall 28

One iteration of the unrolled loop runs in 28 clock cycles. Therefore,

7 clock cycles per iteration of original loop – slower than scheduled original loop!
Scheduling the Unrolled Loop
Unrolled and scheduled code for the MIPS pipeline:
Loop: L.D F0,0(R1)
L.D F6,-8(R1)
L.D F10,-16(R1)
L.D F14,-24(R1)
ADD.D F4,F0,F2
ADD.D F8,F6,F2
ADD.D F12,F10,F2
ADD.D F16,F14,F2
S.D F4,0(R1)
S.D F8,-8(R1)
DADDUI R1,R1,#-32
S.D F12,-16(R1)
BNE R1,R2,Loop
S.D F16,-24(R1) ;8-32 = -24
Executing the Unrolled and
Scheduled Loop
Clock cycle issued
Loop: L.D F0,0(R1) 1
L.D F6,-8(R1) 2
L.D F10,-16(R1) 3
L.D F14,-24(R1) 4
ADD.D F4,F0,F2 5
ADD.D F8,F6,F2 6
ADD.D F12,F10,F2 7
ADD.D F16,F14,F2 8
S.D F4,0(R1) 9
S.D F8,-8(R1) 10
DADDUI R1,R1,#-32 11
S.D F12,-16(R1) 12
BNE R1,R2,Loop 13
S.D F16,-24(R1) 14

No stalls! One iteration of the unrolled loop runs in 14 clock cycles. Therefore,
3.5 clock cycles per iteration of original loop vs. 6 cycles for scheduled but not unrolled loop
Notes
 Scheduling code (if possible) to avoid stalls is always a win
and optimizing compilers typically generate scheduled
assembly
 Unrolling loops can be advantageous but there are potential
problems
 growth in code size
 register pressure: aggressive unrolling and scheduling requires
allocation of multiple registers
Enhancing Loop-Level
Parallelism
 Consider the previous running example:
 for (i=1000; i>0; i=i-1)
x[i] = x[i] + s;
 there is no loop-carried dependence – where data used in a later
iteration depends on data produced in an earlier one
 in other words, all iterations could (conceptually) be executed in
parallel
 Contrast with the following loop:
 for (i=1; i<=100; i=i+1) {
A[i+1] = A[i] + C[i]; /* S1 */
B[i+1] = B[i] + A[i+1]; /* S2
*/ }
 what are the dependences?
 A Loop with Dependences
 For the loop:
 for (i=1; i<=100; i=i+1) {
A[i+1] = A[i] + C[i]; /* S1 */
B[i+1] = B[i] + A[i+1]; /* S2
*/ }
 what are the dependences?
 There are two different dependences:
 loop-carried:
 S1 computes A[i+1] using value of A[i] computed in previous
iteration
 S2 computes B[i+1] using value of B[i] computed in previous
iteration
 not loop-carried:
 S2 uses the value A[i+1] computed by S1 in the same iteration A[i-1]

 The loop-carried dependences in this case force successive A[i]

iterations of the loop to execute in series. Why?
 S1 of iteration i depends on S1 of iteration i-1 which in turn
A[i+1]
depends on …, etc.
 Another Loop with
Dependences
 Generally, loop-carried dependences hinder ILP
 if there are no loop-carried dependences all iterations could be
executed in parallel
 even if there are loop-carried dependences it may be possible to
parallelize the loop – an analysis of the dependences is required…
 For the loop:
 for (i=1; i<=100; i=i+1) {
A[i] = A[i] + B[i]; /* S1 */
B[i+1] = C[i] + D[i]; /* S2 */
}
 what are the dependences?
 There is one loop-carried dependence:
 S1 uses the value of B[i] computed in a previous iteration by S2
B[i]
 but this does not force iterations to execute in series. Why…?
 …because S1 of iteration i depends on S2 of iteration i-1…, and
the chain of dependences stops here! A[i]
Parallelizing Loops with Short
Chains of Dependences
 Parallelize the loop:
 for (i=1; i<=100; i=i+1) {
A[i] = A[i] + B[i]; /* S1 */
B[i+1] = C[i] + D[i]; /* S2 */
}
 Parallelized code:
 A[1] = A[1] + B[1];
for (i=1; i<=99; i=i+1) {
B[i+1] = C[i] + D[i];
A[i+1] = A[i+1] + B[i+1];
}
B[101] = C[100] + D[100];

 the dependence between the two statements in the loop is no

longer loop-carried and iterations of the loop may be executed in
parallel
Another Example
 Analyze the loop:
 for (i=1; i<=100; i=i+1) {
A[i] = A[i] + B[i]; /* S1 */
B[i+1] = C[i] + D[i]; /* S2 */
C[i+1] = E[i] + D[i]; /* S3 */
}