Scribd
Upload
110 views16 pages

Lecture 30 GPU Programming Loop Parallelism

The document discusses data level parallelism and programming GPUs. It explains that functions for the GPU device are declared with _device_ or _global_, which allocates variables to GPU memory. It provides an example of invoking the DAXPY operation on 256 threads per block. It also summarizes key aspects of the Nvidia GPU instruction set architecture like PTX, conditional branching, memory structures, and techniques for detecting and enhancing loop level parallelism.

Uploaded by

Udai Valluru
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPSX, PDF, TXT or read online on Scribd
110 views16 pages

Lecture 30 GPU Programming Loop Parallelism

The document discusses data level parallelism and programming GPUs. It explains that functions for the GPU device are declared with _device_ or _global_, which allocates variables to GPU memory. It provides an example of invoking the DAXPY operation on 256 threads per block. It also summarizes key aspects of the Nvidia GPU instruction set architecture like PTX, conditional branching, memory structures, and techniques for detecting and enhancing loop level parallelism.

Uploaded by

Udai Valluru
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPSX, PDF, TXT or read online on Scribd
You are on page 1/ 16

CKV

Advanced VLSI Architecture


MEL G624

Lecture 30: Data Level Parallelism


CKV
Programming the GPU

To distinguish between functions for GPU (device) and


System processor (host)

_device_ or _global_ => GPU Device

_host_ => System processor (host)

Variables declared in _device_ or _global_ functions are


allocated to GPU memory
Function Call: name<<<dimGrid, dimBlock>>>..parameters list..)
threadIdx: Identifier for threads per block
blockIdx: Identifier for blocks
blockDim: Number of threads per block
CKV
Programming the GPU
//Invoke DAXPY
daxpy(n,2.0,x,y);
//DAXPY in C
void daxpy(int n, double a, double* x, double* y){
for (int i=0;i<n;i++)
y[i]= a*x[i]+ y[i]
}

//Invoke DAXPY with 256 threads per Thread Block


_host_
int nblocks = (n+255)/256;
daxpy<<<nblocks, 256>>> (n,2.0,x,y);
//DAXPY in CUDA
_device_
void daxpy(int n,double a,double* x,double* y){
int i=blockIDx.x*blockDim.x+threadIdx.x;
if (i<n) y[i]= a*x[i]+ y[i]
}
CKV
Nvidia GPU Instruction Set Architecture

Instruction Set target of Nvidia compiler is an abstraction


of the hardware instruction set
PTX (Parallel Thread Execution)
Stable Instruction set for compilers

Compatibility across generations of GPU


PTX describe operations on single CUDA thread
One-to-one map with H/W instruction
One PTX expand to many machine instructions

PTX uses virtual registers (compiler maps physical registers to thread)


CKV
Nvidia GPU Instruction Set Architecture
//Invoke DAXPY with 256 threads per Thread Block
_host_
int nblocks = (n+255)/256;
daxpy<<<nblocks, 256>>> (n,2.0,x,y);
//DAXPY in CUDA
_device_
void daxpy(int n,double a,double* x,double* y){
int i=blockIDx.x*blockDim.x+threadIdx.x;
if (i<n) y[i]= a*x[i]+ y[i]
}

shl.s32 R8, blockIdx, 8 ; Thread Block ID * Block size (256 or 28)


add.s32 R8, R8, threadIdx ; R8 = i = my CUDA thread ID
shl.s32 R8, R8, 3 ;Byte offset
ld.global.f64 RD0, [X+R8] ; RD0 = X[i]
ld.global.f64 RD2, [Y+R8] ; RD2 = Y[i]
mul.f64 R0D, RD0, RD4 ; Product in RD0 = RD0 * RD4 (scalar a)
add.f64 R0D, RD0, RD2 ; Sum in RD0 = RD0 + RD2 (Y[i])
st.global.f64 [Y+R8], RD0 ; Y[i] = sum (X[i]*a + Y[i])
CKV
Conditional Branching
Like vector arch, GPU branch hardware uses internal masks

Branch synchronization stack


Entries consist of masks for each SIMD lane
i.e. which threads commit their results (all threads execute)

Instruction markers to manage when a branch diverges into


multiple execution paths
Push on divergent branch
.. and when paths converge
Act as barriers, Pops stack

Per‐thread‐lane 1‐bit predicate register,


CKV
Conditional Branching
if (X[i] != 0)
X[i] = X[i] – Y[i];
else X[i] = Z[i];

ld.global.f64 RD0, [X+R8] ; RD0 = X[i]


setp.neq.s32 P1, RD0, #0 ; P1 is predicate register 1
@!P1, bra ELSE1, *Push ; Push old mask, set new mask bits
; if P1 false, go to ELSE1
ld.global.f64 RD2, [Y+R8] ; RD2 = Y[i]
sub.f64 RD0, RD0, RD2 ; Difference in RD0
st.global.f64 [X+R8], RD0 ; X[i] = RD0
@P1, bra ENDIF1, *Comp ; complement mask bits
; if P1 true, go to ENDIF1

ELSE1: ld.global.f64 RD0, [Z+R8] ; RD0 = Z[i]


st.global.f64 [X+R8], RD0 ; X[i] = RD0
ENDIF1: <next instruction>, *Pop ; pop to restore old mask
CKV
NVIDIA GPU Memory Structures
Each SIMD Lane has private section of off-chip DRAM
“Private memory”, not shared by any other lanes

Contains stack frame, spilling registers, and private variables

Recent GPUs cache this in L1 and L2 caches

Each MT SIMD processor also has “Local memory” that is on


chip
Shared by SIMD lanes / threads within a block only

The off-chip memory shared by SIMD processors is “GPU


Memory”
Host can read and write GPU memory
CKV
NVIDIA GPU Memory Structures
CKV
Detecting and Enhancing Loop Level Parallelism
Focuses on determining whether data accesses in later
iterations are dependent on data values produced in
earlier iterations
Loop‐carried dependence

Loop‐level parallelism has no loop‐carried dependence


for (i=999; i>=0; i=i‐1)
x[i] = x[i] + s;

No Loop‐carried dependence
CKV
Detecting and Enhancing Loop Level Parallelism

for (i=0; i<100; i=i+1) {


A[i+1] = A[i] + C[i]; /* S1 */
B[i+1] = B[i] + A[i+1]; /* S2 */
}

S1 uses value computed by S1 in earlier iteration


Iteration i computes A[i+1] (& B[i+1]) which is used in iteration i+1

Loop‐carried dependence

S2 uses value computed by S1 in same iteration


No Loop‐carried dependence
CKV
Detecting and Enhancing Loop Level Parallelism
for (i=0; i<100; i=i+1) {
A[i] = A[i] + B[i]; /* S1 */
B[i+1] = C[i] + D[i]; /* S2 */
}

S1 uses value computed by S2 in previous iteration, but


this dependence is not circular.

Neither statement depend on itself

Although S1 Depends on S2, S2 does not depend on S1,


Interchanging the two statements will not affect the
execution of S2
CKV
Detecting and Enhancing Loop Level Parallelism
for (i=0; i<100; i=i+1) {
A[i] = A[i] + B[i]; /* S1 */
B[i+1] = C[i] + D[i]; /* S2 */
}

A[0] = A[0] + B[0];


for (i=0; i<99; i=i+1) {
B[i+1] = C[i] + D[i]; /*S2*/
A[i+1] = A[i+1] + B[i+1]; /*S1*/
}
B[100] = C[99] + D[99];
CKV
Detecting and Enhancing Loop Level Parallelism
for (i=0;i<100;i=i+1) {
A[i] = B[i] + C[i];
D[i] = A[i] * E[i];
}

The second reference to A needs not be translated to a load


instruction.
The two reference are the same. There is no intervening memory
access to the same location

A more complex analysis, i.e. Loop‐carried dependence


analysis + data dependence analysis, can be applied in the
same basic block to optimize
CKV
Detecting and Enhancing Loop Level Parallelism
Recurrence is a special form of loop‐carried dependence.

for (i=1;i<100;i=i+1) {
Y[i] = Y[i‐1] + Y[i];
}

Detecting a recurrence is important

Some vector computers have special support for


executing recurrence

It my still be possible to exploit a fair amount of ILP


CKV

Thank You for Attending

You might also like