GPU Programming

using BU Shared Computing Cluster

Scientific Computing and Visualization

Boston University
 GPU Programming

• GPU – graphics processing unit

• Originally designed as a graphics processor

• Nvidia's GeForce 256 (1999) – first GPU

o single-chip processor for mathematically-intensive tasks

o transforms of vertices and polygons
o lighting
o polygon clipping
o texture mapping
o polygon rendering
 GPU Programming

Modern GPUs are present in

 Embedded systems
 Personal Computers
 Game consoles
 Mobile Phones
 Workstations
 GPU Programming

Traditional GPU workflow

Vertex processing Blending, Z-buffering

Triangles, Lines,
Shading, Texturing
Points
 GPU Programming

GPGPU
1999-2000 computer scientists from various fields started using
GPUs to accelerate a range of scientific applications.

GPU programming required the use of graphics APIs such as

OpenGL and Cg.

2002 James Fung (University of Toronto) developed OpenVIDIA.

NVIDIA greatly invested in GPGPU movement and offered a

number of options and libraries for a seamless experience for C, C+
+ and Fortran programmers.
 GPU Programming

GPGPU timeline
In November 2006 Nvidia launched CUDA, an API that allows to
code algorithms for execution on Geforce GPUs using C
programming language.

Khronus Group defined OpenCL in 2008 supported on AMD,

Nvidia and ARM platforms.

In 2012 Nvidia presented and demonstrated OpenACC - a set of

directives that greatly simplify parallel programming of
heterogeneous systems.
 GPU Programming

CPU GPU

CPUs consist of a few cores GPUs consist of hundreds or

optimized for serial processing thousands of smaller, efficient cores
designed for parallel performance
 GPU Programming

SCC CPU SCC GPU

Intel Xeon X5650: NVIDIA Tesla M2070:

Clock speed: 2.66 GHz Core clock: 1.15GHz
4 instructions per cycle Single instruction
CPU - 6 cores 448 CUDA cores

2.66 x 4 x 6 = 1.15 x 1 x 448 =

63.84 Gigaflops double precision 515 Gigaflops double precision
 GPU Programming

SCC CPU SCC GPU

Intel Xeon X5650: NVIDIA Tesla M2070:

Memory size: 288 GB Memory size: 3GB total
Bandwidth: 32 GB/sec Bandwidth: 150 GB/sec
 GPU Programming

GPU Computing Growth

2008 2013

100M x 4.3 430M

CUDA-capable GPUs CUDA-capable GPUs

150K x 10.67 1.6M

CUDA downloads CUDA downloads

1 x 50 50
Supercomputer Supercomputers

4,000 x 9.25 37,000

Academic Papers
Academic Papers
 GPU Programming

GPU Acceleration

Applications
GPU-accelerated OpenACC Programming
libraries Directives Languages

Seamless linking to GPU- Simple directives for easy Most powerful and flexible
enabled libraries. GPU-acceleration of new way to design GPU
and existing applications accelerated applications

cuFFT, cuBLAS,
C/C++, Fortran,
Thrust, NPP, IMSL, PGI Accelerator
Python, Java, etc.
CULA, cuRAND, etc.
 GPU Programming

GPU Accelerated Libraries

powerful library of parallel algorithms and data
structures;

provides a flexible, high-level interface for GPU

programming;

For example, the thrust::sort algorithm delivers

5x to 100x faster sorting performance than STL
and TBB
 GPU Programming

GPU Accelerated Libraries

a GPU-accelerated version of the complete

standard BLAS library;

cuBLAS
6x to 17x faster performance than the latest
MKL BLAS

Complete support for all 152 standard BLAS

routines

Single, double, complex, and double complex

data types

Fortran binding
 GPU Programming

GPU Accelerated Libraries

cuSPARSE NPP cuFFT cuRAND

 GPU Programming

OpenACC Directives
• Simple compiler directives
• Works on multicore CPUs & many core
GPUs
• Future integration into OpenMP

Program myscience

... serial code ... CPU

!$acc compiler Directive
do k = 1,n1
do i = 1,n2
... parallel code ... GPU
enddo
enddo
$acc end compiler Directive

End Program myscience

 GPU Programming

CUDA
Programming language extension to C/C++ and FORTRAN;

Designed for efficient general purpose computation on GPU.

__global__ void kernel(float* x, float* y, float* z, int n){

int idx= blockIdx.x * blockDim.x + threadIdx.x;
if(idx < n) z[idx] = x[idx] * y[idx];
}
int main(){
...
cudaMalloc(...);
cudaMemcpy(...);
kernel <<<num_blocks, block_size>>> (...);
cudaMemcpy(...);
cudaFree(...);
...
}
 GPU Programming

MATLAB with GPU-acceleration

Use GPUs with MATLAB through Parallel Computing Toolbox

• GPU-enabled MATLAB functions such as fft, filter, and several linear algebra
operations

• GPU-enabled functions in toolboxes: Communications System Toolbox, Neural

Network Toolbox, Phased Array Systems Toolbox and Signal Processing Toolbox

• CUDA kernel integration in MATLAB applications, using only a single line of MATLAB
code

A=rand(2^16,1); A=gpuArray(rand(2^16,1));

B=fft(A); B=fft(A);
 GPU Programming

Will Execution on a GPU Accelerate My Application?

Computationally intensive—The time spent on computation significantly

exceeds the time spent on transferring data to and from GPU memory.

Massively parallel—The computations can be broken down into

hundreds or thousands of independent units of work.