Introduction to the

CUDA Platform
 CUDA Parallel Computing Platform
www.nvidia.com/getcuda

OpenACC Programming
Programming Libraries
Directives Languages
Approaches “Drop-in” Easily Accelerate
Maximum Flexibility
Acceleration Apps

Nsight IDE CUDA-GDB

Development Linux, Mac and Windows debugger
GPU Debugging and NVIDIA Visual
Environment Profiling Profiler

Open Compiler Enables compiling new languages to CUDA

platform, and CUDA languages to other
Tool Chain architectures

Dynamic HyperQ GPUDirect

SMX
Parallelism
Hardware
Capabilities
© NVIDIA 2013
3 Ways to Accelerate Applications

Applications

OpenACC Programming
Libraries
Directives Languages

“Drop-in” Easily Accelerate Maximum

Acceleration Applications Flexibility

© NVIDIA 2013
3 Ways to Accelerate Applications

Applications

OpenACC Programming
Libraries
Directives Languages

“Drop-in” Easily Accelerate Maximum

Acceleration Applications Flexibility

© NVIDIA 2013
 Libraries: Easy, High-Quality
Acceleration
• Ease of use: Using libraries enables GPU acceleration without in-depth
knowledge of GPU programming

• “Drop-in”: Many GPU-accelerated libraries follow standard APIs, thus

enabling acceleration with minimal code changes

• Quality: Libraries offer high-quality implementations of functions

encountered in a broad range of applications

• Performance: NVIDIA libraries are tuned by experts

© NVIDIA 2013
 Some GPU-accelerated Libraries

NVIDIA cuBLAS NVIDIA cuRAND NVIDIA cuSPARSE NVIDIA NPP

Vector Signal GPU Accelerated Matrix Algebra

Image Processing Linear Algebra on GPU and NVIDIA cuFFT
Multicore

Building-block Sparse Linear C++ STL

ArrayFire Matrix
Algorithms for Algebra Features for
IMSL Library Computations
CUDA CUDA
© NVIDIA 2013
 3 Steps to CUDA-accelerated
application
• Step 1: Substitute library calls with equivalent CUDA library calls
saxpy ( … ) cublasSaxpy ( … )

• Step 2: Manage data locality

- with CUDA: cudaMalloc(), cudaMemcpy(), etc.
- with CUBLAS: cublasAlloc(), cublasSetVector(), etc.

• Step 3: Rebuild and link the CUDA-accelerated library

nvcc myobj.o –l cublas

© NVIDIA 2013
 Explore the CUDA (Libraries)
Ecosystem
• CUDA Tools and Ecosystem
described in detail on NVIDIA
Developer Zone:
developer.nvidia.com/cuda-tools-ecosystem

© NVIDIA 2013
3 Ways to Accelerate Applications

Applications

OpenACC Programming
Libraries
Directives Languages

“Drop-in” Easily Accelerate Maximum

Acceleration Applications Flexibility

© NVIDIA 2013
 OpenACC Directives
CPU GPU

Simple Compiler hints

Program myscience Compiler Parallelizes

... serial code ...
!$acc kernels code
do k = 1,n1 OpenACC
do i = 1,n2
compiler
... parallel code ...
enddo Hint Works on many-core
enddo
!$acc end kernels GPUs & multicore CPUs
...
End Program myscience

Your original
Fortran or C
code
© NVIDIA 2013
 OpenACC
The Standard for GPU Directives

• Easy: Directives are the easy path to accelerate

compute intensive applications

• Open: OpenACC is an open GPU directives standard,

making GPU programming straightforward and
portable across parallel and multi-core processors

• Powerful: GPU Directives allow complete access to the

massive parallel power of a GPU

© NVIDIA 2013
 Directives: Easy & Powerful
Real-Time Object Valuation of Stock Interaction of Solvents
Detection Portfolios using Monte and Biomolecules
Global Manufacturer of Carlo University of Texas at San Antonio
Navigation Systems
Global Technology Consulting
Company

5x in 40 Hours 2x in 4 Hours 5x in 8 Hours

“Optimizing code with directives is quite easy, especially compared to CPU threads or writing
CUDA kernels. The most important thing is avoiding restructuring of existing code for
production applications.
” -- Developer at the Global Manufacturer of
Navigation Systems © NVIDIA 2013
Start Now with OpenACC Directives
Sign up for a free trial of
the directives compiler
now!
Free trial license to PGI
Accelerator

Tools for quick ramp

www.nvidia.com/gpudirectives

© NVIDIA 2013
3 Ways to Accelerate Applications

Applications

OpenACC Programming
Libraries
Directives Languages

“Drop-in” Easily Accelerate Maximum

Acceleration Applications Flexibility

© NVIDIA 2013
 GPU Programming Languages
Numerical analytics MATLAB, Mathematica, LabVIEW

Fortran OpenACC, CUDA Fortran

C OpenACC, CUDA C

C++ Thrust, CUDA C++

Python PyCUDA, Copperhead

F# Alea.cuBase
© NVIDIA 2013
 Rapid Parallel C++ Development

• Resembles C++ STL

• High-level interface // generate 32M random numbers on host
• Enhances developer thrust::host_vector<int> h_vec(32 << 20);
thrust::generate(h_vec.begin(),
productivity h_vec.end(),
• Enables performance rand);
portability between GPUs and
// transfer data to device (GPU)
multicore CPUs thrust::device_vector<int> d_vec = h_vec;
• Flexible
// sort data on device
• CUDA, OpenMP, and TBB thrust::sort(d_vec.begin(), d_vec.end());
backends
// transfer data back to host
• Extensible and customizable thrust::copy(d_vec.begin(),
• Integrates with existing d_vec.end(),
h_vec.begin());
software
• Open source

http://developer.nvidia.com/thrust or http://thrust.googlecode.com
 Learn More
These languages are supported on all CUDA-capable GPUs.
You might already have a CUDA-capable GPU in your laptop
or desktop PC!
CUDA C/C++ GPU.NET
http://developer.nvidia.com/cuda-toolkit http://tidepowerd.com

Thrust C++ Template Library

http://developer.nvidia.com/thrust MATLAB
http://www.mathworks.com/discovery/
matlab-gpu.html
CUDA Fortran
http://developer.nvidia.com/cuda-toolkit

Mathematica
PyCUDA (Python) http://www.wolfram.com/mathematica/new
http://mathema.tician.de/software/pycuda -in-8/cuda-and-opencl-support/

© NVIDIA 2013
 Getting Started
• Download CUDA Toolkit & SDK: www.nvidia.com/getcuda

• Nsight IDE (Eclipse or Visual Studio): www.nvidia.com/nsight

• Programming Guide/Best Practices:

• docs.nvidia.com

• Questions:
• NVIDIA Developer forums: devtalk.nvidia.com
• Search or ask on: www.stackoverflow.com/tags/cuda

• General: www.nvidia.com/cudazone

© NVIDIA 2013