Sampling & Filtering

Christoph Garth
Scientific Visualization Lab
A Bit of Theory
 Throughout the course, we have observed artifacts during rendering, both in ray tracing
and rasterization.

Staircase pattern Aliasing

These artifacts result due to the pointwise sampling of the light signal.
Computer Graphics – Sampling & Filtering– A Bit of Theory 12–1
 Observe: Image synthesis is a pointwise sampling of a piecewise continuous signal.

• The generated image consists of samples of the light signals at discrete points on a
virtual sensor (pixel raster).
• The inter-pixel distance defines the frequency (amount of the detail) which can be
reconstructed (represented).
• Undersampling of the image – using too few samples – results in aliasing artifacts.

Aliasing describes the effect that a high-frequency signal appears as a low-frequency

signal due to undersampling.

Mathematically: The frequency of the signal exceeds the Nyquist frequency.

Computer Graphics – Sampling & Filtering– A Bit of Theory 12–2

 The Fourier Transform decomposes a signal (function) (e.g. over [0, 1]) into its constituent
frequencies. Roughly speaking, the signal can be written as a series
∞
a0 X
f (x) = + (an cos(2πnx) + bn sin(2πnx))
2 n=1

Illustration: step function as a sum of cosines

Intuition: The frequency of the signal is the largest n for which an , bn 6= 0.

Computer Graphics – Sampling & Filtering– A Bit of Theory 12–3

 The Nyquist frequency of a discrete signal processing system is half of the sampling rate.

Shannon sampling theorem: A signal with a maximum frequency f must be sampled with
frequency 2f to allow correct reconstruction.

Undersampling leads to aliasing – the reconstruction appears to be of lower frequency

than the original signal.

Computer Graphics – Sampling & Filtering– A Bit of Theory 12–4

 Aliasing artfiacts

spatial aliasing temporal aliasing

sample sample sample

time

Illustration of temporal aliasing (“wagon wheel effect”) → video, video.

Computer Graphics – Sampling & Filtering– A Bit of Theory 12–5

 Illustration: reconstruction errors in rasterization

original rendered

Computer Graphics – Sampling & Filtering– A Bit of Theory 12–6

 Attention: Perspective foreshortening increases the signal frequency emitted by
an object with increasing distance.

d0 d1 d2
camera

Computer Graphics – Sampling & Filtering– A Bit of Theory 12–7

 Illustration: Aliasing artifacts in perspective

high frequencies
aliasing

≈ Nyquist frequency

low frequencies
correct reconstruction

Computer Graphics – Sampling & Filtering– A Bit of Theory 12–8

 How can aliasing (undersampling) artifacts be avoided or mitigated?

Reason: Sampling frequency lower than signal frequency.

Strategy 1: Increase sampling rate

• Supersampling: Increase of the sampling rate.

Strategy 2: Reduce signal frequency

• Prefiltering: Reduce the signal frequency by low-pass filtering.

• Postfiltering: Filter after reconstruction to remove aliasing artifacts (e.g.
selective blurring).

Computer Graphics – Sampling & Filtering– A Bit of Theory 12–9

Supersampling
 Supersampling: increase sampling rate

• ray tracing: send several rays per pixel and calculate average
• rasterization: virtually increase resolution of pixel raster, then average

Practice:

• E.g. 4x4 rays per pixel, calculate average of all 16 colors

• reduced artifacts, but does not in princple “solve” aliasing of edges (edge →
step function → unlimited frequency in principle)
• steps (e.g. edges) are blured or smoothed

Computer Graphics – Sampling & Filtering– Supersampling 12–10

 Example: 2 × 2 regular supersampling

virtual pixel raster

(2 × 2 subsamples)

No staircase effect is apparent, but edges are not sharp.

Note: This is also often called full-scene anti-aliasing (FSAA).

Computer Graphics – Sampling & Filtering– Supersampling 12–11

 Example: supersampling with weighted averaging

1/16 1/8 1/16 1/16 1/8 1/16

1/8 1/4 1/8 1/8 1/4 1/8

1/16 1/8 1/16 1/16 1/8 1/16

1/16 1/8 1/16 1/16 1/8 1/16

1/8 1/4 1/8 1/8 1/4 1/8

1/16 1/8 1/16 1/16 1/8 1/16

Weighted sampling is derived from pre-filtering (smoothing) the light signal using a
Gaussian filter.

Computer Graphics – Sampling & Filtering– Supersampling 12–12

 Example: jittered sampling – (irregularly / randomly) shifted sample points

jittered regular

Computer Graphics – Sampling & Filtering– Supersampling 12–13

 Effectiveness of jittered sampling:

• The human visual perception is very sensitive for regular structures, but less
sensitive for high frequency noise.

Stochastic Supersampling: Choose samples randomly within the pixel.

• Alias frequencies convert to noise and are thus less noticeable.

• Problem: sample clustering

Computer Graphics – Sampling & Filtering– Supersampling 12–14

 Poisson-disk sampling
• Enforce a minimal distance to each sample.

Jittered sampling
• Random shift of samples on a regular grid.

Stratified sampling
• Create random samples per regular sections of the pixel.

Computer Graphics – Sampling & Filtering– Supersampling 12–15

 Illustration: effect of regular and jittered supersampling

regular, 1x1

regular, 3x3

regular, 7x7

jittered, 3x3

jittered, 7x7

Computer Graphics – Sampling & Filtering– Supersampling 12–16

 Example: comparison of supersampling strategies

Computer Graphics – Sampling & Filtering– Supersampling 12–17

 Example: comparison of supersampling strategies

Computer Graphics – Sampling & Filtering– Supersampling 12–18

 Example: comparison of supersampling strategies

Computer Graphics – Sampling & Filtering– Supersampling 12–19

 Example: comparison of supersampling strategies

Computer Graphics – Sampling & Filtering– Supersampling 12–20

 Multi-sample AA (MSAA) only supersamples the depth and stencil buffers, and blends
fragments with the background color based on the ratio of overlapped samples. Different
patterns are in use (x2, x4, x8, Quincunx,. . . ).

MSAA is supported transparently on modern GPUs, but consumes additional

memory-bandwidth and may hence affect performance negatively.

Computer Graphics – Sampling & Filtering– Supersampling 12–21

Postfiltering
 Postfiltering: Smothing of edge artifacts after rendering.

E.g. FXAA (Fast approXimate Anti-Aliasing)

Usually used in combination of MSAA.

There are many postfiltering techniques for real-time rendering: MLAA, SMAA, CSAA, . . . ,
some vendor-specific → Details elsewhere.
Computer Graphics – Sampling & Filtering– Postfiltering 12–22
Prefiltering
 Textures are very sensitive to aliasing effects:

• The pre-image of one pixel in screen-coordinates can cover several texels in texture
space (minification).
• Undersampling occurs if only one texel in the pre-image is evaluated.
• Mitigation: multiple samples (expensive) or pre-filtering.

t xs

Pixel

Texel

s ys

Computer Graphics – Sampling & Filtering– Prefiltering 12–23

 Texture sampling rate is determined by the size at which objects appear.

Computer Graphics – Sampling & Filtering– Prefiltering 12–24

 Mip-Mapping: Pre-filtered textures in a multi-esolution pyramid (level of detail (LOD)).

...
l=3
Construction of levels: l=2

• Average of 2x2 texels of

l=1
lower level.
v
Storage overhead: l=0
1 1 1 u
+ + ··· = = 33%
4 16 3

Computer Graphics – Sampling & Filtering– Prefiltering 12–25

 Illustration: the maximum frequency decreases for each level.

...

256x256 128x128 64x64 32x32

LOD0 LOD1 LOD2 LOD3

Computer Graphics – Sampling & Filtering– Prefiltering 12–26

 Different LOD levels are used for different parts of the scene.

Computer Graphics – Sampling & Filtering– Prefiltering 12–27

 Principle of mip-mapping:
• When a fragment is

image space
generated during the
rendering, then the level of
detail is calculated such that
texel size corresponds to the
pixel size. The resulting
texture is interpolated at this

texture space
level.
• How to calculate the size of a
texel relative to the pixel
size?

Computer Graphics – Sampling & Filtering– Prefiltering 12–28

 Illustration: pixel size in texture space

 
δu δv
δy , δy
     
δu δv δu δv
d ≈ max , , ,
δx δx δy δy
l = log2 d
δu δv


δx , δx
δu
≈ ux+1,y − ux,y
δx
δu
≈ ux,y+1 − ux,y
δy
u, v: texture coordinates of the fragment
x, y: image pixel coordinates

l < 0 ⇒ less texel per pixel: l > 0 ⇒ more texel per pixel:
magnification minification

Computer Graphics – Sampling & Filtering– Prefiltering 12–29

 Illustration: mip-mapping

Computer Graphics – Sampling & Filtering– Prefiltering 12–30

 without mipmapping with mipmapping
Computer Graphics – Sampling & Filtering– Prefiltering 12–31
 Interpolation between resolution levels:

• Mipmaps store only discrete levels.

• The pixel size l is usually between two levels ⇒ linear interpolation between
levels (trilinear filtering).

Sample 0
assign
Select resolution interpolate

Sample 1

Computer Graphics – Sampling & Filtering– Prefiltering 12–32

 Realization in OpenGL:

glBindTexture( GL_TEXTURE_2D, textureId );

// set nearest-neighbor interpolation for minification

glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST );

// set bilinear interpolation for magnification

glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );

// set trilinear filtering / mipmapping for minification

glTexParameteri( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR );
// tell OpenGL to automatically generate Mipmap textures
glGenerateMipmap( GL_TEXTURE_2D );

Computer Graphics – Sampling & Filtering– Prefiltering 12–33

 GL_TEXTURE_MIN_FILTER = GL_NEAREST, GL_TEXTURE_MAG_FILTER = GL_NEAREST;

undersampling /
minfication

aliasing
stepping effects
magnification

Computer Graphics – Sampling & Filtering– Prefiltering 12–34

 GL_TEXTURE_MIN_FILTER = GL_LINEAR, GL_TEXTURE_MAG_FILTER = GL_LINEAR;

undersampling /
minfication

aliasing
stepping effects
magnification

Computer Graphics – Sampling & Filtering– Prefiltering 12–35

 GL_TEXTURE_MIN_FILTER = GL_LINEAR, GL_TEXTURE_MAG_FILTER = GL_LINEAR_MIPMAP_LINEAR;

undersampling /
minfication

aliasing
stepping effects
magnification

Computer Graphics – Sampling & Filtering– Prefiltering 12–36

 Illustration: mipmap levels

Computer Graphics – Sampling & Filtering– Prefiltering 12–37

 Extension: lower resolution separately in u- and v-directions (Anisotropic Mipmapping).

Avoids minification/magnification if
the sample rate is different in u- and
v-direction.

Computer Graphics – Sampling & Filtering– Prefiltering 12–38

 Illustration: anisotropic mipmapping

isotropic anisotropic
same level for different levels
x- and y-direction for x- and y-direction

Computer Graphics – Sampling & Filtering– Prefiltering 12–39

 Summary

Real-time rendering:
Real-time applications (e.g. games) use a combination of supersampling, prefiltering and
postfiltering, for example typically:

• anistropic mipmapping for textured surfaces

• MSAA
• postfiltering using FXAA

Ray / Path tracing:

• supersampling
• in combination with mipmapping
(requires ray differentials for screen-space derivatives)

Computer Graphics – Sampling & Filtering– Prefiltering 12–40

the end