Sampling Theory 101

Sampling Theory 101
Oliver Kreylos
[email protected]
Center for Image Processing and Integrated Computing (CIPIC)

Department of Computer Science
University of California, Davis
CIPIC Seminar 11/06/02 – p.1

Outline
• Things that typically go wrong
• Mathematical model of sampling
• How to do things better

Things That Typically Go Wrong

Where Does Sampling Occur?
Almost all data we are dealing with is discrete

• Evaluation of sampled functions at arbitrary
sites
• Volume rendering
• Isosurface extraction
• Ray tracing
• ...

What Went Wrong Here?
Typical ray tracing example:

Sampling and Reconstruction



• Undersampling?
• Sampling “below Nyquist rate”?
• Quick solution: Double sampling rate


• Things get better, but are still bad
• Isn’t sampling above the Nyquist rate
supposed to solve all problems?

Sampling Theory
To understand what went wrong on the last slide,

we need a mathematical model of sampling.

Mathematical Model of Sampling

Choice of Domain
• Process of sampling and reconstruction is
best understood in frequency domain
• Use Fourier transform to switch between time
and frequency domains
• Function in time domain: signal
• Function in frequency domain: spectrum

Fourier Transform I
• Many functions f : R → R can be written as
sums of sine waves
X
f (x) = aω sin(ωx + θω )
ω
• ω = 2π · frequency is angular velocity,

• aω is amplitude, and
• θω is phase shift

Fourier Transform II
• Moving to complex numbers simplifies
notation:
cos(ωx) + i sin(ωx) = eiωx
(Euler identity )
• One complex coefficient cω eiωx encodes both
amplitude and phase shift
• Moving to integral enlarges class of
representable functions

Fourier Transform III
• Now, for almost all f :
Z
f (x) = F (ω)eiωx dω
• F (ω) is the spectrum of f (x), and the above

operator is the inverse Fourier transform
• Its inverse, the Fourier transform, is
Z
F (ω) = f (x)e−iωx dx

Some Signals and Their Spectra
Single sine wave f (x) = sin(ωx):

Sum of two sine waves

f (x) = sin(ωx) + 0.5 sin(2ωx):

Box function:

Wider box function:

Triangle function:

Comb function:

Wider comb function:

Convolution I
• The convolution operator is a generalized
formula to express weighted averaging of an
input signal f and a weight function or filter
kernel g:
Z
(f ∗ g)(x) = f (t) · g(x − t) dt
• One important application of convolution is

reconstructing sampled signals

Convolution II
Z
(f ∗ g)(x0 ) = f (t) · g(x0 − t) dt
f f f
x0 x x0 x x0 x
g f f*g
x x0 x x0 x

Convolution III
Linear interpolation can be interpreted as

convolution:
f f
x x
g f
x x

Function Reconstruction
Constant interpolation:

Linear interpolation:

Catmull-Rom interpolation:

Cubic B-spline approximation:

The Convolution Theorem
• The Convolution Theorem relates convolution
and Fourier transform:
(f ∗ g) ↔ F · G
• Convolutions can be computed by going
through the frequency domain:
(f ∗ g) = IFT(FT(f ) · FT(g))

Sampling in Time Domain
• Sampling a function f means multiplying it
with a comb function c with tap distance d (or
sample frequency ω = 2π/d):
s=f ·c
• Reconstructing a sampled function means
convolving it with a suitable filter kernel
f0 = s ∗ k
• What happens to the function’s spectrum in
the process?
Sampling in Frequency Domain
• The spectrum of s is the convolution of F
and C, the spectra of f and c
• C is a comb function with tap distance 2π/d
• S consists of shifted copies of F , each 2π/d
apart

Sampling Process - Time Domain

Sampling Process - Frequency Domain

Sampling in Frequency Domain
• If the sampling frequency is ω, the replicated
copies of f ’s spectrum are ω apart
• If the highest-frequency component in f is ωf ,
f ’s spectrum covers the interval [−ωf , ωf ]
• If ωf ≥ ω/2, then the replicated spectra
overlap!
• This means, information is lost

The Sampling Theorem
• “If f is a frequency-limited function with
maximum frequency ωf , then f must be
sampled with a sampling frequency larger
than 2ωf in order to be able to exactly
reconstruct f from its samples.”
• This theorem is sometimes called Shannon’s
Theorem
• 2ωf is sometimes called Nyquist rate

Reconstruction
• The only difference between F and S is that S
is made of shifted copies of F
• If those extra, higher-frequency, spectra could
be removed from S, f would “magically”
reappear
• Reconstructing f means low-pass-filtering s!

Reconstruction Process - Frequency Dom

Reconstruction Process - Time Domain

Reconstruction Problem
• Practical problem: The optimal reconstruction
filter, the sinc function, has infinite support
• Evaluating it in the time domain involves
computing a weighted average of infinitely
many samples
• Practical reconstruction filters must have finite
support
• But: A perfect low-pass filter with finite
support does not exist!

The Quest For Better Filters

Commonly Used Filters
Constant filter:

Linear filter:

Catmull-Rom filter:

B-Spline filter:

Sampling Theory Filters
Truncated sinc filter:

Lanczos filter:

Full sinc filter:

Sampling Theory Applications

• Sampling theory filters (Lanczos etc.) can be
used to reconstruct n-dimensional functions
sampled on regular grids
• Oversampling improves reconstruction quality
with “bad” filters, but at a high cost

Function Resampling
• When resampling a sampled function to a
different grid (shifted and/or scaled), it is
important to use a good reconstruction filter
• When downsampling a function, it is
important to use a good low-pass filter to cut
off frequencies above half the new sampling
frequency

Volume Rendering
• Basic question: How many samples to take
per cell?

Volume Rendering
• Basic question: How many samples to take
per cell?
• Sampling theory says “1,” but integration

along ray might demand more

Ray Tracing
• In ray tracing, the function to be sampled is
typically not frequency-limited
• Practical solution: Shoot multiple rays per
pixel (oversampling) and sample down to
image resolution
• Using a good low-pass filter is important

Ray Tracing

Conclusions

Conclusions
• Knowing the mathematical background of
sampling and reconstruction helps to
understand common problems and devise
solutions
• Ad-hoc solutions usually do not work well
• This stuff should be taught in class!

The End

Sampling Theory 101

Uploaded by

Copyright:

Available Formats

Sampling Theory 101

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sampling Theory 101

Uploaded by

Copyright:

Available Formats

Sampling Theory 101

Center for Image Processing and Integrated Computing (CIPIC)

CIPIC Seminar 11/06/02 – p.1

CIPIC Seminar 11/06/02 – p.2

CIPIC Seminar 11/06/02 – p.3

Almost all data we are dealing with is discrete

CIPIC Seminar 11/06/02 – p.4

Typical ray tracing example:

CIPIC Seminar 11/06/02 – p.5

CIPIC Seminar 11/06/02 – p.6

CIPIC Seminar 11/06/02 – p.6

CIPIC Seminar 11/06/02 – p.6

CIPIC Seminar 11/06/02 – p.6

CIPIC Seminar 11/06/02 – p.6

CIPIC Seminar 11/06/02 – p.6

To understand what went wrong on the last slide,

CIPIC Seminar 11/06/02 – p.7

CIPIC Seminar 11/06/02 – p.8

CIPIC Seminar 11/06/02 – p.9

• ω = 2π · frequency is angular velocity,

CIPIC Seminar 11/06/02 – p.10

CIPIC Seminar 11/06/02 – p.11

• F (ω) is the spectrum of f (x), and the above

CIPIC Seminar 11/06/02 – p.12

Single sine wave f (x) = sin(ωx):

CIPIC Seminar 11/06/02 – p.13

Sum of two sine waves

CIPIC Seminar 11/06/02 – p.13

CIPIC Seminar 11/06/02 – p.13

Wider box function:

CIPIC Seminar 11/06/02 – p.13

CIPIC Seminar 11/06/02 – p.13

CIPIC Seminar 11/06/02 – p.13

Wider comb function:

CIPIC Seminar 11/06/02 – p.13

• One important application of convolution is

CIPIC Seminar 11/06/02 – p.14

CIPIC Seminar 11/06/02 – p.15

Linear interpolation can be interpreted as

CIPIC Seminar 11/06/02 – p.16

CIPIC Seminar 11/06/02 – p.17

CIPIC Seminar 11/06/02 – p.17

CIPIC Seminar 11/06/02 – p.17

Cubic B-spline approximation:

CIPIC Seminar 11/06/02 – p.17

CIPIC Seminar 11/06/02 – p.18

CIPIC Seminar 11/06/02 – p.20

CIPIC Seminar 11/06/02 – p.21

CIPIC Seminar 11/06/02 – p.22

CIPIC Seminar 11/06/02 – p.23

CIPIC Seminar 11/06/02 – p.24

CIPIC Seminar 11/06/02 – p.25

CIPIC Seminar 11/06/02 – p.26

CIPIC Seminar 11/06/02 – p.27

CIPIC Seminar 11/06/02 – p.28

CIPIC Seminar 11/06/02 – p.29

CIPIC Seminar 11/06/02 – p.30

CIPIC Seminar 11/06/02 – p.30

CIPIC Seminar 11/06/02 – p.30

CIPIC Seminar 11/06/02 – p.30