BBM 413

Fundamentals of
Image Processing

Erkut Erdem
Dept. of Computer Engineering
Hacettepe University

Frequency Domain
Techniques – Part1
Review - Point Operations
• Smallest possible neighborhood is of size 1x1
• Process each point independently of the others
• Output image g depends only on the value of f
at a single point (x,y)
• Transformation function T remaps the sample’s value:
s = T(r)
where
– r is the value at the point in question
– s is the new value in the processed result
– T is a intensity transformation function
 1 1 1

Review – Spatial Filtering g[× ,× ] 1 1 1

1 1 1

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0
0 0
0 0
0 90
90 0
0 90
90 90
90 90
90 0
0 0
0

0
0 0
0 0
0 90
90 90
90 90
90 90
90 90
90 0
0 0
0

0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0

0
0 0
0 90
90 0
0 0
0 0
0 0
0 0
0 0
0 0
0

0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
0

h[ m, n] = å g[ k , l ] f [ m + k , n + l ]
k ,l Slide credit: S. Seitz
 1 1 1

Review – Spatial Filtering g[× ,× ] 1 1 1

1 1 1

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = å g[ k , l ] f [ m + k , n + l ]
k ,l Slide credit: S. Seitz
 1 1 1

Review – Spatial Filtering g[× ,× ] 1 1 1

1 1 1

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = å g[ k , l ] f [ m + k , n + l ]
k ,l Slide credit: S. Seitz
 1 1 1

Review – Spatial Filtering g[× ,× ] 1 1 1

1 1 1

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = å g[ k , l ] f [ m + k , n + l ]
k ,l Slide credit: S. Seitz
 1 1 1

Review – Spatial Filtering g[× ,× ] 1 1 1

1 1 1

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 90 0 90 90 90 0 0

0 0 0 90 90 90 90 90 0 0

0 0 0 0 0 0 0 0 0 0

0 0 90 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = å g[ k , l ] f [ m + k , n + l ]
k ,l Slide credit: S. Seitz
 1 1 1

Review – Spatial Filtering g[× ,× ] 1 1 1

1 1 1

f [.,.] h[.,.]
0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10

0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20

0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30

0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30

0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30

0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20

0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10

0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

h[ m, n] = å g[ k , l ] f [ m + k , n + l ]
k ,l Slide credit: S. Seitz
 Review – Spatial Filtering

1 0 -1
2 0 -2
1 0 -1

Sobel

Slide credit: J. Hays

 Review – Spatial Filtering
Fill in the blanks: Filtering A
Operator
a) _ = D * B
b) A = _ * _
c) F = D * _
d) _ = D * D B
E
G
F C

H I D

Slide credit: D. Hoiem

Today
• Frequency domain techniques
• Images in terms of frequency
• Fourier Series
• Convolution Theorem
 Why does the Gaussian give a nice smooth image,
but the square filter give edgy artifacts?

Gaussian Box filter

Slide credit: D. Hoiem

Why does a lower resolution image still make sense
to us? What do we lose?

Image: http://www.flickr.com/photos/igorms/136916757/ Slide credit: D. Hoiem

How is it that a 4MP image can be compressed to a
few hundred KB without a noticeable change?

Slide credit: J. Hays

Answer to these questions?
• Thinking images in terms of frequency.
The"2D"(Con6nuousBTime)"Fourier"Transform
The"2D"(Con6nuousBTime)"Fourier"Transform
• Treat images as infinite-size, continuous periodic
...&
...&
functions.

Carpet&image&(black&&&white)&
Carpet&image&(black&&&white)&

=&=& ...&
...&
...&
...&

...&
...&
 Jean Baptiste Joseph Fourier (1768-1830)
had crazy idea (1807):
Any univariate function can be
rewritten as a weighted sum of
sines and cosines of different
frequencies.

Slide credit: A. Efros

 Jean Baptiste...the
Joseph Fourier
manner in (1768-1830)
which the author arrives at these
equations is not exempt of difficulties and...his
had crazy idea (1807):
analysis to integrate them still leaves something to be
Any univariate functiondesired
can be on the score of generality and even rigour.
rewritten as a weighted sum of
sines and cosines of different
frequencies.

• Don’t believe it?

– Neither did Lagrange, Laplace
Laplace, Poisson and
other big wigs
– Not translated into
English until 1878!

Legendre
Lagrange
 Jean Baptiste Joseph Fourier (1768-1830)
had crazy idea (1807):
Any univariate function can be
rewritten as a weighted sum of
sines and cosines of different
frequencies.

• Don’t believe it?

– Neither did Lagrange,
Laplace, Poisson and
other big wigs
– Not translated into
English until 1878!
• But it’s (mostly) true!
– called Fourier Series
– there are some subtle
restrictions
Slide credit: A. Efros
 A sum of sines
Our building block:

Asin(wx + f )
Add enough of them to get
any signal f(x) you want!

Slide credit: A. Efros

Frequency Spectra
• example: g(t) = sin(2πf t) + (1/3)sin(2π(3f) t)

= +

Slide credit: A. Efros

Frequency Spectra

Slide credit: A. Efros

Frequency Spectra

= +

Slide credit: A. Efros

Frequency Spectra

= +

Slide credit: A. Efros

Frequency Spectra

= +

Slide credit: A. Efros

Frequency Spectra

= +

Slide credit: A. Efros

Frequency Spectra

= +

Slide credit: A. Efros

Frequency Spectra

¥
1
= Aå sin(2p kt )
k =1 k

Slide credit: A. Efros

Frequency Spectra

Image credit: Lucas V. Barbosa

Example: Music
• We think of music in terms of frequencies at different
magnitudes.

Slide credit: D . Hoeim

Other signals
• We can also think of all kinds of other signals the same way

xkcd.com
Slide credit: J. Hays
Fourier Transform
We want to understand the frequency w of our signal. So, let s
reparametrize the signal by w instead of x:

f(x) Fourier F(w)

Transform

For every w from 0 to inf, F(w) holds the amplitude A and

phase f of the corresponding sine Asin(wx + f )
• How can F hold both? Complex number trick!

F (w ) = R(w ) + iI (w )
-1 I (w )
A = ± R(w ) + I (w )
2 2
f = tan
R(w )
We can always go back:

F(w) Inverse Fourier f(x)

Transform Slide credit: A. Efros
 Fourier Transform
• Fourier transform stores the magnitude and phase at each
frequency
– Magnitude encodes how much signal there is at a particular frequency
– Phase encodes spatial information (indirectly)
– For mathematical convenience, this is often notated in terms of real and
complex numbers

I (w )
Amplitude: A = ± R(w ) + I (w ) Phase: f = tan
2 2 -1

R(w )
Discrete Fourier transform

• Forward transform
M −1 N−1
1
F(u, v) =
MN
∑ ∑ f (x, y)e − j 2 π (ux/M +vy/N )

x=0 y=0

for u = 0,1, 2,..., M −1, v = 0,1, 2,..., N −1

• Inverse transform
M -1 N -1
f ( x, y ) = åå F (u, v)e j 2p (ux / M + vy / N ) Euler’s definition of eiθ
u =0 v =0

for x = 0,1,2,..., M - 1, y = 0,1,2,..., N - 1

u, v : the transform or frequency variables
x, y : the spatial or image variables

Slide credit: B. Freeman and A. Torralba

The Fourier Transform
• Represent function on a new basis
– Think of functions as vectors, with many
components
– We now apply a linear transformation to transform
the basis
• dot product with each basis element

• In the expression, u and v select the basis

element, so a function of x and y becomes a
function of u and v
−i 2 π ( ux +vy )
• basis elements have the form e

Slide credit: S. Thrun

 How to interpret a 2-d Fourier
Spectrum
How to interpret 2D Fourier
Spectrum
Vertical orientation Low spatial frequencies

45 deg.

0 Horizontal
orientation

0 fmax High
spatial
fx in cycles/image frequencies

Log
Log power spectrum
power spectrum

Slide credit: B. Freeman and A. Torralba

Fourier basis element
e −i 2 π ( ux +vy)
example, real part

Fu,v(x,y)

Fu,v(x,y)=const. for (ux+vy)=const.

Vector (u,v)
• Magnitude gives frequency
• Direction gives orientation.

Slide credit: S. Thrun

Here u and
v are larger
than in the
previous
slide.

Slide credit: S. Thrun

And larger still...

Slide credit: S. Thrun

2D FFT

Sinusoid with frequency = 1 and its FFT

Slide credit: M. H. Yang

2D FFT

Sinusoid with frequency = 3 and its FFT

Slide credit: M. H. Yang

2D FFT

Sinusoid with frequency = 5 and its FFT

Slide credit: M. H. Yang

2D FFT

Sinusoid with frequency = 10 and its FFT

Slide credit: M. H. Yang

2D FFT

Sinusoid with frequency = 15 and its FFT

Slide credit: M. H. Yang

2D FFT

Sinusoid with varying frequency and their FFT

Slide credit: M. H. Yang

Rotation

Sinusoid rotated at 30 degrees and its FFT

Slide credit: M. H. Yang

2D FFT

Sinusoid rotated at 60 degrees and its FFT

Slide credit: M. H. Yang

2D FFT

Slide credit: M. H. Yang

 Fourier analysis in images

Intensity Image

Fourier Image

http://sharp.bu.edu/~slehar/fourier/fourier.html#filtering
Slide credit: A. Efros More: http://www.cs.unm.edu/~brayer/vision/fourier.html
 Signals can be composed

+ =

http://sharp.bu.edu/~slehar/fourier/fourier.html#filtering
Slide credit: A. Efros More: http://www.cs.unm.edu/~brayer/vision/fourier.html
 Some important Fourier Transforms
Image
Magnitude FT

Slide credit: B. Freeman and A. Torralba

 Some important Fourier Transforms
Image
Magnitude FT

Slide credit: B. Freeman and A. Torralba

TheThe Fourier
Fourier Transform
Transform of some
of some
well-known
Image
images
important
Log(1+Magnitude FT) images

Slide credit: B. Freeman and A. Torralba

Fourier Amplitude Spectrum

A B C

1 2 3

fx(cycles/image pixel size) fx(cycles/image pixel size) fx(cycles/image pixel size)

Slide credit: B. Freeman and A. Torralba

 Fourier transform magnitude

What in the image causes the dots?

Slide credit: B. Freeman and A. Torralba
Masking out the fundamental and
harmonics from periodic pillars

Slide credit: B. Freeman and A. Torralba

The Convolution Theorem

• The Fourier transform of the convolution of two

functions is the product of their Fourier transforms

F[ g * h] = F[ g ] F[ h]
• The inverse Fourier transform of the product of two
Fourier transforms is the convolution of the two inverse
Fourier transforms
-1 -1 -1
F [ gh] = F [ g ] * F [ h]
• Convolution in spatial domain is equivalent to
multiplication in frequency domain!
Slide credit: A. Efros
 Properties of Fourier Transforms

• Linearity

• Fourier transform of a real signal is symmetric about

the origin

• The energy of the signal is the same as the energy of

its Fourier transform

Slide credit: J. Hays

Filtering in spatial domain 1 0 -1
2 0 -2
1 0 -1

* =

Slide credit: D. Hoiem

Filtering in frequency domain

FFT

FFT

=
Inverse FFT

Slide credit: D. Hoiem

 2D convolution theorem example

f(x,y) |F(sx,sy)|

h(x,y) |H(sx,sy)|

g(x,y) |G(sx,sy)|

Slide credit: A. Efros

 Filtering
Why does the Gaussian give a nice smooth image, but
the square filter give edgy artifacts?
Gaussian Box filter

Slide credit: A. Efros

Filtering
Gaussian

Slide credit: A. Efros

Filtering

Box Filter

Slide credit: A. Efros

Fourier Transform pairs
The picture can't be
displayed.

Slide credit: A. Efros

Low-pass, Band-pass, High-pass filters
low-pass:

High-pass / band-pass:

Slide credit: A. Efros

Edges in images

Slide credit: A. Efros

 Phase
Phase and
and Magnitude
Phase and Magnitude Magnitude Image with cheetah phase
Image with cheetah phase
• Curious fact Image
(and withmagnitude)
zebra cheetah phase
(and zebra magnitude)
(and zebra magnitude)
– all natural images have about the same
magnitude transform
– hence, phase seems to matter, but
magnitude largely doesn t Phase and Magnitude
Image with cheetah phase
(and zebra magnitude)
• Demonstration
– Take two pictures, swap the phase Image with zebra phase
transforms, compute the inverse - what Image with
Image with zebra
zebra phase
phase
(and cheetah magnitude)
does the result look like? (and
(andcheetah magnitude)
cheetah magnitude)

Image with zebra phase

(and cheetah magnitude)

Computer Vision - A Modern Approach - Set: Pyramids and Texture - Slides by D.A. Forsyth
Computer Vision - A Modern Approach - Set: Pyramids and Texture - Slides by D.A. Forsyth
Slide credit: B. Freeman and A. Torralba
Slide credit: B. Freeman and A. Torralba
This is the
magnitude
transform of
the cheetah
picture

Slide credit: B. Freeman and A. Torralba

Slide credit: B. Freeman and A. Torralba
This is the
magnitude
transform of
the zebra
picture

Slide credit: B. Freeman and A. Torralba

Reconstruction
with zebra
phase, cheetah
magnitude

Slide credit: B. Freeman and A. Torralba

Reconstruction
with cheetah
phase, zebra
magnitude

Slide credit: B. Freeman and A. Torralba

Clues from Human Perception
• Early processing in humans filters for various orientations and scales of
frequency
• Perceptual cues in the mid-high frequencies dominate perception
• When we see an image from far away, we are effectively subsampling it

Early Visual Processing: Multi-scale edge and blob filters

Slide credit: J. Hays
CS 4495 Computer Vision – A. Bobick Frequency and Fourier Transform

Campbell-Robson contrast
Campbell-Robson contrast sensitivity curve
sensitivity curve

The higher
The higher the frequency thesensitive
the less frequency the less
human sensitive
visual system is...
human visual system is… Slide credit: J. Hays
 Lossy Image Compression (JPEG)
CS 4495 Computer Vision – A. Bobick Frequency and Fourier Transform

Lossy Image Compression (JPEG)

Block-based Discrete Cosine Transform (DCT) on 8x8 Slide credit: A. Bobick

 CS 4495 Computer Vision – A. Bobick

Using
Using DCT
DCT in JPEG
in JPEG
• The
• The firstfirst coefficient
coefficient is theisDC
B(0,0)
B(0,0) thecomponent,
DC component, the
the average
average intensity
intensity
• The top-left coeffs represent low frequencies, the bottom
• The top-left coeffs represent low frequencies,
theright
bottom – high
rightfrequencies
– high frequencies

Slide credit: A. Bobick

 CS 4495 Computer Vision – A. Bobick

Using
Image DCT in JPEG
compression using DCT
• The
• DCT first coefficient
enables B(0,0) is by
image compression theconcentrating
DC component,mostthe
image
average in
information intensity
the low frequencies
• The top-left coeffs represent low frequencies, the bottom
• Loose unimportant image info (high frequencies) by cutting
right
B(u,v) at – high frequencies
bottom right
• The decoder computes the inverse DCT – IDCT

Slide credit: A. Bobick

CS 4495 Computer Vision – A. Bobick Frequency and Fourier Transform

JPEG compression comparison

JPEG compression comparison

89k
89k 12k
12k

Slide credit: A. Bobick

 Things to Remember
• Sometimes it makes sense to think of images
and filtering in the frequency domain
– Fourier analysis

• Can be faster to filter using FFT for large

images (N logN vs. N2 for auto-correlation)

• Images are mostly smooth

– Basis for compression

Slide credit: J. Hays

 Practice question
1. Match the spatial domain image to the Fourier magnitude
image
1 2 3 4 5

A C E
D

Slide credit: J. Hays

Summary
• Frequency domain techniques
• Images in terms of frequency
• Fourier Series
• Convolution Theorem
Next Week
• Sampling
• Gabor wavelets
• Steerable filters