Motion illusion, rotating snakes

Slide credit Fei Fei Li

Slide credit Fei Fei Li
Slide credit Fei Fei Li
 09/11/2013
Image Filtering

Computer Vision
James Hays, Brown Many slides by Derek Hoiem
Next three classes: three views of filtering

• Image filters in spatial domain

– Filter is a mathematical operation of a grid of numbers
– Smoothing, sharpening, measuring texture

• Image filters in the frequency domain

– Filtering is a way to modify the frequencies of images
– Denoising, sampling, image compression

• Templates and Image Pyramids

– Filtering is a way to match a template to the image
– Detection, coarse-to-fine registration
Image filtering

• Image filtering: compute function of local

neighborhood at each position

• Really important!
– Enhance images
• Denoise, resize, increase contrast, etc.
– Extract information from images
• Texture, edges, distinctive points, etc.
– Detect patterns
• Template matching
Example: box filter

g[ , ]
1 1 1

1 1 1

1 1 1

Slide credit: David Lowe (UBC)

Image filtering 1 1 1

g[ , ] 1 1 1

1 1 1

f [.,.] h[.,.]

h[ m, n]   g[ k , l ] f [ m  k , n  l ]
k ,l Credit: S. Seitz
Image filtering 1 1 1

g[ , ] 1 1 1

1 1 1

f [.,.] h[.,.]

h[ m, n]   g[ k , l ] f [ m  k , n  l ]
k ,l Credit: S. Seitz
Image filtering 1 1 1

g[ , ] 1 1 1

1 1 1

f [.,.] h[.,.]

h[ m, n]   g[ k , l ] f [ m  k , n  l ]
k ,l Credit: S. Seitz
Image filtering 1 1 1

g[ , ] 1 1 1

1 1 1

f [.,.] h[.,.]

h[ m, n]   g[ k , l ] f [ m  k , n  l ]
k ,l Credit: S. Seitz
Image filtering 1 1 1

g[ , ] 1 1 1

1 1 1

f [.,.] h[.,.]

h[ m, n]   g[ k , l ] f [ m  k , n  l ]
k ,l Credit: S. Seitz
Image filtering 1 1 1

g[ , ] 1 1 1

1 1 1

f [.,.] h[.,.]

h[ m, n]   g[ k , l ] f [ m  k , n  l ]
k ,l Credit: S. Seitz
Image filtering 1 1 1

g[ , ] 1 1 1

1 1 1

f [.,.] h[.,.]

h[ m, n]   g[ k , l ] f [ m  k , n  l ]
k ,l Credit: S. Seitz
Image filtering g[ , ] 1 1 1
1 1 1
1 1 1

f [.,.] h[.,.]

h[ m, n]   g[ k , l ] f [ m  k , n  l ]
k ,l Credit: S. Seitz
Box Filter

g[ , ]
What does it do?
• Replaces each pixel with 1 1 1
an average of its
neighborhood 1 1 1

1 1 1
• Achieve smoothing effect
(remove sharp features)

Slide credit: David Lowe (UBC)

Smoothing with box filter
One more on board…
Practice with linear filters

?
0 0 0
0 1 0
0 0 0

Original

Source: D. Lowe
Practice with linear filters

0 0 0
0 1 0
0 0 0

Original Filtered
(no change)

Source: D. Lowe
Practice with linear filters

?
0 0 0
0 0 1
0 0 0

Original

Source: D. Lowe
Practice with linear filters

0 0 0
0 0 1
0 0 0

Original Shifted left

By 1 pixel

Source: D. Lowe
Practice with linear filters

-
0 0 0 1 1 1
0
0
2
0
0
0
1
1
1
1
1
1
?
(Note that filter sums to 1)
Original

Source: D. Lowe
Practice with linear filters

-
0 0 0 1 1 1
0 2 0 1 1 1
0 0 0 1 1 1

Original Sharpening filter

- Accentuates differences with local
average

Source: D. Lowe
Sharpening

Source: D. Lowe
Other filters

1 0 -1
2 0 -2
1 0 -1
Sobel

Vertical Edge
(absolute value)
Other filters

1 2 1
0 0 0
-1 -2 -1
Sobel

Horizontal Edge
(absolute value)
How could we synthesize motion blur?

theta = 30; len = 20;

fil = imrotate(ones(1, len), theta, 'bilinear');
fil = fil / sum(fil(:));
figure(2), imshow(imfilter(im, fil));
Filtering vs. Convolution
g=filter f=image
• 2d filtering
– h=filter2(g,f); or
h=imfilter(f,g);
h[ m, n]   g[ k , l ] f [ m  k , n  l ]
k ,l

• 2d convolution
– h=conv2(g,f);
h[ m, n]   g[ k , l ] f [ m  k , n  l ]
k ,l
Key properties of linear filters

Linearity:
filter(f1 + f2) = filter(f1) + filter(f2)

Shift invariance: same behavior regardless of

pixel location
filter(shift(f)) = shift(filter(f))

Any linear, shift-invariant operator can be

represented as a convolution
Source: S. Lazebnik
 More properties
• Commutative: a * b = b * a
– Conceptually no difference between filter and signal
– But particular filtering implementations might break this equality

• Associative: a * (b * c) = (a * b) * c
– Often apply several filters one after another: (((a * b1) * b2) * b3)
– This is equivalent to applying one filter: a * (b1 * b2 * b3)

• Distributes over addition: a * (b + c) = (a * b) + (a * c)

• Scalars factor out: ka * b = a * kb = k (a * b)

• Identity: unit impulse e = [0, 0, 1, 0, 0],

a*e=a
Source: S. Lazebnik
 Important filter: Gaussian

• Weight contributions of neighboring pixels by nearness

0.003 0.013 0.022 0.013 0.003

0.013 0.059 0.097 0.059 0.013
0.022 0.097 0.159 0.097 0.022
0.013 0.059 0.097 0.059 0.013
0.003 0.013 0.022 0.013 0.003

5 x 5,  = 1

Slide credit: Christopher Rasmussen

Smoothing with Gaussian filter
Smoothing with box filter
Gaussian filters
• Remove “high-frequency” components from the
image (low-pass filter)
– Images become more smooth
• Convolution with self is another Gaussian
– So can smooth with small-width kernel, repeat, and
get same result as larger-width kernel would have
– Convolving two times with Gaussian kernel of width σ
is same as convolving once with kernel of width σ√2
• Separable kernel
– Factors into product of two 1D Gaussians

Source: K. Grauman
Separability of the Gaussian filter

Source: D. Lowe
Separability example

2D convolution
(center location only)

The filter factors

into a product of 1D
filters:

Perform convolution =
along rows: *

Followed by convolution =
along the remaining column: *

Source: K. Grauman
Separability
• Why is separability useful in practice?
Some practical matters
Practical matters
How big should the filter be?
• Values at edges should be near zero
• Rule of thumb for Gaussian: set filter half-width to
about 3 σ
Practical matters
• What about near the edge?
– the filter window falls off the edge of the image
– need to extrapolate
– methods:
• clip filter (black)
• wrap around
• copy edge
• reflect across edge

Source: S. Marschner
 Q?
Practical matters

– methods (MATLAB):
• clip filter (black): imfilter(f, g, 0)
• wrap around: imfilter(f, g, ‘circular’)
• copy edge: imfilter(f, g, ‘replicate’)
• reflect across edge: imfilter(f, g, ‘symmetric’)

Source: S. Marschner
 Practical matters
• What is the size of the output?
• MATLAB: filter2(g, f, shape)
– shape = ‘full’: output size is sum of sizes of f and g
– shape = ‘same’: output size is same as f
– shape = ‘valid’: output size is difference of sizes of f and g
full same valid
g g
g g
g g

f f f

g g
g g
g g

Source: S. Lazebnik
Median filters

• A Median Filter operates over a window by

selecting the median intensity in the window.
• What advantage does a median filter have over
a mean filter?
• Is a median filter a kind of convolution?

© 2006 Steve Marschner • 45 Slide by Steve Seitz

 Comparison: salt and pepper noise

© 2006 Steve Marschner • 46 Slide by Steve Seitz

Project 1: Hybrid Images
Gaussian Filter! A. Oliva, A. Torralba, P.G. Schyns,
“Hybrid Images,” SIGGRAPH 2006

Laplacian Filter!

unit impulse Gaussian Laplacian of Gaussian

 Take-home messages

• Linear filtering is sum of dot

product at each position
1 1 1

1 1 1

– Can smooth, sharpen, translate 1 1 1

(among many other uses)

• Be aware of details for filter size,

extrapolation, cropping
Practice questions
1. Write down a 3x3 filter that returns a positive
value if the average value of the 4-adjacent
neighbors is less than the center and a
negative value otherwise

2. Write down a filter that will compute the

gradient in the x-direction:
gradx(y,x) = im(y,x+1)-im(y,x) for each x, y
 Practice questions Filtering Operator

a) _ = D * B
3. Fill in the blanks: b) A = _ * _
A

c) F = D * _
d) _ = D * D

B
E
G
F C

H I D
Next class: Thinking in Frequency