Linear Filters

Hybrid Images, Oliva et al., http://olivalab.mit.edu/hybridimage.htm

 Image filtering

Hybrid Images, Oliva et al., http://olivalab.mit.edu/hybridimage.htm

Announcements
• Assignment 1 will be uploaded this weekend you need to
submit it on teams by next Saturday 12:00 am.
Image transformations
• As with any function, we can apply operators to an image

g (x,y) = f (x,y) + 20 g (x,y) = f (-x,y)

• Today we’ll talk about a special kind of operator, convolution

(linear filtering)
Filters
• Filtering
– Form a new image whose pixel values are a combination of the original
pixel values
• Why?
– To get useful information from images
• E.g., extract edges or contours (to understand shape)
– To enhance the image
• E.g., to remove noise
• E.g., to sharpen and “enhance image”
– A key operator in Convolutional Neural Networks
Question: Noise reduction
• Given a camera and a still scene, how can you reduce noise?

Source: S. Seitz
Question: Noise reduction
• Given a camera and a still scene, how can you reduce noise?

Take lots of images and average them!

What’s the next best thing?
Source: S. Seitz
Image filtering
• Modify the pixels in an image based on some function of a
local neighborhood of each pixel

10 5 3 Some function
4 5 1 7
1 1 7

Local image data Modified image data

Source: L. Zhang
Linear filtering
• One simple version of filtering: linear filtering (cross-correlation,
convolution)
– Replace each pixel by a linear combination (a weighted sum) of its neighbors
• The prescription for the linear combination is called the “kernel” (or
“mask”, “filter”)

10 5 3 0 0 0
4 6 1 0 0.5 0 8
1 1 8 0 1 0.5

Local image data kernel Modified image data

Source: L. Zhang
Cross-correlation
Let be the image, be the kernel (of size 2k+1 x
2k+1), and be the output image

This is called a cross-correlation operation:

• Can think of as a “dot product” between local

neighborhood and kernel for each pixel
Convolution
• Same as cross-correlation, except that the kernel is “flipped”
(horizontally and vertically)

This is called a convolution operation:

• Convolution is commutative and associative

Convolution

Adapted from F. Durand

Mean filtering

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10
0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20

=
0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30

*
1 1 1
1 1 1 0 0 0 90 90 90 90 90 0 0 0 30 50 80 80 90 60 30
0 0 0 90 0 90 90 90 0 0 0 30 50 80 80 90 60 30
1 1 1
0 0 0 90 90 90 90 90 0 0 0 20 30 50 50 60 40 20
0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10

0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Linear filters: examples

0 0 0

* 0
0
1
0
0
0

Original

Source: D. Lowe
What image operation does filtering with this kernel
perform?
([0 0 0; 0 1 0; 0 0 0])

ⓘ Start presenting to display the poll results on this slide.

Linear filters: examples

0 0 0

=
* 0
0
1
0
0
0

Original Identical image

Source: D. Lowe
Linear filters: examples

0 0 0

* 1
0
0
0
0
0

Original

Source: D. Lowe
What image operation does filtering with this
kernel perform? ([0 0 0; 1 0 0; 0 0 0])

ⓘ Start presenting to display the poll results on this slide.

Linear filters: examples

0 0 0

=
* 1
0
0
0
0
0

Original Shifted left by 1 pixel

Source: D. Lowe
Linear filters: examples

1 1 1

=
* 1
1
1
1
1
1

Original Blur (with a mean filter)

Source: D. Lowe
Linear filters: examples

-
0 0 0 1 1 1

* 0
0
2
0
0
0
1
1
1
1
1
1
=
Sharpening filter
Original
(accentuates edges)

Source: D. Lowe
Sharpening

Source: D. Lowe
Smoothing with box filter revisited

Source: D. Forsyth
Gaussian kernel

Source: C. Rasmussen
Gaussian filters

= 1 pixel = 5 pixels = 10 pixels = 30 pixels

Mean vs. Gaussian filtering
Gaussian filter
• Removes “high-frequency” components from the image
(low-pass filter)
• Convolution with self is another Gaussian

* =
– Convolving twice with Gaussian kernel of width
= convolving once with kernel of width

Source: K. Grauman
Sharpening revisited
• What does blurring take away?

– =
original smoothed (5x5) detail
(This “detail extraction” operation is also
Let’s add it back: called a high-pass filter)

+α =
original detail sharpened
Photo credit: https://www.flickr.com/photos/geezaweezer/16089096376/
Sharpen filter
+
blurred
image unit impulse
image
(identity kernel
with single 1 in
center, zeros
elsewhere)

scaled impulse Gaussian Sharpen filter

Sharpen filter

unfiltered

filtered
“Optical” convolution
Camera shake

= *
Source: Fergus, et al. “Removing Camera Shake from a Single Photograph”, SIGGRAPH 2006

Bokeh: Blur in out-of-focus regions of an image.

Source: https://www.diyphotography.net/diy_create_your_own_bokeh/
Filters: Thresholding
Linear filters
• Can thresholding be implemented with a linear filter?
Can thresholding be implemented with a
linear filter?

ⓘ Start presenting to display the poll results on this slide.

Questions?