Lecture 1-Image filtering

Reading
• Szeliski, Chapter 3.1-3.2
What is an image?
 What is an image?

Digital Camera

We’ll focus on these in this course

Also image formation The Eye

Source: A. Efros
 What is an image?
• A grid (matrix) of intensity values
255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 20 0 255 255 255 255 255 255 255

255 255 255 75 75 75 255 255 255 255 255 255

=
255 255 75 95 95 75 255 255 255 255 255 255

255 255 96 127 145 175 255 255 255 255 255 255

255 255 127 145 175 175 175 255 255 255 255 255

255 255 127 145 200 200 175 175 95 255 255 255

255 255 127 145 200 200 175 175 95 47 255 255

255 255 127 145 145 175 127 127 95 47 255 255

255 255 74 127 127 127 95 95 95 47 255 255

255 255 255 74 74 74 74 74 74 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255

(common to use one byte per value: 0 = black, 255 = white)

 What is an image?
• We can think of a (grayscale) image as a
function, f, from R2 to R:
– f (x,y) gives the intensity at position (x,y)
f (x, y)

– A digital image is a discrete (sampled, quantized)

version of this function
 Image transformations
• As with any function, we can apply operators
to an image

g (x,y) = f (x,y) + 20

• We’ll talk about a special kind of operator,

convolution (linear filtering)
 Filters
• Filtering
– Form a new image whose pixels are a combination
of the original pixels
• Why?
– To get useful information from images
• E.g., extract edges or contours (to understand shape)
– To enhance the image
• E.g., to remove noise
• E.g., to sharpen or to “enhance image”
 Image Processing problems
• Image Restoration
– denoising
– deblurring
• Image Compression
– JPEG, JPEG2000, MPEG..
• Computing Field Properties
– optical flow
– disparity
• Locating Structural Features
– corners
– edges
 Question: Noise reduction
• Given a camera and a still scene, how can
you reduce noise?

Take lots of images and average them!

What’s the next best thing?
Source: S. Seitz
 Image filtering
• Modify the pixels in an image based on some
function of a local neighborhood of each pixel

10 5 3 Some function
4 5 1 7
1 1 7

Local image data Modified image data

Source: L. Zhang
 Linear filtering
• One simple version of filtering: linear filtering
(cross-correlation, convolution)
– Replace each pixel by a linear combination (a weighted
sum) of its neighbors
• The prescription for the linear combination is called
the “kernel” (or “mask”, “filter”)
10 5 3 0 0 0
4 6 1 0 0.5 0 8
1 1 8 0 1 0.5
Local image data kernel Modified image data

Source: L. Zhang
 Cross-correlation
Let be the image, be the kernel (of
size 2k+1 x 2k+1), and be the output
image

This is called a cross-correlation operation:

• Can think of as a “dot product” between

local neighborhood and kernel for each pixel
 Convolution
• Same as cross-correlation, except that the
kernel is “flipped” (horizontally and vertically)

This is called a convolution operation:

 Mean filtering

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10
0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20

1 1 1 0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30

1
1
1
1
1
1
*
0
0
0
0
0
0
0
0
0
90
90
90
90
0
90
90
90
90
90
90
90
90
90
90
0
0
0
0
0
0
= 0
0
0
30
30
20
50
50
30
80
80
50
80
80
50
90
90
60
60
60
40
30
30
20
0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10
0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
Mean filtering/Moving average
 Linear filters: examples

0 0 0

=
* 0
0
1
0
0
0

Original Identical image

Source: D. Lowe
 Linear filters: examples

0 0 0

=
* 1
0
0
0
0
0

Original Shifted left

By 1 pixel

Source: D. Lowe
 Linear filters: examples

1 1 1

=
* 1
1
1
1
1
1

Original Blur (with a mean filter)

Source: D. Lowe
 Linear filters: examples

-
0 0 0 1 1 1

=
* 0
0
2
0
0
0
1
1
1
1
1
1

Sharpening filter
Original

Source: D. Lowe
Sharpening

Source: D. Lowe
 Gaussian Kernel

0.003 0.013 0.022 0.013 0.003

0.013 0.059 0.097 0.059 0.013
0.022 0.097 0.159 0.097 0.022
0.013 0.059 0.097 0.059 0.013
0.003 0.013 0.022 0.013 0.003

5 x 5,  = 1

• Constant factor at front makes volume sum to 1 (can be ignored, as

we should re-normalize weights to sum to 1 in any case)
Source: C. Rasmussen
Gaussian Kernel

Source: C. Rasmussen
Gaussian filters

Increasing
Mean vs. Gaussian filtering

Mean filter may garble

high-frequency signal
 Gaussian filter
• Removes “high-frequency” components from
the image (low-pass filter)
• Convolution with self is another Gaussian

* =

Source: K. Grauman
 Sharpening revisited
• What does blurring take away?

– =
original smoothed (5x5) detail

Let’s add it back:

+α =
original detail sharpened
Source: S. Lazebnik
Filters: Thresholding