Computer Vision

CP – 7003

Lecture # 02

Image Processing Review

Spring 2013
 Today
► Point vs Neighborhood Operations
► Filtering
► Morphology
► Edge Detection

 Readings
 Book:  Digital Image Processing, Gonzalez & Woods
► Intro: Ch. 3, 9, 10

Computer Vision Lecture # 2 2

 What is an image?

Digital Camera

We’ll focus on these in this class

(More on this process later) The Eye 3

Computer Vision Lecture # 2 Source: A. 3Efros

 What is an image?
►A grid (matrix) of intensity values
255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 20 0 255 255 255 255 255 255 255

255 255 255 75 75 75 255 255 255 255 255 255

=
255 255 75 95 95 75 255 255 255 255 255 255

255 255 96 127 145 175 255 255 255 255 255 255

255 255 127 145 175 175 175 255 255 255 255 255

255 255 127 145 200 200 175 175 95 255 255 255

255 255 127 145 200 200 175 175 95 47 255 255

255 255 127 145 145 175 127 127 95 47 255 255

255 255 74 127 127 127 95 95 95 47 255 255

255 255 255 74 74 74 74 74 74 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255

255 255 255 255 255 255 255 255 255 255 255 255

(common to use one byte per value: 0 = black, 255 =

Computerwhite)
Vision Lecture # 2 4
 What is an image?
► We can think of a (grayscale) image as a
function, f, from R2 to R (or a 2D signal):
 f (x,y) gives the intensity at position (x,y)
f (x, y)

snoop
3D view

 A digital image is a discrete (sampled, quantized)

Computer Visionversion of this function
Lecture # 2 5
 Image transformations
► As with any function, we can apply
operators to an image

g (x,y) = f (x,y) + 20 g (x,y) = f (-x,y)

Computer Vision Lecture # 2 6

 Point Operations

Computer Vision Lecture # 2 7

 Point Processing
Nonlinear
Nonlinear Lower
Lower
Original
Original Darken
Darken Lower
Lower Contrast
Contrast Contrast
Contrast

Nonlinear
Nonlinear Raise
Raise
Invert
Invert Lighten
Lighten Raise
Raise Contrast
Contrast Contrast
Contrast

Computer Vision Lecture # 2 8

 Point Processing
Nonlinear
Nonlinear Lower
Lower
Original
Original Darken
Darken Lower
Lower Contrast
Contrast Contrast
Contrast

x x - 128 x / 2 ((x / 255.0) ^ 0.33) * 255.0

Nonlinear
Nonlinear Raise
Raise
Invert
Invert Lighten
Lighten Raise
Raise Contrast
Contrast Contrast
Contrast

255 - x x + 128 x * 2 ((x / 255.0) ^2) * 255.0

Computer Vision Lecture # 2 9

 Neighborhood Operations

Computer Vision Lecture # 2 10

 Neighborhood operations

Image Edge detection Blur

Computer Vision Lecture # 2 11

 Question: Noise reduction
► Given a camera and a still scene, how
can you reduce noise?

Take lots of images and average them!

Computer Vision Lecture # 2 What’s the next best thing? 12
 Image filtering
► Modify the pixels in an image based on some
function of a local neighborhood of each pixel

10 5 3
4 5 1 7
1 1 7

Local image data Modified image data

Computer Vision Lecture # 2 13

Source: L. Zhang
 Linear filtering
► One simple version: linear filtering
(cross-correlation, convolution)
 Replace each pixel by a linear combination of its
neighbors
► The prescription for the linear combination is
called the “kernel” (or “mask”, “filter”)
10 5 3 0 0 0
4 6 1 0 0.5 0 8
1 1 8 0 1 0.5

Local image data kernel Modified image data

Computer Vision Lecture # 2 14

Source: L. Zhang
 Cross-correlation
Let be the image, be the kernel (of
size 2k+1 x 2k+1), and be the output
image

This is called a cross-correlation operation:

Computer Vision Lecture # 2 15

 Convolution
► Same as cross-correlation, except that the
kernel is “flipped” (horizontally and
vertically)

This is called a convolution operation:

► Convolution is commutative and

associative
Computer Vision Lecture # 2 16
 Convolution

Computer Vision Lecture # 2 17

Adapted from F. Durand
 Mean filtering

0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 20 10
0 0 0 90 90 90 90 90 0 0 0 20 40 60 60 60 40 20
0 0 0 90 90 90 90 90 0 0 0 30 60 90 90 90 60 30

*
1 1 1
1
1
1
1
1
1
0
0
0
0
0
0
0
0
0
90
90
90
90
0
90
90
90
90
90
90
90
90
90
90
0
0
0
0
0
0
= 0
0
0
30
30
20
50
50
30
80
80
50
80
80
50
90
90
60
60
60
40
30
30
20

0 0 0 0 0 0 0 0 0 0 10 20 30 30 30 30 20 10

0 0 90 0 0 0 0 0 0 0 10 10 10 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

Computer Vision Lecture # 2 18

 Linear filters: examples

0 0 0

* 0
0
1
0
0
0
=
Original Identical image

Computer Vision Lecture # 2 19

Source: D. Lowe
 Linear filters: examples

0 0 0

* 1
0
0
0
0
0
=
Original Shifted left
By 1 pixel

Computer Vision Lecture # 2 20

Source: D. Lowe
 Linear filters: examples

1 1 1

* 1
1
1
1
1
1
=
Original Blur (with a mean filter)

Computer Vision Lecture # 2 21

Source: D. Lowe
 Linear filters: examples

-
0 0 0 1 1 1

* 0
0
2
0
0
0
1
1
1
1
1
1
=
Sharpening filter
Original (accentuates edges)

Computer Vision Lecture # 2 22

Source: D. Lowe
 Sharpening

Computer Vision Lecture # 2 23

Source: D. Lowe
 Smoothing with box filter revisited

Computer Vision Lecture # 2 24

Source: D. Forsyth
 Gaussian Kernel

Computer Vision Lecture # 2 Source: C. Rasmussen

25
 Gaussian filters

= 1 pixel = 5 pixels = 10 pixels = 30 pixels

Computer Vision Lecture # 2 26

 Gaussian filter
• Removes “high-frequency” components
from the image (low-pass filter)
• Convolution with self is another Gaussian

* =
 Convolving two times with Gaussian kernel of
width = convolving once with kernel of
width
Computer Vision Lecture # 2 27
Source: K. Grauman
 Sharpening

Computer Vision Lecture # 2 28

Source: D. Lowe
 Sharpening revisited
► What does blurring take away?

– =
original smoothed (5x5) detail

Let’s add it back:

+α =
original detail sharpened
Computer Vision Lecture # 2 29
Source: S. Lazebnik
 Sharpen filter
blurred
image unit impulse
image
(identity)

scaled impulse Gaussian

Computer Vision Lecture # 2 Laplacian of Gaussian30
 Sharpen filter

unfiltered

filtered

Computer Vision Lecture # 2 31

 Morphological Operations
► Erosion
► Dilation
► Opening
► Closing
► Thinning
► Thickening
► Connected Component Labeling
► Region Filling
► Convex Hull

Computer Vision Lecture # 2 32

 Edge detection

• Convert a 2D image into a set of curves

– Extracts salient features of the scene
– More compact than pixels
Computer Vision Lecture # 2 33
 Origin of Edges
surface normal discontinuity

depth discontinuity

surface color discontinuity

illumination discontinuity

► Edges are caused by a variety of factors

Computer Vision Lecture # 2 34

 Characterizing edges
• An edge is a place of rapid change in the
image intensity function
intensity function
image (along horizontal scanline) first derivative

edges correspond to
Computer Vision
Source: L. Lazebnik Lecture # 2 extrema of derivative35
 Image derivatives
• How can we differentiate a digital image
F[x,y]?
– Option 1: reconstruct a continuous image, f, then
compute the derivative
– Option 2: take discrete derivative (finite
difference)
How would you implement this as a linear filter?

: 1 -1 : -1
1

Computer Vision Lecture # 2 Source: S.36

Seitz
 Image gradient
► The gradient of an image:
The gradient points in the direction of most rapid increase in intensity

The edge strength is given by the gradient magnitude:

The gradient direction is given by:

• how does this relate to the direction of the edge?

Computer Vision Lecture # 2 Source: Steve37
Seitz
 Image gradient

Computer Vision Lecture # 2 39

 Effects of noise

Noisy input image

Where is the edge?

Computer Vision Lecture # 2 Source: S. Seitz 40
 Solution: smooth first
f

f*h

Computer Vision To find edges, look for peaks in

Lecture # 2 41

Source: S. Seitz
 Sobel operator: example

Computer Vision Lecture # 2 Source: Wikipedia

42
 Example

► original image (Lena)

Computer Vision Lecture # 2 43
 Finding edges

gradient magnitude
Computer Vision Lecture # 2 44
 Finding edges

where is the edge?

thresholding
Computer Vision Lecture # 2 45
 Non-maximum supression

► Check if pixel is local maximum along gradient direction

 requires interpolating pixels p and r
Computer Vision Lecture # 2 46
 Finding edges

thresholding
Computer Vision Lecture # 2 47
 Finding edges

thinning
Computer Vision
(non-maximum suppression)
Lecture # 2 48
 Canny edge detector
MATLAB: edge(image,‘canny’)

1. Filter image with derivative of Gaussian

2. Find magnitude and orientation of

gradient

3. Non-maximum suppression

4. Linking and thresholding (hysteresis):

 Define two thresholds: low and high
 Use the high threshold to start edge curves
and the low threshold to continue them

Computer Vision Lecture # 2 49

Source: D. Lowe, L. Fei-Fei
 Canny edge detector
► Still one of the most widely used edge
detectors in computer vision
J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern
Analysis and Machine Intelligence, 8:679-714, 1986.

► Depends on several parameters:

: width of the Gaussian blur
high threshold
low threshold

Computer Vision Lecture # 2 50

 Canny edge detector

original Canny with Canny with

• The choice of depends on desired behavior

– large detects “large-scale” edges
– small detects fine edges

Computer Vision Lecture # 2 Source: S. Seitz51

 Images as vectors
► Very important idea!

2D image Scanline (1D signal)

(A 2D, n x m image can be represented by a vector

of length nm formed by concatenating the rows) Vector
Computer Vision Lecture # 2 52
 Working With Matlab
► Tutorial (thanks to David Kriegman and Serge Belongie)
 http://cseweb.ucsd.edu/~sjb/classes/matlab/matlab.intro.html
► More tutorials: thanks to Martial Hebert:
 Operations
► http://www.cs.cmu.edu/afs/cs/academic/class/15385-
12/www/lec_slides/matlab_ops_tutorial.m
 Programming
► http://www.cs.cmu.edu/afs/cs/academic/class/15385-
s12/www/lec_slides/matlab_prog_tutorial.m
 Working with Images
► http://www.cs.cmu.edu/afs/cs/academic/class/15385-
s12/www/lec_slides/matlab_image_tutorial.m

Computer Vision Lecture # 2 53

 Acknowledgements
►I have planned and made this course
material from the courses taught by
following computer vision experts.
 Steve Seitz, Washington University
 Richard Szeliski, Microsoft Research
 Noah Snavely, Cornell University
 Sirinavasa Narasimhan, Carnegie Mellon
University
 Antonio Torralba, MIT

Computer Vision Lecture # 2 54