09 Image Filtering Edge Detection 09 PDF

Download as pdf or txt
Download as pdf or txt
You are on page 1of 15

What is image filtering?

Image Filtering & Edge Detection


Reading: Chapter 7 and 8, F&P

Modify the pixels in an image based on some function of a local neighborhood of the pixels.

Some function

Linear Functions

Simplest: linear filtering.

Replace each pixel by a linear combination of its neighbors.

The prescription for the linear combination is called the convolution kernel.

Convolution

Key properties

Let I be the image and g be the kernel. The output of convolving I with g is denoted I g

Linearity:

filter(I1 + I2 ) = filter(I1) + filter(I2) same behavior regardless of pixel location filter(shift(I)) = shift(filter(I)) Any linear shift-invariant operator can be represented as a convolution

f [ m, n] = I g = I[ m k , n l ] g[ k , l ]
k ,l

Shift invariance:

Theoretical result:

Properties in more detail

Commutative: a * b = b * a

Conceptually no difference between filter and signal Often apply several filters one after another: (((a * b1) * b2) * b3) This is equivalent to applying one filter: a * (b1 * b2 * b3)

Associative: a * (b * c) = (a * b) * c

Distributes over addition: a * (b + c) = (a * b) + (a * c) Scalars factor out: ka * b = a * kb = k (a * b) Identity: unit impulse e = [, 0, 0, 1, 0, 0, ], a*e=a

Yucky details

What is the size of the output? MATLAB: filter2(g, I, shape)


shape = full: output size is sum of sizes of I and g shape = same: output size is same as I shape = valid:output size is of difference sizes for I & g
full g same g I g g g g g I g valid g

Implementation details

Implementation details

What about near the edge?


What about near the edge?


the filter window falls off the edge of the image need to extrapolate methods:

the filter window falls off the edge of the image need to extrapolate methods (MATLAB):

clip filter (black) wrap around copy edge reflect across edge

clip filter (black): wrap around: copy edge: reflect across edge:

imfilter(f, g, 0) imfilter(f, g, circular) imfilter(f, g, replicate) imfilter(f, g, symmetric)

Source: S. Marschner

Source: S. Marschner

Linear filtering (warm-up slide)

Linear filtering (warm-up slide)

coefficient

1.0

?
original

coefficient

1.0

original

Pixel offset

Pixel offset

0 0 0 0 1 0 0 0 0

Filtered (no change)

Linear filtering

Shift

coefficient

1.0

coefficient

Pixel offset original original

1.0

Pixel offset

0 0 0 0 0 1 0 0 0

shifted

Linear filtering

Blurring

coefficient

0.3
0

?
original
Box filter:

coefficient

0.3
0

Pixel offset original

Pixel offset

1 1 1 1 1 1 1 1 1

Blurred (filter applied in both dimensions).

Blur examples
coefficient 8 impulse 2.4 0.3
0

Blur examples
8 impulse coefficient 2.4 0.3
0

original

Pixel offset

filtered 8 edge

original

Pixel offset coefficient 8 0.3


0

filtered

4 filtered

original

Pixel offset

Smoothing with box filter revisited


Smoothing with box filter revisited


Smoothing with an average actually doesnt compare at all well with a defocused lens Most obvious difference is that a single point of light viewed in a defocused lens looks like a fuzzy blob; but the averaging process would give a little square

Smoothing with an average actually doesnt compare at all well with a defocused lens Most obvious difference is that a single point of light viewed in a defocused lens looks like a fuzzy blob; but the averaging process would give a little square Better idea: to eliminate edge effects, weight contribution of neighborhood pixels according to their closeness to the center, like so:

Source: D. Forsyth

fuzzy blob

Gaussian Kernel

Choosing kernel width

Gaussian filters have infinite support, but discrete filters use finite kernels

0.003 0.013 0.022 0.013 0.003

0.013 0.059 0.097 0.059 0.013

0.022 0.097 0.159 0.097 0.022

0.013 0.059 0.097 0.059 0.013

0.003 0.013 0.022 0.013 0.003

5 x 5, = 1

Constant factor at front makes volume sum to 1 (can be ignored, as we should re-normalize weights to sum to 1 in any case)

Source: C. Rasmussen

Source: K. Grauman

Gaussian filtering

A Gaussian kernel gives less weight to pixels further from the center of the window
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 0 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Example: Smoothing with a Gaussian

1 2 1

2 4 2

1 2 1

This kernel is an approximation of a Gaussian function:

Mean vs. Gaussian filtering

Separability of the Gaussian filter

Source: D. Lowe

Separability example
2D convolution (center location only)

Gaussian filters

Remove high-frequency components from the image (low-pass filter) Convolution with self is another Gaussian

The filter factors into a product of 1D filters:

So can smooth with small-width kernel, repeat, and get same result as larger-width kernel would have Convolving two times with Gaussian kernel of width is same as convolving once with kernel of width sqrt(2) Factors into product of two 1D Gaussians Useful: can convolve all rows, then all columns How does this change the computational complexity?

Separable kernel

Perform convolution along rows:

Followed by convolution along the remaining column:

=
Source: K. Grauman

Linear vs. quadratic in mask size

Source: K. Grauman

Review: Linear filtering

Noise

What are the defining mathematical properties of a convolution? What is the difference between blurring with a box filter and blurring with a Gaussian? What happens when we convolve a Gaussian with another Gaussian? What is separability? How does separability affect computational complexity?

Original Salt and pepper noise

Salt and pepper noise: contains random occurrences of black and white pixels Impulse noise: contains random occurrences of white pixels Gaussian noise: variations in intensity drawn from a Gaussian normal distribution
Source: S. Seitz

Impulse noise

Gaussian noise

Gaussian noise

Reducing Gaussian noise

Mathematical model: sum of many independent factors Good for small standard deviations Assumption: independent, zero-mean noise

Source: K. Grauman

Smoothing with larger standard deviations suppresses noise, but also blurs the image

Reducing salt-and-pepper noise


3x3 5x5 7x7

Alternative idea: Median filtering

A median filter operates over a window by selecting the median intensity in the window

Whats wrong with the results?

Is median filtering linear?

No. Not a convolution

Source: K. Grauman

Median filter

Median filter
Salt-and-pepper noise Median filtered

What advantage does median filtering have over Gaussian filtering?

Robustness to outliers

Source: K. Grauman

MATLAB: medfilt2(image, [h w])


Source: K. Grauman

Median vs. Gaussian filtering


3x3 5x5 7x7

Linear filtering (warm-up slide)

Gaussian

2.0 1.0

Median

original

Linear filtering (no change)

Linear filtering
2.0 1.0 0.33

2.0

original

Filtered (no change)

original

(Remember blurring)

Sharpening

2.0 coefficient

0.3
0 0

0.33
0

Pixel offset original Blurred (filter applied in both dimensions). original

Sharpened original

Sharpening

Sharpening example

1.7 8 coefficient

11.2 8

original

-0.3

-0.25 Sharpened (differences are accentuated; constant areas are left untouched).

Sharpening

Sharpening

0 0 0 0 2 0 0 0 0
Original

1 1 1 1 1 1 1 1 1

?
Original

0 0 0 0 2 0 0 0 0

1 1 1 1 1 1 1 1 1

(Note that filter sums to 1)

Sharpening filter - Accentuates differences with local average


Source: D. Lowe

Source: D. Lowe

Sharpening

Unsharp mask filter


I + ( I I g ) = ( 1 + ) I I g = I (( 1 + ) e g )
image blurred image unit impulse (identity)

before

after

unit impulse

Gaussian

Laplacian of Gaussian

Sharpening Revisited

Edge detection

What does blurring take away?

original smoothed (5x5)

=
detail

Goal: Identify sudden changes (discontinuities) in an image

Lets add it back:

Intuitively, most semantic and shape information from the image can be encoded in the edges More compact than pixels

+
original detail

=
sharpened

Ideal: artists line drawing (but artist is also using object-level knowledge)

Source: D. Lowe

Origin of Edges
surface normal discontinuity depth discontinuity surface color discontinuity illumination discontinuity

Characterizing edges

An edge is a place of rapid change in the image intensity function


image intensity function (along horizontal scanline) first derivative

Edges are caused by a variety of factors


edges correspond to extrema of derivative
Source: Steve Seitz

Image gradient

Differentiation and convolution

The gradient of an image:


The gradient points in the direction of most rapid change in intensity

Recall, for 2D function, f(x,y):


f f (x + , y) f (x, y ) = lim x 0

We could approximate this as


f f (xn +1 , y ) f (xn , y) x x

The gradient direction is given by:

how does this relate to the direction of the edge? perpendicular

This is linear and shift invariant, so must be the result of a convolution.

(which is obviously a convolution)


-1 1

The edge strength is given by the gradient magnitude


Source: D. Forsyth, D. Lowe

Finite differences: example

Finite difference filters

Other approximations of derivative filters exist:

Which one is the gradient in the x-direction (resp. y-direction)?

Source: K. Grauman

Effects of noise

Effects of noise

Consider a single row or column of the image

Finite difference filters respond strongly to noise

Plotting intensity as a function of position gives a signal

Image noise results in pixels that look very different from their neighbors Generally, the larger the noise the stronger the response

What is to be done?

How to compute a derivative?

Where is the edge?

Source: D. Forsyth

Effects of noise

Solution: smooth first

Finite difference filters respond strongly to noise

Image noise results in pixels that look very different from their neighbors Generally, the larger the noise the stronger the response Smoothing the image should help, by forcing pixels different to their neighbors (=noise pixels?) to look more like neighbors

What is to be done?

Source: D. Forsyth

Where is the edge?

Look for peaks in

Derivative theorem of convolution

Derivative of Gaussian filter

Differentiation is convolution, and convolution is d d associative: ( f g) = f g dx dx This saves us one operation:


f

* [1 -1] =

d g dx
f d g dx
Source: S. Seitz

Derivative of Gaussian filter

Summary: Filter mask properties

Filters act as templates

Highest response for regions that look the most like the filter Dot product as correlation Values positive Sum to 1 constant regions are unchanged Amount of smoothing proportional to mask size Opposite signs used to get high response in regions of high contrast Sum to 0 no response in constant regions High absolute value at points of high contrast
Source: K. Grauman

Smoothing masks

x-direction

y-direction

Derivative masks

Which one finds horizontal/vertical edges?

Tradeoff between smoothing and localization

Implementation issues

1 pixel

3 pixels

7 pixels

Smoothed derivative removes noise, but blurs edge. Also finds edges at different scales.
Source: D. Forsyth

The gradient magnitude is large along a thick trail or ridge, so how do we identify the actual edge points? How do we link the edge points to form curves?
Source: D. Forsyth

Laplacian of Gaussian

2D edge detection filters


Laplacian of Gaussian

Consider

Gaussian Laplacian of Gaussian operator

derivative of Gaussian

is the Laplacian operator:

Where is the edge?

Zero-crossings of bottom graph

MATLAB demo
g = fspecial('gaussian',15,2); imagesc(g) surfl(g) gclown = conv2(clown,g,'same'); imagesc(conv2(clown,[-1 1],'same')); imagesc(conv2(gclown,[-1 1],'same')); dx = conv2(g,[-1 1],'same'); imagesc(conv2(clown,dx,'same')); lg = fspecial('log',15,2); lclown = conv2(clown,lg,'same'); imagesc(lclown) imagesc(clown + .2*lclown)

Edge finding

We wish to mark points along the curve where the magnitude is biggest. We can do this by looking for a maximum along a slice normal to the curve (non-maximum suppression). These points should form a curve. There are then two algorithmic issues: at which point is the maximum, and where is the next one?

Source: D. Forsyth

Non-maximum suppression
At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values.

Predicting the next edge point


Assume the marked point is an edge point. Then we construct the tangent to the edge curve (which is normal to the gradient at that point) and use this to predict the next points (here either r or s).

Source: D. Forsyth

Source: D. Forsyth

Designing an edge detector

Canny edge detector

Criteria for an optimal edge detector:

Good detection: the optimal detector must minimize the probability of false positives (detecting spurious edges caused by noise), as well as that of false negatives (missing real edges) Good localization: the edges detected must be as close as possible to the true edges Single response: the detector must return one point only for each true edge point; that is, minimize the number of local maxima around the true edge

This is probably the most widely used edge detector in computer vision Theoretical model: step-edges corrupted by additive Gaussian noise Canny has shown that the first derivative of the Gaussian closely approximates the operator that optimizes the product of signal-to-noise ratio and localization

J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679-714, 1986.
Source: L. Fei-Fei Source: L. Fei-Fei

Canny edge detector


1. 2. 3.

The Canny edge detector

Filter image with derivative of Gaussian Find magnitude and orientation of gradient Non-maximum suppression:

Thin multi-pixel wide ridges down to single pixel width Define two thresholds: low and high Use the high threshold to start edge curves and the low threshold to continue them

4.

Linking and thresholding (hysteresis):


MATLAB: edge(image, canny)


Source: D. Lowe, L. Fei-Fei

original image (Lena)

The Canny edge detector

The Canny edge detector

norm of the gradient

thresholding

The Canny edge detector

Hysteresis thresholding

original image

thinning

(non-maximum suppression)

high threshold (strong edges)

low threshold (weak edges)

hysteresis threshold
Source: L. Fei-Fei

Effect of (Gaussian kernel spread/size)

Edge detection is just the beginning


image human segmentation gradient magnitude

original

Canny with

Canny with

The choice of depends on desired behavior


large detects large scale edges small detects fine features

Berkeley segmentation database:


http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbe nch/

Source: S. Seitz

You might also like