09 Image Filtering Edge Detection 09 PDF
09 Image Filtering Edge Detection 09 PDF
09 Image Filtering Edge Detection 09 PDF
Modify the pixels in an image based on some function of a local neighborhood of the pixels.
Some function
Linear Functions
The prescription for the linear combination is called the convolution kernel.
Convolution
Key properties
Let I be the image and g be the kernel. The output of convolving I with g is denoted I g
Linearity:
filter(I1 + I2 ) = filter(I1) + filter(I2) same behavior regardless of pixel location filter(shift(I)) = shift(filter(I)) Any linear shift-invariant operator can be represented as a convolution
f [ m, n] = I g = I[ m k , n l ] g[ k , l ]
k ,l
Shift invariance:
Theoretical result:
Commutative: a * b = b * a
Conceptually no difference between filter and signal Often apply several filters one after another: (((a * b1) * b2) * b3) This is equivalent to applying one filter: a * (b1 * b2 * b3)
Associative: a * (b * c) = (a * b) * c
Distributes over addition: a * (b + c) = (a * b) + (a * c) Scalars factor out: ka * b = a * kb = k (a * b) Identity: unit impulse e = [, 0, 0, 1, 0, 0, ], a*e=a
Yucky details
shape = full: output size is sum of sizes of I and g shape = same: output size is same as I shape = valid:output size is of difference sizes for I & g
full g same g I g g g g g I g valid g
Implementation details
Implementation details
the filter window falls off the edge of the image need to extrapolate methods:
the filter window falls off the edge of the image need to extrapolate methods (MATLAB):
clip filter (black) wrap around copy edge reflect across edge
clip filter (black): wrap around: copy edge: reflect across edge:
Source: S. Marschner
Source: S. Marschner
coefficient
1.0
?
original
coefficient
1.0
original
Pixel offset
Pixel offset
0 0 0 0 1 0 0 0 0
Linear filtering
Shift
coefficient
1.0
coefficient
1.0
Pixel offset
0 0 0 0 0 1 0 0 0
shifted
Linear filtering
Blurring
coefficient
0.3
0
?
original
Box filter:
coefficient
0.3
0
Pixel offset
1 1 1 1 1 1 1 1 1
Blur examples
coefficient 8 impulse 2.4 0.3
0
Blur examples
8 impulse coefficient 2.4 0.3
0
original
Pixel offset
filtered 8 edge
original
filtered
4 filtered
original
Pixel offset
Smoothing with an average actually doesnt compare at all well with a defocused lens Most obvious difference is that a single point of light viewed in a defocused lens looks like a fuzzy blob; but the averaging process would give a little square
Smoothing with an average actually doesnt compare at all well with a defocused lens Most obvious difference is that a single point of light viewed in a defocused lens looks like a fuzzy blob; but the averaging process would give a little square Better idea: to eliminate edge effects, weight contribution of neighborhood pixels according to their closeness to the center, like so:
Source: D. Forsyth
fuzzy blob
Gaussian Kernel
Gaussian filters have infinite support, but discrete filters use finite kernels
5 x 5, = 1
Constant factor at front makes volume sum to 1 (can be ignored, as we should re-normalize weights to sum to 1 in any case)
Source: C. Rasmussen
Source: K. Grauman
Gaussian filtering
A Gaussian kernel gives less weight to pixels further from the center of the window
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 90 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 0 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 90 90 90 90 90 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 2 1
2 4 2
1 2 1
Source: D. Lowe
Separability example
2D convolution (center location only)
Gaussian filters
Remove high-frequency components from the image (low-pass filter) Convolution with self is another Gaussian
So can smooth with small-width kernel, repeat, and get same result as larger-width kernel would have Convolving two times with Gaussian kernel of width is same as convolving once with kernel of width sqrt(2) Factors into product of two 1D Gaussians Useful: can convolve all rows, then all columns How does this change the computational complexity?
Separable kernel
=
Source: K. Grauman
Source: K. Grauman
Noise
What are the defining mathematical properties of a convolution? What is the difference between blurring with a box filter and blurring with a Gaussian? What happens when we convolve a Gaussian with another Gaussian? What is separability? How does separability affect computational complexity?
Salt and pepper noise: contains random occurrences of black and white pixels Impulse noise: contains random occurrences of white pixels Gaussian noise: variations in intensity drawn from a Gaussian normal distribution
Source: S. Seitz
Impulse noise
Gaussian noise
Gaussian noise
Mathematical model: sum of many independent factors Good for small standard deviations Assumption: independent, zero-mean noise
Source: K. Grauman
Smoothing with larger standard deviations suppresses noise, but also blurs the image
A median filter operates over a window by selecting the median intensity in the window
Source: K. Grauman
Median filter
Median filter
Salt-and-pepper noise Median filtered
Robustness to outliers
Source: K. Grauman
Gaussian
2.0 1.0
Median
original
Linear filtering
2.0 1.0 0.33
2.0
original
original
(Remember blurring)
Sharpening
2.0 coefficient
0.3
0 0
0.33
0
Sharpened original
Sharpening
Sharpening example
1.7 8 coefficient
11.2 8
original
-0.3
-0.25 Sharpened (differences are accentuated; constant areas are left untouched).
Sharpening
Sharpening
0 0 0 0 2 0 0 0 0
Original
1 1 1 1 1 1 1 1 1
?
Original
0 0 0 0 2 0 0 0 0
1 1 1 1 1 1 1 1 1
Source: D. Lowe
Sharpening
before
after
unit impulse
Gaussian
Laplacian of Gaussian
Sharpening Revisited
Edge detection
=
detail
Intuitively, most semantic and shape information from the image can be encoded in the edges More compact than pixels
+
original detail
=
sharpened
Ideal: artists line drawing (but artist is also using object-level knowledge)
Source: D. Lowe
Origin of Edges
surface normal discontinuity depth discontinuity surface color discontinuity illumination discontinuity
Characterizing edges
Image gradient
Source: K. Grauman
Effects of noise
Effects of noise
Image noise results in pixels that look very different from their neighbors Generally, the larger the noise the stronger the response
What is to be done?
Source: D. Forsyth
Effects of noise
Image noise results in pixels that look very different from their neighbors Generally, the larger the noise the stronger the response Smoothing the image should help, by forcing pixels different to their neighbors (=noise pixels?) to look more like neighbors
What is to be done?
Source: D. Forsyth
* [1 -1] =
d g dx
f d g dx
Source: S. Seitz
Highest response for regions that look the most like the filter Dot product as correlation Values positive Sum to 1 constant regions are unchanged Amount of smoothing proportional to mask size Opposite signs used to get high response in regions of high contrast Sum to 0 no response in constant regions High absolute value at points of high contrast
Source: K. Grauman
Smoothing masks
x-direction
y-direction
Derivative masks
Implementation issues
1 pixel
3 pixels
7 pixels
Smoothed derivative removes noise, but blurs edge. Also finds edges at different scales.
Source: D. Forsyth
The gradient magnitude is large along a thick trail or ridge, so how do we identify the actual edge points? How do we link the edge points to form curves?
Source: D. Forsyth
Laplacian of Gaussian
Consider
derivative of Gaussian
MATLAB demo
g = fspecial('gaussian',15,2); imagesc(g) surfl(g) gclown = conv2(clown,g,'same'); imagesc(conv2(clown,[-1 1],'same')); imagesc(conv2(gclown,[-1 1],'same')); dx = conv2(g,[-1 1],'same'); imagesc(conv2(clown,dx,'same')); lg = fspecial('log',15,2); lclown = conv2(clown,lg,'same'); imagesc(lclown) imagesc(clown + .2*lclown)
Edge finding
We wish to mark points along the curve where the magnitude is biggest. We can do this by looking for a maximum along a slice normal to the curve (non-maximum suppression). These points should form a curve. There are then two algorithmic issues: at which point is the maximum, and where is the next one?
Source: D. Forsyth
Non-maximum suppression
At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values.
Source: D. Forsyth
Source: D. Forsyth
Good detection: the optimal detector must minimize the probability of false positives (detecting spurious edges caused by noise), as well as that of false negatives (missing real edges) Good localization: the edges detected must be as close as possible to the true edges Single response: the detector must return one point only for each true edge point; that is, minimize the number of local maxima around the true edge
This is probably the most widely used edge detector in computer vision Theoretical model: step-edges corrupted by additive Gaussian noise Canny has shown that the first derivative of the Gaussian closely approximates the operator that optimizes the product of signal-to-noise ratio and localization
J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679-714, 1986.
Source: L. Fei-Fei Source: L. Fei-Fei
Filter image with derivative of Gaussian Find magnitude and orientation of gradient Non-maximum suppression:
Thin multi-pixel wide ridges down to single pixel width Define two thresholds: low and high Use the high threshold to start edge curves and the low threshold to continue them
4.
thresholding
Hysteresis thresholding
original image
thinning
(non-maximum suppression)
hysteresis threshold
Source: L. Fei-Fei
original
Canny with
Canny with
Source: S. Seitz