Edge Detection

Definition of Edges
• Edges are significant local changes of intensity in an
image.
 What Causes Intensity Changes?
• Geometric events
– surface orientation (boundary) discontinuities
– depth discontinuities
– color and texture discontinuities

• Non-geometric events surface normal discontinuity

– illumination changes
– specularities depth discontinuity

– shadows color discontinuity

– inter-reflections
illumination discontinuity
 Goal of Edge Detection
• Produce a line “drawing” of a scene from an image of that
scene.
 Why is Edge Detection Useful?
• Important features can be extracted from the edges of an
image (e.g., corners, lines, curves).
• These features are used by higher-level computer vision
algorithms (e.g., recognition).
Effect of Illumination
 Edge Descriptors
• Edge direction:
perpendicular to the direction
of maximum intensity change
(i.e., edge normal)
• Edge strength: related to
the local image contrast along
the normal.
• Edge position: the image
position at which the edge is
located.
 Modeling Intensity Changes

• Step edge: the image intensity abruptly changes from

one value on one side of the discontinuity to a
different value on the opposite side.
 Modeling Intensity Changes (cont’d)

• Ramp edge: a step edge where the intensity change is

not instantaneous but occur over a finite distance.
 Modeling Intensity Changes (cont’d)

• Ridge edge: the image intensity abruptly changes

value but then returns to the starting value within
some short distance (i.e., usually generated by lines).
 Modeling Intensity Changes (cont’d)

• Roof edge: a ridge edge where the intensity change is

not instantaneous but occur over a finite distance (i.e.,
usually generated by the intersection of two surfaces).
 Main Steps in Edge Detection

(1) Smoothing: suppress as much noise as possible,

without destroying true edges.

(2) Enhancement: apply differentiation to enhance the

quality of edges (i.e., sharpening).
Main Steps in Edge Detection (cont’d)

(3) Thresholding: determine which edge pixels

should be discarded as noise and which should be
retained (i.e., threshold edge magnitude).

(4) Localization: determine the exact edge location.

 Edge Detection Using Derivatives

• Often, points that lie on an edge

are detected by:

(1) Detecting the local maxima or

minima of the first derivative.
1st derivative

(2) Detecting the zero-crossings

of the second derivative.
2nd derivative
 Image Derivatives

• How can we differentiate a digital image?

– Option 1: reconstruct a continuous image, f(x,y), then

compute the derivative.

– Option 2: take discrete derivative (i.e., finite

differences)

Consider this case first!

Edge Detection Using First Derivative
1D functions (not centered at x)

(centered at x)

(upward) step edge ramp edge

(downward) step edge roof edge

 Edge Detection Using Second Derivative

• Approximate finding maxima/minima of gradient

magnitude by finding places where:
Edge Detection Using Second Derivative (cont’d)
1D functions:

(centered at x+1)

Replace x+1 with x (i.e., centered at x):

Edge Detection Using Second Derivative (cont’d)
Edge Detection Using Second Derivative (cont’d)
(upward) step edge

(downward) step edge

ramp edge

roof edge
Edge Detection Using First Derivative (Gradient)

2D functions:
• The first derivate of an image can be computed using the
gradient:

f
 Gradient Representation
• The gradient is a vector which has magnitude and direction:
(approximation)

or f f
| || |
x y

• Magnitude: indicates edge strength.

• Direction: indicates edge direction.

– i.e., perpendicular to edge direction
 Approximate Gradient

• Approximate gradient using finite differences:

 Approximate Gradient (cont’d)
• Cartesian vs pixel-coordinates:
- j corresponds to x direction
- i to -y direction
 Approximating Gradient (cont’d)

• We can implement and using the following masks:

(x+1/2,y)

good approximation
(x,y+1/2)
at (x+1/2,y) *
*

good approximation
at (x,y+1/2)
 Approximating Gradient (cont’d)

• A different approximation of the gradient:

good approximation
(x+1/2,y+1/2)

• and can be implemented using the following masks:

 Another Approximation
• Consider the arrangement of pixels about the pixel (i, j):

3 x 3 neighborhood:

• The partial derivatives can be computed by:

• The constant c implies the emphasis given to pixels closer to

the center of the mask.
 Prewitt Operator

• Setting c = 1, we get the Prewitt operator:

 Sobel Operator

• Setting c = 2, we get the Sobel operator:

Edge Detection Steps Using Gradient

(i.e., sqrt is costly!)

Example (using Prewitt operator)

Note: in this example, the

divisions by 2 and 3 in the
computation of fx and fy
are done for normalization
purposes only
Another Example

d
I
dx

d
I
dy
Another Example (cont’d)
2 2
 d   d 
  I   I 
 dx   dy 

 Threshold 100
 Practical Issues
• Noise suppression-localization tradeoff.
– Smoothing depends on mask size (e.g., depends on σ for
Gaussian filters).
– Larger mask sizes reduce noise, but worsen localization (i.e.,
add uncertainty to the location of the edge) and vice versa.
smaller mask larger mask
Effect Smoothing on Derivates

Where is the edge??

Effect of Smoothing on Derivatives (cont’d)
Combine Smoothing with Differentiation
(i.e., saves one operation)
 Mathematical Interpretation of combining
smoothing with differentiation
• Numerical differentiation is an ill-posed problem.
- i.e., solution does not exist or it is not unique or it does not
depend continuously on initial data)

• Ill-posed problems can be solved using “regularization”

- i.e., impose additional constraints

• Smoothing performs image interpolation.

 Practical Issues (cont’d)
• Choice of threshold.
gradient magnitude

low threshold high threshold

 Practical Issues (cont’d)

• Edge thinning and linking.

 Edge Linking and Boundary Detection
Local Processing: Example

In this example,
we can find the
license plate
candidate after
edge linking
process.
 Hough Transform

Straight line case

Consider a single isolated edge point (xi, yi)
• There are an infinite number of lines that could pass through the points
Each of these lines can be characterized by some particular equation

y i =mx i +c

42
 Line detection

Mathematical model of a line:

y Y = mx + n Y1=m x1+n

Y2=m x2+n
P(x1,y1)
P(x2,y2)
x
YN=m xN+n

43
 Image and Parameter Spaces

y Y = mx slope
Y1=m x1+n
Y =+m’x
n + n’
m’
Y2=m x2+n
n
n’
intercept
x m
YN=m xN+n

Image Space Parameter Space

Line in Img. Space ~ Point in Param. Space

44
 Hough Transform Technique
Given an edge point, there is an infinite number of lines
passing through it (Vary m and n).
These lines can be represented as a line in parameter space.
slope
y n = (-x) m + y

P(x, y) n
intercept
x m

Parameter Space
45
 n

m
Edge Linking and Boundary Detection
Hough Transform Example
 Local Analysis proximity

• Two nearby pixels possessing similar edges

properties may be joined together and thereby a long
continuous contour can be formed. Provided
• d( (x, y), (x’, y’)) < td
• | (g(x,y)- g(x’,y’) | < tg
• |θ(x,y) - θ(x’,y’) | <tθ
 Graph Searching

• Connect two points P and Q

• Path between P and Q are through the set of points nk
for k= 1, 2, 3, ……..
• Let (nk) be the cost function for the path from P to Q
passing through the intermediate point nk.
• Thus the for and edge node ni objective is to optimize
the cost (ni) .
 Hough Transform Technique

Given a set of collinear edge points, each of them have

associated a line in parameter space.
These lines intersect at the point (m, n) corresponding to the
parameters of the line in the image space.

50
 Hough Transform slope-intercept
parametrization
An Edge Pixel in Real Space would vote into Hough Space all possible
lines that contain that point
y = kx + q
Continue to Add Votes for different Edge Pixels
Intersection gives Equation for line
Edge Detected Image (real space)
Hough Space

51
 HT - parametric representation
y = kx + q
(x,y) - co-ordinates
k - gradient
q - y intercept
Any straight line is characterized by k & q
use : ‘slope-intercept’ or (k,q) space not (x,y) space
(k,q) - parameter space
(x,y) - image space
can use (k,q) co-ordinates to represent a line

52
 Looking at it backwards …

Image space
Fix (m,n), Vary (x,y) - Line Y = mx + n

Fix (x1,y1), Vary (m,n) – Lines thru a Point Y1=m x1+n

P(x1,y1)
x
53
 Looking at it backwards …

Parameter space

Y1=m x1+nCan be re-written as: n = -x1 m + Y1

Fix (-x1,y1), Vary (m,n) - Line n = -x1 m + Y1

slope
m’
n
n’
intercept
m

54
Hough Transform for lines

55
 Image Parameter Spaces

Image Space Parameter Space

Lines Points
Points Lines
Collinear points Intersecting lines

56
Hough Transform
Philosophy
H.T. is a method for detecting straight lines, shapes and
curves in images.

•Main idea:
Map a difficult pattern problem into a simple peak detection
problem

57
 Criteria for Optimal Edge Detection

• (1) Good detection

– Minimize the probability of false positives (i.e., spurious edges).
– Minimize the probability of false negatives (i.e., missing real
edges).

• (2) Good localization

– Detected edges must be as close as possible to the true edges.

• (3) Single response

– Minimize the number of local maxima around the true edge.
 Canny edge detector
• Canny has shown that the first derivative of the Gaussian
closely approximates the operator that optimizes the
product of signal-to-noise ratio and localization.
(i.e., analysis based on "step-edges" corrupted by "Gaussian noise“)

J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern

Analysis and Machine Intelligence, 8:679-714, 1986.
Steps of Canny edge detector
Steps of Canny edge detector (cont’d)

(and direction)
Canny edge detector - example
original image
Canny edge detector – example (cont’d)
Gradient magnitude
Canny edge detector – example (cont’d)
Thresholded gradient magnitude
Canny edge detector – example (cont’d)
Thinning (non-maxima suppression)
 Non-maxima suppression

• Check if gradient magnitude at pixel location (i,j)

is local maximum along gradient direction
Non-maxima suppression (cont’d)
Warning: requires checking
interpolated pixels p and r

(i,j)
 Hysteresis thresholding
• Standard thresholding:

- Can only select “strong” edges.

- Does not guarantee “continuity”.

gradient magnitude low threshold high threshold

 Hysteresis thresholding (cont’d)
• Hysteresis thresholding uses two thresholds:
- low threshold tl
- high threshold th (usually, th = 2tl)

th
tl th
tl

• For “maybe” edges, decide on the edge if neighboring

pixel is a strong edge.
 Hysteresis thresholding/Edge Linking
Idea: use a high threshold to start edge curves and a low
threshold to continue them.

Use edge
“direction” for
linking edges
Hysteresis Thresholding/Edge Linking (cont’d)

(using tl and th)

Note: large gaps are still difficult to bridge.

(i.e., more sophisticated algorithms are required)
Second Derivative in 2D: Laplacian
Second Derivative in 2D: Laplacian (cont’d)
Variations of Laplacian
Laplacian - Example

detect zero-crossings
 Properties of Laplacian

• It is an isotropic operator.
• It is cheaper to implement than the gradient (i.e., one
mask only).
• It does not provide information about edge direction.
• It is more sensitive to noise (i.e., differentiates twice).
 Laplacian of Gaussian (LoG)
(Marr-Hildreth operator)
• To reduce the noise effect, the image is first smoothed.

• When the filter chosen is a Gaussian, we call it the LoG

edge detector.
σ controls smoothing

(inverted
LoG)
• It can be shown that:

2σ2
Laplacian of Gaussian (LoG) - Example
(inverted LoG)
(inverted LoG)

filtering zero-crossings
 Decomposition of LoG
• It can be shown than LoG can be written as follows:

2 g ( x, y )

• 2D LoG convolution can be implemented using 4, 1D

convolutions.

2 g ( x, y )* I ( x, y )
Decomposition of LoG (cont’d)
Steps
 Difference of Gaussians (DoG)
• The Laplacian of Gaussian can be approximated by the
difference between two Gaussian functions:

approximation
actual LoG
Difference of Gaussians (DoG) (cont’d)

(a) (b) (b)-(a)

 Gradient vs LoG
• Gradient works well when the image contains sharp
intensity transitions and low noise.
• Zero-crossings of LOG offer better localization, especially
when the edges are not very sharp.
step edge

ramp edge
 Gradient vs LoG (cont’d)

LoG behaves poorly at corners

 Directional Derivative

f

• The partial derivatives of f(x,y) will give the slope

∂f/∂x in the positive x direction and the slope
∂f /∂y in the positive y direction.

• We can generalize the partial derivatives to calculate

the slope in any direction (i.e., directional derivative).
 Directional Derivative (cont’d)

• Directional derivative computes intensity changes

in a specified direction.

Compute
derivative
in direction u
Directional Derivative (cont’d)
Directional
derivative is a
linear
combination of
(From vector calculus)
partial
derivatives.

+ =
Directional Derivative (cont’d)

||u||=1

+ =
cosθ sinθ
 Higher Order Directional Derivatives

f f
f ( x, y )  cos   sin 
'

x y

2 2 2
 f  f  f
f ( x, y )  2 cos   2
'' 2
cos  sin   2 sin 2 
x xy y

3 f 3 f 3 f 3 f
f''' ( x, y )  cos3   3 cos 2  sin   3 cos  sin 2
 sin 3 
x3 x 2 y xy 2
y 3
Edge Detection Using Directional Derivative

• What direction would you use for edge detection?

Direction of gradient:

 f 
 x   cos  
    
 f   sin  
 y 
 
 Second Directional Derivative
(along gradient direction)
2 2 2
 f  f  f
f'' ( x, y )  2
cos 2
  2 cos  sin   2
sin 2

x xy y

 f 
 x   cos  
    
 f   sin  
 y 
 
Edge Detection Using Second Derivative

Laplacian:

or

(i) the second directional derivative is equal to zero and

(ii) the third directional derivative is negative.
Properties of Second Directional Derivative
(along gradient direction)
 Facet Model
• Assumes that an image is an array of samples of a
continuous function f(x,y).
• Reconstructs f(x,y) from sampled pixel values.
• Uses directional derivatives which are computed
analytically (i.e., without using discrete approximations).

z=f(x,y)
 Facet Model (cont’d)

• For complex images, f(x,y) could

contain extremely high powers
of x and y.
• Idea: model f(x,y) as a piece-
wise function.
• Approximate each pixel value by
fitting a bi-cubic polynomial in
a small neighborhood around the
pixel (facet).
 Facet Model (cont’d)

Steps
(1) Fit a bi-cubic polynomial to a small neighborhood of
each pixel (this step provides smoothing too).

(2) Compute (analytically) the second and third

directional derivatives in the direction of gradient.

(3) Find points where (i) the second derivative is equal to

zero and (ii) the third derivative is negative.
 Fitting bi-cubic polynomial

• If a 5 x 5 neighborhood is used, the masks below can

be used to compute the coefficients.
– Equivalent to least-squares (e.g., SVD)
 Analytic computations of second and
third directional derivatives

• Using polar coordinates

 Compute analytically second and
third directional derivatives

• Gradient angle θ (with positive y-axis at (0,0)):

Locally approximate surface

by a plane and use the normal
to the plane to approximate
the gradient.
 Computing directional derivatives (cont’d)

• The derivatives can be computed as follows:

Second derivative equal

to zero implies:

Third derivative
negative implies:
Edge Detection Using Facet Model (cont’d)

Steps
 Anisotropic Filtering
(i.e., edge preserving smoothing)
• Symmetric Gaussian smoothing tends to blur out edges
rather aggressively.

• An “oriented” smoothing operator would work better:

(i) Smooth aggressively perpendicular to the gradient
(ii) Smooth little along the gradient

• Mathematically formulated using diffusion

equation.
Anisotropic filtering - Example
result using
anisotropic filtering
 Effect of scale (i.e., σ)

original

– Small σ detects fine features.

– Large σ detects large scale edges.
 Multi-scale Processing

• A formal theory for handling image structures at

different scales.

• Process images multiple scales.

• Determine which structures (e.g., edges) are most

significant by considering the range of scales over
which they occur.
 Multi-scale Processing (cont’d)
σ=1

σ=4
σ=2

σ=16
σ=8

• Interesting scales: scales at which important structures are

present.
e.g., in the image above, people can be detected at scales [1.0 - 4.0]
 Scale Space (Witkin 1983)

Gaussian • Detect and plot the

filtered signal zero-crossing of a 1D
function over a continuum
of scales σ.

• Instead of treating zero-

crossings at a single scale as
a single point, we can now
σ treat
them at multiple scales as
contours.
x A. Witkin, "Scale-space filtering", 8th Int. Joint Conf. Art. Intell.,
Karlsruhe, Germany,1019–1022, 1983
Scale Space (cont’d)

• Properties of scale space

(assuming Gaussian
smoothing):
– Zero-crossings may shift with
increasing scale ().
– Two zero-crossing may merge
with increasing scale.
– A contour may not split into
two with increasing scale.
Multi-scale processing (cont’d)
Multi-scale processing (cont’d)
 Edge detection is just the beginning…
image human segmentation gradient magnitude

• Berkeley segmentation database:

http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/