Basics Edge detectors

Edge Detection
CS-477 Computer Vision

Dr. Mohsin Kamal

Associate Professor
[email protected]

School of Electrical Engineering and Computer Science (SEECS)

National University of Sciences and Technology (NUST), Pakistan

1
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

1 Basics

2 Edge detectors

2
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Edge detection includes a variety of mathematical methods that

aim at identifying edges, curves in a digital image at which the
image brightness changes sharply or, more formally, has
discontinuities.

3
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

1 Basics

2 Edge detectors

4
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Examples

5
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Application of edge

Edge is the basic building block for analyzing an image

The main aim of finding an edge is to reduce the
information.
There are many pixels in an image, among which many are
irrelevant.

6
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Edge detection in images

At edges intensity or color changes
In other words, discontinuity is observed

7
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

What is an edge?

Discontinuity of intensities in the image

Edge models
Step
Roof
Ramp
Spike

8
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Detecting Discontinuities

Image derivatives
 
∂f f (x+)−f (x) ∂f f (xn+1 )−f (x)
∂x = lim  → ∂x ≈ ∆x
→0
Convolve image with derivative filters
Backward difference: [−1 1]
Forward difference: [1 − 1]
Central difference: [−1 0 1]

9
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Derivatives in two-dimensions

Definition   
∂x = lim f (x+,y )−f (x,y )
∂f (x,y ) ∂f (x,y )
∂y = lim f (x,y +)−f (x,y )

→0 →0
Approximation
∂f (x,y ) f (xn+1 ,ym )−f (xn ,ym ) ∂f (x,y ) f (xn ,ym+1 )−f (xn ,ym )
∂x ≈ ∆x ∂y ≈ ∆y
Convolution kernels  
  1
fx = 1 −1 fy =
−1

10
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Image derivatives

Image I

 
1
Iy = I ∗
 
Ix = I ∗ 1 −1 −1
11
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Image derivatives
An edge is a place of rapid change in the image intensity function

12
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Image derivatives

13
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Derivatives and noise

Effects of noise:
Consider a single row or column of the image
- Plotting intensity as a function of position gives a signal

14
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Derivatives and noise

What needs to be done?

Strongly affected by noise Neighbouring pixels look
alike
Reason: Image noise
Pixel along an edge look
results in pixels that look
alike
very different from their Image smoothing should
neighbors help
The larger the noise, the Force pixels different
stronger the response to their neighbors
(possibly noise) to
look like neighbors.

15
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Derivatives and noise

16
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Image smoothing

Expect pixels to "be like" their neighbors

Relatively few reflectance changes
Generally expect noise to be independent from pixel to pixel
Smoothing suppresses noise

17
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Image smoothing

Gaussian smoothing

−(x 2 +y 2 )
g(x, y ) = e 2σ 2

Scale of Gaussian
As σ increases, more pixels are involved in average
As σ increases, image is more blurred
As σ increases, noise is more effectively suppressed

18
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Image smoothing

Examples of Gaussian smoothing

19
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

1 Basics

2 Edge detectors

20
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Goal: Identify visual changes (discontinuities) in an image

Intuitively, semantic information is encoded in edges
Edges are caused by variety of factors as provided in Figure
below

21
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Gradient operators
Prewit
Sobel
Laplacian of Gaussian (Marr-Hildreth)
Gradient of Gaussian (Canny)

22
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Prewitt and Sobel edge detector

Compute derivatives
In x and y directions
Find gradient magnitudes
Threshold gradient magnitude

23
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Prewitt edge detector

Vertical Edges:  
1 0 −1
hx = 1 0 −1
1 0 −1

Horizontal Edges:  
−1 −1 −1
hx =  0 0 0
1 1 1

Gradient:
Gx = I ∗ hx
G y = I ∗ hy

24
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Sobel edge detector

The central row in hx and hy has more weights
It is performing weighted averaging
Vertical Edges:  
1 0 −1
hx = 2 0 −2
1 0 −1
Horizontal Edges:  
−1 −2 −1
hx =  0 0 0 
1 2 1
Gradient:
Gx = I ∗ h x
Gy = I ∗ h y

25
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Sobel edge detector

26
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Sobel edge detector

27
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Sobel edge detector

Drawback
Edges are not clear

28
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Marr Hildreth edge detector (Laplacian of Gaussian)

Smooth image by Gaussian filter → S

Apply Laplacian to S
Used in mechanics, electromagnetics, wave theory, quantum
mechanics and Laplace equation
Find zero crossings
If there is no change in the intensity, the derivative will be
zero.
Scan along each row, record an edge point at the location of
zero-crossing.
Repeat above step along each column

29
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Marr Hildreth edge detector (Laplacian of Gaussian)

Mathematically, Gaussian smoothing is represented as,

S =g∗I (1)

where,
1 −(x 2 +y 2 )
g=√ e 2σ 2 (2)
2πσ
Next step is to find the Laplacian of S. This is represented as,

∂2 ∂2
∆2 S = S + S (3)
∂x 2 ∂y 2

30
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Marr Hildreth edge detector (Laplacian of Gaussian)

Derivation of the Laplacian of Gaussian (LoG)

∆2 S = ∆2 (g ∗ I) = (∆2 g) ∗ I (4)

where ∆2 g is represented as

x2 + y2 2 2
 
2 1 − x +y2
∆ g = −√ 2− e 2σ (5)
2πσ 3 σ2

This means, rather than convolve the image with the gaussian
and take the second order derivative of the result, you can
actually take the 2nd derivative of the gaussian itself.

31
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Marr Hildreth edge detector (Laplacian of Gaussian)

Gaussian
−x 2
1-D Gaussian: g(x) = e 2σ2
x -3 -2 -1 0 1 2 3
g(x) .011 .13 .6 1 .6 .13 .011
−(x 2 +y 2 )
2-D Gaussian: g(x, y ) = e 2σ 2 for σ = 2

Here the mask is multiplied with 255 to have integer values from range 0-255. 32
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Marr Hildreth edge detector (Laplacian of Gaussian)

LOG Filter

2 +y 2
−x
 
1 x 2 +y 2
∆2 g = − √ 2− σ2
e 2σ 2
2πσ 3

33
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Marr Hildreth edge detector (Laplacian of Gaussian)

Finding zero crossings (1/2)

When the first derivative is at a maxima or minima the

second derivative is 0
Pixels where a zero crossing occurs are marked as edges
(if the slope of the crossing exceeds a threshold)
Four cases of zero-crossings
Zero crossings occur in the image wherever a positive value
is followed by a negative value, or vice versa. Or there may
even be a 0 in between a positive and negative value.

34
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Marr Hildreth edge detector (Laplacian of Gaussian)

Finding zero crossings (2/2)

The absolute difference between the negative and positive

values gives you the slope of the crossing, which is a
measure of the strength of the edge. Slope of zero-crossing
{a, -b} is |a+b|.
To mark an edge
Compute slope of zero-crossing
Apply a threshold to slope

35
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Marr Hildreth edge detector (Laplacian of Gaussian)

On the separability of LoG

Similar to separability of Gaussian filter

- Two-dimensional Gaussian can be separated into 2
one-dimensional Gaussians
h(x, y ) = I(x, y ) ∗ g(x, y ) n2 mults
h(x, y ) = (I(x, y ) ∗ g1 (x)) ∗ g2 (y ) 2n mults
where,
x2
 
−
g(x) = e 2σ 2 and n represents the mask
.011
 
 .13 
 .6 
 
 
g1 = g(x) = .011 .13 .6 1 .6 .13 .011 g2 = g(y ) =  1 
 
 .6 
 
 .13 
.011

36
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Marr Hildreth edge detector (Laplacian of Gaussian)

On the separability of LoG

∆2 S = ∆2 (g ∗ I) = (∆2 g) ∗ I = I ∗ (∆2 g)

Requires n2 multiplications

∆2 S = (I ∗ gxx (x)) ∗ g(y ) + (I ∗ gyy (y )) ∗ g(x)

Requires 4n multiplications

37
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Marr Hildreth edge detector (Laplacian of Gaussian)

Separability

38
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Marr Hildreth edge detector (Laplacian of Gaussian)

Example

39
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Marr Hildreth edge detector (Laplacian of Gaussian)

Algorithm

Compute LoG
Use 2D filter i.e., ∆2 g(x, y ), OR
Use 4 1D filters i.e., g(x), gxx (x), g(y ), gyy (y )
Find zero-crossings from each row,
Find slope of zero-crossings
Apply threshold to slope and mark edges

40
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

Robust to noise
Localization
Too many or too less responses

41
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

Line detection

We can use binary edge mask to detect lines in images

42
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

Line detection

43
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

Parameter optimization: Least Squared Error Solution

Fitting a line to a set of data points...

Can the line equation provide an estimated edge in this case?

44
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

Line fitting: least squared error solution

Step 1: Identify the model

Equation of line: y = mx + c
Step 2: Set up an error term which will give the goodness of
every point with respect to the (unknown) model
Error induced by the i th point:
ei =P mxi + c − yi
Error for whole data: E P= i ei2
E = i (mxi + c − yi )2
Step 3: Differentiate Error w.r.t. parameters, put equal to
zero and solve for minimum point

45
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

Line fitting: least squared error solution

E = (mxi + c − yi )2
P

∂E
Pi
∂m = (mxi + c − yi )xi = 0
i
∂E
P
∂c = (mxi + c − yi ) = 0
iP
xi2
P P 
xi
  xi yi
i i m i
P P  = P 
xi 1 c yi
i i i

46
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

Least squared error solution

Disadvantages are:
Multiple lines
Not robust to noise

47
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

Another approach
RANSAC (RANdom SAmple Consensus1 )
- It is an iterative method to estimate parameters of a
mathematical model from a set of observed data that contains
outliers, when outliers are to be accorded no influence on the
values of the estimates.
- Roboust against noise

1
https://en.wikipedia.org/wiki/Random_sample_consensus 48
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

RANSAC (RANdom SAmple Consensus)

Algorithm:
1 Sample (randomly) the number of points required to fit the model
2 Solve for model parameters using samples
3 Score by the fraction of inliers within a preset threshold of the model
Repeat 1-3 until the best model is found with high confidence

49
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

RANSAC (RANdom SAmple Consensus)

Algorithm:
1 Sample (randomly) the number of points required to fit the model (n = 2)
2 Solve for model parameters using samples
3 Score by the fraction of inliers within a preset threshold of the model
Repeat 1-3 until the best model is found with high confidence

50
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

RANSAC (RANdom SAmple Consensus)

Algorithm:
1 Sample (randomly) the number of points required to fit the model (n = 2)
2 Solve for model parameters using samples
3 Score by the fraction of inliers within a preset threshold of the model
Repeat 1-3 until the best model is found with high confidence

51
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

RANSAC (RANdom SAmple Consensus)

Algorithm:
1 Sample (randomly) the number of points required to fit the model (n = 2)
2 Solve for model parameters using samples
3 Score by the fraction of inliers within a preset threshold of the model (NI = 6)
Repeat 1-3 until the best model is found with high confidence

52
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

RANSAC (RANdom SAmple Consensus)

Algorithm:
1 Sample (randomly) the number of points required to fit the model (n = 2)
2 Solve for model parameters using samples
3 Score by the fraction of inliers within a preset threshold of the model (NI = 6)
Repeat 1-3 until the best model is found with high confidence

NI = 14
53
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge

RANSAC (RANdom SAmple Consensus)

Summary of RANSAC
Select minimum number of random points from data needed
to estimate the model
Estimate the model from selected random points
Check how many other points are consistent with the fitted
model (Consistent Set)
If consistent set is large enough, estimate model from all
points in the consistent set
If the consistent set is larger than the previous best model,
make the current model as the best model
Repeat a number of times

54
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Quality of an Edge
Pros
Robust to outliers
Applicable for larger number of objective function
parameters
Optimization parameters are easier to choose than other
approaches
Cons
Computational time grows quickly with fraction of outliers
and number of parameters
Needs to be adapted further to allow multiple fits
Common applications
Computing a homography(e.g., image stitching)
Estimating fundamental matrix (relating two views)
55
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

Criterion 1: Good Detection: The optimal detector must

minimize the probability of false positives as well as false
negatives.
Criterion 2: Good Localization: The edges detected must
be as close as possible to the true edges.
Single Response Constraint: The detector must return
one point only for each edge point.

56
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

Steps

1 Smooth image with Gaussian filter

2 Compute derivative of filtered image
3 Find magnitude and orientation of gradient
4 Apply "Non-maximum Suppression"
5 Apply "Hysteresis Threshold"

57
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

Step 1 and 2

1 Smoothing
s = I ∗ g(x, y ) = g(x, y ) ∗ I
2 +y 2
−x
where, g(x, y ) = √1 e 2σ 2
2πσ
2 Derivative
∇S = ∇(g ∗ "
I) =#(∇g) ∗ I
∂g  
g
where ∇g = ∂g = x
∂x
gy
  ∂y   
g g ∗I S
∇S = x ∗ I = x = x
gy gy ∗ I Sy

58
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

Canny Edge Detector Derivative of Gaussian

59
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

60
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

Step 3: Gradient magnitude and gradient direction

q
Magnitude = |∇S| = Sx2 + Sy2
Sy
Direction = θ = tan−1 Sx

61
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

Step 4: Non maximum suppression

We wish to mark points along the curve where the magnitude is

largest. We can do this by looking for a maximum along a slice
normal to the curve (non-maximum suppression). These points
should form a curve. There are then two algorithmic issues: at
which point is the maximum, and where is the next one?
62
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

Suppress the pixels in |∇S| which are not local maximum

|∇S|(x, y ) > |∇S|(x 0 , y 0 )


 |∇S|(x, y ),
M(x, y ) = |∇S|(x, y ) > |∇S|(x 00 , y 00 )
0, otherwise


x 0 and x 00 are the neighbors of x along

normal direction to an edge.

63
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

q
|∇S| = Sx2 + Sy2 M(x, y )

For visualization
M ≥ Threshold = 25
64
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

Step 5: Hysteresis Thresholding

If the gradient at a pixel is

above "High", declare it as an ’edge pixel’
below "Low", declare it as a "non-edge-pixel"
between "Low" and "High"
Consider its neighbors iteratively then declare it an "edge
pixel" if it is connected to an ’edge pixel’ directly or via
pixels between "Low" and "High".

65
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

Connectedness:

66
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

67
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

Scan the image from left to right, top-bottom.

The gradient magnitude at a pixel is above a high threshold
declare that as an edge point
Then recursively consider the neighbors of this pixel.
If the gradient magnitude is above the low threshold declare
that as an edge pixel.

68
CS-477 Computer Vision by Dr. Mohsin Kamal
Basics Edge detectors

Canny Edge Detector

q
|∇S| = Sx2 + Sy2 M(x, y )

M ≥ 25 High = 35, Low = 15

69
CS-477 Computer Vision by Dr. Mohsin Kamal