GLA UNIVERSITY

DIGITAL IMAGE PROCESSING

By:
Dr. Ankush Agarwal
Outline

• Morphological Image Processing

– Introduction, Logical Operations involving Binary Images,
– Dilation and Erosion, Opening and Closing, The Hit-or-Miss Transformation,
– Morphological Algorithms – Boundary Extraction, Region Filling, Extraction of
Connected Components, Convex Hull, Thinning, Thickening
• Image Segmentation
– Point, Line & Edge detection, Thresholding, Region-based Segmentation,
– Region Extraction - Pixel Based Approach & Region Based Approach,
– Edge and Line Detection - Basic Edge Detection, Canny Edge Detection,
– Edge Linking - Hough Transform
• Representation & Description
– Representation - Boundary Following, Chain Codes,
– Boundary Descriptors - Shape Numbers

© ankush agarwal 2
SEGMENTATION

© ankush agarwal 3
Segmentation

• Image segmentation is a technique to extract the attributes of the

image
• Segmentation attempts to partition the pixels of an image into
groups that strongly correlate with the objects in an image
• Mostly used in computer vision application

Image Segmentation Attributes

© ankush agarwal 4
Segmentation

• Separate image into coherent objects

© ankush agarwal 5
Segmentation

• The purpose of image segmentation is to partition an image into

meaningful regions with respect to a particular application
• The segmentation is based on various factors that are taken from
the image, which includes
– Intensity, Color, Texture, Depth, Shape, Size, Association, etc

© ankush agarwal 6
Segmentation

• Segmentation algorithms are based on 2 properties of intensity

values
Segmentation

Discontinuity Similarity

• Discontinuity: pixel, line, edges

• Similarity: region growing, split

© ankush agarwal 7
Segmentation

• Let v be the spatial domain on which the image is defined

• Image segmentation divides v into n regions Ri (i = 1 to n), such
that n
i 1 Ri  v
Ri Rj  

© ankush agarwal 8
Segmentation (Basic Idea)

• All the image segmentation methods assume that

– the intensity values are different in different regions
– within each region of the corresponding object, the intensity values are
similar

• For this, we need to apply threshold

© ankush agarwal 9
Thresholding

• Gray level thresholding is the simplest segmentation process

• Thresholding is computationally inexpensive and fast
• Thresholding as a transformation function
T = T[x, y, p(x, y), f(x, y)]
– where
– f(x, y) is the gray level of the point (x, y)
– p(x, y) denotes some local property (average of a pixels centred at (x, y))

© ankush agarwal 10
Thresholding (Types)

• Global Thresholding
– If T is the only function of f(x, y), then the threshold is called global
threshold
– Apply the same threshold to the whole image

• Local Thresholding
– If T is a function of both f(x, y) & p(x, y), then the threshold is called local
– The threshold depends on local property

• Dynamic (Adaptive) Thresholding

– The threshold depends on spatial coordinates (x, y)

© ankush agarwal 11
Thresholding

• Thresholding is the transformation of an input image f to an

output (segmented) binary image g
1 if f (x, y)  T
g (x, y)  
0 otherwise
– where, T is threshold

© ankush agarwal 12
Thresholding

• Eg:
– Disk = 255 and background = 127
– Segment the image into two regions: disk and background
– What will be the threshold values?

© ankush agarwal 13
Global Thresholding

• Based on the histogram of an image

• Partition the image histogram using a single global threshold
• The success of this technique strongly depends on how well the
histogram can be partitioned

© ankush agarwal 14
Global Thresholding (Algorithm)

• Select an initial estimate for T (typically the average)

• Segment the image using T to produce two groups of pixels:
– G1 consisting of pixels with grey levels >T
– G2 consisting pixels with grey levels ≤ T
• Compute the average grey levels of pixels in G1 to give μ1 and G2
to give μ2
• Compute a new threshold value T = (μ1+ μ2)/2

• Repeat steps 2–4 until the difference in T in successive iterations

is less than a predefined limit

© ankush agarwal 15
Global Thresholding (Problem)

• Single value thresholding only works for bimodal histograms (two

peaks)
• Images with other kinds of histograms need more than one
threshold

© ankush agarwal 16
Global Thresholding (Problem)

© ankush agarwal 17
Global Thresholding (Problem)

© ankush agarwal 18
Adaptive Thresholding

• The threshold for each pixel depends on its location within an

image, this technique is said to adaptive

© ankush agarwal 19
SEGMENTATION (SIMILARITY)

© ankush agarwal 20
Region

• A group of connected pixels with similar properties

• For correct interpretation, image must be partitioned into regions
that correspond to objects or parts of an object

© ankush agarwal 21
Region based approach

• Let R represent the entire image region

• Segmentation R into n sub regions, R1, R2, ..., Rn, such that
n
i 1 Ri  R
Ri is a connected region, i  1, 2,..., n
Ri R j   , for all i and j , i  j

© ankush agarwal 22
Region based approach

• The fundamental drawback of histogram based region detection is

that histograms provide no spatial information
• Region growing approaches exploit the important fact that pixels
which are close together have similar gray values

© ankush agarwal 23
Region growing

• Choose the seed pixels (1 for every segment)

• Check the neighboring pixels and add them to the region if they
are similar to the seed
• Repeat previous step for each of the newly added pixels, stop if no
more pixels can be added
1 1 9 9 9
1 1 9 9 9
5 1 9 9 9
5 5 5 3 9
3 3 3 3 3

© ankush agarwal 24
Region splitting

• Split starts from the assumption that the entire image is

homogeneous
• If this is not true (by the homogeneity criterion), then split image
into four sub images
• This splitting procedure is repeated recursively until we split the
image into homogeneous regions
I1 I2
I1 I2
I I41 I42
I3 I4 I3
I43 I44
© ankush agarwal 25
Region splitting

• If an image of dimensions N x N, that is in the powers of 2 (N =

2n):
– All regions produced by the splitting algorithm are squares having
dimensions M x M, where M is a power of 2 as well (M = 2m, M<= n)
– Since the procedure is recursive, it produces an image that can be
described by a tree, in which each nodes have four child (except leaf
node)
– Such a tree is called a Quadtree

© ankush agarwal 26
Region splitting

R
I1 I2

I41 I42 I1 I2 I3 I4
I3
I43 I44
I41 I41 I41 I41

© ankush agarwal 27
Region splitting

• Advantage
– Created regions are adjacent and homogenous
• Disadvantage
– Over splitting, since no merge is performed
– More clusters

• Improvement
– Split and Merge

© ankush agarwal 28
Region splitting and merging

• After splitting
• Merging phase
– If 2 adjacent regions are homogenous, they are merged
• Repeat merging step, until no further merging is possible

© ankush agarwal 29
SEGMENTATION (DISCONTINUITY)

© ankush agarwal 30
Segmentation (Discontinuity)

• There are basic three types of grey level discontinuities

– Points
– Lines
– Edges

• We typically find discontinuities using masks and correlation

• Discrete form of derivative is used

© ankush agarwal 31
Segmentation (Discontinuity)

• The filter is expected to be isotropic

– response of the filter is independent of the direction of discontinuities in
an image
• The digital implementation of the 2-Dimensional Laplacian is
obtained by summing 2 components
0 1 0
 f  f2 2
 f  2  2
2
1 -4 1
x y
0 1 0

© ankush agarwal 32
Segmentation (Discontinuity)

0 1 0 0 -1 0
1 -4 1 -1 4 -1  90° isotropic

0 1 0 0 -1 0

1 1 1 -1 -1 -1
45° isotropic  1 -8 1 -1 8 -1
1 1 1 -1 -1 -1
© ankush agarwal 33
Segmentation (DiscontinuityPoint)

• A point has been detected at the location on the mask center if

|R| >= T
– where
– T is a nonnegative threshold
– R is the sum of products of the coefficients with the gray levels contained
in the region encompassed by the mark

© ankush agarwal 34
Segmentation (Discontinuity  Point)

7 7 7 7 7 7 7 -1 -1 -1

7 10 7 7 7 7 7 -1 8 -1 24 -3
3 0 0 0

7 7 7 7 7 7 7 -1 -1 -1 -3
3 -3
3 0 0 0

7 7 7 7 7 7 7 P 0 0 3 3 3

7 7 7 7 4 7 7 0 0 3 24
-24 3

7 7 7 7 7 7 7 0 0 3 3 3

7 7 7 7 7 7 7

I Rp =I º|IPº P|

© ankush agarwal 35
Segmentation (Discontinuity  Line)

• The next level of complexity is to detect lines

• The masks below will extract lines that are one pixel thick and
running in a particular direction

-1 -1 -1 -1 -1 2 -1 2 -1 2 -1 -1
2 2 2 -1 2 -1 -1 2 -1 -1 2 -1
-1 -1 -1 2 -1 -1 -1 2 -1 -1 -1 2

© ankush agarwal 36
Segmentation (Discontinuity  Line)

1 1 1 1 1 1 1 1 -1 -1 -1
1 1 1 1 1 1 1 3 2 2 2 0 0 0 0 2 2
1 1 1 1 1 1 3 1 -1 -1 -1 0 0 0 2 2 0
1 1 1 1 1 3 1 1 H 6 6 6 0 0 2
3 3 3 3 3 1 1 1 12 12 12 6 2 2
1 1 1 1 1 1 1 1 6 6 6 4 2 0
1 1 1 1 1 1 1 1 0 0 0 0 0 0
1 1 1 1 1 1 1 1
I |I º H|

© ankush agarwal 37
Segmentation (Discontinuity  Line)

1 1 1 1 1 1 1 1 -1 -1 2
1 1 1 1 1 1 1 3 -1 2 -1 0 0 0 0 2 4
1 1 1 1 1 1 3 1 2 -1 -1 0 0 0 2 4 12
1 1 1 1 1 3 1 1 D45º 0 0 0 0 12 4
3 3 3 3 3 1 1 1 0 0 0 6 4 2
1 1 1 1 1 1 1 1 0 0 0 4 2 0
1 1 1 1 1 1 1 1 0 0 0 0 0 0
1 1 1 1 1 1 1 1
I |I º D45º|

© ankush agarwal 38
Segmentation (Discontinuity  Line)

1 1 1 1 1 1 1 1 -1 2 -1
1 1 1 1 1 1 1 3 -1 2 -1 0 0 0 0 2 2
1 1 1 1 1 1 3 1 -1 2 -1 0 0 0 2 2 0
1 1 1 1 1 3 1 1 V 0 0 0 0 0 2
3 3 3 3 3 1 1 1 0 0 0 0 2 2
1 1 1 1 1 1 1 1 0 0 0 2 2 0
1 1 1 1 1 1 1 1 0 0 0 0 0 0
1 1 1 1 1 1 1 1
I |I º V|

© ankush agarwal 39
Segmentation (Discontinuity  Line)

1 1 1 1 1 1 1 1 2 -1 -1
1 1 1 1 1 1 1 3 -1 2 -1 0 0 0 0 4 4
1 1 1 1 1 1 3 1 -1 -1 2 0 0 0 4 4 0
1 1 1 1 1 3 1 1 D-45º 0 0 0 6 0 4
3 3 3 3 3 1 1 1 0 0 0 0 4 4
1 1 1 1 1 1 1 1 0 0 0 2 4 0
1 1 1 1 1 1 1 1 0 0 0 0 0 0
1 1 1 1 1 1 1 1
I |I º D-45º|

© ankush agarwal 40
Segmentation (Discontinuity  Line)

1 1 1 1 1 1 1 1
1 10 10 10 10 14 14 3
1 10 10 10 14 14 12
3 1
1 16 16 16 16 12
3 14 1
3 12
3 12
3 12
3 36 14 14 1
1 16 16 16 14 14 10 1
1 10 10 10 10 10 10 1
1 1 1 1 1 1 1 1

© ankush agarwal 41
Segmentation (Discontinuity  Edge)

• Edges characterize boundaries

• Edges in images are areas with strong intensity contrasts – a
jump in intensity from one pixel to the next
• Edge detecting in an image significantly reduces the amount of
data and filters out useless information, while preserving the
important structural properties in an image

© ankush agarwal 42
Segmentation (Discontinuity  Edge)

• Edge linking
– process takes an unordered set of edge pixels produced by an edge
detector as i/p to form an ordered list of edgels

• Edge following
– process takes the entire edge strength or gradient image as i/p &
produces geometric primitives such as lines or curves

© ankush agarwal 45
Segmentation (Discontinuity  Edge  Derivative)

• First Order Derivative/Gradient Methods

– Robert operator
– Sobel operator
– Prewitt operator
• Second Order Derivative
– Laplacian
– Laplacian of Gaussian
– Difference of Gaussian
• Optimal Edge Detection
– Canny Edge Detection

© ankush agarwal 46
Segmentation (Discontinuity  Edge  Derivative)

1 1 1 1 2 2 2
Robert -1 -2 -1 -1 -1 -1
1 1 1 1 2 2 2
1 1 1 1 2 2 2 1 0 0 0 0 0 0 0

2 2 2 2 1 1 1 0 -1 1 2 1 1 1 1

2 2 2 2 1 1 1 G45º SH PH
2 2 2 2 1 1 1 -1 0 1 -1 0 1
0 1
2 2 2 2 1 1 1 -2 0 2 -1 0 1
-1 0
I -1 0 1 -1 0 1
G-45º
SV PV
Sobel Prewitt

© ankush agarwal 47
Segmentation (Discontinuity  Edge  Derivative)

• Motivation
– Detect sudden changes in image intensity
– Gradient: sensitive to intensity changes
• Gradient T
 f f 
f   
 x y 

© ankush agarwal 48
Segmentation (Discontinuity  Edge  Derivative)

• The gradient of the image I at location (x, y) is the vector

 I (x, y) 
Gx   x 
I      
 y 
G I (x, y) 
 y 
• Magnitude
| I | Gx2  G y2
• Gradient
 Gy 
 (x, y)  tan 1  
 x
G

© ankush agarwal 49
Segmentation (Discontinuity  Edge  Derivative)

• Eg:
– Find the strength & the direction of the edge at the highlighted pixel
– Pixels in gray are 0 and white are 1

Solution
• Derivative is computed by using a 3x3 neighbourhood
– subtract the pixels in the top row from bottom row (x direction)
– similarly obtain the derivative in the y direction
-1 -1 -1 -1 0 1
0 0 0 -1 0 1
1 1 1 -1 0 1
© ankush agarwal 50
Segmentation (Discontinuity  Edge  Derivative)

• Eg:
– Find the strength & the direction of the edge at the highlighted pixel
– Pixels in gray are 0 and white are 1

 I (x, y) 
Gx   x   2 
I       
G y   I (x, y)   2 
 y 

| I | Gx2  G y2  2 2

 Gy 
 (x, y)  tan 
1
  1  45
 x
G

© ankush agarwal 51
Simple edge detection using gradient

• Compute gradient vector at each pixel by convolving image with

horizontal and vertical derivative filters
• Compute gradient magnitude at each pixel
• If magnitude at a pixel exceeds a threshold, report a possible edge
point

© ankush agarwal 52
CANNY EDGE DETECTOR

© ankush agarwal 53
Canny (Characteristics)

• Good Detection : Robustness to noise

– The optimum detector must minimize the probability of false positive as
well as false negative
• Good Localization:
– The edge must be as close as possible to the true edges
• Strong Response Constraint:
– Not too many or too few responses: The detector must return one point
only for each point

© ankush agarwal 54
Canny (Steps)

• Smoothing with Gaussian Filter

• Compute Derivative of filter image
• Find magnitude & orientation of gradient
• Apply Non Maxima Suppression
• Apply Hysteresis Threshold

© ankush agarwal 55
Canny (Steps – Gaussian filter)

• Smoothing with Gaussian filter on Image I(r,c)

© ankush agarwal 56
Canny (Steps - Derivative)

© ankush agarwal 57
Canny (Steps - Magnitude and Orientation)

© ankush agarwal 58
Canny (Steps - Apply Non Maxima Suppression)

© ankush agarwal 59
Canny (Steps - Apply Non Maxima Suppression)

Exploring Pixels for NonMaxima Suppression

Colour Coded Orientation

Gradient Magnitude after

Gradient Magnitude Additive Noise: Dithering NonMaxima Suppression
© ankush agarwal 60
Canny (Steps – Apply Hysteresis Thresholding)

© ankush agarwal 61
Canny (Steps – Apply Hysteresis Thresholding)

© ankush agarwal 62
Canny

• Eg:
– Compute derivative

© ankush agarwal 63
Canny

• Eg:
– Compute magnitude and orientation

-4 -3 -1 0 0 2 1 1 0 0
-3 -4 -3 -1 0 1 2 3 1 0
0 -1 -3 -4 -3 0 1 3 2 1
0 0 -1 -3 -4 0 0 1 1 2
0 0 0 0 -1 0 0 0 0 1

© ankush agarwal 64
Canny

• Eg:
– Apply Non-maxima suppression

4.5 3.2 1.4 0 0 -27 -18 -45 D D

3.2 4.5 4.2 1.4 0 -18 -27 -45 -45 D
0 1.4 4.2 4.5 3.2 D -45 -45 -27 -18
0 0 1.4 3.2 4.5 D D -45 -18 -27
0 0 0 0 1.4 D D D D -45

© ankush agarwal 65
Canny

• Eg:
– Apply Hysteresis Threshold

-27 -18 -45 D D 4.5 0 0 0 0

-18 -27 -45 -45 D 0 4.5 4.2 0 0
D -45 -45 -27 -18 0 0 4.2 4.5 0
D D -45 -18 -27 0 0 0 0 4.5
D D D D -45 0 0 0 0 1.4

© ankush agarwal 66
HOUGH TRANSFORM

© ankush agarwal 67
Hough Transform

• Image and Parameter space

y = mx+c c = -mx+y
variable: x, y variable: m, c
parameter: m, c parameter: x, y
© ankush agarwal 68
Hough Transform

• Eg:

© ankush agarwal 69
Hough Transform

• Step 1:
– Known
• -1.2<=m<=-0.8
• step_m = 0.1

• m = -0.8, -0.9, -1.0, -1.1, -1.2

• c = -mx+y

© ankush agarwal 70
Hough Transform

• Step 2:
– Compute c

(x,y) = (4,1) (x,y) = (3,2) (x,y) = (2,3) (x,y) = (1,4)

c = -4m+1 c = -3m+2 c = -2m+3 c = -m+4
m c m c m c m c
-0.8 4.2 -0.8 4.4 -0.8 4.6 -0.8 4.8
-0.9 4.6 -0.9 4.7 -0.9 4.8 -0.9 4.9
-1.0 5.0 -1.0 5.0 -1.0 5.0 -1.0 5.0
-1.1 5.4 -1.1 5.3 -1.1 5.2 -1.1 5.1
-1.2 5.8 -1.2 5.6 -1.2 5.4 -1.2 5.2

© ankush agarwal 71
Hough Transform

• Step 3:
– Find min and max of c
(x,y) = (4,1) (x,y) = (3,2) (x,y) = (2,3) (x,y) = (1,4)
• Range: 4.2 to 5.8 c = -4m+1 c = -3m+2 c = -2m+3 c = -m+4
• Interval: 0.1 m c m c m c m c
-0.8 4.2 -0.8 4.4 -0.8 4.6 -0.8 4.8
-0.9 4.6 -0.9 4.7 -0.9 4.8 -0.9 4.9
-1.0 5.0 -1.0 5.0 -1.0 5.0 -1.0 5.0
-1.1 5.4 -1.1 5.3 -1.1 5.2 -1.1 5.1
-1.2 5.8 -1.2 5.6 -1.2 5.4 -1.2 5.2

© ankush agarwal 72
Hough Transform

• Step 4:
– Voting

© ankush agarwal 73
Hough Transform

• Step 5:
– Find equation

• m =-1.0
• c = 5.0
• y = -x+5

© ankush agarwal 74
Hough Transform

• Problem
– The slope of line parallel to y – axis is infinity, the computer can not
handle it.
– So, we follow another parameterization of a line
– p = x cos θ + y sin θ

© ankush agarwal 75
Hough Transform

y
6
5
4
3
p = x cos θ + y sinθ
2
Edgels: (0,5), (1,4), (3,2), (3,3), (5,0)
1
0
0 1 2 3 4 5 6 x

© ankush agarwal 76
Hough Transform

p = x cos θ + y sinθ

p = 0 cos θ + 5 sin θ (0,5)

p = 1 cos θ + 4 sin θ (1,4)
p = 3 cos θ + 2 sin θ (3,2)
p = 3 cos θ + 3 sin θ (3,3)
p = 5 cos θ + 0 sin θ (5,0)

© ankush agarwal 77
Hough Transform

0 cos θ + 5 sin θ 1 cos θ + 4 sin θ 3 cos θ + 2 sin θ 3 cos θ + 3 sin θ 5 cos θ + 0 sin θ

0 0 1 3 3 5

30 2.5 2.9 3.9 4 4.3

45 3.5 3.5 3.5 4.2 3.5

60 4.3 3.9 3.2 4 2.5

90 5 4 2 3 0

© ankush agarwal 78
Hough Transform

• The equation for the edge will be:

3.5 = x cos 45 + y sin 45

© ankush agarwal 79