Digital Image Processing

2nd Edition
S.
Sridhar

© Oxford University Press 2016. All rights

reserved.
 Chapter 9
Image
Segmentation

Dr. Poornima B V
Assistant Professor
Dept. of CSE
© Oxford University Press 2016. All rights
reserved.
 WHAT YOU LEARN IN THE COURSE

MODULE-V
Image Segmentation: Introduction, classification, detection of
discontinuities, Edge detection (up to canny edge
detection(included)).
Image Morphology: Need of Morphological Processing,
Morphological operators, Hit or miss transform.

Computer Graphics and Fundamentals of Image 3

 Image Segmentation

• Segmentation is the process of partitioning

a digital image into multiple regions and
extracting meaningful regions known as
regions of interest (ROI) for further image

© Oxford University Press 2016. All rights reserved.

• Image segmentation algorithms are based on either
discontinuity principle or similarity principle.
• The discontinuity principle is to extract regions that differ
in properties such as intensity, color, texture.
• Mostly abrubt changes in intensity among the regions
result in extraction of edges.
• The idea behind similarity principle is to group pixels
based on a common property to extract coherent region.
An image can be partitioned into many regions R1,R2…Rn.

20 20 10 10 10

20 20 10 10 10

20 20 10 10 10

20 20 10 10 10

15 15 10 10 10

15 15 10 10 10
 20 20 10 10 10

R1
20 20 10 10 10
R3
20 20 10 10 10

20 20 10 10 10

15 15 10 10 10

R2 15 15 10 10 10

R
Formal Definition of Image Segmentation

© Oxford University Press 2016. All rights reserved.

CLASSIFICATION OF ALGORITHMS

© Oxford University Press 2016. All rights reserved.

 Types of Segmentation Algorithms
Based on user Interaction
• Manual: User selects the region. Often it is difficult to replicate
• Semiautomatic: User provides a “seed” using which the algorithm
creates segmentation
• Automatic: No user input required
 Types of Segmentation Algorithms
• Contextual (region-based or global) algorithms
• Group pixels based on common properties by exploiting the
relationships that exist among the pixels.
• Non-contextual (pixel-based or local) algorithms
• Ignore the relationships between pixels or
features
• Instead identify the discontinuities that exist in
the images such as isolated lines or edges
 DETECTION OF DISCONTINUITIES

• The basic types of grey level discontinuities in a digital image

are:
1. Points
2. Lines
3. Edges
 DETECTION OF DISCONTINUITIES- Point
Detection

An isolated point is a point whose grey level is significantly different from its background

© Oxford University Press 2016. All rights reserved.

Working of a generic spatial mask
Point Detection

© Oxford University Press 2016. All rights reserved.

 Line Detection
4 types of masks used:
1. Horizontal
2. Vertical
3. +45◦
4. -45◦
Responses Ri from each of the masks are obtained
• R1 is the response for moving the mask from left to the right of
the image.
• R2 is the response for moving the mask from top to the bottom of
the image.
• R3 is the response of the mask along +45 degree line
• R4 is the response of the mask with respect to a line of -45 degree.
EDGE DETECTION

• Edge detection is a technique used in image processing and

computer vision to identify points in an image where the
brightness or intensity changes sharply.
• These points often correspond to boundaries of objects, texture
changes, or surface discontinuities—essentially, the "edges"
within an image.
Why is Edge Detection Important?
• Edges represent significant local changes in intensity and
are critical for:
• Object detection and recognition
• Image segmentation
• Scene understanding
• Feature extraction in computer vision
 EDGE DETECTION
• Edges play an important role in many applications.
• Edge is a set of connected pixels that lies on the boundaries between the
regions that differ in gray value.
• The pixels on an edge are called edge point
• They provide object outline and correspond to discontinuities in depth, surface
orientation or change in material properties

There are 4 types of edges:

1. Step edge: Abrupt change in pixel intensity
2. Ramp edge: Gradual change in intensity
3. Spike edge: quick change in the intensity and
returns to original intensity level
4. Roof edge: Non instantaneous over a short
edge
• Pixel intensity refers to the brightness or color value of a
pixel in a digital image.
• It represents how light or dark a pixel is and is a key
component in understanding and processing images,
especially in grayscale or black-and-white formats.
1. In Grayscale Images
• Each pixel has a single intensity value, usually ranging
from 0 to 255:
• 0 = black
• 255 = white
• Values in between represent varying shades of gray.
2. In Color Images (e.g., RGB)
• Each pixel is made up of three intensity values: one each
for Red, Green, and Blue.
• Each channel typically has values from 0 to 255:
• Example: (255, 0, 0) is bright red
• Example: (0, 255, 0) is bright green
• Example: (0, 0, 0) is black, and (255, 255, 255) is white
3. Why Pixel Intensity Matters
Pixel intensity is used for:
• Image enhancement (e.g., brightness adjustment)
• Thresholding (e.g., turning a grayscale image into black
and white)
• Edge detection (detects sharp changes in pixel intensity)
• Object recognition and tracking
Stages in Edge Detection
• In edge detection, the goal is to find significant transitions
in pixel intensity—essentially the outlines of objects in an
image.
• This process typically involves three main steps: filtering,
differentiation, and localization.
• Each plays a specific role in ensuring accurate and noise-
resistant edge detection.
 1. Filtering (Noise Reduction)
• Purpose: Remove unwanted noise that can interfere with
edge detection.
• Real-world images often contain noise (random variations in
intensity), which can create false edges.
• Filtering smooths the image to reduce these unwanted
variations while preserving important structural details.
Common technique:
• Gaussian filter (blurring): Applies a Gaussian kernel to the
image to reduce noise.
Differentiation (Finding Intensity Gradients)
Purpose: Detect regions in the image where intensity changes
rapidly—these are potential edges.
• This step involves calculating the gradient (rate of change)
of pixel intensities.
• A high gradient means a sharp change in brightness → likely
an edge.
Common techniques:
• Sobel, Prewitt, Roberts operators (use convolution kernels
to approximate gradients)
• Calculate gradient magnitude and direction:
Localization (Pinpointing the Exact Edge Position)
• Purpose: Precisely determine where the edges are located in
the image.
• This step refines edge candidates by:
• Thinning them to a single-pixel width (called non-
maximum suppression)
• Suppressing false edges caused by noise or color
variation
Techniques:
• Non-Maximum Suppression: Only keeps the strongest
gradient in the direction of the edge.
• Thresholding (like in Canny): Applies high and low thresholds
to decide which gradients are real edges.
 Stages in Edge Detection
Differentiation
• This phase distinguishes the edge pixels from neighbourhood
pixels.
• If difference between 2 neighbouring pixels = 0 then No
transition.
• The non zero difference indicates the presence of edge point.
A point is an edge point if:
1. First derivative is greater than a specified threshold and
2. There is a sign change in the second derivative(Zero
crossing)
• The first derivative is
Compare neighboring pixels:
If two neighboring pixels are very similar, the difference is 0
or close to 0 → No edge.
If the difference is big, it means there's a sharp change →
Possible edge.
First Derivative (Detects change):
Think of it like checking how fast the pixel value is changing.
If this change is bigger than a threshold (a set number), it
might be an edge.
Pixels: 50 52 51 49 48 → Small changes → No edge
Pixels: 50 52 51 90 91 → Big jump → Edge here!

Second Derivative (Zero crossing):

•This checks where the direction of change flips.
•If the brightness was increasing and then suddenly starts
decreasing, or vice versa, this causes a zero crossing — a
sign of an edge.
• Consider 1D image f(x) = 60 60 60 100 100 100. What are the first
and second derivatives?
1st derivative:
0 0 40 0 0
2nd derivative:
0 40 -40 0
Edge Detection

© Oxford University Press 2016. All rights reserved.

 Edge Detection
•
|gx|+|gy|

max(gx,gy)
 Edge Detection
•
• A generic gradient based algorithm can be given as:
1. Read the image, smooth it
2. Convolve the image f with gx
3. Convolve the image f with gy
4. Compute edge magnitude and edge orientation
5. Compare edge magnitude with a threshold value. If the edge
magnitude is higher than the threshold, it is a possible edge point
Types of Edge Detectors

© Oxford University Press 2016. All rights reserved.

First-order Edge Detection Operators
 Roberts operator
Roberts kernel are derivatives wrt diagonal elements. They are also
called cross gradient operators and are mathematically given as:
 Prewitt
It uses central difference
of the neighboring pixels.
= f(x+1,y) – f(x-1,y)
2
This method is very sensitive to noise. Hence averaging is needed before
applying the Prewitt’s operator

© Oxford University Press 2016. All rights reserved.

 Sobel
It also uses central difference and can be viewed as an approximation of first order
Gaussian derivative.

© Oxford University Press 2016. All rights reserved.

© Oxford University Press 2016. All rights reserved.
Template matching

Template matching is a technique used in image processing to find

parts of an image that match a pattern or object (called a template).
Imagine you’re looking for a small piece of an image (like a letter “A”
or a face) inside a larger image — template matching helps you do
that.

A template matching mask is basically a small matrix (like 3×3,

5×5, etc.) that represents the pattern you’re looking for.
You slide this mask over the image, compare it to each region, and
measure how similar the mask is to that part of the image.
 Template Matching Masks: Kirsch
Template matching filters are direction sensitive filters
• Kirsch Masks are also called compass masks because they are obtained by taking
one mask and rotating it to 8 major directions.
• Each mask is applied to the image using convolution.
• Magnitude of the final edge is the maximum value of all the eight masks
• Direction is the direction associated with the mask that produces maximum
magnitude

© Oxford University Press 2016. All rights reserved.

• The Kirsch operator is an edge detection technique used in image
processing. It detects edges by looking in eight compass directions
(N, NE, E, SE, S, SW, W, NW) using special 3×3 masks.
© Oxford University Press 2016. All rights reserved.
© Oxford University Press 2016. All rights reserved.
Frei-Chen

Any image can be considered to be the

weighted sum of 9 Frei-Chen Masks.

• The first four masks represent edge space,

• The next four represent line subspace and
• The last one represent average subspace
 Robinson Compass
Feature / Aspect Kirsch Mask Frei-Chen Mask
Mask
Compass-based edge Compass-based edge Gradient-based edge
Type
detection detection detection
8 (N, NE, E, SE, S, SW, 8 (N, NE, E, SE, S, SW, 9 masks (8 for features
Number of Directions
W, NW) W, NW) + 1 for smoothing)
Mask Size 3×3 3×3 3×3
Values Used +5, 0, -3 +1, 0, -1 √2, 1, 0, -1, -√2
Strong emphasis on the Balanced, combines
Lighter emphasis than
Emphasis edge direction gradient & edge
Kirsch
(sharper) features
Directional Sensitivity High Moderate Moderate to High
Less smooth, more Smoothest among the
Smoothness Smoother than Kirsch
sensitive to noise three
Higher (due to stronger Slightly higher due to
Computational Cost Lower (simpler masks)
weights) square roots
Feature extraction and
Strong edge detection, General-purpose edge
Use Case noise-tolerant
especially noisy images detection
detection
Edge magnitude (max Edge magnitude (max Multiple outputs
Output Type
of 8 directions) of 8 directions) (edges + smoothing)
Second-order Derivative Filters
 Second-order Derivative Filters
•
 Second-order Derivative Filters

Laplacian masks are shown in the figure below.

The mask shown in

a) Is sensitive to horizontal and vertical edges
b) Is used to recognize diagonal edges
c) Is the addion of variants in (a) and (b)
d) Two times the mask in (a) subtracted to the mask in (b) yields the mask in the variant (d)
© Oxford University Press 2016. All rights reserved.
 Laplacian of Gaussian (Marr–Hildrith) operator
• Used to minimize the noise susceptibility of the Laplacian operator, Laplacian of
Gaussian (LoG) is used.
• The image is blurred using Gaussian operator
• Then the Laplacian operator is used
• The Gaussian operator reduces the noise and Laplacian detect the edges
 The Laplacian of Gaussian is then defined as

LoG = +

The LoG Algorithm:

1. Generate the mask and apply LoG to the image
Note: As σ increases, wider convolution masks are required for better performance
2. 2. Detect zero crossing
 Difference of Gaussian filter
The LoG filter can be approximated by taking two differently sized
Gaussians. The Difference of Gaussian (DoG) filter is given as :
 Difference of Gaussian filter

•
Feature / Aspect LoG (Laplacian of Gaussian) DoG (Difference of Gaussians)
Full Name Laplacian of Gaussian Difference of Gaussians
Approximation of LoG using
Type Second-order derivative operator
Gaussian filters
1. Smooth with Gaussian → 2. Apply 1. Blur image with two Gaussians
Process
Laplacian (σ₁, σ₂) → 2. Subtract
Detect edges using zero-crossings of Approximate LoG to detect edges
Purpose
second derivative efficiently
Zero-crossings in the difference of
Edge Detection Method Zero-crossings in the Laplacian
blurred images
Noise Sensitivity Low (due to initial Gaussian smoothing) Also low, depending on σ values
Higher (requires calculating second Lower (just Gaussian blurs and
Computational Cost
derivatives) subtraction)
Accuracy More accurate but slower Slightly less accurate but faster
Two Gaussian kernels with
Kernel Single custom LoG kernel
different σ values
Classical edge detectors, theory-based
Used In Efficient implementations like SIFT
methods
Feature detection, blob detection
Application Example Edge detection with precise localization
(e.g., scale-space in SIFT)
 Canny edge detection
The Canny approach is based on optimizing trade-off between two performance criteria
and can be described as follows:

© Oxford University Press 2016. All rights reserved.

 Algorithm
• The canny edge detection consists of 5 steps:
1. Noise reduction
• Convolve the image with the Gaussian filter.
2. Gradient calculation
• Compute the gradient of the resultant smooth image.
• Store the edge magnitude and edge orientation separately in two arrays,
M(x, y) and α (x,y), respectively.
3. Non Maximum suppression
• This is done to thin out the edges The algorithm finds the pixels with maximum
value in the edge direction
4. Double threshold
• High threshold identifies strong pixels
• Low threshold identifies non relevant pixels
• Pixels with intensity values between these thresholds are called weak pixels
5. Edge tracking by hysteresis
• Weak pixels are transformed to strong pixel if it is in the neighbourhood of strong
pixel
simple explanation can be found here:
https://youtu.be/hUC1uoigH6s
© Oxford University Press 2016. All rights reserved.
© Oxford University Press 2016. All rights reserved.
 Chapter 11

Image Morphology

© Oxford University Press 2016. All rights

reserved.
• Morphological operations are a set of image processing
techniques that process images based on their shapes. They are
primarily applied to binary images, though they can also be
extended to grayscale images.
• These operations are commonly used in computer vision and
image analysis tasks such as noise removal, image enhancement,
object segmentation, and shape analysis.
• Morphological operations are based on the mathematical
morphology framework, and they typically involve a structuring
element (a small shape or kernel) that probes the image.
 Need for morphological processing
• Mathematical morphology is a very powerful tool for analysing the
shapes of the objects present in images.
• Necessary for object recognition
• Morphological operators take a binary image and a mask known as a
structuring element as inputs.
• Then the set operators such as intersection, union, inclusion, and
complement can be applied to the images.

(A Ս B) Ս C = A Ս (B Ս C)
(A Ո B) Ո C = A Ո (B Ո C)
(A Ս B) Ո (A Ս C)
•
 Structuring Elements
• Structuring elements are small images that are used to probe the
original image.
• For basic morphological operators, the structuring elements have
ones, zeros and don’t cares. These elements select or supress the
features of a certain shape. Thus a morphological operator is defined
by its structuring element and the applied set operator.
 Structuring Elements
Some standard structuring elements are:
• Line segment
• It is characterized by the length and orientation
• Disk
• Can be described for digital grids
• Set of points
• Composite structuring elements
• Two structuring elements that share a common origin
• Elementary structured mask
• These are square grids. Smallest size is 2x2, but typically, it is 3x3.
• It has the origin in the center of the matrix.
• It is then shifted over the image, and at each pixel of the image, its elements
are compared with the set of underlying pixels.
• Fit: When all the pixels in the structuring element cover the
pixels of the object, we call it Fit.

• Hit: When at least one of the pixels in the structuring element

cover the pixels of the object, we call it Hit.

• Miss: When no pixel in the structuring element cover the pixels

of the object, we call it miss
 1 1

1 1 1 1 1

1 1 1 1

1 1 1 1

1 1 1 1
1
 MORPHOLOGICAL OPERATORS
• The basic morphological operators are
dilation and erosion.

© Oxford University Press 2016. All rights reserved.

 Dilation

© Oxford University Press 2016. All rights reserved.

 Dilation A + B
• Dilation expands the image pixels, or it adds pixels on object
boundaries.
• First, we traverse the structuring element over the image object to
perform an dilation operation.
• The output pixel values are calculated using the following equation.
Pixel (output) = 1 {if HIT}
Pixel (output) = 0 {otherwise
• Properties:
1. It can repair breaks .
2. It can repair intrusions
•
 Erosion

© Oxford University Press 2016. All rights reserved.

Erosion A - B

• Erosion removes pixels on object boundaries.

• We traverse the structuring element over the image object to perform
an erosion operationThe output pixel values are calculated using the
following equation.
Pixel (output) = 1 {if FIT}
Pixel (output) = 0 {otherwise}
1. It can split apart joint objects
2. It can strip away extrusions
Properties of dilation and erosion

© Oxford University Press 2016. All rights reserved.

Approaches to Dilation and Erosion operations

• Hit and Miss approach

• Convolution approach
Operation Meaning
“Does this whole shape (structuring element) fit
Erosion
here?” → If yes, keep it; else, erase it.
“Is any part of this shape present here?” → If yes,
Dilation
grow it; else, nothing changes.
Feature Erosion Dilation
Keeps a pixel white if all Sets a pixel white if any
Definition neighboring pixels under the neighboring pixel under
SE are white the SE is white
Effect on objects Shrinks white shapes Expands white shapes
Foreground pixels Decrease Increase
Background pixels Spread into the object Shrink
All 1s in SE must match 1s in At least one 1 in SE must
Binary rule
image match 1 in image
Noise removal Removes small white noise Fills small gaps
Can build new
Shape connection Can break narrow connections
connections
Opening/Closing

© Oxford University Press 2016. All rights reserved.

• The opening operation will smooth the object image,
break down the bridge, and remove small objects,
whereas the closing operation will create small breaks.
• In image processing, opening and closing are two
fundamental morphological operations used mainly on
binary images, though they can be extended to grayscale
images as well.
• They are particularly useful for noise removal, shape
smoothing, and image segmentation tasks.
 Opening Operation

•Removes small objects or noise from the foreground (typically

white pixels).
•Smoothens contours of objects.
•Breaks narrow thin connections between regions.
Closing Operation

Purpose:
• Fills small gaps in the foreground.
• Connects nearby objects.
• Smoothens contours from the outside.
Properties of Opening/Closing
Hit or miss Transform
• The hit-or-miss transform is a morphological operation
used to detect specific patterns in binary images by
comparing the image with two structuring elements.
• It works by checking if one structuring element (a "hit"
element) perfectly matches the foreground pixels and
another structuring element (a "miss" element) perfectly
matches the background pixels in a specific region of the
image.
• If both conditions are met, the pixel at the center of the
structuring elements is set to foreground, indicating a
match.
Hit or miss Transform

© Oxford University Press 2016. All rights reserved.

Applications of Hit and Miss Transform
Feature Hit-or-Miss Opening / Closing
Main goal Find exact shapes/patterns Clean up the image
What it does Detects a match Removes noise / fills gaps
Input 2 structuring elements 1 structuring element
Affects image? No (just highlights matches) Yes (modifies the image)
Use case Shape detection Smoothing / noise removal
Applications of Hit and Miss Transform