Image

Segmentation
 Image Segmentation

 Image segmentation divides an image into regions that are connected

and have some similarity within the region and some difference between
adjacent regions.
 The goal is usually to find individual objects in an image.
 For the most part there are fundamentally two kinds of approaches to
segmentation: discontinuity and similarity.
– Similarity may be due to pixel intensity, color or texture.
– Differences are sudden changes (discontinuities) in any of these, but especially sudden
changes in intensity along a boundary line, which is called an edge.
Fundamentals
 Detection of Discontinuities
 There are three kinds of discontinuities of intensity: points, lines and edges.
Detection of Discontinuities
Detection of Discontinuities
 The most common way to look for discontinuities is to scan a small mask over the
image. The mask determines which kind of discontinuity to look for.
Detection of Discontinuities
Point Detection
Detection of Discontinuities
Point Detection
 Detection of Discontinuities
Line Detection
 Only slightly more common than point detection is to find a one pixel wide line in
an image.
Detection of Discontinuities
Line Detection
 Detection of Discontinuities
Line Detection
 For digital images the only three point straight lines are only horizontal, vertical, or
diagonal (+ or –45°).
Detection of Discontinuities
Line Detection
Detection of Discontinuities
Edge Detection
Detection of Discontinuities
Edge Detection
Detection of Discontinuities
Edge Detection
Detection of Discontinuities
Edge Detection
 Detection of Discontinuities
Gradient Operators
 First-order derivatives:
– The gradient of an image f(x,y) at location (x,y) is defined as the vector:

– The magnitude of this vector:

– The direction of this vector:

– It points in the direction of the greatest rate of change of f at location (x,y)

Detection of Discontinuities
Gradient Operators
Different Edge masks
Different Edge masks
Different Edge masks
Different Edge masks
Different Edge masks
 Detection of Discontinuities
Gradient Operators: Example
 Detection of Discontinuities
Gradient Operators: Example
 Detection of Discontinuities
Gradient Operators: Example
 Detection of Discontinuities
Gradient Operators: Example
 Advanced Techniques for edge
detection
Marr-Hildreth edge detector

• It takes into account factors such as image noise and the nature of edges.

• Time period – 1980, MIT

• During that time: edge detection was mainly based on small operators such as the Sobel,
Robert, Prewitt masks
 Marr-Hildreth edge detector

 Marr and Hildreth argued that

1) Intensity changes are dependent of image scale and so their detection requires the
use of operators of different sizes and
2) That a sudden intensity change will give rise to a peak or trough in the first derivative
or, equivalently, to zero crossing in the second derivative.

Two salient feature:

1)It should be differential operator capable of computing first or second order derivatives
at every point in an image
2)It should be capable of being tuned to act at any desired scale, so that large operator
can be used to detect blurry edges and small operators to detect sharply focused fine
detail
 Marr-Hildreth edge detector

• Marr and Hildreth argued that the most satisfactory operator is the Laplacian of a
Gaussian (LoG)

2 2
𝜕 𝜕
∇2 = 2 + 2
𝜕𝑥 𝜕𝑦

𝑥 2 +𝑦 2
−
𝐺(𝑥, 𝑦) = 𝑒 2𝜎2
 Marr-Hildreth edge detector

2
𝑥 + 𝑦 − 2𝜎 2 2 𝑥 2 +𝑦 2
2 −
∇ 𝐺(𝑥, 𝑦) = 𝑒 2𝜎2
𝜎4

This is known as LoG operator

Marr-Hildreth edge detector
 Marr-Hildreth edge detector

Fundamental ideas:
• the Gaussian part blurs the image, reducing structures/noise

• unlike average masks, Gaussian filter is smooth in both the spatial and frequency
domains

• less artifacts are introduced

 Marr-Hildreth edge detector

Fundamental ideas:
• second part: the Laplacian part is isotropic, invariant to rotation
• no need to use multiple masks like first derivative cases, since isotropic
 Marr-Hildreth edge detector

• Finally, the edge detection is performed as

2
𝑔(𝑥, 𝑦) = ∇ 𝐺(𝑥, 𝑦) ∗ 𝑓(𝑥, 𝑦)

𝑔(𝑥, 𝑦) = ∇2 𝐺(𝑥, 𝑦) ∗ 𝑓(𝑥, 𝑦)

The locations of edges is found by the zero-crossings of g (x, y).
 Marr-Hildreth edge detector

Algorithm:
1. Filter the input image with an nxn Gaussian lowpass filter
2. Compute the Laplacian of the image resulting from Step 1.
3. Find the zero crossings of the image from Step 2.
 Marr-Hildreth edge detector

• Size of the mask: about 99.7% of the volume under a 2-D Gaussian surface lies
between ±3𝜎 about the mean.

• The value of n should be the smallest odd integer greater than or equal to 6 𝜎
 Marr-Hildreth edge detector

How to find the zero crossings:

• Consider a 3x3 neighborhood centered at any pixel p.
• A zero crossing at p implies that the signs of at least two of its opposing neighboring
pixels must differ (out of four cases).

• Sometimes, the absolute value of opposing neighbor’s numerical difference must also
exceed the threshold.
Marr-Hildreth edge detector
 Marr-Hildreth edge detector

• It is possible to approximate the LoG filter by a difference of Gaussians (DoG)

𝑥 2 +𝑦 2 𝑥 2 +𝑦 2
1 −
2𝜎 2 1 −
2𝜎 2
𝐷𝑜𝐺(𝑥, 𝑦) = 𝑒 1 − 𝑒 2
2𝜋𝜎12 2𝜋𝜎22
𝜎1 > 𝜎2
The value of σ for the LoG
Marr-Hildreth edge detector
 Marr-Hildreth edge detector

• Gives false edges

• Poor localization at curved edges

• Now of historical interest

• Canny edge detector gives better performance

 Canny edge detector

• Best edge detection method

• Increase in implementation complexity

 Canny edge detector

• Canny’s approach based on three basic objectives

• Low error rate – all edges should be found, no false edges
• Edge points should be well localized
• Single edge point response – only one point for each true edge
point/Minimize no. of false maxima around true edge.
 Canny edge detector

• First smoothing the image with a circular 2-D Gaussian function, then computing
the gradient image 𝑥 2 +𝑦 2
−
𝐺(𝑥, 𝑦) = 𝑒 2𝜎2
𝑓𝑠 (𝑥, 𝑦) = 𝐺(𝑥, 𝑦) ∗ 𝑓(𝑥, 𝑦)

𝑀(𝑥, 𝑦) = 𝑔𝑥2 + 𝑔𝑦2

𝑔𝑦
𝛼(𝑥, 𝑦) = tan−1
𝑔𝑥
𝜕𝑓𝑠
𝑤ℎ𝑒𝑟𝑒 𝑔𝑥 =
𝜕𝑥
𝜕𝑓𝑠
𝑔𝑦 =
𝜕𝑦
 Canny edge detector: Single
response requirement
• Since M(x, y) is generated using the gradient, it contains wide ridges
around local maxima
• Next step is to thin those ridges, using Non-maxima suppression
• Specify a number of discrete orientations of the edge normal (gradient
vector)
• In a 3x3 region, we can define four orientations, horizontal, vertical,
+45, -45
Canny edge detector
 Canny edge detector
• Let d1, d2, d3, and d4 denote four basic edge normal directions
for a 3x3 region
• Find the direction dk that is closet to 𝛼(𝑥, 𝑦)

• If the value of M(x, y) is less than at least one of its two

neighbors along dk,
gN(x, y)=0 (suppression);
otherwise let gN(x, y)=M(x, y)
 Canny edge detector: Post Processing

• Final operation is to threshold gN(x, y) to reduce false edge points

• Using Hysteresis Thresholding – low and high thresholds

• The ratio of high to low threshold should be 2:1 or 3:1
 Canny edge detector

𝑔𝑁𝐻 (𝑥, 𝑦) = 𝑔𝑁 (𝑥, 𝑦) ≥ 𝑇𝐻 Strong edge

𝑔𝑁𝐿 (𝑥, 𝑦) = 𝑔𝑁 (𝑥, 𝑦) ≥ 𝑇𝐿 Weak edge

All the nonzero pixels of former image will be contained in the later image

𝑔𝑁𝐿 (𝑥, 𝑦) = 𝑔𝑁𝐿 (𝑥, 𝑦) − 𝑔𝑁𝐻 (𝑥, 𝑦)

 Canny edge detector
• After thresholding operations, all strong edge pixels in 𝑔𝑁𝐻 (𝑥, 𝑦)
are marked as valid edge pixels

• These edge pixels have gaps (broken edges); so longer edges are
formed using the next procedure
 Canny edge detector
• Locate the next unvisited edge pixel p in 𝑔𝑁𝐻 (𝑥, 𝑦)

•All the weak edge pixels in 𝑔𝑁𝐿 (𝑥, 𝑦) that are 8-connected to p, are
marked as valid edge pixels

• If all non-zero pixels in 𝑔𝑁𝐻 (𝑥, 𝑦) have been visited go to next step, else
go to step 1.
 Canny edge detector
• Set rest weak edge pixels in 𝑔𝑁𝐿 (𝑥, 𝑦) that were not marked as valid
edge pixels to zero

• Final edge image is formed by appending to 𝑔𝑁𝐿 (𝑥, 𝑦) all the nonzero
pixels from 𝑔𝑁𝐻 (𝑥, 𝑦)
 Canny edge detector
Summary
• Smooth the input image with a Gaussian filter
• Compute the gradient magnitude and angle images
• Apply Non-maxima suppression to the gradient image
• Use Double thresholding (Hysteresis Thresholding) and connectivity
analysis to detect and link edges
Canny edge detector
Canny edge detector
 Edge Linking

 Edge detection should yield sets of pixels lying only on edges.

 These pixels seldom characterize edges completely because of noise,
breaks in the edges due to nonuniform illumination, and other effects that
introduce spurious discontinuities in intensity values.
 Edge detection typically is followed by linking algorithms designed to
assemble edge pixels into meaningful edges and/or region boundaries.
Edge Linking using local Processing
 Edge Linking using local Processing
 Simplified Approach: A simplification particularly well suited for real time applications
consists of the following steps
❑ Compute the gradient magnitude and angle arrays, M(x, y) and α(x, y), of the input
image, f (x, y).
❑ Form a binary image, g(x, y), whose value at any point (x, y) is given by:

where TM is a threshold, A is a specified angle direction, and ±TA defines a “band” of

acceptable directions about A.
❑ Scan the rows of g and fill all gaps in each row that do not exceed a specified length, K.
❑ To detect gaps in any other direction, θ, rotate g by this angle and apply the horizontal
scanning procedure in Step 3. Rotate the result back by −θ.
Edge Linking using local Processing
 Global Processing using
Hough Transforms
 Hough Transforms takes the images created by the edge detection operators
 Most of the time, the edge map generated by the edge detection algorithms is
disconnected
 HT can be used to connect the disjointed edge points
 It is used to fit the points as plane curves
 Plane curves are lines, circles, and parabolas

 The line equation is y = mx + c

 However, the problem is that there are infinite line passing through one points
 Therefore, an edge point in an x-y plane is transformed to a c-m plane
 Now equation of line is c = (-x)m + y
Hough Transforms
 Hough Transforms
 All the edge points in the x-y plane need to be fitted
 All the points are transformed in c-m plane
 The objective is to find the intersection point
 A common intersection point indicates that the edges points which are part of the line
 If A and B are the points connected by a line in the spatial domain, then they will be
intersecting lines in the Hough Space (parameter plane)
 The principle lines in the x-y plane can be found by identifying points in parameter
space where large numbers of parameter-space lines intersect.
Hough Transforms
Hough Transforms
Hough Transforms
 Thresholding
 Assumption: the range of intensity levels covered by objects of interest is different from
the background.
 Thresholding
The Role of Noise
Image Smoothing prior to
Thresholding
 Thresholding
The Role of Illumination
Basic Global Thresholding
Iterative Global thresholding
Image Partitioning
 Region-Based Segmentation

• Edges and thresholds sometimes do not give good results for segmentation.
• Region-based segmentation is based on the connectivity of similar pixels in a region.
– Each region must be uniform.
– Connectivity of the pixels within the region is very important.
• There are two main approaches to region-based segmentation: region growing and
region splitting.
 Region Growing
 Thresholding still produces isolated image
 Region growing algorithms works on principle of similarity
 It states that a region is coherent if all the pixels of that region are homogeneous with
respect to some characteristics such as color, intensity, texture, or other statistical
properties
 Thus idea is to pick a pixel inside a region of interest as a starting point (also known as a
seed point) and allowing it to grow
 Seed point is compared with its neighbors, and if the properties match , they are
merged together
 This process is repeated till the regions converge to an extent that no further merging is
possible
 Region Growing Algorithm

 It is a process of grouping the pixels or subregions to get a bigger region present in an

image
 Selection of the initial seed: Initial seed that represent the ROI should be given typically
by the user. Can be chosen automatically. The seeds can be either single or multiple
 Seed growing criteria: Similarity criterion denotes the minimum difference in the grey
levels or the average of the set of pixels. Thus, the initial seed ‘grows’ by adding the
neighbors if they share the same properties as the initial seed
 Terminate process: If further growing is not possible then terminate region growing
process
Region Splitting and Merging
Region Splitting and Merging