48

Image Segmentation
fundamentals in Digital Image
Processing
© Dr. Dafda
❖ Introduction
❑ What is Image Segmentation?
• Image segmentation divides an image into regions that are connected and
have some similarity within the region and some difference between
adjacent regions.
• The goal is usually to find individual objects in an image.
• For the most part there are fundamentally two kinds of approaches to
segmentation: discontinuity and similarity.
– Similarity may be due to pixel intensity, color or texture.
– Differences are sudden changes (discontinuities) in any of these, but
especially sudden changes in intensity along a boundary line, which is
called an edge.
❖ Image segmentation fundamentals
• Segmentation subdivides an image into its constituent regions or objects, until
the objects of interest in an application have been isolated.
• Segmentation partitions image R into subregions R1, R2, …, Rn such that:
✓ R1∪R2∪…∪Rn=R
✓ Each Ri is a connected set, i = 1,2,…..,n
✓ Ri∩Rj = Ø for all i, j where i # j
✓ Q(Ri)=TRUE for every i
✓ Q(Ri ∪ Rj)=FALSE for any two adjacent regions
• Segmentation problem: to partition the image into regions satisfying above
conditions.
❖ Two principal approaches:

❑ Edge-based Segmentation
- Partition an image based on abrupt
changes in intensity(edges).

❑ Region-based Segmentation
- Partition an image into regions that
are similar according to a set of pre-
defined criteria.
Thank You
49

Point, Line and Edge

detection in DIP and its
implementation in MATLAB
© Dr. Dafda
❑ Image Segmentation:

For the most part there are fundamentally two

kinds of approaches to segmentation:
discontinuity and similarity.
❖ Discontinuity
The strategy is to partition an image based on
abrupt changes in intensity
➢ Detection of gray level discontinuities:
(Point, Line & Edge) – Point detection
– Line detection
– Edge detection
• Gradient operators
• Gaussian based operators
❖ Similarity
The strategy is to partition an image into regions
that are similar according to a set of predefined
criteria.
❑ Detection of Discontinuities:
• Detect the three basic types of gray-level discontinuities
– points , lines , edges
• Use the image sharpening techniques
– The first order derivatives produce thicker edges
– The second order derivatives(Laplacian operation) have a strong response
to fine detail, such as thin lines and isolated points, and noise.

• Can be done by running a mask through the image

❑ Point detection:
• Steps for point detection:
1. Apply Laplacian filter to the image to obtain R(x, y).
2. Create binary image by threshold.

where T is a non-negative threshold.

❑ Line detection:
• A special mask is needed to detect a special type of line.
• Examples:
- Horizontal mask has high response when a line is passed through
the middle row of the mask.
❑ Edge detection:
• Edge detection is the approach for segmenting images based on
abrupt changes in intensity. It is used to detect the boundaries or to
find size or location of an object in an image.

• What is an edge?
- an edge is a set of connected pixels that lie on the boundary
between two regions.
- an edge is a “local” concept whereas a region boundary, owing
to the way, it is defined, is a more global idea.
❑ Edge models:
- Step edge (Ideal edge), ramp edge (thick edge), and roof edge
• An image may all the three types of edges.
 First and Second order derivatives response at the edge:

1. The magnitude of the first derivative can be used to detect the presence of an edge at a point.
2. Sign of second derivative indicates which side of the edge the pixel is on.
3. Zero crossings of the second derivative can be used to locate the centers of thick edges.
Second derivative produces two values for every edge in an image. An imaginary straight line
joining the extreme positive and negative values of the second derivative would cross zero near
the midpoint of the edge. Zero-crossing point is the center of thick edges.
 ❑ Steps in edge detection
1. Image smoothing for noise
reduction.
2. Detection of edge points (this
extracts from an image all points
that are potential candidates to
become edge points).
3. Edge localization (this selects from
candidate edge points only the
points that are true members of
the set of points comprising an
edge).
❑ Image Gradient (1st order derivatives)
❑ Gradient (1st order derivatives) Operators and Masks
Prewitt and Sobel masks for Diagonal edges
Example of use of Gradient
❖ Laplacian (Second order derivative) for edge detection
❖ Conclusions for Edge detections:

• Gradient based classical operators (first derivatives) are simple in

implementation and does fast computations.
• Robert operator is simplest but it is most sensitive to noise.
• The response of Sobel and Prewitt operators are almost similar, but still
Sobel produces better output due to more resistance against noise than
Prewitt.
• For noisy images, Laplacian of Gaussian and Canny operators (Second
derivatives) must be used.
Thank You
 50

Edge Linking and Boundary Detection,

Hough Transform and its
implementation in MATLAB
© Dr. Dafda
❖ Edge Linking
• An edge detection algorithm (Roberts, Sobel, Prewitt, LoG etc.) enhance
the edges. When implemented, there are normally breaks in lines. Due
to this reason, these are generally followed by linking procedures to
assemble edge pixels into meaningful edges.

• There are two basic approaches for edge linking:

(1) Local Processing. (This is a simplest approach for linking pixels in a small
neighborhood)
(2) Global Processing via the Hough Transform. (Here, we attempt to link edge
pixels that lie on specified curves. The Hough transform is designed to detect lines, using the
parametric representation of a line.)
❖ Local Processing
• Analyze the characteristics of pixels in ✓ pixels (s,t) and (x,y) are similar and linked if,
a small neighborhood Sxy (say, 3x3,
5x5) about every edge pixels (x, y) in
an image that have undergone edge
detection.
• All points that share some common
properties are linked together. These • Algorithm steps:
are:
a) Strength/magnitude of the gradient.
b) Direction of the gradient.

• This local processing is expensive. A record has to be kept of

all linked points by, for example, assigning a different label to
every set of linked points.
❖ Example of local processing:
❖ Hough Transform (Global Processing)
• One powerful global method for detecting edges is called the Hough transform.
• Let us suppose that we are looking for straight lines in an image.
• If we take a point (x',y') in the image, all lines which pass through that pixel have the form,
for varying values of m and c.

• However, this equation can also be written as,

where we now consider x' and y' to be constants, and m and c as varying.
❖ Hough Transform (Global Processing)
• This is a straight line on a graph of c against m as shown in Fig. below.

• Each different line through the point (x',y') corresponds to one of the points on
the line in (m,c) space.
❖ Hough Transform (Global Processing)
• Now consider two pixels p and q in (x,y) space which lie on the same line.
• For each pixel, all of the possible lines through it are represented by a single
line in (m,c) space.

• Thus the single line in (x,y) space which goes through both pixels lies on the
intersection of the two lines representing p and q in (m,c) space, as shown in
figure. If p and q are the points connected by a line in the spatial domain,
then they will be intersecting lines in the Hough Space (m-c plane).
❖ Hough Transform (Global Processing)
• Taking this one step further, all pixels which lie on the same line in (x,y) space
are represented by lines which all pass through a single point in (m,c) space.
• The single point through which they all pass gives the values of m and c in the
equation of the line y=mx+c.
To detect straight lines in an image, we do:
1. Quantize (m,c) space into a two-dimensional array A for appropriate steps
of m and c.
2. Initialize all elements of A(m,c) to zero.
3. For each pixel (x',y') which lies on some edge in the image, we add 1 to all elements
of A(m,c) whose indices m and c satisfy y'=mx'+c.
4. Search for elements of A(m,c) which have large values. Each one found corresponds
to a line in the original image.
• One useful property of the Hough transform is that the pixels which lie on the
line need not all be contiguous.
❖ Hough Transform (Global Processing)
• For example, all of the pixels lying on the two dotted lines in below Fig.
will be recognized as lying on the same straight line.

• This can be very useful when trying to detect lines with short breaks in them
due to noise, or when objects are partially obstructed as shown.
❖ Hough Transform (Global Processing)

• On the other hand, it can also give misleading results when objects happen to be
aligned by chance, as shown by the two dotted lines in Fig. below,

• Indeed, this clearly shows that one disadvantage of the Hough transform method
is that it gives an infinite line as expressed by the pair of m and c values, rather
than a finite line segment with two well-defined endpoints.
• One practical detail is that the y =mx + c form for representing a straight line
breaks down for vertical lines, when m becomes infinite.
❖ Hough Transform (Global Processing)
• To avoid this problem, it is better to use the alternative formulation given earlier,

as a means of describing straight lines.

• Note, however, that a point in (x,y) space is now represented by a curve
in space rather than a straight line. Otherwise, the method is unchanged.
• The Hough transform can be used to detect other shapes in an image as well as
straight lines.
❖ Hough Transform (Global Processing)
• We attempt to link edge pixels that lie on specified curves. The Hough transform
is designed to detect lines, using the parametric representation of a line.
• Hough Transforms takes the images created by the edge detection operators.
• Hough Transform can be used to connect the disjointed edge points.
• It is used to fit the points as plane curves.
• Plane curves are lines, circles, and parabolas.
• The line equation is y = mx + c
• However, the problem is that there are infinite line passing through one point.
• Therefore, an edge point in an x-y plane is transformed to a m-c plane.
• Now equation of line is c = (-x)m + y
❖ Hough Transform (Global Processing)
• All the edge points in the x-y plane need to be fitted.
• All the points are transformed in m-c plane.
• The objective is to find the intersection point.
• A common intersection point indicates that the edges points which are part of the line.
• If A and B are the points connected by a line in the spatial domain, then they will be
intersecting lines in the Hough Space (m-c plane).
• To check whether they are intersecting lines, the m-c plane is partitioned as
accumulator lines.
• To find this, it can be assumed that the c-m plane can be partitioned as an accumulator
array.
• For every edge point (x,y), the corresponding accumulator element is incremented in
the accumulator array.
• At the end of this process, the accumulator values are checked.
• Significance is that this point gives us the line equation of the (x,y) space.
❖ Hough Transform Steps
1) Load the image
2) Find the edges of the image using any edge detector
3) Quantize the parameter space P
4) Repeat the following for all the pixels of the image:
if the pixel is an edge pixel, then
(a) c = (-x)m + y or calculate ρ
(b) P(c,m) = P(c,m) + 1 or increment position in P
5) Show the Hough Space
6) Find the local maxima in the parameter space
7) Draw the line using the local maxima
• The major problem with this algorithm is that it does not work for vertical lines, as they
have a slope of infinity.
• Solution is, convert line into polar coordinates ρ = x cosӨ + ysinӨ, where Ө is the angle
between the line and x-axis, and ρ is the diameter.
Thank You
51

Thresholding : Global and

Adaptive(local) in DIP and its
implementation in MATLAB
© Dr. Dafda
❑ Image Segmentation:

For the most part there are fundamentally two

kinds of approaches to segmentation:
discontinuity and similarity.
❖ Discontinuity
The strategy is to partition an image based on
abrupt changes in intensity
➢ Detection of gray level discontinuities:
(Point, Line & Edge) – Point detection
– Line detection
– Edge detection
• Gradient operators
• Gaussian based operators
❖ Similarity
The strategy is to partition an image into regions
that are similar according to a set of predefined
criteria.
Image Segmentation
Here the computer tries to
separate objects from the image.
• The spatial domain processes can be denoted by the expression, g(x, y) = T[ f(x, y)], where f(x, y) is the input image,
g(x, y) is the output image, and T is an operator on f defined over a neighborhood of point (x, y).

• The smallest possible neighborhood is of size 1 x 1. Here, g depends only on the value of f at a single point (x, y)
and T becomes s = T(r), where, for simplicity in notation, s and r are variables denoting, respectively, the
intensity of g and f at any point (x, y).

Contrast stretching Thresholding

T function T function
Image with a 3 X 3 spatial mask
❖ Segmentation by thresholding
• Thresholding is the simplest segmentation method. Thresholding is used to
produce regions of similarity within the given image, based on some threshold
criteria T. Hence it partitions/segments an image into different objects.
• The pixels are partitioned depending on their intensity value.
• Single thresholding:
g(x, y) = 1, if f (x, y) > T
0, if f (x, y) ≤ T
• Multiple thresholding:
g(x, y) = a, if f (x, y) > T2
b, if T1 < f (x, y) ≤ T2
c, if f (x, y) ≤ T1
❖ Thresholding types:
• Thresholding operation can be thought of as an operation, such that,
T=T[x, y, p(x, y), f(x, y)], where f(x, y) is the gray level of input pixel at (x, y) and
p(x, y) denotes some local property of this point (x, y), e.g. the average level of a
neighborhood centered on (x, y).
• If the thresholding operation depends only on the gray scale value, it is called
global thresholding. If the neighborhood property is also taken into account, it is
called local thresholding. If T depends on pixel coordinates also, T is called
dynamic/adaptive thresholding. Thresholding is called adaptive when a different
threshold is used for different regions in the image.
Thresholding

Global Local
T = T [ f(x, y) ] T=T [ p(x, y), f(x, y) ]
❖ How to select T ? OR How to choose Threshold value T ?
• Assuming that the background and the object occupy comparable
areas in the image, a good initial value of T is the average gray level of
the image.
1. Select an initial estimate for T.
2. Segment the image into two group of pixels G1 and G2 using T.
3. Compute the average gray level values of G1 and G2, say the values
obtained are μ1 and μ2.
4. Compute a new threshold value as Tnew=(μ1+μ2)/2,
5. Compare if |Tnew-T|>z0 ( predefined threshold z0) then T=Tnew and go
to step 2, else stop
❖ Basic Global Thresholding
• Global thresholding having a single threshold (T) is used when the objects are
easily differentiated from each other.
❖ The Role of Illumination
❖ Adaptive (Local) thresholding

• Global thresholding often fails

in the case of uneven
illumination.

• The solution is to divide the

image into sub-images, and
determine T for each sub-
image. This method is known
as adaptive(local) thresholding.
❖ Adaptive (Local) thresholding
❖ Basic Adaptive Thresholding:
• Subdivide original image into
small areas.
• Utilize a different threshold to
segment each sub-image.
• As the threshold used for each
pixel depends on the location of
the pixel in terms of the sub-
images, this type of thresholding
is adaptive(local).
• Further subdivision can
improve the quality of
adaptive thresholding.

• But this comes at the cost

of additional
computational
complexity.
Thank You
52

Region-Based Segmentation
in DIP and its implementation in
MATLAB
© Dr. Dafda
❑ Image Segmentation:

(Point, Line & Edge)

❖ Why to use Region based segmentation ?
• Fig. 10.41 shows the histogram of Fig. 10.40 (a). It is difficult to segment
the defects by thresholding methods. (Applying region growing methods
are better in this case.)
❖ Region-Based Segmentation-Basic Formulation
• Let R represent the entire image region.
• Segmentation is a process that partitions R into subregions, R1,R2,…,Rn,
such that

• where P(Rk): a logical predicate defined over the points in set Rk.
• For example: P(Rk)=TRUE if all pixels in Rk have the same gray level.
❖ Region-Growing
• Edges and thresholds sometimes do not give good results for segmentation.
Thresholding still produces isolated image.
• Region growing algorithms works on principle of similarity.
• It states that a region is coherent if all the pixels of that region are
homogeneous with respect to some characteristics such as colour, intensity,
texture, or other statistical properties.
• Thus idea is to pick a pixel inside a region of interest as a starting point (also
known as a seed point) and allowing it to grow.
• Seed point is compared with its neighbours, and if the properties match , they
are merged together.
• This process is repeated till the regions converge to an extent that no further
merging is possible.
❖ Region-Growing Algorithm
• It is a process of grouping the pixels or subregions to get a bigger
region present in an image.
• Selection of the initial seed: Initial seed that represent the ROI
should be given typically by the user. Can be chosen automatically.
The seeds can be either single or multiple.
• Seed growing criteria: Similarity criterion denotes the minimum
difference in the grey levels or the average of the set of pixels. Thus,
the initial seed ‘grows’ by adding the neighbours if they share the
same properties as the initial seed.
• Terminate process: If further growing is not possible then terminate
region growing process.
❖ Region-Growing Example
• Consider image shown in figure:

• Assume seed point indicated by underlines. Let the seed pixels 1

and 9 represent the regions C and D, respectively.
• Subtract pixel from seed value.
• If the difference is less than or equal to 4 (i.e. T=4), merge the pixel
with that region. Otherwise, merge the pixel with the other region.
❖ Region Splitting
• Entire image is assumed as a single region. Then the
homogeneity(similarity) test is applied, where pixels that are similar are
grouped together. If the conditions are not met, then the regions are split
into four quadrants, else leave the region as it is.

• Split and continue the subdivision process until some stopping criteria is
fulfilled. The stopping criteria often occur at a stage where no further
splitting is possible.

• This process is repeated for each quadrant until all the regions meet the
required homogeneity criteria. If the regions are too small, then the
division process is stopped.
❖ Region Splitting
• To explain this in terms of graph theory, we call each region a node.
• This technique has a convenient representation in the form of a
quadtree structure.
• Quadtree: a tree in which nodes have exactly four descendants.
❖ Region Merging
• Region merging is opposite to region splitting.
• Here we start from the pixel level and consider each of them as a
homogeneous region.
• At any level of merging, we check if the four adjacent regions satisfy the
homogeneity property. If yes, they are merged to form a bigger region,
otherwise the regions are left as they are.
• This is repeated until no further region exits that requires merging.
❖ Region Splitting and Merging
• Splitting or merging might not produce good results when applied
separately. Better results can be obtained by interleaving merge and
split operations.
• The split and merge procedure is as follows:
• First there is a large region (possible the entire image).
a) Split into four disjoint quadrants any region Ri for which P(Ri) =
FALSE.
b) Merge any adjacent regions Rj and Rk for which P(Rj U Rk) =
TRUE. (the quadtree structure may not be preserved).
c) Stop when no further merging or splitting is possible.
❖ Region Splitting and Merging Example
Thank You