MODULE-3

PART-I

Representation & Description

Ms. Neethu Radha Gopan, Asst. Prof., Dept. of ECE,

Rajagiri School of Engineering & Technology, Kakkanad
Syllabus
Representation

 After an image has been segmented into regions, it is usually represented and
described in a form suitable for further computer processing.
 Representing a region involves 2 choices - We can represent the region in terms of its
(i) external characteristics (boundary).
(ii) internal characteristics (the pixels comprising the region).
 An external representation is chosen when the primary focus is on shape
characteristics.
 An internal representation is chosen when the primary focus is on region properties
like color & texture.
 After representation, next we describe the region based on the chosen representation.
Representation Approaches:
Boundary Representation
1. Boundary (Border) Following

 Most of the algorithms require that the points in the boundary of a region be
ordered in a clockwise or anti clockwise direction.
 Hence we go for a boundary algorithm whose output is an ordered sequence of
points.
 Also known as Moore boundary tracking.
 Assume a binary image in which object & background points are labelled 1 & 0
respectively.
 Given a binary region R or its boundary, an algorithm for following the border of
R is given as follows:
1.
   Letthe starting point be the uppermost left most point in the image that is
labelled 1. Denote as the west neighbor of . Clearly is always a background
point. Examine the 8 neighbors of , starting at and proceeding in a clockwise
direction. Let denote the first neighbor encountered whose value is 1 and let be
the (background) point immediately preceding in the sequence. Store the
locations of & for use in step 5.
2. Let b = and c = .
3. Let the 8 neighbors of b, starting at c and proceeding in a clockwise direction be
denoted by Find the first labelled 1.
4. Let b = and c =
5. Repeat steps 3 & 4 until b = and the next boundary point found is The sequence
of b points found when the algorithm stops constitute the set of ordered boundary
points.
  𝒄 𝟏
 𝒃𝟏   ,

 Starting Point Background Point

Set of ordered
Boundary Points
2. Chain Codes

 Chain codes are used to represent a

boundary by a connected sequence
of straight line segments of a
specified length and direction.
 Based on 4 or 8 adjacency
 The direction of each segment is
coded by using a numbering
scheme.
 A boundary formed as a sequence
of such directional numbers is
referred to as a Freeman chain code.
 Chain codes can be generated by following a
boundary in the clockwise direction and
assigning a direction to the segments
connecting every pair of pixels

 This method is unacceptable due to 2 reasons:

1. The resulting chain tends to quite long.

2. Any small disturbances along the boundary
due to noise or imperfect segmentation causes
changes in the code that may not be related to
the principal shape feature of the boundary.
 This can be overcome by resampling the
boundary by selecting a larger grid spacing.
 The code can be normalized wrt the starting point as follows:
1. Treat the chain code as a circular sequence of direction numbers and redefine the
starting point so that the resulting sequence of numbers forms an integer of
minimum magnitude.
We can also normalize for rotation by using the first difference of the chain code
instead of the code itself.
 This difference is obtained by counting the number of direction changes in the
counter clockwise direction that separate two adjacent elements of the code.
 For eg: the first difference of the 4 direction chain code 10103322 is 3133030.
 If we treat the code as a circular sequence, then the first element of difference is
computed by using the transition between the last and first components of the
chain. Hence the result is 33133030
3. Polygonal approximation

 A digital boundary can be approximated with arbitrary accuracy by a polygon.

 Captures the essence of the boundary shape with the fewest possible polygon
segments.
 For a closed boundary, the approximation becomes exact when the number of
segments of the polygon is equal to the number of points in the boundary
 Minimum – Perimeter Polygon (MPP) is the most powerful in representing a
boundary.
Foundation:
 The idea is to enclose a boundary (think of it as a rubber band) by a set of
concatenating cells.
 The boundary is allowed to shrink but it is constrained by the inner and outer
walls of the bounding region defined by the cells.
 Ultimately, the shape shrinking provides the MPP.
 The vertices of the MPP coincide with corners of the inner or the outer wall.
 The shape of the object enclosed by the inner wall of the light gray cells is
shown in dark gray.
 Traversing the boundary (counterclockwise) encounters convex (white dots)
or concave (black dots) vertices.
 The vertices of the MPP coincide either with convex vertices in the inner
wall or with the “mirrors” of the concave vertices in the outer wall.
MPP algorithm
1.
   Form a list of coordinates of each vertex.
2. Label each vertex as W (convex) or B (concave).
3. Form list of the mirrors of B vertices.
4. Vertices must be in sequential order.
5. The first vertex is the uppermost leftmost vertex. It is always a W .
6. The algorithm uses a white crawler and a black crawler crawling along the convex
(W) and mirrored concave (B) vertices respectively.
7. Initialization: = = .
8. is the last vertex examined & is the current vertex being examined.

 One
  of the 3 conditions can exist between , & the 2 crawler points.
Case 1: sgn( , , ) > 0 , lies to the positive side of the line through ( , ).
Case 2: sgn( , , ) ≤ 0, lies to the negative side of the line through ( , )
or is collinear with it and sgn( , , ) ≥ 0 , lies to the positive side of the line
through ( , ) or is collinear with it .
Case 3: sgn( , , ) < 0, lies to the negative side of the line through ( , ).
 If condition 1 holds, the next MPP vertex is & we let = . Then reinitialize the algorithm
by setting = = and continue with the next vertex after .
 If the condition 2 holds, becomes a candidate MPP vertex. We set if is convex, otherwise
we set = . Then continue with the next vertex in the list.
 If condition 3 holds, the next MPP vertex is & we let = . Then reinitialize the algorithm
by setting = = and continue with the next vertex after .
4. Boundary Segments

 Decomposing a boundary into segments

 Decomposition reduces boundary’s complexity, thus simplifies description process.
 Useful when the boundary contains one or more concavities, that carry shape
information.
 Convex Hull of the region enclosed by the boundary is a powerful tool.
 The convex hull of an arbitrary set S is the smallest convex set containing S.
 The set difference H-S is called the convex deficiency D of the set S.
 The figure shows an object (set S) and its convex deficiency (shaded region).
 The region boundary can be partitioned by following the contour of S and marking the
points at which a transition is made into or out of a component of the convex deficiency.
a) A region S and its convex deficiency b) Partitioned Boundary
Boundary Descriptors


 There
  are several methods to describe the boundary of a region.
1. Some Simple Descriptors
 Length : The length of a boundary is one of the simplest descriptors. The number of
pixels along a boundary gives a rough approximation of its length.
 Diameter : The diameter of a boundary B is defines as
, where D is a distance measure and are points
on the boundary.
 Major Axis : The largest diameter is the major axis.
 Minor Axis : The minor axis of a boundary is defined as the line perpendicular to the
major axis.
 Basic Rectangle : The box consisting of major & minor axes with points
intersecting the border forms a basic rectangle.
 Eccentricity : The ratio of the major to the minor axis is called eccentricity of the
boundary and is also a useful descriptor.
 Curvature defined as the rate of change of slope of boundary segments is another
descriptor.
2. Shape Numbers

 The first difference of a chain coded boundary depends on the starting point.
 First differences are computed by treating the chain as a circular sequence.
 The shape number of such a boundary, based on the 4 directional code is defined as the
first difference of smallest magnitude.
 The order n of a shape number is defined as the number of digits in its representation.
 n is even for a closed boundary.
 Eg: All closed shapes of order n=4, 6 and 8 are shown in figure.
Generation of Shape Number

 Figure 1 represents a boundary.

 Suppose n= 18 is specified for the boundary.
 To obtain the shape number of the boundary, find a basic rectangle that encloses the
boundary.
 The closest rectangle of order n= 18 is a 3×6 rectangle with the subdivision shown.
 Align the chain code directions with the grid.
 Final step is to obtain the chain code and use its first difference to compute the shape
number.
3. Fourier Descriptors


 Consider
  a K-point digital boundary in the x-y plane.
 Starting at an arbitarary point traversing the boundary in the counter clockwise
direction, we encounter the co-ordinates , ,….. .
 The boundary can be represented as a sequence of co-ordinates , k=0,1,…K-1.
 s(k) can be represented as a complex number and the DFT is computed.

  𝐾 −1 𝑗2 𝜋 𝑢
− 𝑘
𝐾
𝑎 ( 𝑢 ) = ∑ 𝑠(𝑘)𝑒 , 𝑢=𝑜 , 1 , … . 𝐾 −1
𝑘 =0
 The complex coefficients a(u) are called the Fourier descriptors of the boundary
 The inverse Fourier transform of these coefficients restores s(k).
  𝐾 −1 𝑗 2 𝜋 𝑢𝑘
1 𝐾
𝑠 (𝑘 )= ∑ 𝑎 (𝑢)𝑒 , 𝑘=𝑜 , 1 ,… . 𝐾 −1
𝐾 𝑢 =0
 The table summarizes the Fourier descriptors for a boundary sequence s(k) that
undergoes rotation, translation, scaling & changes in starting point.
4. Statistical Moments


 The
  shape of boundary segments can be described using statistical moments like mean,
variance & higher order moments.
 A boundary segment may be represented as a 1D function g(r) by connecting the end
points and rotating the line segment to be horizontal.
 We can treat the amplitude of g as a discrete random variable ‘v’ representing discrete
amplitude in the range [0, A-1] then the statistical moment of v (about its mean) is
calculated as:

 An
  alternative approach is to normalize g(r) to unit area and represent it as a
histogram.
 Now g() is the probability of value .
 Here r is treated as the random variable and moments are given by
Regional Descriptors

1.
  Some Simple Descriptors
 Area of the region - number of pixels in the region.
 Perimeter of region - length of its boundary
 Compactness -
 Circularity Ratio – Ratio of area of the region to the area of a circle having the same
perimeter.
 Other regional descriptors include statistics on the intensity levels of the pixels in the
region
– Mean & median of the intensity values
– Minimum & Maximum intensity value
– Number of pixels with values above or below the mean
2. Topological Descriptors

 Useful for global descriptions of regions in the image

plane.
 Topology is the study of properties of a figure that are
unaffected by any deformation.
 These properties do not depend on any distance measure.
 Different topological descriptors:
1. Holes in a region
2. Number of Connected Components
3. The number of holes (H) and connected components
(C) in a figure is used to define the Euler number E
E = C-H
 Regions represented by straight line segments have a simple interpretation in terms
of the Euler number.
 Figure shows a polygonal network.
 The number of vertices (V) and the number of edges (Q) and the number of faces
(F) give the Euler formula: V – Q + F = C – H = E
3. Texture Descriptors


 Texture
  provides measure of properties such as smoothness, coarseness & regularity.
 3 principal approaches used to describe texture of a region - statistical, structural &
spectral.
a) Statistical Approaches: 2 types: Histogram approach & co-occurrence matrix.
 Simplest method for describing texture is to use statistical moments of the intensity
histogram of an image or region.
 Let z be a random variable denoting intensity and , i=0,1,2…L-1 be the
corresponding histogram, where L is the number of distinct intensity levels. The nth
moment of z about mean is given by

 The
  second moment or variance is a measure of intensity contrast used to
establish descriptors of relative smoothness.
 Eg: the measure is 0 for areas of constant intensity (variance is 0) and approaches
1 for larger values of .
 The third moment is a measure of skewness of the histogram while the fourth
moment is a measure of its relative flatness.
 Some additional texture measures based on histograms include measure of
uniformity given by
, (maximum for uniform images)
 And average entropy E. Entropy is a measure of variability and is 0 for a constant
image.

 Measure
  of texture computed using only the histogram carries no information
regarding the relative position of pixels wrt each other.
 Let Q be an operator that defines the position of two pixels relative to each other.
 Consider an image f, with L possible gray levels.
 Let G be a matrix whose element is the number of times that pixel pairs with
intensities and occur in f in the position specified by Q.
 G is called co-occurrence matrix.
 Figure shows an example how to construct a co-occurrence matrix with L=8 &
position operator Q defined as one pixel immediately to the right.
 Since G depends on Q, the presence of intensity texture patterns can be detected
by choosing an appropriate position operator and analyzing the elements of G.
Eg: the element (1,1) of G is 1, because there is only 1 occurrence in f of a pixel value 1
having a pixel valued 1 immediately to its right.
The element (6,2) of G is 3, because there are 3 occurrences in f of a pixel value 6
having a pixel valued 2 immediately to its right.
 A set of useful
descriptors for analyzing
the elements of G are
listed
b) Structural Approaches:

 Suppose
  we have a rule , which indicates that the symbol ‘S’ can be rewritten as ‘aS’.
Three repetitions of the rule yield the string aaaS.
 If ‘a’ represents a circle and the meaning “circles to the right” is assigned to a
string of the form ‘aaa…’ then the rule allows the generation of the pattern:

  Suppose we add new rules, for example: S bA, A cA, A c, A

bS, S a, where b means “circle down” & c means “circle to the
left”.
 Now a string ‘aaabccbaa’ can be generated that corresponds to a
3x3 matrix of circles.
 Larger texture patterns can be generated in a similar manner.
c) Spectral Approaches:


 The
  Fourier transform (FT) is a useful for description of the directionality of periodic
or almost periodic structures.
 The 3 features of the Fourier Spectrum that are useful for texture description are:
1. Peaks in the FT give the principal direction of patterns.
2. The location of peaks gives the fundamental period of patterns.
3. The FT is symmetric around the origin and only half of the frequency plane needs
to be considered.
 The spectrum is expressed in polar coordinates S(r,θ) for simplification
 (θ) is a 1D function for a given direction (r) and (r) is a 1D function for a given
direction (θ).

 Analyzing
  (r) for a fixed value of θ yields the behavior of the spectrum along a
radial direction from the origin.
 Analyzing (θ) for a fixed value of r yields the behavior of the spectrum along a
circle of radius r which is centered at the origin.
 A more global description is obtained by integrating these functions:

with being a the radius of a circle centered at the origin.

 The result constitute a pair of values [S(r), S(θ)] for each pair of coordinates (r,θ)
that gives the energy description of texture for the region .
Relational Descriptors


 Applies
  to both boundary and regions.
 Example: the staircase structure has been extracted from an image and we want to
describe it.
 We employ two primitive elements ‘a’ & ‘b’ and a set of rules.
 One simple approach is to use a recursive relationship involving these variables.
1.

 S and A are variables and ‘a’ and ‘b’

are constants.
 Rule 1 indicates that the starting symbol S can be replaced by ‘a’ and variable A.
 Rules 2 and 3 indicate that variable A in turn can be replaced by ‘b’ and ‘S’ or ‘b’
alone.
 Replacing A by ‘bS’ leads back to the first rule and the procedure may be repeated.
 Replacing A by ‘b’ terminates the process as there are no more variables to be
processed.
 These strings are 1D structures.
 Applications of the rules to images requires methods for reducing 2D positional relations
to 1D relations.
 An approach is to follow the contour of an object and code the result with head-to-tail
segments of specified direction and length.
 Another approach is to describe sections of an image by directed line segments which
can be joined in other ways, besides head-to-tail connections.
Step by step generation of a specific shape

Set of specific primitives consisting of

line segments in 4 directions

Each composite region has a

single tail & single head

 String
  descriptors are best suited for applications where connectivity of primitives
can be expressed in a continuous manner.
 Tree descriptors better represent non contiguous textured regions.
 A tree ‘T’ is a finite set of one or more nodes for which
1. There is a unique node $ - designated the root and
2. The remaining nodes are portioned into m disjoined sets each of which is a tree
called subtree of T.
 Important information in a tree
1. A set of words describing the node (e.g.
image region or boundary).
2. Relation between a node and its
neighbors
 REFERENCES
44

1. R. C. Gonzalez, R. E. Woods, Digital Image Processing, Pearson

Education, 2007.
45 End of Part I

THANK YOU!