Representation and

Description
Image analysis fundamental steps
 Representation and description
• After segmentation one needs to represent objects in order to describe them

• External (boundary or shape):

• Representation: Polygon of the boundary
• Description: The circumference

• Internal (regional)
• Representation: Pixels inside the object
• Description: The average color
OR use both methods
 Representation

• Describes the region based on a selected

representation:
• Representation  boundary or textural features
• Description  length, orientation, the number of concavities in the boundary,
statistical measures of region
BOUNDARY PREPROCESSING

The segmentation techniques yield raw data in the form of pixels along a
boundary or pixels contained in a region

It is standard practice to use schemes that compact the segmented data into
representations that facilitate the computation of descriptors
BOUNDARY FOLLOWING (TRACING)

Boundary-following algorithm is the one whose output is an ordered sequence of

points. We assume
(1) that we are working with binary images in which object and background
points are labeled 1 and 0, respectively; and
(2) that images are padded with a border of 0’s to eliminate the possibility of an
object merging with the image border.
Note that c in Step 4 is always a background point because nk is the first 1-valued point found in the
clockwise scan. This algorithm is referred to as the Moore boundary tracing algorithm after Edward F.
Moore, a pioneer in cellular automata theory
Figure 11.1 illustrates the first few steps of the algorithm. It is easily verified (see Problem 11.1) that
continuing with this procedure will yield the correct boundary, shown in Fig. 11.1(f), whose points are ordered
in a clockwise sequence.
CHAIN CODES
Chain codes are used to represent a boundary by a connected sequence of straightline
segments of specified length and direction

We assume in this section that all curves are closed, simple curves (i.e., curves that are closed
and not self intersecting).
Freeman Chain Codes

Typically, a chain code representation is based on 4- or 8-connectivity of the segments

The direction of each segment is coded by using a numbering scheme, as in Fig. 11.3
A boundary code formed as a sequence of such directional numbers is referred to as a
Freeman chain code.
Digital images usually are acquired and processed in a grid format with equal spacing in
the x- and y-directions, so a chain code could be generated by following a boundary in,
say, a clockwise direction and assigning a direction to the segments connecting every
pair of pixels

This level of detail generally is not used for two principal reasons:
(1) The resulting chain would be quite long and
(2) any small disturbances along the boundary due to noise or imperfect segmentation
would cause changes in the code that may not be related to the principal shape
features of the boundary
An approach used to address these problems is to resample the boundary by
selecting a larger grid spacing, as in Fig. 11.4(a).

Then, as the boundary is traversed, a boundary point is assigned to a node of

the coarser grid, depending on the proximity of the original boundary point to
that node, as in Fig. 11.4(b)

The resampled boundary obtained in this way can be represented by a 4- or 8-

code

Figure 11.4(c) shows the coarser boundary points represented by an 8-

directional chain code. It is a simple matter to convert from an 8-code to a 4-
code and vice versa
BOUNDARY APPROXIMATIONS USING MINIMUM-PERIMETER POLYGONS

A digital boundary can be approximated with arbitrary accuracy by a polygon

For a closed curve, the approximation becomes exact when the number of
segments of the polygon is equal to the number of points in the boundary, so
each pair of adjacent points defines a segment of the polygon

The goal of a polygonal approximation is to capture the essence of the shape

in a given boundary using the fewest possible number of segments
SIGNATURES
A signature is a 1-D functional representation of a 2-D boundary and may be generated in various
ways
One of the simplest is to plot the distance from the centroid to the boundary as a function of angle, as
illustrated in Fig. 11.10.
The basic idea of using signatures is to reduce the boundary representation to a 1-D function that
presumably is easier to describe than the original 2-D boundary.
Based on the assumptions of uniformity in scaling with respect to both axes, and that sampling is
taken at equal intervals of u, changes in the size of a shape result in changes in the amplitude values
of the corresponding signature.
One way to normalize for this is to scale all functions so that they always span the same range of
values, e.g., [0,1]

The main advantage of this method is simplicity, but it has the disadvantage that scaling of the
entire function depends on only two values: the minimum and maximum

If the shapes are noisy, this can be a source of significant error from object to object

A more rugged (but also more computationally intensive) approach is to divide each sample by
the variance of the signature, assuming that the variance is not zero—as in the case of Fig.
11.10(a)—or so small that it creates computational difficulties

Using the variance yields a variable scaling factor that is inversely proportional to changes in size
and works much as automatic volume control does

Whatever the method used, the central idea is to remove dependency on size while preserving
the fundamental shape of the waveforms.
Distance versus angle is not the only way to generate a signature.
For example, another way is to traverse the boundary and, corresponding to each point on
the boundary, plot the angle between a line tangent to the boundary at that point and a
reference line

The resulting signature, although quite different from the r(u) curves in Fig. 11.10, carries
information about basic shape characteristics

For instance, horizontal segments in the curve correspond to straight lines along the
boundary because the tangent angle is constant there

A variation of this approach is to use the so-called slope density function as a signature
This function is a histogram of tangent-angle values. Because a histogram is a measure of
the concentration of values, the slope density function responds strongly to sections of the
boundary with constant tangent angles (straight or nearly straight segments) and has deep
valleys in sections producing rapidly varying angles (corners or other sharp inflections).
BOUNDARY FEATURE DESCRIPTORS
SOME BASIC BOUNDARY DESCRIPTORS

The length of a boundary is one of its simplest descriptors

The number of pixels along a boundary is an approximation of its length

For a chain-coded curve with unit spacing in both directions, the number of vertical
and horizontal components plus 2 multiplied by the number of diagonal components
gives its exact length

If the boundary is represented by a polygonal curve, the length is equal to the sum
of the lengths of the polygonal segments
The diameter of a boundary B is defined as

where D is a distance measure and pi and pj are points on the boundary

The value of the diameter and the orientation of a line segment connecting the
two extreme points that comprise the diameter is called the major axis (or
longest chord) of the boundary

That is, if the major axis is defined by points (x1 , y1 ) and (x2 , y2 ), then the
length and orientation of the major axis are given by
The minor axis (also called the longest perpendicular chord) of a boundary is
defined as the line perpendicular to the major axis, and of such length that a
box passing through the outer four points of intersection of the boundary with
the two axes completely encloses the boundary.

The box just described is called the basic rectangle or bounding box, and the
ratio of the major to the minor axis is called the eccentricity of the boundary.
STATISTICAL MOMENTS

Statistical moments of one variable are useful descriptors applicable to 1-D

renditions of 2-D boundaries, such as signatures

To see how this can be accomplished, consider Fig. 11.20 which shows the
signature from Fig. 11.10(b) sampled, and treated as an ordinary discrete
function g(r) of one variable, r.

11.10(b)
Suppose that we treat the amplitude of g as a discrete random variable z and
form an amplitude histogram p(zi), i = 0, 1, 2,…, A - 1, where A is the number
of discrete amplitude increments in which we divide the amplitude scale.

If p is normalized so that the sum of its elements equals 1, then p(zi) is an

estimate of the probability of intensity value zi occurring.

The nth moment of z about its mean is

m is the mean (average) value of z, and m2 is its variance. Generally, only

the first few moments are required to differentiate between signatures of
clearly distinct shapes
An alternative approach is to normalize the area of g(r) in Fig. 11.20 to unity and
treat it as a histogram. In other words, g(ri) is now treated as the probability of
value ri occurring.

In this case, r is treated as the random variable and the moments are
REGION FEATURE DESCRIPTORS

SOME BASIC DESCRIPTORS

The major and minor axes of a region, as well as the idea of a bounding box, are
as defined earlier for boundaries.

The area of a region is defined as the number of pixels in the region

The perimeter of a region is the length of its boundary

When area and perimeter are used as descriptors, they generally make sense only when they
are normalized
A more frequent use of these two descriptors is in measuring compactness of a region, defined
as the perimeter squared over the area:
This is the diameter of a circle having the same area, A, as the region being
processed.

This measure is neither dimensionless nor independent of region size, but it is

independent of orientation and translation.
In a manner analogous to the way we defined compactness and circularity
relative to a circle, we define the eccentricity of a region relative to an ellipse
as the eccentricity of an ellipse that has the same second central moments
as the region

For 1-D, the second central moment is the variance

For 2-D discrete data, we have to consider the variance of each variable as
well as the covariance between them
These are the components of the covariance matrix, which is estimated from
samples using Eq. (11-21) below, with the samples in this case being 2-D
vectors representing the coordinates of the data
TEXTURE

An important approach to region description is to quantify its texture content

While no formal definition of texture exists, intuitively this descriptor provides measures of
properties such as smoothness, coarseness, and regularity (Fig. 11.29 shows some examples)
In this section, we discuss statistical approaches for describing the texture of a region

Statistical approaches yield characterizations of textures as smooth, coarse, grainy, and so on

Some useful additional texture measures based on histograms include a measure of uniformity, defined as
SCALE-INVARIANT FEATURE TRANSFORM (SIFT)

SIFT is an algorithm developed by Lowe [2004] for extracting invariant features from an image

It is called a transform because it transforms image data into scale-invariant coordinates relative
to local image features

When images are similar in nature (same scale, similar orientation, etc), corner detection and
MSERs are suitable as whole image features

However, in the presence of variables such as scale changes, rotation, changes in illumination,
and changes in viewpoint, we are forced to use methods like SIFT

SIFT features (called keypoints) are invariant to image scale and rotation, and are robust across
a range of affine distortions, changes in 3-D viewpoint, noise, and changes of illumination

The input to SIFT is an image. Its output is an n-dimensional feature vector whose elements are
the invariant feature descriptors
The algorithm

SIFT is quite an involved algorithm. There are mainly four steps involved in the SIFT
algorithm

•Scale-space peak selection: Potential location for finding features.

•Keypoint Localization: Accurately locating the feature keypoints.

•Orientation Assignment: Assigning orientation to keypoints.

•Keypoint descriptor: Describing the keypoints as a high dimensional vector.

•Keypoint Matching
Fig 01: Sequence of steps followed in SIFT Detector