Binary Image Analysis

Chapter 3

Dept. of Computer Science & Engineering

Chittagong University of Engineering & Technology
Kaushik Deb, Ph.D.
 Motivation
„ Need less space
„ Simpler computational algorithms
„ May be extended to grayscale images
„ Adequate for many applications
„ Counting
„ OCR
„ Foreground-background separation

Computer Vision 2
 Binary Images
„ Foreground : pixels that belong to
objects of interest
„ Background : Everything else
„ Separation is important
„ Simpler in binary images
„ OCR :
„ Scan in gray scale
„ Convert to binary
„ Hardware or software

Computer Vision 3
Examples

Computer Vision 4
 Binary Image
„ Pixels are 0 or 1
„ 1 foreground
„ 0 background
„ Assume mxn image
„ Indexed from 0 .. m-1 for rows
„ and 0 …. n-1 for columns
„ I[i,j] : refers to individual pixels

Computer Vision 5
 Binary Image Analysis

Binary image analysis

• consists of a set of image analysis operations

that are used to produce or process binary
images, usually images of 0’s and 1’s.

0 represents the background

1 represents the foreground

00010010001000
00011110001000
00010010001000
Computer Vision 6
 What kinds of operations?

„ Separate objects from background

and from one another

„ Aggregate pixels for each object

„ Compute features for each object

Computer Vision 7
Example: red blood cell image
„ Many blood cells are
separate objects
„ Many touch – bad!
„ Salt and pepper
noise from
thresholding
„ How useable is this
data?

Computer Vision 8
 Results of analysis
„ 63 separate
objects
detected
„ Single cells
have area
about 50
„ Noise spots
„ Gobs of cells

Computer Vision 9
 Useful Operations

1. Thresholding a gray-tone image

2. Determining good thresholds

3. Connected components analysis

4. Binary mathematical morphology

5. All sorts of feature extractors

(area, centroid, circularity, …)

Computer Vision 10
 Thresholding
„ Background is black
„ Healthy cherry is bright
„ Bruise is medium dark
„ Histogram shows two
cherry regions (black
background has been
removed)

pixel
counts

0 256
gray-tone values
Computer Vision 11
 Thresholding
„ Common way to convert a gray scale
image to a binary image
„ Binary Thresholding
„ Map the intensities in the image into 0 or 1
„ Multilevel Thresholding
„ Map the intensities in the image into a
finite number (usually small) of intensities
„ Often guided by the histogram

Computer Vision 12
 Histogramming
„ Map the frequency of intensities in an image
„ Table of frequencies
„ Graphical representation
„ Shows first order properties of an image
„ E.g. Bright/dark
„ Modality
„ Number of peaks of a histogram
„ Bimodal, unimodal, etc.

Computer Vision 13
 Histogramming

Image Histogram
Computer Vision 14
Histogram-Directed Thresholding

How can we use a histogram to separate an

image into 2 (or several) different regions?

Is there a single clear threshold? 2? 3?

Computer Vision 15
 Automatic Thresholding: Otsu’s Method

Grp 1 Grp 2
Assumption: the histogram is bimodal

Method: find the threshold t that minimizes

the weighted sum of within-group variances
for the two groups that result from separating
the gray tones at value t.

See text (at end of Chapter 3) for the recurrence relations;

in practice, this operator works very well for true bimodal
distributions and not too badly for others.
Computer Vision 16
Thresholding Example

Computer Vision 17
Another Example

Threshold = 180
Computer Vision 18
 Neighborhood
„ How many neighbors does a pixel have?

NW N NE

W P E

SW S SE

Computer Vision 19
 4/8 Neighborhood

N NW N NE

W E W E

S SW S SE

4-Neighborhood 8-Neighborhood

Computer Vision 20
 Neighborhood Ambiguity
0 0 0 0 0 0 0 0 „ All 1's are 8-neighbors
„ Define a ring
0 0 0 0 0 0 0 0 „ The red 0’s are also 8-
neighbors
0 0 1 1 1 1 0 0 „ Foreground and background
0 0 1 0 0 1 0 0 objects intersect each other
„ Not simple objects

0 0 1 0 0 1 0 0
0 0 0 1 1 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0

Computer Vision 21
 Managing Ambiguity
„ 4-neighborhood for foreground and
8-neighbor for background

„ 8-neighborhood for foreground and

4-neighbor for background

Computer Vision 22
 Applying Masks
„ Basic concept is based on convolution
„ Variety of tasks can be framed in terms
of mask operations
„ Mask = a set of pixels
„ Have associated weights
„ Generally square
„ Has an origin (usually the center pixel)
„ Generally 2D or 1D

Computer Vision 23
 Example of Masks

1 1 1 -1 0 1

1 1 1 -1 0 1 -1 1

1 1 1 -1 0 1

Computer Vision 24
 Mask Operation
„ Overlay the origin of the mask at a pixel
„ Pair wise multiply
„ Pixel intensity of the image
„ Weight of the mask at the pixel
„ Sum the terms
„ Defines the value at that pixel
„ Normalize
„ Divide by the size of the mask
„ Produces a new image

Computer Vision 25
 Example
9 9 9 9 1 1 1
9 9 9 9 1 1 1
-1 0 1
9 9 9 9 1 1 1
9 9 9 9 1 1 1 -1 0 1

9 9 9 9 1 1 1 -1 0 1
9 9 9 9 1 1 1
9 9 9 9 1 1 1

Computer Vision 26
 Example -1 0 1
-1 0 1
9 9 9 9 1 1 1 Mask
9 9 9 9 1 1 1 -1 0 1
9 9 9 9 1 1 1
9 9 9 9 1 1 1
0 0 -24 -24 0
9 9 9 9 1 1 1
0 0 -24 -24 0
9 9 9 9 1 1 1
0 0 -24 -24 0
9 9 9 9 1 1 1
0 0 -24 -24 0
Image 0 0 -24 -24 0

Computer Vision Result 27

 Example
„ Normalize
„ By the number of elements

0 0 -24 -24 0 0 0 -3 -3 0
0 0 -24 -24 0 0 0 -3 -3 0
0 0 -24 -24 0 0 0 -3 -3 0
0 0 -24 -24 0 0 0 -3 -3 0
0 0 -24 -24 0 0 0 -3 -3 0

Computer Vision 28
 Example -1 0 1
-1 0 1
9 9 9 9 1 1 1 Mask
9 9 9 9 1 1 1 -1 0 1
9 9 9 9 1 1 1
0 0 0 0 0 0 0
9 9 9 9 1 1 1
0 0 0 -24 -24 0 0
9 9 9 9 1 1 1
0 0 0 -24 -24 0 0
9 9 9 9 1 1 1
0 0 0 -24 -24 0 0
9 9 9 9 1 1 1
0 0 0 -24 -24 0 0
Image 0 0 0 -24 -24 0 0
0 0 0 0 0 0 0
Computer Vision Result 29
Counting Objects
 Counting Objects
„ Number of foreground objects
„ Counting number of holes
„ Equivalent problem
„ Swap the role of foreground and
background
„ Assumption
„ Objects are 4-connected
„ Foreground pixels are 1’s
„ No interior holes

Computer Vision 31
 Counting Objects
„ External Corners „ Internal Corners
„ Three 0’s and one 1 „ Three 1’s and one 0

0 0 0 0 1 1 1 1
0 1 1 0 1 0 0 1

1 0 0 1 0 1 1 0
0 0 0 0 1 1 1 1

Computer Vision 32
 Counting Objects

„ External corners=4 „ External corners=8

„ Internal corners=0 „ Internal corners=4

„ Number of objects =(External-Internal)/4

Computer Vision 33
 Counting Objects
„ Take the 4 external masks and 4
internal masks
„ Put them at each pixel in the image
„ Count the number of E’s and I’s
„ The number of pixels where they match
„ Determine the number of objects based
on E and I

Computer Vision 34
Counting Objects

Computer Vision 35
Connected Components
 Connected Components
• Given
– A binary image B
– A pair of pixels B[r,c] = B[r ' , c' ] = v (0,1)
• Pixel (r,c) is connected to (r ' ,c' ) if
– there is a sequence of pixels
(r,c) = (r0 ,c0 ), (r1, c1 ), L , (rn ,cn ) = (r',c')
– B[ri ,ci ] = v
– (ri ,ci ) and (r i +1, ci +1 ) are adjacent

(r0,c0 ), (r1, c1 ), L , (rn ,cn ) = (r',c')

is a connected path from (r,c) to (r',c')
Computer Vision 37
 Connected Components
„ A connected component is a set of
pixels with the same intensity value
such that each pair of pixels in the set
is connected
„ Use 4 or 8 neighborhood
„ Use complementary connectedness for
foreground and background
„ Usually foreground is chosen to be the
1 pixels

Computer Vision 38
Connected Components Labeling
„ CCL is a well –known technique in image
processing that scans an image (binary and
gray level) and labels its pixels connected
components based on pixel connectivity. In
other words all pixels in connected
components share similar pixel intensity
values and these components are actually
connected with respect of either 4 or 8 –
neighborhood definition
 Connected Components Labeling
„ A connected component labeling of a
binary image B is a labeled image in which
the value of each pixel is the label of its
connected component
„ Background label is 0
„ Each component has unique label
„ Consecutive labels starting with 1

Computer Vision 40
Connected Components

Computer Vision 41
 Recursive Labeling Algorithm
„ First negate the binary image
„ Finding connected component, becomes one
of finding a pixel which values minus one and
assigning a label
„ Calling procedure search to find its neighbors
that have value minus one and recursive
repeat the process for its neighbors
„ Algorithm 3.2
„ Fig 3.7, 3.8

Computer Vision 42
Connected Components Labeling
Classical Algorithm

„ Take a template and go over the image

„ 2 pass algorithm
„ Make an initial labeling
„ Make a final labeling

Top

Left Current

Computer Vision 43
Connected Components Labeling
Classical Algorithm

Top Top
0 0
Left Current Left Current
0 1 1 1

Top Top
1 1
Left Current Left Current
0 1 1 1
Computer Vision 44
Connected Components Labeling

Computer Vision 45
 Classical Algorithm
First Pass
currentlabel=1
for (i=0; i<nrows; i++)
for(j=0; j<ncols; j++)
if (Image[i,j]==0) break;
if ((top==0) && (left==1) label (current) = label(left);
if ((top==1) && (left==0) label (current) = label(top);
if ((top==0) && (left==0) {
label (current) = currentlabel;
currentlabel++;
}
if ((top==1) && (left==1) {
label (current) = label(left);
equivelent(label(left), label(top));
}

Computer Vision 46
 Classical Algorithm
Second Pass

for (i=0; i<nrows; i++)

for(j=0; j<ncols; j++)
if (Image[i,j]==0) break;
currentlabel=label(i,j);
label(i,j)=equivalentclass(currentlabel);

Computer Vision 47
 Classical Algorithm
Union Find

„ Need to merge different labels for the

same connected component
„ We have pairwise equivalences
„ Need to merge them into equivalent
classes
„ Union Find algorithm is an efficient
algorithm to do this

Computer Vision 48
 Union Find
„ Stores a collection of disjoint sets
„ Efficiently implements some operations
„ Union: Merges two sets into one
„ Find: Find which set an element is in
„ Stores the sets as a tree structure
„ Node: Label
„ Stored as a vector (2D array)

Computer Vision 49
Union Find

Computer Vision 50
Union Find

Computer Vision 51
Union Find

Computer Vision 52
Union Find

Computer Vision 53
 Classical Algorithm
First Pass
currentlabel=1
for (i=0; i<nrows; i++)
for(j=0; j<ncols; j++)
if (Image[i,j]==0) break;
if ((top==0) && (left==1) label (current) = label(left);
if ((top==1) && (left==0) label (current) = label(top);
if ((top==0) && (left==0) {
label (current) = currentlabel;
currentlabel++;
}
if ((top==1) && (left==1) {
label (current) = label(left);
equivelent(label(left),
PARENT[label(top)] label(top));
= label(left);
}

Computer Vision 54
 Classical Algorithm
Second Pass

for (i=0; i<nrows; i++)

for(j=0; j<ncols; j++)
if (Image[i,j]==0) break;
currentlabel=label(i,j);
label(i,j)=PARENT[currentlabel];

Computer Vision 55
Classical Algorithm

Computer Vision 56
 Connected Components Labeling

Once you have a binary image, you can identify and

then analyze each connected set of pixels.

The connected components operation takes in a binary image

and produces a labeled image in which each pixel has the
integer label of either the background (0) or a component.

Computer Vision 57
 Methods for CC Analysis

1. Recursive Tracking (almost never used)

2. Parallel Growing (needs parallel hardware)

3. Row-by-Row (most common)

• Classical Algorithm (see text)

• Efficient Run-Length Algorithm

(developed for speed in real
industrial applications)

Computer Vision 58
 Equivalent Labels

Original Binary Image

0 0 0 1 1 1 0 0 0 0111100001
0 0 0 1 1 1 1 0 0 0111100011
0 0 0 1 1 1 1 1 0 0111100111
0 0 0 1 1 1 1 1 1 0111100111
0 0 0 1 1 1 1 1 1 1111100111
0 0 0 1 1 1 1 1 1 1111100111
0 0 0 1 1 1 1 1 1 1111111111
0 0 0 1 1 1 1 1 1 1111111111
0 0 0 1 1 1 1 1 1 0000011111

Computer Vision 59
 Equivalent Labels

The Labeling Process

0001110000222200003 1≡2
0001111000222200033 1≡3
0001111100222200333
0001111110222200333
0001111111111100333
0001111111111100333
0001111111111111111
0001111111111111111
0001111110000011111

Computer Vision 60
 Run-Length Data Structure
01234
0 11 11
row scol ecol label
1 11 1
2 1 1 1 1 Binary 0 UNUSED 0
3 Image 1 0 0 1 0
4 1111 2 0 3 4 0
3 1 0 1 0
Rstart Rend 4 1 4 4 0
0 1 2 5 2 0 2 0
1 3 4 6 2 4 4 0
2 5 6 Row Index 7 4 1 4 0
3 0 0
Runs
4 7 7
Computer Vision 61
 Run-Length Algorithm

Procedure run_length_classical
{
initialize Run-Length and Union-Find data structures
count <- 0

/* Pass 1 (by rows) */

for each current row and its previous row

{
move pointer P along the runs of current row
move pointer Q along the runs of previous row

Computer Vision 62
 Case 1: No Overlap

Q Q

|/////| |///| |///|

|/////| |////| |/////|
P P

/* new label */
count <- count + 1 /* check Q’s next run */
label(P) <- count Q <- Q + 1
P <- P + 1

Computer Vision 63
 Case 2: Overlap

Subcase 1: Subcase 2:
P’s run has no label yet P’s run has a label that is
different from Q’s run
Q Q
|///////| |/////| |///////| |/////|
|/////////////| |/////////////|
P P
label(P) <- label(Q) union(label(P),label(Q))
move pointer(s) move pointer(s)
}

Computer Vision 64
 Pass 2 (by runs)

/* Relabel each run with the name of the

equivalence class of its label */
For each run M
{
label(M) <- find(label(M))
}

}
where union and find refer to the operations of the
Union-Find data structure, which keeps track of sets
of equivalent labels.

Computer Vision 65
Labeling shown as Pseudo-Color

connected
components
of 1’s from
thresholded
image

connected
components
of cluster
labels

Computer Vision 66
Morphology
 Morphology
„ Study of shape (form and structure)
„ Mathematical morphology was designed
for a set of points
„ Treat the object as binary
„ A shape = a set of points
„ Use shape-based or iconic approach

Computer Vision 68
 Mathematical Morphology

Binary mathematical morphology consists of two

basic operations

dilation and erosion

and several composite relations

closing and opening

conditional dilation
...

Computer Vision 69
 Dilation

Dilation expands the connected sets of 1s of a binary image.

It can be used for

1. growing features

2. filling holes and gaps

Computer Vision 70
 Erosion

Erosion shrinks the connected sets of 1s of a binary image.

It can be used for

1. shrinking features

2. Removing bridges, branches and small protrusions

Computer Vision 71
 Structuring Elements
„ Morphology operations involve
„ Binary image
„ Structuring element
„ Structuring Element
„ Binary image
„ Can be any arbitrary binary image in
general
„ Usually much smaller

Computer Vision 72
 Structuring Elements

A structuring element is a shape mask used in

the basic morphological operations.

They can be any shape and size that is

digitally representable, and each has an origin.

box
hexagon disk
something

box(length,width) disk(diameter)
Computer Vision 73
 Dilation with Structuring Elements

The arguments to dilation and erosion are

1. a binary image B
2. a structuring element S

dilate(B,S) takes binary image B, places the origin

of structuring element S over each 1-pixel, and ORs
the structuring element S into the output image at
the corresponding position.

0000 dilate 0110

1
0110 0111
11
0000 0000
S
B origin B⊕S
Computer Vision 74
 Erosion with Structuring Elements

erode(B,S) takes a binary image B, places the origin

of structuring element S over every pixel position, and
ORs a binary 1 into that position of the output image only if
every position of S (with a 1) covers a 1 in B.

origin
0 0 1 1 0 0 0 0 0 0
1 erode
0 0 1 1 0 0 0 1 1 0
1
0 0 1 1 0 0 0 1 1 0
1
1 1 1 1 1 0 0 0 0 0

B S B S

Computer Vision 75
Common Structuring Elements

Computer Vision 76
 Operations
„ Intersection

A I B = { p | p ∈ A and p ∈ B}
„ Union
A U B = { p | p ∈ A or p ∈ B}
„ Complement

A = { p | p ∈ Ω and p ∉ A}
Ω is a universal binary image (all 1s)
Computer Vision 77
 Translation
„ A : binary image
„ p : pixel
„ Translation of A by p is an image
„ Shifts the origin of A to p

Ap = {a + p | a ∈ A}

Computer Vision 78
Example

Computer Vision 79
Translation

Computer Vision 80
 Dilation

A ⊕ B = U Abi
bi ∈B

B = {b1 , b2 ,.....bn }
Dilation of A by B
Ab1 is the translation of A by a pixel of B

Computer Vision 81
 Dilation
„ Commutative

A⊕ B = B ⊕ A
„ Associative

A ⊕ ( B ⊕ C ) = ( A ⊕ B) ⊕ C

Computer Vision 82
Example

Computer Vision 83
Dilation

Computer Vision 84
 Erosion
„ Opposite of Dilation

AΘB = { p | Bp ⊆ A}
„ 1 at a pixel iff every 1 pixel in the
translation of B to p is also a 1
„ Structuring element is a probe
„ Locate where the structuring element is
located in the image

Computer Vision 85
Erosion

Computer Vision 86
Erosion and Dilation

B' = {− p | p ∈ B}
A ⊕ B = AΘB '

AΘB = A ⊕ B '

Computer Vision 87
Complement of A and probe

Computer Vision 88
 '
AΘB

Computer Vision 89
A⊕ B '

Computer Vision 90
 Opening
„ Erosion followed by dilation
„ Remove all pixels in a region too small
to contain the structuring element
„ Disc shaped probe
„ Eliminate all convex regions and isolated
points

Computer Vision 91
Example

Computer Vision 92
Example

Computer Vision 93
 Closing
„ Opposite of opening
„ Dilation followed by erosion
„ Fill holes smaller than the structuring
element

Computer Vision 94
Example

Computer Vision 95
Example

Computer Vision 96
Example

Computer Vision 97
Gear Tooth Inspection

original
binary
image
How did
they do it?

detected
defects

Computer Vision 98
Example

Computer Vision 99
Example

Computer Vision 100

Shape Features
 Shape Features
„ Organization of points in space
„ Not related to any intensity or color features
„ Common Features
„ Area
„ Centroid
„ Perimeter
„ Elongatedness
„ Aspect Ratio
„ Bounding box

Computer Vision 102

 Area
„ Number of pixels in a component
„ Gives an indication of size
„ Easy to compute

A= ∑1
( r ,c )∈R
where R is a given region

Computer Vision 103

Area and centroid

Computer Vision 104

 Perimeter
„ Boundary points of the object
„ Length of the perimeter is the number
of pixels on the boundary of the object
„ Use 4/8 neighborhood to determine

P4 = {( r , c) ∈ R | N 8 (r , c) − R ≠ φ }
P8 = {(r , c) ∈ R | N 4 (r , c) − R ≠ φ }

Computer Vision 105

 Perimeter
„ Length of the perimeter is an important
measure of the shape
„ First order of approximation
„ Number of pixels on the boundary
„ More accurate approximation
„ Horizontal and vertical moves : 1 unit
„ Diagonal move: 1.4 unit

Computer Vision 106

 Perimeter
„ Order the boundary pixels in a
sequence
P =< (r0 , c0 ),L , (rk , ck ) >
„ Pixels in the sequence are adjacent

P = {k | (ri +1 , ci +1 ) ∈ N 4 (ri , ci )} | +
2 {k | (ri +1 , ci +1 ) ∈ N 8 (ri , ci ) − N 4 (ri , ci )} |
„ Indices are computed modulo k (wrap
around) Computer Vision 107
 Circularity (1)
„ Degree of similarity with a circle
2
P
Circularity =
A
„ Based on continuous planar shapes
„ smallest value for a circle?
„ Make adjustments for digital shapes

Computer Vision 108

 Circularity (2)
μR
Circularity =
σR
μ R :Mean radial distance
σ r :Standard deviation of radial distance
„ Radial distance computed as the distance
between the centroid and a boundary point

Computer Vision 109

 Circularity (2)

1 k −1
μ R = ∑ (r , ck ) − (r , c)
k i =0

[ ]
1
⎛1 k −1
⎞
2 2
σ r = ⎜ ∑ ( r , ck ) − ( r , c ) − μ R ⎟
⎝ k i =0 ⎠

(r , c) is the centroid of the region

Computer Vision 110
Region Properties – Fig 3.18

Computer Vision 111

 Compactedness
„ Approximated by the bending energy

P
1
E = ∫ | κ (s) | ds
2

P0
κ is the curvature

„ Minimized by a circle

Computer Vision 112

 Slope Density Function
„ Use Ψ−s representation for the curve
„ Ψ: Slope of the curve at a given point
„ s: arc length
„ SDF shows the distribution of the slopes
along the boundary

Slope
Slope

Arc length
Arc length
Circle Computer Vision Square 113
 Projections/Signatures
„ Non-information preserving
„ Maxima and minima are indications of
landmarks
„ Horizontal signature
p( x) = ∫ f ( x, y)
y

„ Vertical signature
p ( y ) = ∫ f ( x, y )
x Computer Vision 114
 Bounding Box
„ Gives a rough idea of the location of the
object
„ Smallest rectangle to enclose the shape
„ Extremal points
„ Leftmost, rightmost, topmost and
bottommost
„ Lie on the bounding box boundary
„ May have upto 8 extremal points
„ Occur in opposite pairs

Computer Vision 115

Bounding Box

Computer Vision 116

 Bounding Box
„ Each point of opposite extremal point
pairs define an axis
„ Properties of axes are useful
„ Lengths
„ Orientations
„ Need to make corrections to the lengths
„ Error due to sampling

Computer Vision 117

 Bounding Box

„ Distance between the two pixels =1

„ Distance between the two ends of the

pixels=2
Computer Vision 118
 Bounding Box
„ Extremal axis length

D = (r2 − r1 ) + (c2 − c1 ) + Q(θ )

2 2

(r1 , c1 ) and (r2 , c2 ) are two extremal points

⎧ 1
⎪⎪ cosθ θ < 45 o

Q(θ ) = ⎨ the angle

1 θ:
⎪ θ > 45 o
the angle
⎪⎩ sin θ
Computer Vision 119
 Moments
„ Used to indicate the shape of the
objects
„ Describe the distribution of points in
shape
„ Many moments can be defined
„ Only low order moments are useful

Computer Vision 120

 Second-order Moments
1
„ Row moments μ rr = ∑ (r − r )
A ( r ,c )∈R
2

1
„ Column moments μ cc = ∑ (c − c )
A ( r ,c )∈R
2

1
„ Mixed moments μ rc = ∑ (
A ( r ,c )∈R
r − r )( c − c )

Computer Vision 121

 Moments /

M pq= ∫ ∫x
p
y r ( x, y) dx dy
q

x y

where
⎧1 if ( x, y) is in the object⎫
r ( x, y) = ⎨ ⎬
⎩0 otherwise ⎭

Computer Vision 122

 Moments
„ Defines moments of order p+q
„ Changes with location of the object
„ Discrete case

M pq = ∑∑ x y r ( x, y)
p q

x y

Computer Vision 123

 Moments

M00 = Areaof the object

M 10 = Sum of all x - coordinates
M 01 = Sum of all x - coordinates
M 10 M 01
xc = yc =
M 00 M 00
Computer Vision 124
 Central Moments
„ Moments change with the location of the object
„ Not useful for comparing shapes

M pq= ∑∑ ( x − xc ) ( y − yc ) r ( x, y)
p q

x y

M 01 = M 10 = 0
xc = yc = 0
Computer Vision 125
 Central Moments
„ Centroid forms the origin
„ Orientation of on object is given by

1 −1 2M11 π
θ = tan ( ) + n( )
2 M 20 − M 02 2

„ Eccentricity (elongatedness)
1
(M 20 − M 02 ) 2 + 4M11
e=
M 00

Computer Vision 126

 Moments
„ Row moments

1
M pp= ∑ ( x − xc ) r ( x, y )
p

A x
„ Column moments

1
M qq= ∑ ( y − yc ) r ( x, y )
q

A x
Computer Vision 127
 Ellipses
„ Circular objects get mapped to ellipses
in imaging process
„ Defined by their major and minor axes
„ May be used to approximate elongated
objects
dx + 2exy + fy ≤ 1
2 2

Assumes that the center of the circle is

at the origin
Computer Vision 128
 Ellipses
• May be defined as a function of the
moments also
• Lengths of the major and minor axes
can be derived from the moments

⎛d e⎞ 1 ⎛ μ yy μ xy ⎞
⎜⎜ ⎟⎟ = 2 ⎜
⎜ ⎟
⎝e f ⎠ 4( μ xx μ yy − μ xy ) ⎝ − μ xy μ xx ⎟⎠

Computer Vision 129

 Ellipse Axes
„ Need to consider four cases

μ rc = 0 and μ rr > μ cc
1
Major axis :Orientation = −90 Length = 4 μ rr2
o

1
Minor axis :Orientation = 0 Length = 4 μ cc2
o

„ Angle measured counterclockwise from the

column axis
Computer Vision 130
 Ellipse Axes

Case II: μ rc = 0 and μ rr ≤ μ cc

1
Major axis :Orientation = 0 Length = 4 μ cc2
o

1
Minor axis :Orientation = −90 Length = 4 μ rr2
o

Computer Vision 131

 Ellipse Axes
Case III: μ rc ≠ 0 and μ rr ≤ μ cc
Major axis :
⎧⎪ − 2 μ ⎫⎪
Orientation = tan −1 ⎨ rc
1 ⎬
⎪⎩ μ rr − μ cc + [( μ rr − μ cc ) + 4 μ rc ] 2 ⎪⎭
2 2

Length = ⎧⎨8⎛⎜ μ rr − μ cc + [( μ rr − μ cc ) 2 + 4 μ rc2 ] 2 ⎞⎟⎫⎬

1 2

⎩ ⎝ ⎠⎭

Computer Vision 132

 Ellipse Axes
Case III: μ rc ≠ 0 and μ rr ≤ μ cc

Minor axis :
⎧⎪ − 2 μ ⎫⎪
Orientation = 90 + tan −1 ⎨ rc
1 ⎬
⎪⎩ μ rr − μ cc + [( μ rr − μ cc ) 2 + 4 μ rc2 ] 2 ⎪⎭
1
⎧
Length = ⎨8⎜ μ rr + μ cc − [( μ rr − μ cc ) + 4 μ rc ] 2 ⎞⎟⎫⎬
⎛ 1 2
2 2

⎩ ⎝ ⎠⎭
Computer Vision 133
 Ellipse Axes

Case IV: μ rc ≠ 0 and μ rr > μ cc

Major axis :
1
⎡ μ + μ + [( μ − μ ) 2 + 4 μ 2 ] 2 ⎤ 1 2

−1 ⎢
⎣ ⎥⎦
rr cc cc rr rc
Orientation = tan
− 2 μ rc
1

Length = ⎨8⎜ μ rr + μ cc + [( μ rr − μ cc ) + 4 μ rc ] 2 ⎞⎟⎫⎬

⎧ ⎛ 1 2
2 2

⎩ ⎝ ⎠⎭
Computer Vision 134
 Ellipse Axes

Case IV: μ rc ≠ 0 and μ rr ≤ μ cc

Minor axis :
1
⎡ μ + μ + [( μ − μ ) 2 + 4 μ 2 ] 2 ⎤ 1 2

−1 ⎢
⎣ ⎥⎦
rr cc cc rr rc
Orientation = 90 + tan
− 2 μ rc
1

Length = ⎧⎨8⎛⎜ μ rr + μ cc − [( μ rr − μ cc ) 2 + 4 μ rc2 ] 2 ⎞⎟⎫⎬

1 2

⎩ ⎝ ⎠⎭
Computer Vision 135
 Best Axes /
„ Some objects have natural axis
„ Pencils, Letters l,I, -
„ Axis around which there is least second
order moment
„ Axis of inertia
„ Line about which the object can be spun
with least energy input
„ Circle
„ All axes have same length

Computer Vision 136

 Best Axes
„ Axis must past through the centroid
„ Need to compute moment about any
arbitrary axis
„ Minimize the moments as a function of
angle
„ Gives the axis of least moment
„ Other principal axis is perpendicular to it

Computer Vision 137

Best Axis p79

Computer Vision 138

 Best Axes

1
μ r ,c ,α = ∑ d
A ( r ,c )∈R
2

1
= ∑ ((r − r ) cos β + (c − c) sin β )
A ( r ,c )∈R
2

„ Any axes can be defined by the three

parameters
Computer Vision 139
 Best Axes
„ Axis with the least second moment
2∑ (r − r )(c − c)
tan 2αˆ =
∑ (r − r )(r − r ) − ∑ (c − c)(c − c)
1
2∑ (r − r )(c − c)
= A
1 1
A
∑ (r − r )(r − r ) − ∑ (c − c)(c − c)
A
2 μ rc
=
μ rr − μ cc
„ Symmetric objects (circle/square) results
in zero divide Computer Vision 140
Region Adjacency Graphs p81
„ Relationships between the regions are
important also
„ Region adjacency is one way to show
„ Two regions are adjacent if they have
two pixels that are neighbors
„ Binary images
„ Foreground and background regions
„ All components are adjacent to the
background

Computer Vision 141

 Region Adjacency Graphs
„ Why?
„ Objects can have holes
„ Find connected components for the
background
„ Adjacency makes sense then
„ Dealing with grayscale/color images

Computer Vision 142

 Region Adjacency Graphs
„ A Region Adjacency Graph (RAG) is a
graph
„ Nodes: Components of the image
„ Edges: If two components are adjacent,
then there is an edge between the
corresponding nodes

Computer Vision 143

Region Adjacency Graphs

Computer Vision 144

 Region Adjacency Graphs
„ Algorithm
„ Make one pass of the labeled image
„ At each pixel
„ Check the labels of the neighbors
„ If it is different from the current label
„ Add an edge in the graph it is not already there
„ Efficiency is very very good
„ May have a large number of labels
„ Repetition

Computer Vision 145

Computer Vision 146
 Region Properties

Properties of the regions can be used to recognize objects.

• geometric properties (Ch 3)

• gray-tone properties

• color properties

• texture properties

• shape properties (a few in Ch 3)

• motion properties

• relationship properties (1 in Ch 3)
Computer Vision 147
 Geometric and Shape Properties

• area
• centroid
• perimeter
• perimeter length
• circularity
• elongation
• mean and standard deviation of radial distance
• bounding box
• extremal axis length from bounding box
• second order moments (row, column, mixed)
• lengths and orientations of axes of best-fit ellipse

Computer Vision 148

Perimeter pixels and length

Computer Vision 149

Circularity or elongation

Computer Vision 150

Circularity as variance of “radius”

Computer Vision 151

 Second moments

These quantities are used as simple shape descriptors.

They are invariant to translation and scale change of a
2D shape.
Computer Vision 152
 Region Adjacency Graph

A region adjacency graph (RAG) is a graph in which

each node represents a region of the image and an edge
connects two nodes if the regions are adjacent.

This is jumping ahead a little bit.

We’ll consider this further for structural image analysis.

Computer Vision 153