23/5/2013

2 Object Recognition
Digital Image Processing

One of the most interesting aspects of the world is

Object Recognition that it can be considered to be made up of patterns.

A pattern is essentially an arrangement.

Christophoros Nikou It is characterized by the order of the elements of
[email protected] which it is made, rather than by the intrinsic nature
of these elements

Images taken from: R. Gonzalez and R. Woods. Digital Image Processing, Prentice Hall, 2008 Norbert Wiener
University of Ioannina - Department of Computer Science C. Nikou – Digital Image Processing

3 Introduction 4 Patterns and pattern classes

• Recognition of individual image regions • Pattern: an arrangement of descriptors (or
(objects or patterns). features).
• Introduction to basic techniques. • Pattern class: a family of patterns sharing
– Decision theoretic techniques. some common properties.
• Quantitative descriptors (e.g. area, length…). – They are denoted by ω1, ω2,…, ωW, W being
• Patterns arranged in vectors. the number of classes.
– Structural techniques. • Goal of pattern recognition: assign
• Qualitative descriptors (relational descriptors for patterns to their classes with as little
repetitive structures, e.g. staircase).
• Patterns arranged in strings or trees.
human interaction as possible.
• Central idea: Learning from sample patterns.
C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

5 Pattern vectors 6 Pattern vectors (cont.)

• Historical example • Shape signature represented by the sampled
– Recognition of three amplitude values.
types of iris flowers by the • Cloud of n-dimensional points.
lengths and widths of
their petals (Fisher 1936). • Other shape characteristics could have been
• Variations between and employed (e.g. moments).
within classes. • The choice of descriptors has a profound role in
• Class separability the recognition performance.
depends strongly on the  x1 
choice of descriptors. x 
x  x   2
x   1 
 x2   
 xn 
C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

1
 23/5/2013

7 String descriptors 8 String descriptors (cont.)

• Description of structural relationships. • Staircase pattern described by a head-to-tail
structural relationship.
• Example: fingerprint recognition.
• The rule allows only alternating pattern.
• Primitive components that describe.
• It excludes other types of structures.
fingerprint ridge properties.
• Other rules may be defined.
– Interrelationships of print features (minutiae).
• Abrupt endings, branching, merging,
disconnected segments,…
– Relative sizes and locations of print features.

C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

9 Tree descriptors 10 Decision-theoretic methods

• Hierarchical ordering • They are based on decision (discriminant)
• In the satelite image functions.
example, the structural • Let x=[x1, x2,…, xn]T represent a pattern
relationship is defined as:
“composed of” vector.
• For W pattern classes ω1, ω2,…, ωW, the
basic problem is to find W decision functions
d1(x), d2(x),…, dW (x)
with the property that if x belongs to class ωi:

di (x)  d j (x) for j  1, 2,...,W ; j  i

C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

11 Decision-theoretic methods (cont.) 12 Decision-theoretic methods (cont.)

• The decision boundary separating class ωi • Matching: an unknown pattern is assigned
from class ωj is given by the values of x for to the class to which it is closest with
which di (x) = dj (x) or respect to a metric.
– Minimum distance classifier
dij (x)  di (x)  d j (x)  0 • Computes the Euclidean distance between the
unknown pattern and each of the prototype vectors.
– Correlation
• If x belongs to class ωi : • It can be directly formulated in terms of images
• Optimum statistical classifiers
dij (x)  0 for j  1, 2,...,W ; j  i
• Neural networks

C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

2
 23/5/2013

13 Minimum distance classifier 14 Minimum distance classifier (cont.)

The prototype of each pattern class is the mean It is easy to show that selecting the smallest
vector: 1 distance is equivalent to evaluating the
mj 
Nj
x
x j
j j  1, 2,...,W
functions:
Using the Euclidean distance as a measure of 1
d j (x)  xT m j  mTj m j j  1, 2,...,W
closeness: 2
D j (x)  x  m j j  1, 2,...,W
and assigning x to class ωj if dj (x) yields the
We assign x to class ωj if Dj (x) is the smallest largest numerical value. This formulation
distance. That is, the smallest distance implies the agrees with the concept of a decision function.
best match in this formulation.

C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

15 Minimum distance classifier (cont.) 16 Minimum distance classifier (cont.)

• The decision boundary between classes ωi and ωj
is given by:
dij (x)  di (x)  d j (x)
1
 xT (mi  m j )  (mi  m j )T (mi  m j )  0
2

• The surface is the perpendicular bisector of the

line segment joining mi and mj.
• For n=2, the perpendicular bisector is a line, for
n=3 it is a plane and for n>3 it is called a
hyperplane.
C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

17 Minimum distance classifier (cont.) 18 Minimum distance classifier (cont.)

• In practice, the classifier works well when the • Characters designed on a
distance between means is large compared 9x7 grid.
to the spread of each class. • The characters are
scanned horizontally by a
• This occurs seldom unless the system head that is narrower but
designer controls the nature of the input. taller than the character
which produces a 1D
• An example is the recognition of characters signal proportional to the
on bank checks rate of change of the
– American Banker’s Association E-13B font quantity of the ink.
character set. • The waveforms
(signatures) are different
for each character.
C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

3
 23/5/2013

19 Matching by correlation 20 Matching by correlation (cont.)

• We have seen the definition of correlation • Normalized correlation coefficient:
and its properties in the Fourier domain.
  w(s, t )  w  f ( x  s, y  t )  f xy

g ( x, y )   w( s, t ) f ( x  s, y  t )  ( x, y )  s t
1
 2 2
  w( s, t )  w   f ( x  s, y  t )  f xy  
s t 2

 G (u, v)  F * (u, v)W (u, v)  s t s t 

• This definition is sensitive to scale changes • γ (x,y) takes values in [-1,1].

in both images. • The maximum occurs when the two regions
• Instead, we use the normalized correlation are identical.
coefficient.
C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

21 Matching by correlation (cont.) 22 Matching by correlation (cont.)

• The compared
  w(s, t )  w  f ( x  s, y  t )  f xy  windows may be
 ( x, y )  s t
1 seen as random
 2 2
  w( s, t )  w   f ( x  s, y  t )  f xy  
2 variables.
 s t s t  • The correlation
coefficient
• It is robust to changes in measures the linear
the amplitudes. dependence
• Normalization with between X and Y. E [( X  mX )(Y  mY )]
respect to scale and  ( X ,Y ) 
 X X
rotation is a challenging
task.

C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

23 Matching by correlation (cont.) 24 Optimum statistical classifiers

Detection of the eye of the hurricane • A probabilistic approach to recognition.
• It is possible to derive an optimal
Image Template approach, in the sense that, on average, it
yields the lowest probability of committing
classification errors.
• The probability that a pattern x comes
from class ωj is denoted by p(ωj /x).
Correlation
coefficients
Best match • If the classifier decides that x came from
ωj when it actually came from ωi it incurs a
loss denoted by Lij.
C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

4
 23/5/2013

25 Bayes classifier (cont.) 26 Bayes classifier (cont.)

• As pattern x may belong to any of W W

classes, the average loss assigning x to rj (x)   Lkj p(x / k ) P(k )

k 1
ωj is:
• p(x/ωj) is the pdf of patterns of class ωj
W W
1
rj (x)   Lkj p(k / x)   Lkj p(x / k ) P(k )
k 1 p(x) k 1 (class conditional density).
• P(ωj) is the probability of occurrence of
Because 1/p(x) is positive and common to class ωj (a priori or prior probability).
all rj(x) the expression reduces to:
• The classifier evaluates r1(x), r2(x),…,
W
rj (x)   Lkj p(x / k ) P(k ) rW(x), and assigns pattern x to the class
k 1 with the smallest average loss.
C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

27 Bayes classifier (cont.) 28 Bayes classifier (cont.)

• The classifier that minimizes the total • The loss for a wrong decision is generally
average loss is called the Bayes classifier. assigned to a non zero value (e.g. 1)
• It assigns an unknown pattern x to classs • The loss for a correct decision is 0.
ωi if: Lij  1   ij
ri (x)  rj (x), for j  1, 2,...,W
Therefore,
or
W W

W W rj (x)   Lkj p(x / k ) P(k )  (1   jk ) p(x / k ) P(k )

L ki p(x / k ) P(k )   Lqj p(x / q ) P(q ), for all j, j  i k 1 k 1

k 1 q 1  p(x)  p(x /  j ) P( j )

C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

29 Bayes classifier (cont.) 30 Bayes classifier (cont.)

• The Bayes classifier assigns pattern x to classs • The probability of occurrence of each class P(ωj)
ωi if: must be known.
p(x)  p(x / i ) P(i )  p(x)  p(x /  j ) P( j ) – Generally, we consider them equal, P(ωj)=1/W.

or
• The probability densities of the patterns in each
p(x / i ) P(i )  p(x /  j ) P( j ) class P(x/ωj) must be known.
– More difficult problem (especially for multidimensional
which is the computation of decision functions: variables) which requires methods from pdf estimation.
– Generally, we assume:
d j (x)  p(x /  j ) P( j ), j  1, 2,...,W • Analytic expressions for the pdf.
• The pdf parameters may be estimated from sample patterns.
It assigns pattern x to the class whose decision
• The Gaussian is the most common pdf.
function yields the largest numerical value.
C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

5
 23/5/2013

31
Bayes classifier for Gaussian pattern 32
Bayes classifier for Gaussian pattern
classes classes (cont.)
• We first consider the 1-D case for W=2 classes. • In the n-D case:
( xm j ) 2
1
 1  ( x m j )T Cj 1 ( x m j )

d j (x)  p(x /  j ) P( j ) 

1
e
2 2j
P( j ), j  1, 2,..., W p(x /  j )  1/2
e 2

2 j (2 ) n /2
Cj

• Each density is specified by its mean vector and

its covariance matrix:

• For P(ωj)=1/2: m j  E j [ x]
C j  E j [(x  m j )(x  m j )T ]

C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

33
Bayes classifier for Gaussian pattern 34
Bayes classifier for Gaussian pattern
classes (cont.) classes (cont.)
• Approximation of the mean vector and covariance • It is more convenient to work with the natural
matrix from samples from the classes: logarithm of the decision function as it is
monotonically increasing and it does not change
1 the order of the decision functions:
mj  x
Nj x j d j (x)  ln  p(x /  j ) P( j )   ln p(x /  j )  ln P( j )
1
Cj 
Nj
 (x  m
x j
j )(x  m j ) T
1 1
 ln P( j )  ln C j  (x  m j )Cj 1 (x  m j )T
2 2

• The decision functions are hyperquadrics.

C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

35
Bayes classifier for Gaussian pattern 36
Bayes classifier for Gaussian pattern
classes (cont.) classes (cont.)
• If all the classes have the same covarinace Cj=C, • The minimum distance classifier is optimum in the
j=1,2,…,W the decision functions are linear Bayes sense if:
(hyperplanes): − The pattern classes are Gaussian.
1 − All classes are equally to occur.
d j (x)  ln P( j )  x C m j  mTj C1m j
T 1
− All covariance matrices are equal to (the same multiple
2 of) the identity matrix.
• Gaussian pattern classes satisfying these
• Moreover, if P(ωj)=1/W and Cj=I: conditions are spherical clouds (hyperspheres)
1 • The classifier establishes a hyperplane between
d j (x)  xT m j  mTj m j
2 every pair of classes.
which is the minimum distance classifier − It is the perpendicular bisector of the line segment
decision function. joining the centers of the classes
C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

6
 23/5/2013

37
Application to remotely sensed 38
Application to remotely sensed
images images (cont.)
• 4-D vectors.
• Three classes
− Water
− Urban development
− Vegetation
• Mean vectors and
covariance matrices
learnt from samples
whose class is known.
− Here, we will use
samples from the image
to learn the pdf
parameters

C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

Application to remotely sensed

39 40 Structural methods
images (cont.)
• Matching shape numbers
• String matching

C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

41 Matching shape numbers 42 Reminder: shape numbers

• Examples. All closed shapes of order n=4, 6 and 8.
• The degree of similarity, k, between two
• First differences are computed by treating the chain
shapes is defined as the largest order for as a circular sequence.
which their shape numbers still coincide.
− Reminder: The shape number of a boundary is
the first difference of smallest magnitude of its
chain code (invariance to rotation).
− The order n of a shape number is defined as
the number of digits in its representation.

C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

7
 23/5/2013

43 Matching shape numbers (cont.) 44 Matching shape numbers (cont.)

• Let a and b denote two closed shapes which are • Alternatively, the distance between two shapes a
represented by 4-directional chain codes and s(a) and b is defined as the inverse of their degree of
and s(b) their shape numbers. similarity:
1
• The shapes have a degree of similarity, k, if: D ( a, b) 
k

s j (a)  s j (b) for j  4, 6,8,..., k • It satisfies the properties:

s j (a)  s j (b) for j  k  2, k  4,... D ( a, b)  0
D(a, b)  0, iff a  b
• This means that the first k digits should be equal. D(a, c)  max[ D(a, b), D(b, c)]
• The subscript indicates the order. For 4-directional chain
codes, the minimum order for a closed boundary is 4.

C. Nikou – Digital Image Processing C. Nikou – Digital Image Processing

45 String matching

• Region boundaries a and b are code into strings

denoted a1a2a3 …an and b1b2b3 …bm.
• Let p represent the number of matches between
the two strings.
− A match at the k-th position occurs if ak=bk.
• The number of symbols that do not match is:
q  max(| a |,| b |)  p

• A simple measure of similarity is:

number of matches p p
R  
number of mismatches q max(| a |,| b |)  p
C. Nikou – Digital Image Processing