Image Analysis and Object Recognition

Basic Concepts from

Image Processing to Image Understanding

BOX

Lectures SoSe 2024

(Course notes for internal use only!)
 Overview
• Shape detection Fitting model into image

– Hough transform
• Line fitting
• Circle fitting known radius and unkown radius
• General contours

– Fourier descriptors is used for shape detection.

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 2
Hough: Model fitting
 Example: Line fitting
• Why fit lines?
– Many (artificial) objects are characterized by presence of straight lines

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 4
 Difficulty of line fitting
this is the data.

• Noise in measured edge points, orientations:

– how to detect underlying parameters?
• Only some parts of each line detected,
and some parts are missing:
– how to find a line that bridges missing
information?
• Extra edge points (clutter), multiple models:
– which points go with which line, if any?

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 5
 Hough transform
• Find imperfect objects with an early type of a voting scheme
Basically : Fitting noised data into a model

• General outline:
– Discretize dual parameter space into bins (accumulator array)
– For each feature point in the image, put a vote in every bin in the
parameter space that could have generated this point
– Find bins that have the most votes
x b

y m
y = mx + b 5

Image space Hough (parameter) space

Data will be available in pixels
© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 6
Hough: Duality of points and lines
y b

This line has one point in parameter space.

b0

x m0 m
Image space Hough (parameter) space

Connection between image (x, y) and Hough (m, b) spaces

 A line in the image corresponds to one point in Hough space
 To go from image space to Hough space:
 Given a set of points (x, y), find all (m, b) such that y = mx + b

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 7
Hough: Duality of points and lines
y b
y0

x0 x m
Image space Hough (parameter) space

Connection between image (x, y) and Hough (m, b) spaces

 What does a point (x , y ) in the image space map to?
0 0
 Answer: the solutions of b = -x0m + y0
 This is a line in Hough space

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 8
 Hough transform: Lines
y b
(x1, y1)
b = –x0m + y0
(x0, y0)

b = –x1m + y1

x m
Image space Hough (parameter) space

What are the line parameters for the line

that contains both (x0, y0) and (x1, y1)?
 It is the intersection of the lines
b = –x0m + y0 and b = –x1m + y1

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 9
 Voting
• It is not feasible to check all combinations of features
by fitting a model to each possible subset.

• Voting is a general technique where we let each feature

vote for all models that are compatible with it
– Cycle through features, cast votes for model parameters.
– Look for model parameters that receive a lot of votes.

• Noise and clutter features will cast votes too, but typically their votes
should be inconsistent with the majority of “good” features.

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 10
 Example: Voting 1/2
y m

accumulator
array

x b
m 1 0 0 0 1 0
0 1 0 1 0 0
0 0 2 0 0 0
0 1 0 1 0 0
1 0 0 0 1 0
b
© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 11
 Example: Voting 2/2
y m

x b
m 1 2 0 1 2 2
1 3 3 3 3 2
1 3 7 5 2 0
3 3 1 2 2 1
2 1 0 1 2 1
b
© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 12
Parameter space representation
• Problems with the (m, b) space:
– Unbounded parameter domain
– Vertical lines require infinite m
• Alternative: polar representation (Hessian normal form)
This has to be used in assignment.

x cosθ + y sinθ = ρ

ρ (rho) is the shortest distance

from the line to origin
rho shouldn´t be more than diagonal of the image.

θ (theta) is the angle the

perpendicular (normal vector)
makes with the x-axis.

Each point will add a sinusoid in the (θ, ρ) parameter space

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 13
 Example: Polar representation
More points are required to come to conclusion on the rho and theta values.

y ρ

ρ
θ
x θ

image space Hough (parameter) space

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 14
 Algorithm outline
• Initialize discreet accumulator
H(0…179, -d…d) for d = sqrt(w²+h²)
to all zeros
• for each edge point (x, y) in the image ρ
for θ = 0 to 179 Only looping around angle. No need of rho.
ρ = x cos θ + y sin θ
H(θ, ρ) = H(θ, ρ) + 1
end θ
end
• Find the value(s) of (θ, ρ)
where H(θ, ρ) is a local maximum
– The detected line in the image is given by
ρ = x cos θ + y sin θ

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 15
Extension: Incorporating image gradients
• Recall: when we detect an
edge point, we also know
its gradient direction
• This means that the line
is uniquely determined!

• Modified Hough transform:

For each edge point (x, y)
θ = gradient orientation at (x, y)
ρ = x cos θ + y sin θ Instead of divided theta, we calculate the gradient at the point.
This will speed up the process.

H(θ, ρ) = H(θ, ρ) + 1
end
© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 16
Example: Straight line detection

original edge detection found lines

parameter space
Visualize the parameter space.

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 17
 Example: Detected lines

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 18
 Practical details
• Try to get rid of irrelevant features Things to consider in implementation.

– Take only edge points with significant gradient magnitude

• Choose a good grid / discretization
– Too coarse: large votes obtained when too many
different lines correspond to a single bucket
– Too fine: miss lines because some points that are not
exactly collinear cast votes for different buckets

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 19
 Example: Basic illustration
Perfect lines -> perfect parameters

features votes

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 20
 Effect of noise
Peak gets fuzzy and hard to locate
Use smoothing to avoid this clusters.
Run smoothing filter through accumulator array.

features votes
© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 21
 Practical details
• Try to get rid of irrelevant features
– Take only edge points with significant gradient magnitude
• Choose a good grid / discretization
– Too coarse: large votes obtained when too many
different lines correspond to a single bucket
– Too fine: miss lines because some points that are not
exactly collinear cast votes for different buckets
• Increment neighboring bins
(smoothing in accumulator array)
• Who belongs to which line?
– Tag the votes See next page for its application

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 22
Example: Image  Canny  line segments

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 23
 Example: Line interpretation

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 24
 Example: A more complicated image?

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 25
Extension: Cascaded Hough transform
• Let’s go back to the original (m, b) parameterization
• A line in the image maps to a pencil of lines in the
Hough space
• What do we get with parallel lines or a pencil of lines?
– Collinear peaks in the Hough space!
• So we can apply a second Hough transform to the
output of the first transform to find vanishing points

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 26
Example: Cascaded Hough transform

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 27
Hough: Circle Fitting
Finding circles by Hough transform

Equation of a circle: y

( xi − a ) 2 + ( yi − b) 2 = r 2 r
b (xi, yi)

If radius r is known:
Accumulator array H (a, b) x
a
(2D Hough space)
b

a
© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 29
 Example: Finding coins

original edges (note noise)

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 30
 Example: Finding coins

Hr1 (Penny) Hr2 (Quarters)

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 31
 Example: Finding coins

Note: because of the

different sizes,
a separate Hough
transform was used!

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 32
 Circles with unknown radius
• Main idea: The gradient direction θ at an
edge pixel points from/to the center of the circle
• Circle equations:
a = xi - r cos(θ) a, b, r are parameters
b = yi - r sin(θ)

(xi, yi)
r
(a, b)

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 33
 Hough transform for circles
2D image space 3D Hough parameter space (volume)

y r

−r ⋅∇f ( x, y )

(x,y)
a
+ r ⋅∇f ( x, y )

x b
© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 34
 Example: Variable circle radius

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 35
Hough: Universal transform
 Generalized Hough transform
• We want to find a shape defined by its boundary points
p and a reference point a
• For every boundary point, we can compute the
displacement vector r = a – p as a function of the
gradient orientation θ

θ r(θ)
p

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 37
Example: Generalized Hough transform
image of model r table

P2 P3 Φ ri
0 - 40°
90° 90° 40 - 80°
offline phase
80 --120°
80 120°
r2 r3 120 - 160°
(model 160 - 200°
generation) r1
315° o 200 - 240°
240 - 280°
P1 280 - -320°
280 320°
320 - 360°

online phase r3 r2
r2 r3
search image
(object
recognition)
r1

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 38
 Hough transform
• Pros
– Can deal with non-locality and occlusion
– Can detect multiple instances of a model in a single pass
– Some robustness to noise: noise points unlikely to
contribute consistently to any single bin
• Cons
– Complexity of search time increases exponentially with
the number of model parameters
– Non-target shapes can produce wrong peaks in
parameter space
– It’s hard to pick a good grid size

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 39
Fourier descriptors
 Recognition using Fourier descriptors
- Given: N points at the boundary of a closed 2D region
- Coordinate system is interpreted as the complex plane
- Complex number
xi + j yi y point on

imaginary
region
boundary
is i-th of the N points region (N = 24)

- 1D-DFT of the sequence x

(using the FFT) of N points
real
will be referred to as
Fourier descriptor F
of the contour
- Manipulations in frequency domain allow elimination
of the dependence on position, size and orientation.
© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 41
 Normalization 1/3
• Fourier component F0 corresponds to centroid ≙ translation
• Leaving out F0 when comparing shapes
→ translation invariance

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 42
 Normalization 2/3
• F1 corresponds to the radius
• Standard size: Fourier component F1 = 1
• Then normalization is a division of all coefficients by the absolute value of F1
→ scale invariance

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 43
 Normalization 3/3
• Orientation and starting position affect only the phase:
Simple solution: Remove all phase information and consider only
the absolute values of the descriptor
→ rotation invariance and invariance regarding starting point.

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 44
Similarity of Fourier descriptors

Reconstruction of the shapes of letters “L” and “T” using 2, 3, 4 and 8 Fourier components

• Shape differences are measurable by

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 45
 Example: Detect aero planes

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 46
 Example: Detect aero planes
• 512 contour points
• Calculation of the normalized Fourier descriptor
• Work only on the first halve (256 elements),
the second halve must be mirrored!
• Fine details can be hidden by leaving out high frequency
components
• Keep the first low frequency components,
set the rest to 0
• Then, the inverse Fourier transform provides an
approximation of the original data
→ The first 32 low-frequency components are adequate
for the classification of the aero plane shapes
© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 47
 Example: Recognizing leaves

Berlin university study.

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 48
 Example: Recognizing leaves

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 49
 Example: Classification result

© Volker Rodehorst Lecture Image Analysis & Object Recognition - SoSe 2024 50