BSB663

Image Processing
Pinar Duygulu

Slides are adapted from

Selim Aksoy
Fitting
Fitting
• We’ve learned how to detect
edges, corners, blobs. Now
what?
• We would like to form a
higher-level, more compact
representation of the
features in the image by
grouping multiple features
according to a simple model
Fitting
• Choose a parametric model to represent a set of
features

simple model: lines simple model: circles

complicated model: car

Source: K. Grauman
Fitting: Issues Case study: Line detection

• Noise in the measured feature locations

• Extraneous data: clutter (outliers), multiple lines
• Missing data: occlusions
Canny edge detector

Adapted from Martial Hebert, CMU

Canny edge detector
Canny edge detector
• The Canny operator gives single-pixel-wide images with good
continuation between adjacent pixels.
• It is the most widely used edge operator today; no one has done
better since it came out in the late 80s. Many implementations are
available.
• It is very sensitive to its parameters, which need to be adjusted for
different application domains.
Edge linking
• Hough transform
• Finding line segments
• Finding circles
• Model fitting
• Fitting line segments
• Fitting ellipses
• Edge tracking
Fitting: main idea
• Choose a parametric model to represent a set of features
• Membership criterion is not local
• Cannot tell whether a point belongs to a given model just by looking at that point
• Three main questions:
• What model represents this set of features best?
• Which of several model instances gets which feature?
• How many model instances are there?
• Computational complexity is important
• It is infeasible to examine every possible set of parameters and every possible combination of
features

Adapted from Kristen Grauman

Example: line fitting
• Why fit lines?
• Many objects characterized by presence of straight lines

Adapted from Kristen Grauman

Difficulty of line fitting
• Extra edge points (clutter),
multiple models:
• which points go with which line, if
any?
• Only some parts of each line
detected, and some parts are
missing:
• how to find a line that bridges
missing evidence?
• Noise in measured edge points,
orientations:
• how to detect true underlying
parameters?

Adapted from Kristen Grauman

Voting
• It is not feasible to check all combinations of features by fitting a
model to each possible subset.
• Voting is a general technique where we let each feature vote for all
models that are compatible with it.
• Cycle through features, cast votes for model parameters.
• Look for model parameters that receive a lot of votes.
• Noise and clutter features will cast votes too, but typically their votes
should be inconsistent with the majority of “good” features.

Adapted from Kristen Grauman

Hough transform
• The Hough transform is a method for detecting lines or
curves specified by a parametric function.
• If the parameters are p1, p2, … pn, then the Hough
procedure uses an n-dimensional accumulator array in
which it accumulates votes for the correct parameters
of the lines or curves found on the image.

image m accumulator

y = mx + b
Adapted from Linda Shapiro, U of Washington
Hough transform: line segments
y b

b0

x m0 m
Image space Hough (parameter) space
Connection between image (x,y) and Hough (m,b) spaces
 A line in the image corresponds to a point in Hough space
 To go from image space to Hough space:
 given a set of points (x,y), find all (m,b) such that y = mx + b

Adapted from Steve Seitz, U of Washington

Hough transform: line segments
y b
y0

x0 x m
Image space Hough (parameter) space
Connection between image (x,y) and Hough (m,b) spaces
 A line in the image corresponds to a point in Hough space
 To go from image space to Hough space:
 given a set of points (x,y), find all (m,b) such that y = mx + b
 What does a point (x0, y0) in the image space map to?
 Answer: the solutions of b = -x0m + y0
 This is a line in Hough space Adapted from Steve Seitz, U of Washington
Hough transform: line segments
y b
(x1, y1)
y0
(x0, y0)

b = –x1m + y1
x0 x m
Image space Hough (parameter) space

What are the line parameters for the line that contains both (x0,
y0) and (x1, y1)?
 It is the intersection of the lines b = –x0m + y0 and
b = –x1m + y1

Adapted from Steve Seitz, U of Washington

Hough transform: line segments
• y = mx + b is not suitable (why?)
• The equation generally used is:
d = r sin(θ) + c cos(θ).

c

d is the distance from the line to origin.
d

θ is the angle the perpendicular makes

r with the column axis.

Adapted from Linda Shapiro, U of Washington

Hough transform: line segments

Adapted from Shapiro and Stockman

Hough transform: line segments
Hough transform: line segments
Hough transform: line segments
Hough transform: line segments

Adapted from Kristen Grauman

Hough transform: circles
• Main idea: The gradient vector at an edge pixel points
the center of the circle.
• Circle equations:
• r = r0 + d sin(θ) r0, c0, d are parameters
• c = c0 + d cos(θ)

d
*(r,c)

Adapted from Linda Shapiro, U of Washington

Hough transform: circles

Adapted from Shapiro and Stockman

Hough transform: circles

https://www.mathworks.com/help/images/ref
/imfindcircles.html

http://shreshai.blogspot.com.tr/2015/01/matl
ab-tutorial-finding-center-pivot.html

http://docs.opencv.org/2.4/doc/tutorials/imgp
roc/imgtrans/hough_circle/hough_circle.html
Hough transform: circles

Zhao et al., “Oil Tanks Extraction from High

Resolution Imagery Using a Directional and
Weighted Hough Voting Method”, Journal
of the Indian Society of Remote Sensing,
September 2015
Model fitting
• Mathematical models that fit data not only reveal important structure in the
data, but also can provide efficient representations for further analysis.
• Mathematical models exist for lines, circles, cylinders, and many other shapes.
• We can use the method of least squares for determining the parameters of the
best mathematical model fitting the observed data.
Model fitting: line segments

Adapted from Martial Hebert, CMU

Model fitting: line segments
Model fitting: line segments
Model fitting: line segments
Model fitting: line segments
• Problems in fitting:
• Outliers
• Error definition (algebraic vs. geometric distance)
• Statistical interpretation of the error (hypothesis testing)
• Nonlinear optimization
• High dimensionality (of the data and/or the number of model parameters)
• Additional fit constraints
Model fitting: ellipses
Model fitting: ellipses

Adapted from Andrew Fitzgibbon, PAMI 1999

Model fitting: ellipses

Adapted from Andrew Fitzgibbon, PAMI 1999

Edge tracking
• Mask-based approach uses masks to identify the following events:
• start of a new segment,
• interior point continuing a segment, junction
• end of a segment,
• junction between multiple segments,
• corner that breaks a segment into two.

corner

Adapted from Linda Shapiro, U of Washington

Edge tracking: ORT Toolkit
• The algorithm is called Strider and is like a spider moving along
pixel chains of an image, looking for junctions and corners.
• It identifies them by a measure of local asymmetry.
• When it is moving along a straight or curved segment with no
interruptions, its legs are symmetric about its body.
• When it encounters an obstacle (i.e., a corner or junction) its legs are no
longer symmetric.
• If the obstacle is small (compared to the spider), it soon becomes
symmetrical.
• If the obstacle is large, it will take longer.
• The accuracy depends on the length of the spider and the size of
its stride.
• The larger they are, the less sensitive it becomes.
Edge tracking: ORT Toolkit
The measure of asymmetry is the angle
between two line segments.

L1: the line segment from pixel 1 of the spider

angle 0 to pixel N-2 of the spider
here
L2: the line segment from pixel 1 of the spider
to pixel N of the spider

The angle must be <= arctan(2/length(L2))

Longer spiders allow less of an angle.
Adapted from Linda Shapiro, U of Washington
Edge tracking: ORT Toolkit
• The parameters are the length of the spider and the
number of pixels per step.
• These parameters can be changed to allow for less
sensitivity, so that we get longer line segments.
• The algorithm has a final phase in which adjacent
segments whose angle differs by less than a given
threshold are joined.
• Advantages:
• Works on pixel chains of arbitrary complexity.
• Can be implemented in parallel.
• No assumptions and parameters are well understood.
Example: building detection

by Yi Li @ University of Washington
Example: building detection
Example: object extraction
by Serkan Kiranyaz
Tampere University of Technology
Example: object extraction
Example: object extraction
Example: object extraction
Example: object recognition
• Mauro Costa’s dissertation at the University of Washington for
recognizing 3D objects having planar, cylindrical, and threaded
surfaces:
• Detects edges from two intensity images.
• From the edge image, finds a set of high-level features and their relationships.
• Hypothesizes a 3D model using relational indexing.
• Estimates the pose of the object using point pairs, line segment pairs, and
ellipse/circle pairs.
• Verifies the model after projecting to 2D.
Example: object recognition

Example scenes used. The labels “left” and “right” indicate the direction of the light source.
Example: object recognition
Example: object recognition
Example: object recognition

1 coaxials-
multi encloses
1 1 2 3
encloses
2 ellipse
e e e c
encloses

3 parallel coaxial 2 3 3 2
lines Relationship graph and the corresponding 2-graphs.
Example: object recognition
• Learning phase: relational
indexing by encoding each 2-
graph and storing in a hash
table.
• Matching phase: voting by each
2-graph observed in the image.
Example: object recognition
Incorrect hypothesis

1. The matched features of

the hypothesized object
are used to determine its
pose.
2. The 3D mesh of the object
is used to project all its
features onto the image.
3. A verification procedure
checks how well the object
features line up with edges
on the image.