Scribd
Upload
406 views46 pages

CS4670: Computer Vision: Lecture 5: Feature Detection and Matching

This lecture discusses feature detection and matching. It begins with announcements about an upcoming quiz and project on feature detection and matching. It then covers reading assignments and motivation for feature extraction in applications like automatic panorama stitching. The lecture discusses why local features are extracted and what makes a good feature. It provides detailed mathematical explanations of the Harris corner detection algorithm, including computation of the second moment matrix and interpretation of its eigenvalues to classify image points. Local features detected using the Harris operator are shown.

Uploaded by

Danni Le
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
406 views46 pages

CS4670: Computer Vision: Lecture 5: Feature Detection and Matching

This lecture discusses feature detection and matching. It begins with announcements about an upcoming quiz and project on feature detection and matching. It then covers reading assignments and motivation for feature extraction in applications like automatic panorama stitching. The lecture discusses why local features are extracted and what makes a good feature. It provides detailed mathematical explanations of the Harris corner detection algorithm, including computation of the second moment matrix and interpretation of its eigenvalues to classify image points. Local features detected using the Harris operator are shown.

Uploaded by

Danni Le
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 46

CS4670: Computer Vision

Noah Snavely

Lecture 5: Feature detection and


matching
Announcements

• Quiz on Monday

• Next project: Feature detection and matching (TBA)

• Newsgroup: cornell.class.cs4670

• Late policy: 1 free day over the semester, otherwise


-30% per day
Reading
• Szeliski: 4.1
Feature extraction: Corners and blobs
Motivation: Automatic panoramas

Credit: Matt Brown


Motivation: Automatic panoramas

HD View
http://research.microsoft.com/en-us/um/redmond/groups/ivm/HDView/HDGigapixel.htm

Also see GigaPan:


http://gigapan.org/
Why extract features?
• Motivation: panorama stitching
– We have two images – how do we combine them?
Why extract features?
• Motivation: panorama stitching
– We have two images – how do we combine them?

Step 1: extract features


Step 2: match features
Why extract features?
• Motivation: panorama stitching
– We have two images – how do we combine them?

Step 1: extract features


Step 2: match features
Step 3: align images
Image matching

by Diva Sian

by swashford
Harder case

by Diva Sian by scgbt


Harder still?

NASA Mars Rover images


Answer below (look for tiny colored squares…)

NASA Mars Rover images


with SIFT feature matches
Feature Matching
Feature Matching
Invariant local features
Find features that are invariant to transformations
– geometric invariance: translation, rotation, scale
– photometric invariance: brightness, exposure, …

Feature Descriptors
Advantages of local features
Locality
– features are local, so robust to occlusion and clutter
Quantity
– hundreds or thousands in a single image
Distinctiveness:
– can differentiate a large database of objects
Efficiency
– real-time performance achievable
More motivation…
Feature points are used for:
– Image alignment (e.g., mosaics)
– 3D reconstruction
– Motion tracking
– Object recognition
– Indexing and database retrieval
– Robot navigation
– … other
What makes a good feature?

Snoop demo
Want uniqueness
Look for image regions that are unusual
– Lead to unambiguous matches in other images

How to define “unusual”?


Local measures of uniqueness
Suppose we only consider a small window of pixels
– What defines whether a feature is a good or bad
candidate?

Credit: S. Seitz, D. Frolova, D.


Local measure of feature uniqueness
• How does the window change when you shift it?
• Shifting the window in any direction causes a big
change

“flat” region: “edge”: “corner”:


no change in all no change along the significant change in
directions edge direction all directions
Credit: S. Seitz, D. Frolova, D.
Harris corner detection: the math

Consider shifting the window W by (u,v)


• how do the pixels in W change?
• compare each pixel before and after by W
summing up the squared differences
(SSD)
• this defines an SSD “error” E(u,v):
Small motion assumption
Taylor Series expansion of I:

If the motion (u,v) is small, then first order approximation is good

Plugging this into the formula on the previous slide…


Corner detection: the math

Consider shifting the window W by (u,v) W


• define an SSD “error” E(u,v):
Corner detection: the math

Consider shifting the window W by (u,v) W


• define an SSD “error” E(u,v):

• Thus, E(u,v) is locally approximated as a quadratic error function


The second moment matrix
The surface E(u,v) is locally approximated by a quadratic form.

Let’s try to understand its shape.


E(u,v)

Horizontal edge:
v
u
E(u,v)

Vertical edge:
v
u
General case
We can visualize H as an ellipse with axis lengths
determined by the eigenvalues of H and orientation
determined by the eigenvectors of H

max, min : eigenvalues of H


Ellipse equation: direction of the
fastest change
u 
[u v] H    const direction of the
slowest change
v 
(max)-1/2
(min)-1/2
Corner detection: the math

xmin

xmax

Eigenvalues and eigenvectors of H


• Define shift directions with the smallest and largest change in error
• xmax = direction of largest increase in E
 max = amount of increase in direction xmax
• xmin = direction of smallest increase in E
 min = amount of increase in direction xmin
Corner detection: the math
How are max, xmax, min, and xmin relevant for feature detection?
• What’s our feature scoring function?
Corner detection: the math
How are max, xmax, min, and xmin relevant for feature detection?
• What’s our feature scoring function?
Want E(u,v) to be large for small shifts in all directions
• the minimum of E(u,v) should be large, over all unit vectors [u v]
• this minimum is given by the smaller eigenvalue (min) of H
Interpreting the eigenvalues
Classification of image points using eigenvalues of M:

2 “Edge”
2 >> 1 “Corner”
1 and 2 are large,
1 ~ 2;
E increases in all
directions

1 and 2 are small;


E is almost constant “Flat” “Edge”
in all directions region 1 >> 2

1
Corner detection summary
Here’s what you do
• Compute the gradient at each point in the image
• Create the H matrix from the entries in the gradient
• Compute the eigenvalues.
• Find points with large response (min > threshold)
• Choose those points where min is a local maximum as features
Corner detection summary
Here’s what you do
• Compute the gradient at each point in the image
• Create the H matrix from the entries in the gradient
• Compute the eigenvalues.
• Find points with large response (min > threshold)
• Choose those points where min is a local maximum as features
The Harris operator
min is a variant of the “Harris operator” for feature detection

• The trace is the sum of the diagonals, i.e., trace(H) = h11 + h22
• Very similar to min but less expensive (no square root)
• Called the “Harris Corner Detector” or “Harris Operator”
• Lots of other detectors, this is one of the most popular
The Harris operator

Harris
operator
Harris detector example
f value (red high, blue low)
Threshold (f > value)
Find local maxima of f
Harris features (in red)
Weighting the derivatives
• In practice, using a simple window W doesn’t
work too well

• Instead, we’ll weight each derivative value


based on its distance from the center pixel
Questions?
Image transformations
• Geometric

Rotation

Scale

• Photometric
Intensity change

You might also like