Features
Features
Readings
• Szeliski, Ch 4.1
• (optional) K. Mikolajczyk, C. Schmid, A performance evaluation of local descriptors. In PAMI 27(10):1615-1630
- http://www.robots.ox.ac.uk/~vgg/research/affine/det_eval_files/mikolajczyk_
Feature Descriptors
Advantages of local features
Locality
• features are local, so robust to occlusion and clutter
Distinctiveness:
• can differentiate a large database of objects
Quantity
• hundreds or thousands in a single image
Efficiency
• real-time performance achievable
Generality
• exploit different types of features in different situations
More motivation…
Feature points are used for:
• Image alignment (e.g., mosaics)
• 3D reconstruction
• Motion tracking
• Object recognition
• Indexing and database retrieval
• Robot navigation
• … other
What makes a good feature?
Want uniqueness
Look for image regions that are unusual
• Lead to unambiguous matches in other images
• The solution:
x-
x+
• The trace is the sum of the diagonals, i.e., trace(H) = h11 + h22
• Very similar to - but less expensive (no square root)
• Called the “Harris Corner Detector” or “Harris Operator”
• Lots of other detectors, this is one of the most popular
The Harris operator
Harris
operator
Harris detector example
f value (red high, blue low)
Threshold (f > value)
Find local maxima of f
Harris features (in red)
Invariance
Suppose you rotate the image by some angle
• Will you still pick up the same features?
Scale?
Scale invariant detection
Suppose you’re looking for corners
Slide
Slide
from
from
Tinne
Tinne
Tuytelaars
Tuytelaars
Feature descriptors
We know how to detect good points
Next question: How to match them?
?
Feature descriptors
We know how to detect good points
Next question: How to match them?
?
Lots of possibilities (this is a popular research area)
• Simple option: match square windows around the point
• State of the art approach: SIFT
– David Lowe, UBC http://www.cs.ubc.ca/~lowe/keypoints/
Invariance
Suppose we are comparing two images I1 and I2
• I2 may be a transformed version of I1
• What kinds of transformations are we likely to encounter in
practice?
Invariance
Suppose we are comparing two images I1 and I2
• I2 may be a transformed version of I1
• What kinds of transformations are we likely to encounter in
practice?
40 p
ixels 8 pixels
0 2
angle histogram
f1 f2
I1 I2
Feature distance
How to define the difference between two features f1, f2?
• Better approach: ratio distance = SSD(f1, f2) / SSD(f1, f2’)
– f2 is best SSD match to f1 in I2
– f2’ is 2nd best SSD match to f1 in I2
– gives small values for ambiguous matches
f1 f2' f2
I1 I2
Evaluating the results
How can we measure the performance of a feature matcher?
50
75
200
feature distance
True/false positives
50
true match
75
200
false match
feature distance
0.7
0.7
SIFT usage:
Recognize
charging
station
Communicate
with visual
cards
Teach object
recognition