4.01 08 2022 - FeatureDescriptors
4.01 08 2022 - FeatureDescriptors
3) Matching: Determine
correspondence between x2 [ x1( 2) , , xd( 2) ]
descriptors in two views
Image transformations
• Geometric
Rotation
Scale
• Photometric
Intensity change
Invariance and equivariance
• We want corner locations to be invariant to photometric transformations and
equivariant to geometric transformations
– Invariance: image is transformed and corner locations do not change
– Equivariance: if we have two transformed versions of the same image,
features should be detected in corresponding locations
– (Sometimes “invariant” and “equivariant” are both referred to as “invariant”)
– (Sometimes “equivariant” is called “covariant”)
Harris detector: Invariance properties
-- Image translation
R R
threshold
Corner
Slide
Slidefrom
fromTinne
TinneTuytelaars
Tuytelaars
Gaussian pyramid
g g
2 2 (very similar to a Difference of Gaussians (DoG) –
g 2 2
2
i.e. a Gaussian minus a slightly smaller Gaussian)
x y
Laplacian of Gaussian
• “Blob” detector minima
=
*
maximum
image Laplacian
Characteristic scale
• We define the characteristic scale as the scale
that produces peak of Laplacian response
characteristic scale
5
4
Lxx ( ) Lyy ( ) 3
2
List of
(x, y, s)
K. Grauman, B. Leibe
Note: The LoG and DoG operators
covariant
are both rotation equivariant
Local features: main components
1) Detection: Identify the
interest points
Kristen Grauman
Feature descriptors
We know how to detect good points
Next question: How to match them?
?
Answer: Come up with a descriptor for each point,
find similar descriptors between the two images
Feature descriptors
We know how to detect good points
Next question: How to match them?
?
Lots of possibilities
– Simple option: match square windows around the point
– State of the art approach: SIFT
• David Lowe, UBC http://www.cs.ubc.ca/~lowe/keypoints/
Invariance vs. discriminability
• Invariance:
– Descriptor shouldn’t change even if image is
transformed
• Discriminability:
– Descriptor should be highly unique for each point
Rotation invariance for
feature descriptors
• Find dominant orientation of the image patch
– E.g., given by xmax, the eigenvector of H corresponding to max (the
larger eigenvalue)
– Or simply the orientation of the (smoothed) gradient
– Rotate the patch according to this angle
0 2
angle histogram
f1 f2
I1 I2
Feature distance
How to define the difference between two features f1, f2?
• Better approach: ratio distance = ||f1 - f2 || / || f1 - f2’ ||
• f2 is best SSD match to f1 in I2
• f2’ is 2nd best SSD match to f1 in I2
• gives large values for ambiguous matches
f1 f2' f2
I1 I2
Feature distance
• Does the SSD vs “ratio distance” change the
best match to a given feature in image 1?
Feature matching example
feature distance
True/false positives
How can we measure the performance of a feature matcher?
50
true match
75
200
false match
feature distance
0.7
true
# true positives
positive
# matching features (positives)
rate
recall
0.7
Single number: Area
true
# true positives
positive Under the Curve (AUC)
# matching features (positives)
rate E.g. AUC = 0.87
recall 1 is the best