Scribd
Upload
24 views

Lecture 4 1 Feature Descriptors

The document discusses feature descriptors used in computer vision, including Histogram of Gradients (HoG) descriptors like SIFT and SURF, as well as binary descriptors like BRIEF, ORB, BRISK, and FREAK. SIFT extracts a 16x16 patch around keypoints, computes gradients, divides the window into a 4x4 grid, and concatenates the histograms to create a 128-dimensional descriptor. Binary descriptors are more efficient and compare pairwise intensities to generate binary strings using Hamming distance.

Uploaded by

Eli
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
24 views

Lecture 4 1 Feature Descriptors

The document discusses feature descriptors used in computer vision, including Histogram of Gradients (HoG) descriptors like SIFT and SURF, as well as binary descriptors like BRIEF, ORB, BRISK, and FREAK. SIFT extracts a 16x16 patch around keypoints, computes gradients, divides the window into a 4x4 grid, and concatenates the histograms to create a 128-dimensional descriptor. Binary descriptors are more efficient and compare pairwise intensities to generate binary strings using Hamming distance.

Uploaded by

Eli
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 30

Lecture 4.

1
Feature descriptors
Trym Vegard Haavardsholm
Feature descriptors

• Histogram of Gradients (HoG) descriptors

• Binary descriptors

2
Histogram of Gradients (HOG) descriptors

• Scale Invariant Feature Transform (SIFT)


– David G. Lowe.
"Distinctive image features from scale-invariant keypoints.”
IJCV 60 (2), pp. 91-110, 2004

• Speeded Up Robust Features (SURF)


– Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool.
“Surf: Speeded up robust features.”
Computer Vision–ECCV 2006.
Springer Berlin Heidelberg, 2006. 404-417

3
SIFT detector

4
SIFT detections

5
Patch at detected position, scale, orientation

6
SIFT descriptor

• Extract patch around detected keypoint

• Normalize the patch


to canonical scale and orientation

7
SIFT descriptor

• Extract patch around detected keypoint

• Normalize the patch


to canonical scale and orientation

• Resize patch to 16x16 pixels

8
SIFT descriptor

• Compute the gradients

9
SIFT descriptor

• Compute the gradients


– Unaffected by additive intensity change

10
SIFT descriptor

• Compute the gradients


– Unaffected by additive intensity change

• Apply a Gaussian weighting function

11
SIFT descriptor

• Compute the gradients


– Unaffected by additive intensity change

• Apply a Gaussian weighting function


– Weighs down gradients far from the centre
– Avoids sudden changes in the descriptor
with small changes in the window position

12
SIFT descriptor

• Compute the gradients


– Unaffected by additive intensity change

• Apply a Gaussian weighting function


– Weighs down gradients far from the centre
– Avoids sudden changes in the descriptor
with small changes in the window position

• Divide the patch into 16 4x4 pixels squares

13
SIFT descriptor

• Compute gradient direction histograms


over 8 directions in each square

14
SIFT descriptor

• Compute gradient direction histograms


over 8 directions in each square
– Trilinear interpolation
– Robust to small shifts, while preserving
some spatial information

15
SIFT descriptor

• Compute gradient direction histograms


over 8 directions in each square
– Trilinear interpolation
– Robust to small shifts, while preserving
some spatial information

16
SIFT descriptor

• Concatenate the histograms to obtain


a 128 dimensional feature vector

17
SIFT descriptor

• Concatenate the histograms to obtain


a 128 dimensional feature vector

• Normalize to unit length


– Invariant to multiplicative contrast change

• Threshold gradient magnitudes to avoid


excessive influence of high gradients
– Clamp gradients > 0.2
– Renormalize

18
Example: Feature comparison

19
Example: Feature comparison

20
Example: Feature comparison

21
SIFT summary

• Extract a 16x16 patch around detected keypoint


• Compute the gradients and apply a Gaussian weighting function
• Divide the window into a 4x4 grid of cells
• Compute gradient direction histograms over 8 directions in each cell
• Concatenate the histograms to obtain a 128 dimensional feature vector
• Normalize to unit length

22
SIFT summary

http://www.vlfeat.org/ 23
Binary descriptors
• Extremely efficient construction and comparison

• Based on pairwise intensity comparisons


– Sampling pattern around keypoint
– Set of sampling pairs
– Feature descriptor vector is a binary string: BRISK sampling pattern
F= ∑
0≤ a ≤ N
2a T ( Pa )

1 if I ( Par1 ) > I ( Par 2 )


T ( Pa ) = 
 0 otherwise
• Matching using Hamming distance:

L= ∑
0≤ a ≤ N
XOR( Fa1 , Fa2 )
BRISK sampling pairs

24
Binary descriptors

Method Sampling pattern Orientation Sampling pairs


calculation
BRIEF None None Random

ORB None Moments Learned pairs

BRISK Concentric circles, Comparing Short pairs


More points on gradients of long
outer rings pairs
FREAK Overlapping Comparing Learned pairs
concentric circles, gradients of
more points on preselected 45 BRIEF sampling pairs
inner rings pairs

Credit: Gil Levi 25


Binary descriptors

Method Sampling pattern Orientation Sampling pairs


calculation
BRIEF None None Random

ORB None Moments Learned pairs

BRISK Concentric circles, Comparing Short pairs


More points on gradients of long
outer rings pairs
FREAK Overlapping Comparing Learned pairs
concentric circles, gradients of
more points on preselected 45 ORB sampling pairs
inner rings pairs

Credit: Trzcinski, et al, “Learning Image Descriptors with Boosting”, 2015 26


Binary descriptors

Method Sampling pattern Orientation Sampling pairs


calculation
BRIEF None None Random

ORB None Moments Learned pairs


BRISK sampling pattern
BRISK Concentric circles, Comparing Short pairs
More points on gradients of long
outer rings pairs
FREAK Overlapping Comparing Learned pairs
concentric circles, gradients of
more points on preselected 45
inner rings pairs

BRISK sampling pairs

Credit: Gil Levi 27


Binary descriptors

Method Sampling pattern Orientation Sampling pairs


calculation
BRIEF None None Random
FREAK sampling pattern
ORB None Moments Learned pairs

BRISK Concentric circles, Comparing Short pairs


More points on gradients of long
outer rings pairs
FREAK Overlapping Comparing Learned pairs
concentric circles, gradients of
more points on preselected 45
inner rings pairs

FREAK sampling pairs

28
Binary descriptors

• Often achieves very good performance compared to SIFT/SURF


• Much faster than SIFT/SURF

A. Alahi, R. Ortiz, and P. Vandergheynst. FREAK: Fast Retina Keypoint. In IEEE


Conference on Computer Vision and Pattern Recognition,

29
Binary descriptors

• Gil Levi’s CV blog:


– https://gilscvblog.wordpress.com/

30

You might also like