Feature Detection - Fast Feature Detection

Pi19404
March 3, 2013

Contents

Contents
Feature Detection - Fast Feature Detection 3

0.1 Introduction . . . . . . . . . . . . . . . . 0.2 Fast Feature Detection . . . . . . . 0.2.1 Implementation Details . . . . . Location of Points on Circle . Construction of Lookup Table 0.2.2 Corner Filter . . . . . . . . . . . 0.2.3 Comments . . . . . . . . . . . . . . 0.3 Code . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

3 3 5 5 6 8 9 10 11

2 | 11

Feature Detection - Fast Feature Detection

Feature Detection - Fast Feature Detection

0.1 Introduction
In this article we will look at the Fast Feature detection which describes a machine learning approach to corner detection. CORNER detection is used as the nArst step of many vision tasks such as tracking, localisation, SLAM (simultaneous localisation and mapping), image matching and recognition. The aim of spatial feature detection is to identify point in the image that correpond to object in motion that can be tracked in adjacent image frames robustly. Interest points arise as a result of geometric discontinuities such as corner of real world objects but they may also arise from small patches of texture. Features can be denAned as points with low self-similarity in all directions which is used by harris corner detectors or good features to track feature detector. These algorithms define corners using model or algorithm and apply that algorithm to image directly. In the paper [5] a classifier is trained on the model and then classifier is applied to the model.

0.2 Fast Feature Detection

Assume that we have a corner test candidate.To decide whether the candidate point is a corner the region in the neighborhood of the corner is analyze. In the paper [5] they consider a circle of 16 pixels around the corner pixel.

3 | 11

Feature Detection - Fast Feature Detection This implies point may be a isolated corner or linear edge. N contiguous pixels in the circle are analyzed to check if they are all brighter or darker than pixel being analyzed. This is done by checking if all the pixels are greater than p + t or smaller than p t where p is pixel being tested and t is a small threshold. The value of N is chosen to be 12.This enables rejection of of non corners easily. Thus if N=12 the pixels labelled atleast three

(a) figure taken from http: // www. edwardrosten. com

of the 4 pixels labelled 1,9,4,12 must satisfy the test. Thus if point p is a corner then atleast three of the pixels must be all brighter than Ip + t or darker than Ip t. If both the conditions are not satisfied then the point p cannot be a corner. Thus this test can be used to reject corners.If all these criteria are satisfied then we test the entire line segment.

4 | 11

Feature Detection - Fast Feature Detection

0.2.1 Implementation Details

This can be implemented by testing pixels 1,9,4,12 in that order if all satisfy the test then check the entire line segment. The Threshold can be chosen as pixel between 0 and 255.

Location of Points on Circle

Given a point p and image matrix I we need to quickly indentify the location of the different points on the circle. The location of different points relative to point p are given below 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 (0,-3) (1,-3) (2,-2) (3,-1) (3,0) (3,1) (2,2) (1,3) (0,3) (-1,3) (-2,2) (-3,1) (-3,0) (-3,-1) (-2,-2) (-1,-3)

These relative co-ordinates from the center pixel are stored in a vector and can be used to obtain the location of point on circle given the center pixel.
1 2 3 4 5 6 7 8

vector < Point2f > offset ; offset . push_back ( Point2f (0 , -3) ) ; offset . push_back ( Point2f (1 , -3) ) ; offset . push_back ( Point2f (2 , -2) ) ; offset . push_back ( Point2f (3 , -1) ) ; offset . push_back ( Point2f (3 ,0) ) ; offset . push_back ( Point2f (3 ,1) ) ; offset . push_back ( Point2f (2 ,2) ) ;

5 | 11

Feature Detection - Fast Feature Detection

9 10 11 12 13 14 15 16 17

offset . push_back offset . push_back offset . push_back offset . push_back offset . push_back offset . push_back offset . push_back offset . push_back offset . push_back

( Point2f (1 ,3) ) ; ( Point2f (0 ,3) ) ; ( Point2f ( -1 ,3) ) ; ( Point2f ( -2 ,2) ) ; ( Point2f ( -3 ,1) ) ; ( Point2f ( -3 ,0) ) ; ( Point2f ( -3 , -1) ) ; ( Point2f ( -2 , -2) ) ; ( Point2f ( -1 , -3) ) ;

Construction of Lookup Table

If the test pixel is 0 we need to check if all the pixels surronding pixels are greater than 0+t or less than 0-t. If the test pixel is 255 we need to check if all the surrounding pixels are less than 255-t or greater than 255+t. Since t can take value between 0 and 255 .The range of pixel values observed are 255 255. Let k be the neighboring pixel and we want to test if k>p+t . p-k < -t and k<p-t or p-k > t. If k-p<-t we will label the pixel as 1 and is k-p>t we will lable the pixel as 2. The range of values taken by k-p need to be determined. k can take values from 0 -255 and p and take values from 0 -255. thus the minumum values is -255 and the maximum value 255. Thus we construt a lookup table of size 512. for values of k-p from -255 to 255 which satisfy the condition k-p<-t value 1 is assigned to correponding element of look up table. For values of k-p from -255 to 255 which satisfy the condition k-p<-t value 1 is assigned to correponding element of look up table. For values of k-p which do not satisfy both the condition are assigned value 0 in the lookup table.

int threshold_tab [512];

6 | 11

Feature Detection - Fast Feature Detection

2 3

for ( i = -255; i <= 255; i ++ ) threshold_tab [ i +255] = ( int ) ( i < - threshold ? 1 : i > threshold ? 2 : 0) ;

Consider that values of k and p are given .Thus the task is to compute the index k-p and look up the cooresponding values from the lookup table. Thus given a pixel p(x,y) we need to derive all pixel on the circle using the relative offsets and look up their corresponding values from the lookup table. We have the pointer to the location of the pixel p(x,y).Let x and y offsets be denoted by (dx; dy ).Then location of the offset pixel is given by ptr + dy cols + dx Thus a simple check of pixel labels is performed to reject invalid corners. The aim is to check a if contigious 12 pixel with labels either 1 or 2 are observed. This is performed by process of elimination. The following pair of pixels are checked.If both of elements of any of pairs are zero then it cannot be a corner. The pairs of pixels are (1; 9)(3; 11)(5; 13)(7; 15)(2; 10)(4; 12)(6; 14)(8; 16) Let relative buffer index of elements from the center pixel is represented by a and b. Thus terminate processing of current pixel if both the elements of any of the pairs is zero.This can be easily checked by performing a bitwise OR operation elements,if it is zero then it cannot be a corner.
1 2 3 4 5 6 7 8

uchar * ptr = src . data + i * src . step ; // pointer to the row of center pixel a = checks [ k ]; // relative index from center pixel b = checks [ k +1]; int value1 = threshold_tab [ - ptr [ j + a ]+ v1 +255]; int value2 = threshold_tab [ - ptr [ j + b ]+ v1 +255]; flag = flag &( value1 | value2 ) ; // check if both of index pairs are 0 if ( flag ==0) break ;

7 | 11

Feature Detection - Fast Feature Detection

If all the tests are positive we need to check if we obtain a continous sequence of 1 or 2 If we obtain a continuous sequence of length 12 then a corner is detected. This will provide us with a list of corners.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

int count =0; // corner count for ( int k =0; k <2; k ++) // for dark and bright thresholds { count =0; for ( int l =0; l < offset . size () ; l ++) // parsing the circle { int index = j + checks1 [ l ]; // index of element on circle int x = ptr [ index ]; // value of pixe ; int flag1 =( k ==0) ?( x < v1 - threshold ) :( x > v1 + threshold ) ; count =( flag1 ==1) ? count +1:0; // set count to 0 if false if ( count > K ) break ; // 12 continous pixel detected } if ( count > K ) { corners1 . push_back ( Point2f (j , i ) ) ; break ; // since only dark or bright pixels >12 on circle } }

0.2.2 Corner Filter

Typicall A corner filtering operations is performed after corner detection. The aim is to retain the significant corner which as spaced at minimum distance from each other to avoid clustering of corners at same location. In the present implementation we have no measure to indicate a strong or weak corner. In the earlier corner detector [4, 3]the eigen values of the matrix indicated strength of corner pixels.In generic terms we can say that a corner response function is required for evaluate the strength of the corner. Thus whenever a corner pixel is detected we need to compute a measure which

8 | 11

Feature Detection - Fast Feature Detection indicates the strength of the corner. The response function used in the present application is sum of absolute difference of the pixels in contigious arc and the center pixel.

R(x; y )

arc I (p)

I (x; y )

(1)

The criteria used in the paper for corner filtering is to remove adjancet corner pixels which have adjacent corners with high values of response function. However the corner filtering described in the earlier corner detectors [4, 3] will used as the purpose is the same.

0.2.3 Comments
A significant performance improvement is obtained of 4x-10x compared to the earlier corner detectors. The present method gives better performance compared to feature descriptiors But the disadvantage is that lot of irrelevant or invalid corners may be detected. Hence the corner filtering stage may contain a lot of elements and this may lead to slighlty longer time for corner filtering. Some significant corners may also be detected in the textured areas which may necessarily not be useful. This can be seen in the demo videos available in code respository. Another disadvantage of the method is not generic and does not generalize well if a larger neighborhood is selected or N different than 12 is chosen. The choice of ordering of pixels from 1-16 assumes a specific distri-

9 | 11

Feature Detection - Fast Feature Detection bution of appearance of neighborhood about the test point. In the paper [5] a machine learning technique is proposed which learns the criteria for corner filtering using decision trees which is supposed to provides good generalization performance We will look at training procedure and detecting corners using machine learning techniques in the future.

0.3 Code
we define a main feature_detector base class containing methods and data common to all feature detector. the fast class is derived class containing specific implementations of algorithms. The code OpenCV code can be found in code repository https://github.com/ pi19404/m19404/tree/master/FEATURE_DETECTOR or https://code.google. com/p/m19404/source/browse/FEATURE_DETECTOR/

10 | 11

Bibliography

Bibliography
[1] Chris Harris and Mike Stephens.  A combined corner and edge detector. In: In
Proc. of Fourth Alvey Vision Conference.

1988, pp. 147151.

[2]

Ivan Laptev.  On Space-Time Interest Points. In: Int. J. Comput. Vision 64.2-3 (Sept. 2005), pp. 107123.

[3]

issn: 0920-5691. doi: 10.1007/s11263-005-1838url: http://dx.doi.org/10.1007/s11263-005-1838-7. pi19404. Overview and Implementation Harris Corner Detection. url: http :
7. / / pi - virtualworld . blogspot . com / 2013 / 02 / feature - detection overview-of-harris.html.
pi19404. Overview and Implementation of Good Features to Track Feature De-

[4]

url: http://pi-virtualworld.blogspot.com/2013/02/overviewof-good-features-to-track.html.
tector.

[5]

Edward Rosten and Tom Drummond.  Machine learning for high-speed corner detection. In: In European Conference on Computer Vision. 2006, pp. 430443.

[6]

Jianbo Shi and C. Tomasi.  Good features to track. In: Computer Vision and
Pattern Recognition, 1994. Proceedings CVPR '94., 1994 IEEE Computer Society Conference on.

1994, pp. 593 600.

doi: 10.1109/CVPR.1994.323794.

11 | 11