Feature Detection SubPixel Corner Localization Algorithm

Pi19404
March 17, 2013

Contents

Contents
Feature Detection - SubPixel Corner Localization Algorithm 3

0.1 Introduction . . . . . . . . . 0.2 Subpixel Corner Algorithms 0.2.1 Implementation . . . 0.3 Results . . . . . . . . . . . . . 0.4 Code . . . . . . . . . . . . . . References . . . . . . . . . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

. . . . . .

3 3 8 10 12 12

2 | 13

Feature Detection - SubPixel Corner Localization Algorithm

Feature Detection - SubPixel Corner Localization Algorithm

0.1 Introduction
In the article we will look at refining the corners obtained using feature detection techniques using SubPixel Corner Localization Algorithm.

0.2 Subpixel Corner Algorithms

The basic unit of digital image is pixel which corresponds to integer valued location.However for applications like tracking the integer valued locations may not be accurate enough.Hence a floating point valued representation of corner location is requried .This process of obtaining this is called subpixel corner localization. In a discrete image the pixel intensities are only known at integral locations. In a floating point representation of image ,the pixel intensities are required to be continuous or at subpixel accuracy. The feature/corner detectors help in idenfiying signifciant corner point location upto pixel accuracy or at integral locations.This location is used as initial approximation for subpixel corner location. The subpixel corner localization alogrithms uses the initial approximation as well as local image information to identify the true corner point Let the true corner point be denoted as q(x,y).Let p denote any point in the neighborhood of the true corner point.Let vector q p denote the line joining points p and q and I denote the intensity gradient at point p.

The sub pixel corner detector is based on the assumption that

3 | 13

Feature Detection - SubPixel Corner Localization Algorithm that the image gradient at a point close to the center is orthogonal to every vector from the center to that point. For any point p in the neighborhood of true corner point there will exist a the intensity gradient which will be orthogonal to the vector q p .

Two unknowns ie x and y co-ordindates of point p are required to be determined. This is a linear equation in q x and q y the corner which need to be determined.

@A

@A

the true location of

Thus we require more than one equation to determine the true location of the corner. Let us assume that given the location of corner q we are able to find N such points which satisfy the above equations.This leads to a overconsrained problem.The exact solution may not exists which satisfies all the constraints however a approximate solution can be obtained by least square method. Consider q x y be initial estimate of corner point. Consider a small neighborhood W about the corner point .Let p denote the points in the neighborhood of the corner point.
k T k k i

Compute the dot product at each point of the neighborhood. If point is a true corner point then all residuals will be zero. Thus the aim is to find the point q that best minimizes the sum of squares of residual dot product

4 | 13

Feature Detection - SubPixel Corner Localization Algorithm

@ A@ AaH @ AC @ AaH
I x; y q p
@I @x

xk

xi

@I

@y

yk

yi

C qk

C x

@ A a P @ Ar @ A @ A a P @ A @ A C @ A
g xi ; yi Ii qk pi

k +1

;y

k +1

g xi ; yi

@I

@x

xk

xi

@I

@y

yk

xi

C (x

k+1 ;y k+1 )
@x

aP @ A C

g xi ; yi

A @ A C @ A a H
@I @x

xk

xi

@I

@y

yk

yi

@I

@x

xk

2 g xi ; yi Ix

yk

g xi ; yi Iy

A aP @
Ix xi
@I @y

2 Ix g xi ; yi xi

A C
@I @y

Iy Ix g xi ; yi yi

(1)

C (x

k+1 ;y k+1 )
@x

aP @ A a a
C

g xi ; yi

A @ A C @ A a H
@I @x

xk

yk

yi

xk

g xi ; yi Ix Iy

yk

2 g xi ; yi Iy

A aP @

g xi ; yi Ix Iy xi

CP @

2 Iy g xi ; yi yi

Axk

Dxk

C C

Byk

E yk

5 | 13

Feature Detection - SubPixel Corner Localization Algorithm

Axk

Dxk

C C

Byk

E yk

a a

Solving the equations we obtain new values

A D

B E

 
xk yk

 
C F

 
xk yk

a a a

A D

B E

1  
C F

 
xk yk


AE

BD

E D

 
C F

 
xk yk

CE AF

BF CD


(2)

AE

BD

xk

a a a a

AE

BD

@ @

CE

BF

A A

yk

AE

BD

AF

CD

B = D
xk
AE

1 2 @C E BF A
B

yk

AE

1 2 @AF
B

CD

A
Ix Iy ; E

aP @ aP @

2 g xi ; yi Ix ; B 2 g xi ; yi Ix xi

A A

aP C

Ix Iy yi ; D

aP @ A aP @ A C
2 g xi ; yi Iy 2 g xi ; yi Iy yi

Ix Iy xi

To simplify the computation at each iteration the window is choose with center as estimated corner location.The computations are performed on a window translated to the origin.Then the results are translated back to the original position. At each step residual error is computed for new estimated values of corner position. The next step of iterations can be computed will newly estimated corner position as input approximation.

6 | 13

Feature Detection - SubPixel Corner Localization Algorithm A Stopping criteria is required to specify when to stop the iterative process. The stopping criteria can be specified in different ways like Maximum number of iterations ,if residual error falls below threshold,change in estimated values of corner position is below a certian threshold. To give importance to functions near the estimated corners compared to far away points a weighted least squares method can be used where the weights are defined using a gaussian function g x ;y is the values of the weighing function at points x ; y which is centered at the approximate corner position.

Let us look at the computation of derivatives.The rectangular ROI is retrived from the image with subpixel accuracy.The rectangular ROI is a rectangular grid with unit distance between adjacent pixels but the starting/end point can consist of floating point values. The values of pixels at non integer co-ordinates is retrieved using bi-linear interpolation.The 4 neighboring integral pixel location which bound the non integral pixel location are determined. The pixel values at non integral location is computed as weighted combination of neighboring known pixel intensity values. The value at the black spot is the sum of the value at each coloured spot multiplied by the area of the rectangle of the same colour, divided by the total area of all four rectangles Let

xf ; yf

denote the center location and Lower corner lo-

(a) Image

cation the integral approximation of center location x2 ; y1 . The values x lies between x1 ; x2 and the point y lies between y1 ; y2 .
f f

7 | 13

Feature Detection - SubPixel Corner Localization Algorithm

0.2.1 Implementation
The input to the algorithm are as follows : 1. input image 2. approximate corner position 3. neighborhood size 4. stopping criteria 5. weighing function For each corner point win_size ; win_size image.
win

xk ; yk

_size

CP

@ C PA

xk ; yk

extract the rectangular area of size about the corner is extracted from the

1 2 3 4 5 6 7 8 9

vector < Point2f > corners ; // vector of corner locatations Mat image ; // original image int _winSize ; // neighborhood size int x1 = max ( corners [ k ]. x - _winSize -1 ,0) ; int y1 = max ( corners [ k ]. y - _winSize -1 ,0) ; int width = _winSize +2; int height = _winSize +2; // neighborhood size Rect roi = Rect ( x1 , y1 , width , height ) ; // extracing image ROI about the corner point Mat local = image ( roi ) ;

For all the points in the window sobel derivatives in x and y directions are compute which are approximation for the first order derivatives of intensity image along the x and y directions required for computations of corner point location.
1 2 3

Mat Dx , Dy ; cv :: Sobel ( local , Dx , CV_32FC (1) ,1 ,0 , _apertureSize ,1 ,0) ; cv :: Sobel ( local , Dy , CV_32FC (1) ,0 ,1 , _apertureSize ,1 ,0) ;

Compute the parameters A,B,C,D,E,F over the specified neighborhood which are require to estimate the new location of corner points. As mentioned earlier the center location will be translated to origin ie the location of pixels range from winS ize= pertureS ize= ; winS ize= pertureS ize= to winS ize= pertureS ize= ; winS ize= pertureS ize= . The corresponding range of derivative matrix is winS ize= ; winS ize=

PA

PC

PC

PA

PA

8 | 13

Feature Detection - SubPixel Corner Localization Algorithm to winS ize= location is

P @H HA.
;

; winS ize=

PA

The location of current estimate of corner

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

double A =0 , B =0 , C =0 , D =0 , E =0 , F =0; int lx =0 , ly =0; for ( int i = _apertureSize /2; i < _winSize ; i ++) { float * dx_ptr = Dx . ptr < float >( i ) ; float * dy_ptr = Dy . ptr < float >( i ) ; ly =i - _winSize /2 - _apertureSize /2; // ( -1 ,0 ,1) for ( int j = _apertureSize /2; j < _winSize ; j ++) { lx =j - winSize /2 - _apertureSize /2; // ( -1 ,0 ,1) double dxx = dx_ptr [ j ]* dx_ptr [ j ]; double dyy = dy_ptr [ j ]* dy_ptr [ j ]; double dxy = dx_ptr [ j ]* dy_ptr [ j ]; A = A + dxx ; B = B + dxy ; E = E + dyy ; C = C + dxx * lx + dxy * ly ; F = F + dxy * Ix + dyy * ly ; } }

Solving the two equations we obtain new estimate for the center corner location. the residual error is computed with estimated corner location which is required for the stopping criteria.
1 2 3 4 5 6 7

// computing denominator double det =( A *E - B * B ) ; det =1./ det ; // translating back to original corner and adding new estimates estimate_corner . x = estimate_corner . x +( C *E - B * F ) * det ; estimate_corner . y = estimate_corner . y +( A *F - C * D ) * det ; double err = pointDist ( estimate_corner , curr_corner ) ;

Stopping criteria used is number of iterations be less than maximim specificed iterations and the change/distance in the estimated corner co-ordinates wrt to previous corner coordinate. is less than a specified values.

iter < _term . maxCount && dist > _term . epsilon

9 | 13

Feature Detection - SubPixel Corner Localization Algorithm This method of estimating pixels wit subpixel accuracy gives good results if estimated corner lies in the neighborhood of the approximate corner location.The approximate corner locations must be a good estimate of the desired corner for the method to provide good results.In the present algorithm the criteria used is that estimated corner should lie within a neighborhood of the approximate corner for it to be considered a good estimate else the approximate corner value is retained.
1 2 3 4

if ( fabs ( corners [ k ]. x - estimate_corner . x ) > _winSize || fabs ( corners [ k ]. y - estimate_corner . y ) ) { estimate_corner = corners [ k ]; }

The bilinear intepolation to extract the rectangle with subpixel accuracy is performed using OpenCV function.
1 2

Mat local ; cv :: getRectSubPix ( image , Size ( _winSize +2*( _apertureSize /2) , _winSize +2*( _apertureSize /2) ) , curr_corner , local ) ;

The Default window size is 15,aperture used for computing derivatives is 3, The maximum number of iterations is 10,the maximum change in distance is 0.01.

0.3 Results

10 | 13

Feature Detection - SubPixel Corner Localization Algorithm Subtle difference introduced by subpixel algorithm can be observed in the following images Better corner locations are obtained after application of subpixel algorithm. The Light blue points are locations obtained after application of subpixel algorithm and yellow points are locations obtained by feature detection algorithm. In the Sample image it can be seen that better corner locations are obtained after subpixel localization. The real adavantage cannot be observed visual in 8 bit image but can be noted that subpixel accuracy algorithm provide locations of pixels with higher accuracy. This will be used to improve the robustness of vision based tracking applications which will be used in later articles.

(b) Subpixel Algorithm

11 | 13

Feature Detection - SubPixel Corner Localization Algorithm

0.4 Code
The class SubPixelCorner defines a class for SubPixelCorner Alcogrithm The input to algorithm is input image and vector of corner locations. The corner locations is a input/output vector which modifies the updated corner location. Code is available in repository https://github.com/pi19404/m19404/ tree/master/FeatureDetector subdirectory.The File names for code containing SubPixelCorner Algorithms are SubPixelCorner.cpp and SubPixelCorner.hpp.

12 | 13

Bibliography

Bibliography
[1] Mauricio Delbracio, Pablo Mus, and Andrs Almansa.  Non-parametric Subpixel Local Point Spread Function Estimation. In: Image Processing On Line 2012 (2012). [2]

doi: 10.5201/ipol.2012.admm-nppsf.

Chris Harris and Mike Stephens.  A combined corner and edge detector. In: In
Proc. of Fourth Alvey Vision Conference.

1988, pp. 147151.

[3]

S. Koelstra and I. Patras.  The FAST-3D Spatio-Temporal Interest Region Detector. In: Workshop on Image Analysis for Multimedia Interactive Services. 2009, pp. 242245.

[4]

Ivan Laptev.  On Space-Time Interest Points. In: Int. J. Comput. Vision 64.2-3

[5]

url: http://dx.doi.org/10.1007/s11263-005-1838-7. pi19404. Overview and Implementation Harris Corner Detection. url: http:
1838-7. / / pi - virtualworld . blogspot . com / 2013 / 02 / feature - detection overview-of-harris.html.
pi19404. Overview and Implementation Harris Corner Detection.

(Sept. 2005), pp. 107123.

issn:

0920-5691.

doi: 10 . 1007 / s11263 - 005 -

[6]

url: http: / / pi - virtualworld . blogspot . com / 2013 / 02 / feature - detection overview-of-harris.html.

Detector.

[7]

pi19404. Overview and Implementation of Good Features to Track Feature

url: http : / / pi - virtualworld . blogspot . com / 2013 / 02 / overview-of-good-features-to-track.html.

augmented reality. In: Advances in Visual Computing (2005), pp. 294302.

[8]

E. Rosten, G. Reitmayr, and T. Drummond.  Real-time video annotations for

[9]

Edward Rosten and Tom Drummond.  Fusing Points and Lines for High Performance Tracking. In: IN INTERNATIONAL CONFERENCE ON COMPUTER
VISION.

Springer, 2005, pp. 15081515.

[10]

Edward Rosten and Tom Drummond.  Machine learning for high-speed corner detection. In: In European Conference on Computer Vision. 2006, pp. 430443.

[11]

Jianbo Shi and C. Tomasi.  Good features to track. In: Computer Vision and
Pattern Recognition, 1994. Proceedings CVPR '94., 1994 IEEE Computer Society Conference on.

1994, pp. 593 600.

doi: 10.1109/CVPR.1994.323794.

13 | 13