Orientation Histogram

http://phd.serkangenc.com/orientation/orientation.php

Orientation Histogram for Hand Shape Recognition Systems

1. Purpose
The aim of this project is to test the capability, efficiency and the robustness of the algorithm presented in the paper "Orientation Histograms for Hand Gesture Recognition" by W.T. Freeman and M.Roth.

2. Orientation Histogram and Algorithm

The algorithm depends on the feature vector called orientation histogram used in pattern recognition systems by R.K. McConnell. This feature vector forms a histogram based on the orientation of edges in an image. The system formed by Orientation Histogram is a train-based system. First, user must train the system by giving hand postures for commands. The system finds the nearest hand posture/command for a given hand posture.

2.1 Calculation of Orientation Histogram

In the histogram, 360 degree is grouped in 36 groups, each of which is 10 degree. for each pixel (x,y) in an image I { find the gradient of the pixel dx = I(x,y) - I(x+1,y) dy = I(x,y) - I(x,y+1) find gradient direction or orientation = arctan(dx, dy) find gradient magnitute = sqrt( dx*dx + dy*dy) if ( gradient magnitute > threshold ) // threshold is average_gradient_magnitute*2.0 { find the group of gradient direction and increment the frequency } } blur the initial histogram by [1 4 6 4 1] averaging filter.

2.2 Advantages
1. Very Fast: The calculation of orientation histogram can be performed in real time. Furthermore, the comparison of orientation histograms can be performed using Euclidian distance which is very fast to compute for vectors whose dimensions are 36. Therefore, it is convenient to use in applications require real time performance. 2. Robust: Since it calculates the direction of edges, it is robust to illumination changes. Even light condition changes, the edges are about the same. 3. Translation Invariant: The place of the object in the image is not important in calculation of feature vector. Therefore, translation of object in the image does not change the orientation histogram.

2.3 Disadvantages
1. Rotation dependent: As "Orientation Histogram" name implies, it considers the orientation of edges in the image. Therefore, the rotated version of the object produces very different orientation histogram. 2. Limited Vocabulary: Two different hand positions may produce the same orientation histogram. Therefore, it limits the vocabulary of recognition system. As a result, Hand gestures with different histograms should be selected in applications. In my experiments with a hand, vocabulary size which is more than 8 gestures produces wrong results. 3. Object Distance: The object should occupy a considerable amount of area in the image because small

1 of 5

4/5/2011 8:25 PM

Orientation Histogram

http://phd.serkangenc.com/orientation/orientation.php

objects have little impact on orientation histogram.

3. Algorithm
The overall recognition algorithm is as follows: TRAINING 1. Capture image 2. Convert it into gray-scale image 3. Subsample the image ( to make it real time ) 4. Find the Orientation histogram 5. Save it as a training pattern RECOGNITION 1. Capture image 2. Convert it into gray-scale image 3. Subsample the image 4. Find the Orientation histogram 5. Calculate the Euclidian Distance between the new image and the training patterns 6. Find the pattern with minimum distance

4. Implementation
Compiler : Visual C++ 6.0 Operating System : Windows XP/2000 Libraries: Microsoft Vision SDK, MFC

In the program, Orientation Histogram of the captured image is displayed in real time, the red lines show the

2 of 5

4/5/2011 8:25 PM

Orientation Histogram

http://phd.serkangenc.com/orientation/orientation.php

histogram. The power of the histogram is shown by light blue bar. "power of the histogram" is the number of pixels which are edge points whose gradient magnitutes greater than a threshold value.( if there is no edge in the image, orientation histogram will be empty, and power of the signal will be small. ) In the above figure, the hand is oriented in diagonal direction and the corresponding histogram shows that diagonal directions have more frequencies than others. That is, it works well. Software: 1. Capture image as BMP : This button is used to save the current image as a bmp file. 2. Match Pattern: It finds the closest training pattern to the captured image. The result is the closest pattern's number which is given by button "Add a new Pattern" and the picture of the closest training pattern. 3. Add a new Pattern: This button adds the current image as a training pattern, it gives a pattern no for each training pattern. For example, in the figure above, the diagonal hand shape is saved as traing Pattern No 1. 4. Reset Training Set: It is used to clear training set.

A Popup dialog box shows the image of the closest training pattern.

5. EXPERIMENTS
Experimentally, I could find 5-8 meaningful hand gestures whose histograms are different, adding new training pattern to the system results in poor recognition. However, one can find maximum approximately 10-15 artifical patterns, maybe less than this, to form a robust recognition systems. This is just my observation. However, if the recognition system has maximum 6 hand gestures and requires real time performance, this algorithm can be a good candidate this kind of applications, such as computer games.

3 of 5

4/5/2011 8:25 PM

Orientation Histogram

http://phd.serkangenc.com/orientation/orientation.php

The following is a test case in which there are 9 meaningful patterns used, the test is performed on my desk, that is, not a mono-color artificial background.

Training Set

Recognition Mistakes
Out of 50 trials, 8 of them recognized wrong! Input Pattern Recognized Pattern

The empirical results show that 9 patterns result in about 16% error in recognition. As I said before, 9

4 of 5

4/5/2011 8:25 PM

Orientation Histogram

http://phd.serkangenc.com/orientation/orientation.php

meaningful patterns like above is not appropriate for robust systems. First 6 patterns give a good recognition performance.

6. Downloads
To compile the source code, you have to download VisionSDK first. Source Codes and Win32 Binaries Win32 Binaries

7. References
1. "Orientation Histograms for Hand Gesture Recognition" by W.T. Freeman and M.Roth. 2."Computer Vision for Interactive Computer Graphics", IEEE Computer Graphics and Applications, May-June, 1998, pp 42-53.

BACK

5 of 5

4/5/2011 8:25 PM