IAETSD JOURNAL FOR ADVANCED RESEARCH IN APPLIED SCIENCES, VOLUME 4, ISSUE 5, OCT /2017

ISSN (ONLINE): 2394-8442

A REAL TIME STATIC HAND GESTURE RECOGNITION FOR

HUMAN COMPUTER INTERACTION
Vikas Marathe [1], Dimpal Korake [2]

[1] Assistantprofessor, Dept. of E&TC, N.B.N. Sinhgad College of Engineering, Solapur University, India
[2] PG Student [E&TC], Dept. of E&TC, N.B.N. Sinhgad College of Engineering, Solapur University, India

ABSTRACT.

Hand Gesture is invariably used in everyday life style. It is so natural way to communicate. Hand gesture
recognition method is widely used in the application area of controlling mouse and/or keyboard functionality, mechanical
system, 3D World, Manipulate virtual objects, Navigate in a Virtual Environment, Human/Robot Manipulation and
Instruction Communicate at a distance.This system consists of four stages: image acquisition, feature extraction, classification
and recognition. In the first stage input image of hand gestures are acquiesced by digital camera in approximate
frame/snapshot rate. In second stage a rotation, translation, scaling and orientation invariant feature extraction method will
be introduce to extract the feature of the input image based on SURF(speeded up robust method). Finally, an artificial neural
network is use to recognize the hand gestures. The performance of the system is tested on real data.

Keywords Hand gestures, Artificial neural network, JAVA library, Gesture recognition Mouse control operation

I. INTRODUCTION

Gesture recognition has gained a lot of importance since few years. Various applications can be controlled using gestures. Face
gestures like lip movements are used to recognize the language spoken, hand gestures are used in various applications like gaming,
mouse control etc. In an application like robot control using hand gestures, the robot responds to hand gestures given by the human.
This hand sign of humans is visually observed by a robot through a camera. A hand gesture recognition system to be able to
successfully replace a mouse or keyboard needs to be able to precisely detect each finger and hand orientation in real time and should
be robust to various changes in hand measurements, rotation, color and lighting. This is a very complex problem and requires
advanced image processing and computer vision concepts.

In this paper, a novel method is proposed to recognize hand gestures in real time with high accuracy and precision. Two
types of hand gestures are used, static and dynamic. Static gestures make gestures by keeping the hand stable. This system uses static
gestures for mouse interfacing application. In this system the images are captured through the webcam. It is segmented to recognize
the hand region. There are different static hand gestures segmented into five class, these are representing one finger-a class, two finger
-b class, three finger -c class, four finger -d class, five finger -e class. RGB images are converted into gray scale images and these ray
scale images are used for hand gesture recognition

II. SYSTEM MODEL

The system proposed in this paper is designed to recognize hand gesture in real-time. The technique that is used to recognize hand
gesture is based on computer vision. The overall system architecture is shows in figure 1. The whole system of hand gesture
recognition divided into five phases: image acquisition, image pre-processing, feature extraction, classification and hand gesture
recognition. The overall model is designed to be a simple gestural interface prototype for various pc applications.

To Cite This Article: Vikas Marathe and Dimpal Korake ,. A Real Time Static Hand Gesture
Recognition for Human Computer Interaction. Journal for Advanced Research in Applied Sciences.
Volume 4, Issue 5, Oct-2017; Pages: 48-54
49. Vikas Marathe and Dimpal Korake ,. A Real Time Static Hand Gesture Recognition for Human Computer
Interaction. Journal for Advanced Research in Applied Sciences. Volume 4, Issue 5, Oct-2017 Pages: 48-54

`To design static hand gesture recognition for Human Computer Interaction. Steps or the overview of system is as shown in following
figure.

A. Image Acquisition :

The user simply is moving his/her hand into view of the camera to begin interaction with the system. The system uses a
Logitech-c270 digital camera to capture the desk area where the users hand is located. The background of the image is white and has
good lighting system. Image is acquiesced by approximate frame rate/snapshot. The image database contains totally 200 images for 5
classes of Hand gesture, each class containing 40images. The images were captured with different signers, different lighting conditions
and with different orientation. For creating the image database, 10 images for each class were captured with 4 signers and archived. All
the gestures in the hand gesture database are static in nature

CLASS DATABASE

Table 1.[Database]
50. Vikas Marathe and Dimpal Korake ,. A Real Time Static Hand Gesture Recognition for Human Computer
Interaction. Journal for Advanced Research in Applied Sciences. Volume 4, Issue 5, Oct-2017 Pages: 48-54

B. Pre-processing:

The captured hand gesture images are in RGB colour space. In the pre-processing step, the input image is first resized into the size
of 200 x 200 pixels. Image resizing is done to reduce the processing time by the computer and to improve the accuracy of the system.
The resized image is then converted into a gray scale image which has the gray level intensity ranging from 0 to 255. To reduce the
effect of illumination changes and scaling. The gray scale image is normalized using the equation that follows.

y = ((x - min) * 255 / (max - min))

Where x = gray scale intensity of original image, y = gray scale intensity of output image after normalization, min = minimum gray
scale intensity value of the original image, max = maximum gray scale intensity value of the original image.

Algorithm 1
1) Start from top-left corner; repeat for each column and row. If sum of all black pixels in row/column>0, Then save column and row
2) Else dont save column/row.

C. Median filter
The median filter is used to remove noise from an image or signal. Such noise reduction is a typical pre-processing step to improve the
results of later processing (for example, edge detection on an image). Median filtering is used in digital image processing because, under
certain conditions, it preserves edges while removing noise.

D. Feature Extraction:
The aim of this phase is to find and extract features that can be used to determine the meaning of a given gesture. Features lead to
a better recognition of hand gestures even if the hand gesture is captured in a different angle. In this our proposed method we used
edge detection Harris Feature and Speeded-Up Robust Features for classification and hand gesture recognition. To extract the feature
of the hand gesture we developed a rotation, scaling, translation and orientation independent feature extraction method.

1) Laplacian Edge Detection: It wishes to build a morphing algorithm which operates on features extracted from target images
automatically. It is a good beginning to find the edges in the target images. Here, we have accomplished this by implementing a
Laplacian Edge Detector.
In Matlab command edge (I,'Canny') detect edges using the Canny method. The Canny method finds edges by looking for
local maxima of the gradient of I. The edge function calculates the gradient using the derivative of a Gaussian filter. This method uses
two thresholds to detect strong and weak edges, including weak edges in the output if they are connected to strong edges. By using two
thresholds, the canny method is less likely than the other methods to be fooled by noise, and more likely to detect true weak edges.

Algorithm 2:
Step 1: Start with an image of a hand as a sample and that is compared with the various types of other hand images.
Step 2: Blur the image this blurring is accomplished by convolving image with a Gaussian
Step 3: Perform the laplacian on this blurred image. It is necessary to perform the laplacian transformation. The gradient which has a
large peak cantered on the edge. By comparing the gradient to a threshold, through the edge. Whenever the threshold is exceeded up
to 10 In this case, an edge is found, but the edge has become "concentrated" due to the thresholding.

2) Speeded-Up Robust Features:

By using SURF feature we developed a rotation, scaling, translation and orientation independent feature extraction method.
Feature Detection and Feature Extraction:
Feature detection: selects regions of an image that have unique content, such as corners or blobs. Feature detection are used
to find points of interest that we can use for further processing. These points do not necessarily correspond to physical structures,
such as the corners of a table. The key to feature detection is to find features that remain locally invariant so that we can detect them
even in the presence of rotation or scale change.

Feature extraction: involves computing a descriptor, which is typically done on regions centred around detected features.
Descriptors rely on image processing to transform a local pixel neighbourhood into a compact vector representation. This new
representation permits comparison between neighbourhoods regardless of changes in scale or orientation. For feature extraction we
used SURF feature.
51. Vikas Marathe and Dimpal Korake ,. A Real Time Static Hand Gesture Recognition for Human Computer
Interaction. Journal for Advanced Research in Applied Sciences. Volume 4, Issue 5, Oct-2017 Pages: 48-54

Algorithm 3:
SURF algorithm:
SURF (Speed Up Robust Features) algorithm, is based on multi-scale space theory and the feature detector is based on
Hessian matrix. Since Hessian matrix has good performance and accuracy. In image I, x = (x, y) is the given point SURF creates a
stack without 2:1 down sampling for higher levels in the pyramid resulting in images of the same resolution. Due to the use of
integral images, SURF filters the stack using a box filter approximation of second-order Gaussian partial derivatives. Since integral
images allow the computation of rectangular box filters in near constant time. The first step consists of fixing a reproducible
orientation based on information from a circular region around the interest point. And second construct a square region aligned to the
selected orientation, and extract the SURF descriptor from it.

E Classification:
We used a feed-forward multilayer ANNs. For this network, we used 20 hidden neurons by calculating the input vector
dimension that is 33 20 and the output layer has 5 neurons because there are 5 outputs of the network. The network is fully
connected and use back-propagation learning algorithm. Train applies the inputs to the new network, calculates the outputs, compares
them to the associated targets, and calculates a mean square error. If the error goal is met, or if the maximum number of epochs is
reached, the training is stopped and train returns the new network and a training record. Otherwise train goes through another epoch.
The figure 2 shows the training of the network. At the beginning the Sum squared error (sse) is big, but after 100 epochs the error of
the network is very small and the neuronal net can recognize the training data with a very small error. If the train and test set is same
the recognition rates are above 100%, otherwise the results are above 91 %. The overall recognition rates of test data are given bellow.

Gradient = 9.9301e-08, at epoch 209

10
10
Best Training Performance is 3.25 at epoch 209
2
gradient

10
Train
0
Best
10
Mean Squared Error (mse)

-10
10
1
10
Mu = 1e-06, at epoch 209
10
10

0
mu

10
0
10
0 20 40 60 80 100 120 140 160 180 200 -10
209 Epochs 10

Training: R=0.85067
10
Data
9 Fit
Y = T

8
Output ~= 0.72*Target + 1.5

1
1 2 3 4 5 6 7 8 9 10
Target

Fig 2: Training of the Neural Network

 52. Vikas Marathe and Dimpal Korake ,. A Real Time Static Hand Gesture Recognition for Human Computer
Interaction. Journal for Advanced Research in Applied Sciences. Volume 4, Issue 5, Oct-2017 Pages: 48-54

F. Mouse control operation:

There is no inbuilt function in MATLAB which can directly access the mouse drivers of the computer. But MATLAB code
support integration with the other language like c,c++,and JAVA .since java is a machine independent language so it is preferred over
the others. A java object is created and it is linked in the mouse drivers.

Result analysis:
INPUT ACTION PERFOREMED
A class -One finger For left click
B class-Two finger For left top click
C class-Three finger For right click
D class-Four finger For right bottom click
E class-Five finger For centre click

Table 2. [Mouse control operation]

III. EXPERIMENTAL ANALYSIS AND RESULT

To develop a real-time hand gesture recognition system we use MATLAB that is a high performance language for computing. To
get efficient response from this network we use 5 samples for each type of hand gesture. Instead, A data set of 40 images after feature
extraction contains 33 20 elements. We have tested the recognition method with 5 gesture types. In Table 1 the recognition
efficiency of hand gesture classification method with several trainers and tester users can be seen. It is tested by a set of 40 samples per
user
1) If the train and test set is same the recognition rates are 100%` and it is tested by a class of 10 sample per user.
No. of Correct Error rate
Class Gesture Type frames recognition
rate

40 100% 0

40 100% 0

40 100% 0

40 100% 0

E 40 100% 0

OVERALL RESULT 200 100% 0

 53. Vikas Marathe and Dimpal Korake ,. A Real Time Static Hand Gesture Recognition for Human Computer
Interaction. Journal for Advanced Research in Applied Sciences. Volume 4, Issue 5, Oct-2017 Pages: 48-54

We use 40 frames for each type of hand gesture and evaluate correct recognition rate and the error rate of the system. We use
the following equation to find the correct recognition rate:

Correct recognition% = correct recognition / total frames

And find the error rate by using following equation:

Error rate = false recognition / total frames

1) If the train and test set are different the recognition rates are 91%` and it is tested by a class of 10 sample per user.
No.of Correct Error rate
Class Gesture Type frames recognition
rate

20 90% 10%

20 85% 15%

20 95% 5%

20 90% 10%

E 20 95% 5%

OVERALL RESULT 100 91% 9%

Table 2.[Overall result of Testing]

We use 20 frames for each type of hand gesture and evaluate correct recognition rate and the error rate of the system.It
would be more efficient if the system could learn only the faulty detected gestures by interaction with the user. The proposed system
runs in simultaneous real-time at resolution of 1280*700 pixels on a single 1.7GHz Pentium processor. Sometime default frames are
captured by the system. In this case the system cannot recognize hand gestures.

IV.CONCLUSION

The proposed approach applies computer vision methodology that is sinuous, sound act in real time performance. This paper
describes a new method to extract gesture features, which make the system rotation, scaling and translation independent. The gesture
recognition system recognizes 91% hand gestures among the acquiesced frames. It can be made as an effective user interface with
different applications like controlling VLC media player, mouse cursor controlling, home appliance etc.
54. Vikas Marathe and Dimpal Korake ,. A Real Time Static Hand Gesture Recognition for Human Computer
Interaction. Journal for Advanced Research in Applied Sciences. Volume 4, Issue 5, Oct-2017 Pages: 48-54

REFERENCES

[1] M. Zabri Abu Bakar, Rosdiyana Samad, Dwi Pebrianti, Nicolaas Lim Yong AnReal-time Rotation Invariant Hand Tracking Using 3D
Data2014 IEEE International Conference on Control System, Computing and Engineering, 28 - 30 November 2014, Penang, Malaysia.
[2] Zhi-hua Chen,1 Jung-Tae Kim,1 Jianning Liang,1 Jing Zhang,1,2 and Yu-Bo Yuan1 Real-Time Hand Gesture Recognition Using Finger
Segmentation Received 24 April 2014; Accepted 29 May 2014; Published 25 June 2014
[3] Mithun George Jacob, Juan Pablo Wachs, Context-based hand gesture recognition for the operating room, in Pattern Recognition Letters, Volume 36,
15 January 2014, Pages 196- 203, ISSN 0167-8655
[4] Fabio Dominio, Mauro Donadeo, Pietro Zanuttigh, Combining multiple depth-based descriptors for hand gesture Recognition, in Pattern Recognition
Letters, Volume 50, 1 December 2014,Pages 101-111, ISSN 0167-8655
[5] Guillaume Plouffe and Ana-Maria Cretu, Member, IEEE, Static and Dynamic Hand Gesture Recognition in Depth Data Using Dynamic Time
Warping, IEEE, 2015.
[6] Rishabh Agrawal, Nikita Gupta,Real Time Hand Gesture Recognition for Human Computer Interaction, IEEE, 16
[8] Arun Kumar, A Hand gesture recognition for human interaction, vol 4, june 2016.[11]