Hand gesture based communication for Military Application

ABSTRACT
One of the most precious gifts of nature to the human breed is the ability to express him by
responding to the events occurring in his surroundings. Every normal human being sees,
listens and then reacts to the situations by speaking himself out. But there are some less
fortunate ones who are deprived of this valuable gift. Such people, mainly the deaf and the
dumb, rely on some sort of sign language for communicating their feelings to others.
Generally dumb people use sign language for communication, but they find difficulty in
communicating with others who dont understand sign language. So there is a barrier in
communication between these two communities. This project aims to lower this barrier in
communication.
The main aim of the project is to develop a cost effective system which can give voice
to voiceless people with the help of Smart Gloves. With the proposed work sign language is
converted into speech using flex sensor, accelerometer and microcontroller. It means that
using smart gloves communication will not be a barrier between two different communities.

INTRODUCTION
Man is a social being so it is natural for him to interact and communicate. Communication is
a process of exchanging ideas, thoughts, feelings and information in form of verbal or non
verbal message. Over the world 7.6 million of the total population suffers from deafness and
dumbness. But communication for a person who cannot hear is visual, but not auditory. This
person lacks the amenities which a normal person should own. The big reason behind this is
lack of communication, as deaf people are unable to listen and dumb people are unable to
speak. The sign language is an important and only method of communication for deaf-dumb
persons. As sign language is a formal language employing a system of hand gesture for
communication (by the deaf).
It actually becomes the same problem of two persons which knows two different
language, no one of them knows any common language so its becomes a problem to talk with
each other and so they requires a translator physically which may not be always convenient to
arrange and this same kind of problem occurs in between the Normal Person and the Deaf
person or the Normal Person and the Dumb person.

To overcome this problem, this project has been proposed. In this project the
accelerometer and flex sensors are used for sign language for each sign the respective voice
output is obtained via MP3 module and additionally those people can operate home appliance
like fan and lights through hand gesture movement.

DESCRIPTION
Many embedded systems have substantially different designs according to their
functions and utilities. In this project design, structured modular design concept is adopted
and the system is mainly composed of a single microcontroller, LCD, mp3 module, speaker,
relay, switch, GSM, accelerometer sensor and flex sensor.
The microcontroller located at the centre of the block diagram forms the control unit
of the entire project. Embedded within the microcontroller is a program that helps the
microcontroller to take action based on the inputs provided.
In the above block diagram, toggle switch is provided. When toggle switch is slide to
one side, it indicates, military personnel wants to communicate to base. If toggle switch is
slide to another side, it indicates military personnel admitted in military hospital wishes to
communicate or speak to someone.
In this project the accelerometer and flex sensor will placed on hand glove, the hand
glove will be weared by the person who wants to communicate. Tilting of the palm can be
captured by the accelerometer while Flex sensors can measure the bend of the five fingers
when making a sign. For each sign the voice will be recorded and stored in Sd card. .When
the user performs a gesture the signs are captured by the microcontroller which converts the
analog signals to digital values through its 8-channel ADC. For each sign the respective voice
output is obtained and home appliance like fan and blub is turned on and off through signs.
A liquid crystal display (LCD) is a flat panel display, electronic visual display, based
on Liquid Crystal Technology liquid crystal display consists of an array of tiny segments
(called pixels) that can be manipulated to present information. LCDs are used in a wide range
of applications, including computer monitors, television, instrument panels, aircraft cockpit
displays, signage, etc. They are common in consumer devices such as video players, gaming
devices, clocks, watches, calculators, and telephones.
Flex sensors are passive resistive devices that can be used to detect bending or flexing.

An Accelerometer is a device that measures proper acceleration ("g-force"). Proper

acceleration is not the same as coordinate acceleration (rate of change of velocity). For
example, an accelerometer at rest on the surface of the Earth will measure an
acceleration g= 9.81 m/s2 straight upwards. By contrast, accelerometers in free fall (falling
toward the centre of the Earth at a rate of about 9.81 m/s2) will measure zero.
FN-M16P module is a serial MP3 module that is with a perfect integrated MP3 and WMV
decoder chip. It provides micro SD card driver, and supports FAT16 and FAT32 file systems.
It is able to play back specified sound files and realize other functions through simple serial
commands. In the mean time, this module supports AD key control mode that facilitates users
to develop their jobs in some simple applications. Without the cumbersome underlying
operating, easy to use, stable and reliable are the most important features of this module.
Demo points:

For gesture accelerometer and flex sensors are used.

To obtain voice output mp3 module is used.

BLOCK DIAGRAM
LCD

Accelerometer

Hand Gesture

RL78
RENESAS
MICROCONTROLLER

Military/Patient

SW

Sd card

Mp3
module

GSM

Speaker

Components used
Hardwares used

RL78 Renesas Microcontroller

LCD
GSM
MP3 Player
Accelerometer
Speaker
Switch

Softwares used

CubeSuite+
Renesas Flash Programmer
Embedded c

Applications

With little modification this module can be used to establish communication between
dumb, deaf and blind on commercial basis

ADVANTAGES

This is also the best way to communicate for dumb, deaf and blind.
Low cost.
High efficiency.

DISADVANTAGES
In the absence of power supply, the system can be a failure.
If pre-recorded messages are altered then there will be mis-communication.

FUTURE SCOPE
For project demo concern, we have developed a prototype module. In future, this project can
be taken to the product level. To make this project as user friendly and durable, we need to

make it compact and cost effective. Going further, most of the units can be embedded along
with the controller on a single board with change in technology, thereby reducing the size of
the system.

REFERENCE

Andrew Wilson and Aaron Bobick, Learning visual behavior for gesture analysis, In
Proceedings of the IEEE Symposium on Computer Vision, Coral Gables, Florida, pp.

19-21, November 1995.

Ginu Thomas, A Review of Various Hand Gesture Recognition Techniques, VSRD-

IJEECE, Vol. 1 (7), 2011,pp. 374-383.

Surachai, Stewart, Ahmet, Two Hand Tracking using Color Statistical Model with
the Kmeans Embedded Particle Filter for Hand Gesture Recognition, 7th Computer

Information Systems and Industrial Management Applications, 2008, pp. 201-205.

Pham, Nguyen, TuKhoa, A New Approach to Hand Tracking and Gesture
Recognition by a New Feature Type and HMM, Sixth International Conference on
Fuzzy Systems and Knowledge Discovery, IEEE Computer Society, 2009, pp. 3-6.