Handsfree Mouse Control Using Eye Movement

Prof. Premanand Ghadekar, Pragya Korpal, Pooja Chendake ,Raksha Bansal, Apurva
Pawar,Siddhi Bhor
Department of Information Technology ,
Vishwakarma Institute Of Technology ,Pune,India.

---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract – -In this paper, an individual human
computer interface system using eye motion is
introduced. Traditionally, human computer interface
uses mouse, keyboard as an input device. This paper
presents hands free interface between computer and
human. The system we describe is real time, on-
intrusive, fast and affordable technique for tracking
facial features. This technology is intended to replace
the conventional computer screen pointing devices for
the use of disabled. The paper presents a novel idea to
control computer mouse cursor movement with human
eyes It controls mouse-moving by automatically
affecting the position where eyesight focuses on, and
simulates mouse-click by affecting blinking action.
However, the proposed vision-based virtual interface
controls system work on various eye movements such
as eye blinking.
Keywords – Eye tracking, mouse movement, Eye-
blinking detection.
Proposed System
Our system is real time which captures a movement of
mouse cursor through face detection and facial
features. It overcomes the existing system by avoiding
the use of external hardware that caused serious eye
damages. It uses a template matching method for eye
extraction instead of using hardware, even as in
previous system the short blinks of eyes were avoided
or neglected. In this system the hard blink is only used
for selecting particular file or folder .With eye detection
it’s first aim is to captured face for the movement of
mouse cursor. Then it reacts as the mouse does.
Design and Implementation
The proposed system which will accomplish our
objectives for projects are defined in those below
stages of figure system flow.
A. Start: Whenever the user will Start the next module
will open.
Stop: Stop will get the user out of the application.
When user will run the module the camera will open
automatically.
B. Recording Videos and Convert it in Frames:
Whenever the camera will open it will start recording
the video. The videos will record the movement of user.
The recorded videos are now converted into number of
frames. The frames recorded per second would be
100fpm .
C. Converting RGB into grey scale: When the videos are
converted into frames they are in RGB format. These
frames which are in RGB format now converted into the
Grey scale format.
D. Eye Detection & Tracking: As, there are only two
colors it will easy to detect the users eye. After the
conversion of frames into grey-scale the eye will be in
white color.
Now, the eye detection approaches will be applied on
the frames to detect eye. Then the system will detect
the eye it will track the movement of the eye.
After these all above steps are done successfully basic
function such as moving the cursor would be
performed.
 Start

Recording Videos and

convert it in frames

Converting RGB into

grey scale

Eye Detection and

Tracking

Stop Basic Operation

Dataset Used
We have used shape_predictor_68_face_landmarks
dataset. The pose takes the form of 68 landmarks.
These are points on the face such as the corners of
the mouth, along the eyebrows, on the eyes, and so
forth. The face detector we use is made using the
classic Histogram of Oriented Gradients (HOG)
feature combined with a linear classifier, an image
pyramid, and sliding window detection scheme.