A

Technical Seminar Report

On
BLUE EYES
Submitted by
K. PRIYAMVADA (16W91A0527)
Under the Esteemed Guidance of
G. NAGAPPA
Professor, CSE
TO
Jawaharlal Nehru Technological University
In partial fulfilment of the requirements for award of degree of
BACHELOR OF TECHNOLOGY
IN
COMPUTER SCIENCE & ENGINEERING

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

MALLA REDDY INSTITUTE OF ENGINEERING AND TECHNOLOGY
(Sponsored by Malla Reddy Educational society)
(Affiliated to JNTU, Hyderabad)
Maisammaguda, Dhulapally post, Secunderabad-500014.

1
 DECLARATION
I hereby declare that the technical seminar report entitled “BLUE EYES” submitted to Malla
Reddy Institute of Engineering and Technology, affiliated to Jawaharlal Nehru Technological
University Hyderabad (JNTUH), for the award of the degree of Bachelor of Technology in
Computer Science & Engineering is a result of original industrial oriented seminar done by me.
It is further declared that the seminar report or any part thereof has not been previously
submitted to any University or Institute for the award of degree or diploma.

K. PRIYAMVADA
16W91A0527

2
 MALLA REDDY INSTITUTE OF ENGINEERING
& TECHNOLOGY
(Sponsored by Malla Reddy Educational Society)
Affiliated to JNTU, Hyderabad
Maisammaguda, Dhulapally (post via Hakimpet), Sec’Bad-500 014.
Phone: 040-65969674, Cell: 9348161205

Department of Computer Science and Engineering

BONAFIDE CERTIFICATE

This is to certify that this is the bonafide record of the technical seminar report titled “BLUE
EYES” is submitted by K. PRIYAMVADA(16W91A0527) of B.Tech in the partial fulfilment
of the requirements for the degree of Bachelor of Technology in Computer Science and
Engineering, Dept. of Computer Science & Engineering and this has not been submitted for
the award of any other degree of this institution.

INTERNAL GUIDE HEAD OF THE DEPARTMENT

Mr. G. Nagappa Mr. Anantha Raman G R

Professor, CSE

TECHNICAL REVIEW COMMITTEE

3
 ACKNOWLEDGEMENT
First and foremost, I am grateful to the principal Dr. M. Ashok for providing me with all the
resources in the college to make my project a success. I thank him for his valuable suggestions
at the time of seminars which encouraged me to give my best in the project.
I would like to express my gratitude to Mr. Anantha Raman G R, Head of the Department of
Computer Science and Engineering for his support and valuable suggestions during the
dissertation work.
I offer my sincere gratitude to my internal guide Mr. G. Nagappa Assistant Professor of
Computer Science and Engineering who has supported me throughout this project with their
patience and valuable suggestions.
I would also like to thank all the supporting staff of the dept. Of CSE and all other departments
who have been helpful directly or indirectly in making the project a success. I am extremely
grateful to my parents for their blessings and prayers for my completion of project this gave
me strength to do my project.

K. PRIYAMVADA
16W91A0527

4
 TABLE OF CONTENTS

LIST OF FIGURES I
ABSTRACT I1
CONTENTS PAGENO
1. INTRODUCTION 1
1.1 Parts of Blue Eye System 2
1.1.1 Overview 2
1.2 Aims 5
1.3 Tracks Used 5

2. AFFECTIVE COMPUTING 7
2.1 Emotion and Computing 8
2.2 Theory 9
2.3 Result 10

3. MANUAL AND GAZE INPUT CASCADE (MAGIC) POINTING 11

3.1.1 Advantages of Liberal Conservative Approach 13
3.1.2 Disadvantages in Liberal Conservative Approach 13
3.2 Implementing Magic Pointing 14

4.EYE TRACKER 15

5. ARTIFICIAL INTELLIGENT SPEECH RECOGNITION 18

5.1 The Technology 18
5.2 Speech Recognition 18

5
6. APPLICATION OF BLUE EYES TECHNOLOGY 20
6.1 Applications of Artificial Speech Recognition 20

7. ADVANTAGES OF BLUE EYES TECHNOLOGY 21

7.1 The Simple User Interface Tracker 21

8. CONCLUSION 22
9. REFERENCES 23

6
 LIST OF FIGURES

LIST PAGE NO.

FIG 1.1.1 Overview of Data Acquisition Unit and Central System Unit 2
FIG 1.1.1 Data Acquisition Unit 2
FIG 1.1.1 Central System Unit 3
FIG 1.1.1 Data Analysis Model 4
FIG 2 Emotional Mouse 8
FIG 2.1 Physiological Emotions for Emotion Mouse 9
FIG 2.2 Showing Physiological Measure 9
FIG 3 Experimental task: point at paired targets 11
FIG 4.1 Literal Magic Technique: cursor placed on vicinity that fixates on 15
FIG 4.2 Conservation Technique: with Intelligent offset 15
FIG 4.3 Bright and dark pupil images on-axis and illumination. 16
FIG 4.4 MRI Scanning 16
FIG 6.1 POD 20
FIG 6.1 PONG 20

7
 ABSTRACT
The Technology known as Blue Eyes Technology is growing day by day with something new
innovations in order to make something new and useful every day which would give a helping
hand to the human beings. The new inventions coming up have now reached to technology
which can sense the emotions and feelings as well in order to make work easy for human
beings. The main reason for making this technology successful is to make computers sense the
emotions of humans.
The main objective of Blue Eyes Technology is to design and develop smarter devices also, by
creating devices with the help of emotional intelligence. To make this possible, it uses various
sensing devices in order to evaluate the results. Eventually, creating computational devices
with perceptual sensing abilities that could sense human’s behaviour and can provide
physiological information. Blue Eyes Technology basically uses different types of sensors to
identify physical as well as psychological actions and to extract key information.

8
 1. INTRODUCTION
Imagine yourself in a world where humans interact with computers. You are sitting in
front of your personal computer that can listen, talk, or even scream aloud. It has the
ability to gather information about you and interact with you through special techniques
like facial recognition, speech recognition, etc. It can even understand your emotions at
the touch of the mouse. It verifies your identity, feels your presents, and starts interacting
with you.
You ask the computer to dial to your friend at his office. It realizes the urgency of the
situation through the mouse, dials your friend at his office, and establishes a
connection. The term BLUE EYES Technology computational machines that have
perceptual and sensory ability like those of human beings. It uses non-obtrusive
sensing method, employing most modern video cameras and microphones to identifies
the user’s actions through the use of imparted sensory abilities. The machine can
understand what a user wants, where he is looking at, and even realize his physical or
emotional states.
The U.S. computer giant, IBM has been conducting research on the Blue Eyes technology at
its Alma den Research Centre (ARC) in San Jose, Cal if, since 1997. The ARC is IBMs main
laboratory for basic research. The primary objective of the research is to give a computer the
ability of the human being to assess a situation by using the senses of sight, hearing and touch.
Animal survival depends on highly developed sensory abilities. Likewise, human cognition
depends on highly developed abilities to perceive, integrate, and interpret visual, auditory, and
touch information. Without a doubt, computers would be much more powerful if they had even
a small fraction of the perceptual ability of animals or humans. Adding such perceptual abilities
to computers would enable computers and humans to work together more as partners. Toward
this end, the Blue Eyes project aims at creating computational devices with the sort of
perceptual abilities that people take for granted. Thus Blue eyes are the technology to make
computers sense and understand human behaviour and feelings and react in the proper ways.

9
1.1 PARTS OF BLUE EYES SYSTEM
The major parts of blue eye system are Data Acquisition Unit and Central System Unit. The
tasks of Data Acquisition Unit are to maintain Bluetooth connections, to get information from
the sensor and sending it over the wireless connection, to deliver the alarm messages sent from
the Central System Unit to the operator and handle personalized ID cards. Central System Unit
maintains the other side of the Bluetooth connection by performing on-line data analysis,
buffers incoming sensor data, records the conclusions for further exploration and provides
visualization interface.

1.1.1 Overview

Figure: 1.1.1 Overview of Data Acquisition Unit and Central System Unit

HARDWARE
Data Acquisition Unit:
Data acquisition unit is a major part of a blue eyes system. Its main function is to fetch
physiological data from the sensor and to forward it to the central system unit to be processed.
To accomplish the task the device must manage wireless Bluetooth connections (connection
establishment, authentication and termination) and Personal ID cards and PIN codes provides
operator’s authorization.

Figure 1.1.1 Data Acquisition Unit

10
Communication with operator is carried on using a simple 5-key keyboard, a small LCD
Display and a beeper. When an exceptional situation is detected, the device uses them to notify
the operator. Voice data transferred using a small headset, interfaced to DAU with standard
mini-jack plugs.
Data Acquisition Unit comprises several hardware modules
• Atmel 89c52 microcontroller-system core
• Bluetooth module (based on ROK101008)
• HD44780-small LCD display
• 24C16-12CCEEPROM (on are movable ID card)
• MC145483- 13bit PCM codec
• Jazz Multisensory Interface
• Beeper and LED indicators, voltage level monitors and 6AAbatteries.
Central System Unit:
Central System Unit hardware is the second peer of a wireless-network connection. The box
contains a wireless Bluetooth module (based on ROK101008) and a PCM codec which is
commonly used for voice data transmission. The module is interfaced to a PC using a USB,
serial and parallel cable. Audio data is accessed through standard mini-jack sockets.
To program operator’s personal ID cards we developed a simple programming device. The
programmer is interfaced to a PC using serial and PS/2 (power source) ports. The micro
controller (Atmel-89C2051) inside the unit handles the12CEEPROM-programming and
UART transmission.

Figure 1.1.1 Central System Unit

Software:
Blue Eyes Software’s main task is to look after working operator’s physiological functions. To
assure instant reaction on operator’s condition change software performs real time buffering of
incoming data analysis and alarm triggering.
The Blue Eyes software comprises several functional modules:
Connection manager is responsible for managing the wireless communication between the
Data Acquisition Unit and the Central System Unit. The connection manager handles:
• Communication with CSU Hardware.
11
 • Searching for new devices in covered range.
• Connection authentication.
• Establishing Bluetooth connections.
• Incoming data buffering and sending alerts.
Data analysis module analyses the raw sensor data in order to obtain the information
related to operator’s physiological condition. The separately running Data Analysis Model
supervises each of working operators.
The module consists of a number of smaller analysers to extract different types of information.
Each of Analyzer registers appropriate. Operator Manager or another analyser as a date
consumer and acting as a producer, provides result of analysis. The most important analysers
are:
• saccade detector- Monitors eye movements in order to determine level of operator’s
visual attention.
• pulse rate analyser- uses blood oxygenation signal to compute operator’s pulse rate.
• custom analysers- Recognizes other behaviour than those which are built-in system.
The new modules are created using C4.5 decision tree induction algorithm.

Figure 1.1.1 Data Analysis Model

Visualization Module:
User interface for the supervisors is provided by visualization module. It enables a preview of
selected video source and related sound stream. All the incoming alarm messages are instantly
signalled to the supervisor. The visualization module can be sent in an offline mode, where all
the data is fetched from the database. Watching all the recorded physiological parameters,
alarms, video and audio data the supervisor is able to reconstruct source of selected operator’s
duty.
The physiological data is presented using a set of custom-built GUI control:
➢ A pie-chart used to present a percentage of time the operator was actively acquiring
the visual information.
➢ A VU-meter showing the present value of a parameter time series displaying a history
of selected parameter’s value.

12
1.2 AIMS
1) To design smarter & devices.
2) To create &vices with emotional intelligence.
3) To create computational &vices with perceptual abilities.

The idea of giving computers personality or, more accurately, emotional

intelligence" may seem creepy, but the technologists say such machines would offer
important advantages.

Despite their lightning speed and awesome powers of computation, today's PCs are
essentially deaf, dumb, and blind. They can’t see, they can’t hear, and they certainly don't
care a whit how you feel. Every computer user knows frustration of nonsensical error
messages, buggy software, and abrupt system crashes. We might berate computer as if it was
an unruly child, but, of course, the machine respond". It's ironic that people feel like dummies
in front of their computers, when in fact the computer is the dummy," says Rosalind Picard, a
computer science professor at MIT Lab in Cambridge.

A computer endowed with emotional intelligence, on the other hand, could recognize when
its operator is feeling angry or frustrated and try to respond in an appropriate fashion. Such a
computer might slow down or replay a tutorial program for a confused student, or recognize
when a designer burned out and suggest he take a break. It could even play a recording of
Beethoven's "Moonlight Sonata" if it sensed anxiety or serve up a rousing Springsteen anthem
if it detected lethargy. The possible applications of "emotion technology" extend far beyond
desktop.

A car equipped with an affective computing system could recognize when a driver feeling
drowsy and ad-vise pull over, or it might sense when a stressed-out motorist is about to explode
and warn him to slow down and cool off.
Human cognition depends primarily on the ability to perceive, interpret, and integrate
audio-visuals and sensoring information. Adding extraordinary perceptual abilities to
computers would enable computers to work together with human believes as intimate partners.
Researchers are attempting to add more capabilities to computers that will allow them to
interact like humans, recognize human presents, talk, listen, or even guess their feelings.

1.3 TRACKS USED

Our emotional changes are mostly reflected in our heart pulse rate, breathing rate, facial
expressions, eye movements, voice etc. Hence these are the parameters on which blue
technology is being developed. Making computers see and feel Blue Eyes uses sensing
technology to identify a user's actions and to extract key information. This information is then
analyzed to determine the user's physical, emotional, or informational state, which in turn can
be used to help make the user more productive by performing expected actions or by providing
expected information. Beyond making computers more researchers say there is another
compelling reason for giving machines emotional intelligence. Contrary to the

13
common wisdom that emotions contribute to irrational behavior, studies have shown that
feelings actually play a vital role in logical thought and decision- making. Emotionally
impaired people often find it difficult to make decisions because they fail to recognize the
subtle clues and signals--does this make me feel happy or sad, excited or bored? That help
direct healthy thought processes. It stands to reason, therefore, that computers that can emulate
human emotions are more likely to behave rationally, in a manner we can understand. Emotions
are like the weather. We only pay attention to them when there is a sudden outburst, like a
tornado, but in fact they are constantly operating in the background, helping to monitor and
guide our day-to-day activities.
Picard, who is also the author of the groundbreaking book Affective Computing, argues
that computers should operate under the same principle. "They have tremendous mathematical
abilities, but when it comes to interacting with people, they are autistic," she says. "1f we want
computers to be genuinely intelligent and interact naturally with us, we must give them the
ability to recognize, understand, and even to behave' and express emotions." Imagine the
benefit of a computer that could remember that a particular Internet search had resulted in a
frustrating and futile exploration of cyberspace. Next time, it might modify its investigation to
improve the chances of success when a similar request is made.

14
 2.AFFECTIVE COMPUTING

The process of making emotional computers with sensing abilities is known as affective
computing. The steps used in this are:

1)Giving sensing abilities

2)Detecting human emotions

3)Respond properly

The first step, researchers say, is to give ma-chills the equivalent of the eyes, ears, and other
sensory organs that humans use to recognize and express emotion. To that end, computer
scientists are exploring a variety of mechanisms including voice-recognition software that can
discern only what being said but the tone in which it is said; cameras that can track subtle facial
expressions, eye movements, and hand gestures; and biometric sensors that can measure body
temperature, blood pressure, muscle tension, and other physiological signals associated with
emotion.
In the second step, the computers have to detect even the minor variations of our moods.
For e.g. person may hit the keyboard very fast earlier in the happy mood or in the angry mood.
In the third step the computers have to react in accordance MJ with the emotional states.
Various methods of accomplishing affective computing are:

1) Affect Detection
2) Magic Pointing
3) Suitor
4) Emotional Mouse

1) AFFECT DETECTION
This is the method of detecting our emotional states from the expressions on our face.
Algorithms amenable to real time implementation that extract information from facial
expressions and gestures are being explored. Must of the information is extracted from the
position of the eye rows and the corners of mouth.

2) MAGIC POINTING
MAGIC stands for Manual Acquisition with Gam Tracking Technology. A computer with this
technology could move the cursor by following the direction of the user's eyes. This type of
technology will enable the computer to automatically transmit information related to the screen
that user is gazing at. Also, it will enable the computer to determine, from user's expression, if
he or she understood the information on the screen, before automatically deciding to proceed
to the text program. The user pointing is still done by the hand, but the cursor always appears
at the right position as if by MAGIC. By varying input technology and eye tracking, we get
MAGIC Pointing.

3) SUITOR
SUITOR stands for Simple User Interface Tracker. It implements the method for putting
computational devices in touch with their users changing moods. By watching what we page

15
the user is currently browsing, the SUITOR can find information on that topic. The key that
user simply intervals with the computer as usual and computer infers user interest based on
what it sees the user do.

4) EMOTION MOUSE
This is the mouse embedded with sensors that can sense physiological attributes such as
temperature, Body pressure, pulse rate, and touching style, etc. The computer can determine
user's emotional states by a single touch. IBM is still Performing research on this mouse and
will available in the market within the text two or three years. TIR expected is 75%.

Figure 2 Emotional mouse implemented on a real mouse and mouse developed at IBM
Research Lab
One goal of Human Computer Interaction (HCI) is to make an adaptive, smart computer
system. This type of project could possibly include gesture recognition, facial recognition, eye
tracking, speech recognition, etc. Another non- invasive way to obtain information about a
version is through touch. People use their computers to obtain, store and manipulate data using.
In order to start creating smart computers, the computer must start gaining information
about the user. Our proposed method for gaining user information through touch is via a
computer input device, the mouse. From the physiological data obtained from the user, an
emotional state may be determined which would then related to the task user is currently doing
on the computer. Over a period of time, a user model will built in order to gain a sense of the
user's personality.
The scope of the project is to have the computer adapt to user in order to create a better
working environment where user is more productive. The first steps towards realizing this
goal are described here.

2.1 EMOTION AND COMPUTING

Rosalind Picard (1997) describes why emotions are important to the computing community.
There are two aspects of affective computing: giving computer the ability to detect emotions
and giving the computer the ability to express emotions. Not only are emotions crucial for
rational decision making but emotion detection is an important step to an adaptive computer
system. An adaptive, smart computer system has driving our efforts to detect a person's
emotional state.

16
 By matching a person' s emotional state and the context of the expressed emotion, over a
period of time the person's personality is being exhibited. Therefore, by giving the computer a
longitudinal understanding of the emotional state of its user, the computer could adapt working
style which fits with its user's personality. The result of this collaboration could increase
productivity for user. One way of gaining information from a user non-intrusively is by video.

Figure 2.1 Physiological Emotions for Emotion Mouse

Cameras have used to detect a person's emotional state. We have explored gaining information
through touch. One obvious place to put sensors on the mouse.

2.2. THEORY

Based on Paul Elman 's facial expression work, we see a correlation between a person's
emotional state and a person's physiological measurements. Selected works from Elman and
others on measuring facial behavior describes Elman's Facial Action Coding System (Elman
and Rosenberg, 1997).
One of his experiments involved participants attached to devices to record certain
measurements including pulse, galvanic skin response (GSR), temperature, somatic movement
and blood pressure. He then recorded measurements as participants were instructed to mimic
facial expressions which corresponded to six basic emotions. He defined the six basic emotions
as anger, fear, sadness, disgust, joy and surprise.

Figure 2.2 Showing Physiological Measures

17
From this work, Dryer (1993) determined how physiological measures could be used to
distinguish various emotional states. The measures taken were GSR, heart rate, skin
temperature and General Somatic Activity (GSA). These data were then subject to two
analyses. For first analysis, a multidimensional scaling. (MDS) procedure was used to
determine dimensionality of the data.

2.3. RESULT

The data for each subject consisted of scores for four physiological assessments [GSA, GSR,
pulse, and skin temperature, for each of six emotions (anger, disgust, fear, happiness, sadness,
and surprise)] across five minutes baseline and test sessions. GSA data was sampled 80 times
per second, GSR and temperature were reported approximately 34times second and pulse was
recorded as a beat was detected, approximately I time per second. To account for individual
variance in physiology, we calculated difference the baseline and test scores. Scores that
differed by more than one and a half standard deviations from the mean were treated as missing.
By this criterion, twelve score were removed from analysis. The results show theory emotion
mouse work is fundamentally sound.

18
 3.MANUAL AND GAZE INPUT CASCADED

(MAGIC) POINTING

This work explores a new direction in utilizing eye gaze for computer input. Gaze tracking has
long been considered as an alternative or potentially superior pointing method for computer
input. We believe that many fundamental limitations exist with traditional gam pointing. In
particular, it is unnatural to overload a perceptual channel such as vision with a motor control
task. We therefore propose an alternative approach, dubbed MAGIC (Manual and Gaze Input
Cascaded) pointing. With such an approach, pointing appears to the user to a manual task, used
for manipulation and selection. However, a large portion of cursor movement is eliminated by
warping the cursor to the eye gaze area, which encompasses the target.
Two specific MAGIC pointing techniques, one conservative and one were designed,
analyzed, and implemented with an eye tracker we developed. They were then tested in a pilot
study. This early stage exploration showed that the MAGIC pointing techniques might offer
many advantages, including reduced physical effort and fatigue as compared to manual
pointing, greater and naturalness than traditional gaze pointing, and possibly faster speed than
manual pointing.

In our view, there are two fundamental short comings to the existing gaze pointing
techniques, regardless of the maturity of eye tracking technology.

First, given the one-degree size of the fovea and the subconscious jittery motions that the eyes
constantly produce, eye gaze is precise enough to operate UI widgets such as scrollbars,
hyperlinks, and slider handles Second, perhaps more importantly, the eye, as one of our primary
perceptual devices, has not evolved to be a control organ. Sometimes its movements are
voluntarily controlled while at other times it is driven by external events. With the target
selection by dwell time method, considered more natural than selection by blinking [71, one
has to conscious of where one looks and how long one looks at an object. If one does not look
at a target continuously for a set threshold (e.g 200ms), the target will successfully selected.

Figure 3: Experimental task: point at paired targets

19
 Once the cursor position had been redefined, the user would need to only make a small
movement to, and click on, the target with a regular manual input device. We have designed
two MAGIC pointing techniques, one liberal and the other conservative in terms of target
identification and cursor placement.

The liberal MAGIC pointing technique: cursor placed in the vicinity of a target that user
fixates on. Actuate input device, observe the cursor position and decide in which direction to
steer the cursor. The cost to this method is the increased manual movement amplitude.

The conservative MAGIC pointing technique with "intelligent offset". To initiate a pointing
trial, there are two strategies available to the user. One is to follow "virtual inertia" move
from tie cursor's current position towards the new target the way the user looking at. This is
likely the strategy the user will employ, due to the way the user interacts with today's
interface. The alternative strategy, which may be more advantageous but takes time to learn,
is to ignore previous cursor position and make a motion which most convenient and least
effortful to the user for a given input device.

The goal of the conservative MAGIC pointing method is following. Once the user looks at a
target and moves the input device, cursor will appear "out of the blue" in motion towards
target, on side of target opposite to initial vector. In comparison to the liberal approach, this
conservative approach has both pros and cons. While with this technique the cursor would
never over-active and jump to a place the user does not intend to acquire, it may require more
hand-eye coordination effort both the liberal and the conservative.

MAGIC pointing techniques offer the following potential advantages:

1) Reduction of manual stress and fatigue, since the cross screen long-distance cursor
movement is eliminated from manual control.
2) Practical accuracy level. In comparison to traditional pure gaze pointing whose
accuracy is fundamentally limited by the nature of eye movement, the MAGIC pointing
techniques let the hand complete the pointing task, so they can be as any other manual
input techniques.
3) A more natural mental model for the user. The user does not have to aware of
the role of eye gaze. To user, pointing continues to a manual task, with a cursor
conveniently appearing where it needs to be.
4) Speed. Since the need for large magnitude pointing operations is less than with pure
manual cursor control, it is possible that MAGIC pointing will faster than pure manual
pointing.
5) Improved subjective speed and ease-of-use. Sime tlæ manual pointing amplitude is
smaller, user may perceive the MAGIC pointing system to operate faster and more
pleasantly than pure manual control, even if it operates at same speed or more slowly.

The fourth point wants further discussion. According to well accepted Fit’s Law, manual
pointing time Es logarithmically proportional to the A/W ratio, where A is the movement
distance and W is the target sin. In other words, targets which are smaller or farther away
take longer to acquire.

For MAGIC pointing, since target size remains the same but the cursor movement
distance shortened, the pointing time can hence reduced. It is less clear if eye gaze control
follows Fit’s Law. In Ware and Michelin's study, selection time was shown to logarithmically

20
proportional to target distance, thereby conforming to Fits' Law. To the contrary, Silber and
Jacob [9] found that trial completion time with eye tracking input increases little with
distance, therefore defying Fit’s Law.

In addition to problems with today's eye tracking systems, such as delay, error, and
inconvenience, there may also be many potential human factor disadvantages to MAGIC
pointing techniques we have proposed, including the following

1) With the more MAGIC pointing technique, cursor warping can be overactive at times,
since the cursor moves to the new gaze location whenever eye gaze moves more than a
set distance (e.g., 120 pixels) away from the cursor. This could be particularly
distracting when user is trying to read. It is possible to introduce constraint according
to context. For example, when user’s style appears to follow a text reading pattern,
MAGIC pointing can automatically suppressed.
2) With the more conservative MAGIC pointing technique, uncertainty of the exact
location at which the cursor might may force the user, especially a novice, to adopt
strategy: take a touch (use manual input device to activate cursor), wait (for the cursor
to appear), and move (cursor to target manually). Such a strategy may prolong target
acquisition time. The user may have to learn a novel hand-eye coordination pattern to
efficient with this technique. Gaze position reported by eye tracker Eye tracking
boundary with confidence. True target will be within the circle with probability. The
cursor is warped to the boundary of the gaze area, along the initial actuation vector
previews cursor position, far from target initial manual actuation vector.
3) With pure manual pointing techniques, user, knowing the current cursor location, could
conceivably perform his motor in parallel to visual search. Motor action may stan as
soon as the user's gaze settles on a target. With MAGIC pointing techniques, the motor
action computation (decision) cannot stan until cursor appears. This may negate the
time saving gained from the MAGIC pointing technique's reduction of movement
amplitude. Clearly, experimental (implementation and empirical) work Es needed to
validate, refine, or invent alternative MAGIC pointing techniques.

3.1.1. ADVANTAGES OF LIBERAL CONSERVATIVE APPROACH

• Reduction of manual stress and fatigue.

• Practical accuracy level.
• A more natural mental model for user.
• Faster than pure manual pointing.
• Improved subjective speed and ease of use

3.12. DISDVANTAGES IN LIBERAL CONSERVATIVE APPROACH

• Literal approach is distracting when user is trying to real.

• The motor action computation cannot start until the cursor appears.
• In conservative approach, uncertainty of the exact location prolong the target
acquisition time.

21
3.2 IMPLEMENTING MAGIC POINTING

We programmed the two MAGIC pointing techniques on a Windows NT system. The

techniques work independently from the applications. The MAGIC pointing program takes
data from both the manual input device (of any type, such as a mouse) and the eye tracking
system running either on same machine or on another machine connected via serial port. Raw
data from an eye tracker cannot directly used for gaze-based interaction, due to 'Wise from
image processing, eye movement jitters, and samples taken during (ballistic eye movement)
periods.
The goal of filter design in general is to make the best compromise preserving signal
bandwidth and eliminating unwanted noise. In the case of eye tracking, as Jacob argued, eye
information relevant to interaction lies in fixations.

22
 4.EYE TRACKER

Figure 1. The literal MAGIC pointing technique: cursor is placed in

Vicinity of target that user fixates on

Since the goal of this work is to explore MAGIC pointing as a user interface technique,
we started out by purchasing a commercial eye tracker (ASL Mo&1 5000) after a market
survey. In comparison to the system reported in early studies this system is much more
compact and reliable. However, we felt that it was still not robust enough for a variety of
people with different eye such as pupil brightness and correction glasses.

We hence chose to develop and use our own eye tracking system. Available commercial
systems, such as those made by ISCAN Incorporated, LC Technologies, and Applied Science
Laboratories (ASL), rely on a single light source that is positioned either off the camera axis in
the case of the ISCANETL400 systems, or on-axis in the case of the LCT and the ASL ES04

23
systems.

Figure 4.3 Bright (left) and dark (right) pupil images resulting from on-axis and
illumination. The glints, or corneal reflections, from the on- and light sources can easily
identified the bright points in the iris.

Eye tracking data can be acquired simultaneously with MRI scanning using a system that
illuminates the left eye of a subject with an infrared (IR) source, acquires a video image of that
eye, locates the corneal reflection (CR) often IR source, and in real time calculates/
displays/records the gaze direction and pupil diameter.
floor outline

Figure 4.4. MRI scanning

24
Once the pupil has been detected, the corneal reflection is determined from the dark pupil
image. The reflection is then used to estimate the user's point of gam in terms of the screen
coordinates where the user is looking as an initial calibration procedure, similar to that
required by numerical eye trackers.

25
 5.ARTIFICAL INTELLIGENCE SPEECH RECOGNITION

It is important to consider the environment in which the speech recognition system has to
work. The grammar used by the speaker and accepted by the system, noise level, 'Wise type,
position of the microphone, and speed and manner of the user's speech are some that may
affect the quality of speech recognition When you dial the telephone of a big company, you
are likely to hear sonorous voice of a cultured lady who responds to your call with great
courtesy saying "Welcome to company X. Please give me the extension number you want".
You pronounce the extension number your name, and the name of version you want to
contact. If the called version accepts call, the connection is given quickly. This is artificial
intelligence where an automatic call-handling system is used without employing any
telephone operator.

5.1 THE TECHNOLOGY

Artificial intelligence (Al) involves two basic ideas. First, it involves studying the thought
processes of human being. Second, it deals with representing those via machines (like
computers, robots etc). Al is behavior of a machine, which, if performed by a human being,
would be called intelligent. It makes machines smarter and more useful, Es less expensive than
natural intelligence.
Natural language processing (NLP) refers to artificial intelligence methods of
communicating with a computer in a natural language like English. The main objective of NLP
program is to understand input and initiate action. The input words are scanned and matched
against internally stored known words. identification of a key word causes some action to
taken. In this way, one can communicate with the computer in one's language. No special
commands or computer language are required. There is no need to enter programs in a special
language for creating software.

5.2 SPEECH RECOGNITION

The user speaks to the computer through a microphone, which, in used a simple system may
contain a minimum of three filters. The more the of filters used, higher the probability of
recognition. Presently, switched capacitor digital filters are used because these can be custom-
built in integrated circuit form. These are smaller and cheaper than active filters using
operational amplifiers.
The filter output is then fed to the ADC to translate the analogue signal into digital word.
The ADC samples the filter outputs many times a second. Each sample represents different
amplitude of signal evenly spaced vertical lines represent amplitude of the audio filter output
at the instant of sampling. Each value is then convened to a binary proportional to amplitude
of the sample. A central processor unit (CPU) controls input circuits that are fed by the
ADCS.
A large RAM (random access memory) stores all the digital values in a buffer area. This
digital information, representing the spoken word, is now accessed by CPU to it further. The
normal speech has a frequency range of 200 Hz to 7 kHz. Recognizing a telephone call is more
difficult as it has bandwidth limitation of 30 Hz to 33 kHz.

26
 As explained earlier, the spoken words are processed by filters and ADCs. The binary
representation of each of these words a template or standard, against which the future words
are compared. These templates are stored in the memory. Once the storing process is
completed, the system can go into its active mode is capable of identifying spoken words. As
each word is spoken, it is converted into binary equivalent and stored in RAM. The computer
then Stans searching and compares binary input pattern with the templates, t is to be noted that
even if the same speaker talks same text, there are always slight variations in amplitude or
loudness of signal, pitch, frequency difference, time gap, etc. Due to this reason, there is never
a perfect match template and binary input word. The pattern matching process therefore uses
statistical techniques is designed to look for the best fit.
The values of binary input words are from corresponding values in the templates. If both
values are same, difference is zero and there is perfect match. If not, the subtraction produces
some difference or error. The smaller the error, the better the match. When the best match
occurs, the word is identified and displayed on screen or used in some other manner. The search
takes a considerable amount of time, as the CPU has to make many comparisons before
recognition occurs. This necessitates use of very high-speed processors. A large RAM is also
required as even though a spoken word may last only a few hundred milliseconds, but the same
is translated into many thousands of digital words. It is important to note that alignment of
words and templates are to matched correctly in time, before computing the similarity score.
The process, termed as dynamic time warping, recognizes that different speakers pronounce
the same words at different speeds as well as elongate different pans of the same word. This is
important for the speaker-independent recognizers.

27
 6.APPLICATIONS OF BLUE EYE TECHNOLOGY

One of the main benefits of speech recognition system Es that it lets user do other works
simultaneously. The user can concentrate on observation and manual operations, and still
control machinery by voice input commands. Another major application of speech processing
is in military operations. Voice control of weapons is an example. With reliable speech
recognition equipment, pilots can give commands and information to computers by simply
speaking into their microphones—they don't have to use their hands for this purpose. Another
good example is a radiologist scanning hundreds of X-rays, ultra-sonograms, CT scans and
simultaneously dictating conclusions to a speech recognition system connected to word. The
radiologist can focus his attention on the images rather than writing text. Voice recognition
could also used on computers for making airline and hotel reservations. A user requires simply
stating his needs, to make reservation, cancel a reservation, or making enquiries about schedule.

6.1 APPLICATIONS OF ARTIFICIAL SPEECH RECOGNITION

• To control weapons by voice commands.

• Pilot give commands to computers by speaking into microphones.
• For making airline and hotel reservations.
• For making reservations, canceling reservations or making enquiries.
• Can connected to word processors and instead of writing, simply dictate to them.

Some of the blue eyes enabled devices are discussed below:

1)POD

Figure 6.1 POD

The first blue enabled mass production device was the car manufactured by TOYOTA. It
could keep the driver alert and it could tell to go if he is driving too fast and it could pull the
driver When he feels drowsy it could hear driver sort of interesting When he is getting bored.
2)PONG

Figure 6.1 PONG

IBM released a robot designed for demonstrating the new technology. The Blue Eyes robot
is equipped with a computer capable of analyzing a person's glances and other forms of
expressions of feelings, automatically determine text type of action. IBM has released a robot
called PONG, which is equipped with Blue Eyes technology. PONG is capable of perceiving
the person standing in front of it, smiles when the person calls his name, and expresses
loneliness when it loses sight of the person.

28
 7.ADVANTAGES OF BLUE EYE TECHNOLOGY

7.1 THE SIMPLE USER INTERFACE TRACKER

Computers would have much more powerful, had they gained perceptual and sensory
abilities of the living beings on the earth. What needs to developed is an intimate relationship
between computer and the humans. And the Simple User Interest Tracker (SUITOR) is a
revolutionary approach in this direction. By observing the webpage at bedizen is browsing, the
SUITOR can help by fetching more information at his desktop. By simply noticing where the
user's eyes focus on the computer screen, the SUITOR can more precise in determining his
topic of interest.
The Almaden cognitive scientist who invented SUITOR, "The system presents the latest
stock price or business news stories that could affect IBM. If I read the headline off the
ticker, it pops up the story in a browser window. If I start to read the story, it adds related
stories to the ticker. That's the whole idea of an attentive system—one that attends to what
you are doing, typing, reading, so that it can attend to your information needs.

SUITOR:

1) Help by fetching more information at desktop.

2) Notice where the user's eyes focus on the screen.
3) Fills a scrolling ticker on a computer screen with information related to user’s task.

29
 8.CONCLUSION:
The nineties witnessed quantum leaps designing for improved man machine interactions. The
BLUE EYES technology ensures a convenient way of simplifying life by providing more
delicate and user-friendly facilities in computing devices. Now that we have proven method,
the next step is to improve the hardware. Instead of using modules to gather information about
the user, it will be better to use smaller and less intrusive units.
The day is not far when this technology will push its way into your house hold, making
you more lazy. It may even reach your hand-held mobile device. Any way this is only a
technological forecast.

30
 9.REFERENCES

1) Levin J .19 An eye controlled computers.

2) Silbert. L and R. Jacob, The advantage of eye gazing interactions.
3) Richard A Bolt, Eyes at interface.
4) Colin ware, Harutune H. Mikaelian, An evaluation of eye tracker.
5) www.wikipedia.com
6) www.techreview.com
7) www.almaden.ibm.com
8) www.research.ibm.com
9) www.metropolismag.com
10) www.howstuffworks.com

31