An Artificial Vision Based Computer Interface

Miguel Mrquez
1
, Erika Chacn
2
, ngelica Ramrez
3
, ngel Gil
4
Universidad Nacional Experimental del Tchira (UNET). Decanato de Investigacin. Laboratorio de
rototipos. !an "ristbal # $ene%&ela. ''() # *+,-.(.'/** Ext. /(+.
0
mmar1&e%2&net.ed&.ve3
*
eri4a.chacon2&net.ed&.ve3
.
angelica.ramire%2&net.ed&.ve3
/
agil2&net.ed&.ve
Abst ract - This research resen!s an ar!i"icial #isi$n % &ase' c$mu!er in!er"ace, (hich main $&)ec!i#e is
su$r!ing han'icae' e$le (i!h m$!$r 'isa&ili!* in !he use $" ers$nal c$mu!ers+ The s*s!em "acili!a!es !he
use $" s$"!(are alica!i$ns !$ in'i#i'uals !ha! ha#e null $r limi!e' m$&ili!* $" !heir uer lim&s+ ,n $r'er !$
acc$mlish !he $&)ec!i#es, !he s*s!em c$un!s $n a -e& camera !ha! all$(s rec$gnizing !he a!!erns $" m$#emen!
$" !he su&)ec!.s "ace, (hich is l$ca!e' in "r$n! $" !he camera &* means $" real !ime image r$cessing+ The
alica!i$n carries $u! an anal*sis $" !he images "$r !he e/ecu!i$n $" !he &asic "unc!i$ns $" !he m$use such as le"!
&u!!$n clicking an'0$r 'ragging $&)ec!s in !he c$mu!er screen+ The "ace 'e!ec!i$n r$cess is carrie' $u! !hr$ugh
!he imlemen!a!i$n $" an alg$ri!hm !ha! uses a casca'e s$r! ke* $" 1aar%2ike !*e+ The in#es!iga!i$n is &ase' $n
!he resen! kn$(le'ge $" ar!i"icial #isi$n, su&%"iel' $" !he ar!i"icial in!elligence (hich $""ers !$$ls $" anal*sis an'
33 image r$cessing in $r'er !$ in!erre! !he ca!ure' scenes+ The alica!i$n (as 'e#el$e' in C44 !$ &e use'
$n -in'$(s 56 la!"$rm+ Tr*$u! $" !he alica!i$n has &een er"$rme' &* 'i""eren! users, sh$(ing e/cellen!
acce!a!i$n an' sh$r! !ime !raining requiremen!s+
1. Introduction
7$(a'a*s !he ers$nal c$mu!ers ha#e &ec$me a !$$l $" 'ail* use in 'i""eren! ac!i#i!ies $" !he s$cie!*8 such !he*
ha#e 'i#erse 'e#ices !ha! all$( !he in!erac!i$n (i!h !he users, am$ng !hem aears !he eriheral use' "$r 'a!a
inu!, &eing !he ke*&$ar' an' !he m$use !he m$s! $ular+ The in!erac!i$n (i!h !hese eriheral requires $" a
minimum h*sical user caa&ili!* !$ $&!ain an e""icien! use8 ne#er!heless a 19: $" !he ($rl'%(i'e $ula!i$n
resen!0'isla* a! leas! s$me kin' $" m$!$r 'isa&ili!* ;1<, !his makes 'i""icul! !he in!erac!i$n &e!(een !hem an'
'e#ices such like !he m$use, hence !he use $" !he c$mu!ers8 i! is "$r !ha! reas$n !ha! arises !he necessi!* !$
'e#el$ing al!erna!i#e means !ha! "acili!a!e !he in!erac!i$n (i!h c$mu!ers+ =r!i"icial #isi$n $""ers !he necessar*
!$$ls "$r imlemen!ing $ne $" !hese al!erna!i#es &* means $" anal*zing ca!ure' images using a camera -e&+
Thr$ugh !he anal*sis $" !he user "ace ges!ures in "r$n! $" !he camera !he ac!i$n $" !he &asic $era!i$ns $" !he
m$use (ill &e emula!e' &* !he s*s!em+
2. General Description
2.1 General diagram of the System
>igure 1 resen!s a 'iagram sh$(ing !he general $era!i$n 'escri!i$n $" !he alica!i$n, c$nsi'ering !he main
r$cesses $" !he 'e#el$e' alica!i$n+
The alica!i$n uses a -e& camera !$ ca!ure !he scene l$ca!e' in "r$n! $" !he c$mu!er, (here, in "irs! ins!ance,
i! searches "$r a "ace+ ?nce a "ace is l$ca!e' in !he scene, !hen i! searches "$r a c$l$r a!!ern, !ha! mus! &e l$ca!e'
in !he "ace i'en!i"ie' in !he scene+ This research ($rk uses a green rec!angular a!!ern l$ca!e' in !he "$rehea' $"
!he in'i#i'ual+ This a!!ern (ill ser#e as !arge! !$ &e "$ll$(e' &* !he image r$cessing m$'ule $" !he s*s!em an'
all$(s !he user !$ l$ca!e !he curs$r in !he screen &* his0her hea' m$#emen!s+ =n$!her &asic "unc!i$n !$ emula!e
is !he click $" !he m$use, !he alica!i$n c$ns!an!l* re#ie(s !he s!a!e $" !he m$u!h $" !he user, !ha! is !$ sa*,
(he!her i! is $en $r cl$se'+ The s*s!em uses !he "requenc* (i!h (hich !he user $ens an' cl$ses !he m$u!h !$
emula!e !he ressing an' releasing $" !he le"! ush &u!!$n $" !he m$use+ ," !he $en m$u!h image "ul"ills cer!ain
am$un! $" h$!$gram is assume' !ha! !he user (ishes !$ main!ain resse' !he le"! ush &u!!$n, !$ c$n'uc! s$me
ac!i$n such as !$ 'rag a "$l'er+

>igure 1+ General 'iagram $" !he alica!i$n
2.2 Description of the Application
sed !ools" The E/!reme 6r$gramming $r 56 is use', (hich $""ers a se! $" !echniques !ha! c$n"$rms a simle
me!h$'$l$g* "$r s$"!(are 'e#el$men!
;2<
, als$ C44 2anguage (as use' al$ng (i!h !he li&raries $" $en c$'e "$r
!he !rea!men! $" ?enC@ images A?en B$urce C$mu!er @isi$nC
;3<
an' ,62 $" ,n!elD, &$!h $" !hem inclu'ing a
series $" #er* use"ul "unc!i$ns "$r !he 'e#el$men! $" !his kin' $" alica!i$ns+ The r$cesses $" !he alica!i$n
(ere m$'ele' using !he Eni"ie' M$'ele' 2anguage AEM2C+
ser #ace Detection" The alica!i$n c$ns!an!l* ca!ures !he images $" !he scene !ha! is in "r$n! $" !he
c$mu!er screen an' in each images i! search "$r a "ace+ This r$cess $" "ace 'e!ec!i$n is ma'e !hr$ugh !he
imlemen!a!i$n $" an alg$ri!hm !ha! uses a casca'e s$r! ke* $" 1aar%2ike !*e
;4<
, !his s$r! ke* l$$ks "$r !he
charac!eris!ics $" !he "ace in a h$riz$n!al (a*, reas$n (h* i!s single use '$es n$! guaran!ee !he 'e!ec!i$n $" !he
user "ace, 'ue !$ !he "ac! !ha! !he "ace can &e incline'+ There"$re, i" !he alg$ri!hm '$es n$! 'e!ec! a "ace in "irs!
ins!ance, au!$ma!icall* r$!a!es !he image 19F 'egrees cl$ck(ise an' search !he ne( image, un!il c$mle!ing G9F+
," !he "ace is n$! e#en 'e!ec!e', is c$me !$ !ake !he $riginal image, an' alies !he same r$ce'ure, &u! !his !ime
c$un!er cl$ck(ise+
The alg$ri!hm is a&le !$ 'e!ec! m$re !han $ne "ace8 reas$n (h* i! is rec$mmen'e' !ha! (i!hin !he #ie(ing angle
$" !he camera $nl* remains !he alica!i$n user+ ?nce !he "ace is 'e!ec!e', a circular &i! ma !rimme' image $"
!he z$ne (here !he "ace (as 'e!ec!e' is s!$re' "$r la!er anal*ses A>igure 2C+
>igure 2+ >ace 3e!ec!i$n
Detection of the pattern to $e follo%ed" ="!er !he r$ce'ure $" 'e!ec!i$n $" !he "ace is "inishe', !he s*s!em
searches in !he "ace image s!$re' !$ i'en!i"* !he c$l$re' a!!ern !ha! (ill ser#e as !arge! !$ m$#e !he curs$r
!hr$ugh !he c$mu!er screen+ = green rec!angular a!!ern is use', (hich is rec$mmen'e' !$ &e l$ca!e' in !he
user.s "$rehea'+ The green c$l$r (as selec!e' 'ue !$ i!s li!!le resence (i!hin !he range $" c$l$rs !ha! can &e
usuall* "$un' in !he human "ace A>igure 3C+
=n alg$ri!hm !$ anal*ze !he G c$m$nen! in !he RGH sace $" c$l$r (as 'e#el$e' !$ carr* $u! !he 'e!ec!i$n $"
!his a!!ern+ The alg$ri!hm cr$sses !he image i/el &* i/el e/!rac!ing $" each $ne i!s c$m$nen!s RGH (i!h !he
in!en!i$n $" #eri"*ing i" !he* "ul"ill !he "$ll$(ing c$n'i!i$nsI G J R, G J H, AG K RC L 29 an' AG K HC L 29+
,n case a i/el sa!is"ies !he men!i$ne' c$n'i!i$ns, !hen !ha! i/el is c$nsi'ere' as ar! $" !he !arge!, !his (a* i! is
e/!rac!e' all i/els &el$nging !$ !he rec!angular a!!ern, an' !hen !$ $&!ain i!s sace c$$r'ina!es A>igure 4C+

>igure 3+ 6a!!ern !$ "$ll$( >igure 4+ E/!rac!i$n $" !he a!!ern

7e/! s!e is !$ i'en!i"* !he c$$r'ina!es $" !he cen!ral $in! $" !he a!!ern, (hich (ill &e use' !$ !ransla!e !he
$si!i$n $" !he image !$ an* $si!i$n in !he c$mu!er screen+ ," !he in'i#i'ual.s "ace is r$!a!e' an angle
'uring !he r$cess $" "ace i'en!i"ica!i$n, !hen a r$!a!i$n a')us!men! mus! &e '$ne as "$ll$(s+ =ssuming !ha! !he
ini!ial c$$r'ina!es $" !he a!!ern cen!ral $in! are 6A51, M1C, an' !he r$!a!i$n angle $" !he $riginal image,
!hen !he $riginal $si!i$n $" !he cen!ral $in! is gi#en &*I
C c$sA N 1 r 5 =
A1C
C A N 1 sen r 6 =
A2C
&o'ement of the Cursor" The m$#emen! $" !he curs$r is gi#en &* !he !ransla!i$n $" !he cen!ral $in! $" !he
a!!ern using screen c$$r'ina!es+ 6re#i$us !$ !his calcula!i$n a mirr$r !rans"$rma!i$n is alie' !$ !he ca!ure'
image s$ !here is n$! c$n"usi$n &e!(een righ! an' le"! !$ !he user+
,n $r'er !$ manage an e""icien! m$#emen! $" !he curs$r, an' all$( !he user !$ $si!i$n elemen!s in !he screen, a
(eigh!e' a#erage $" !he las! "i#e AOC m$#emen!s carrie' $u! &* !he curs$r (as use'+ This r$ce'ure !akes !he las!
O l$ca!i$ns "r$m !he curs$r an' calcula!es a (eigh!e' $si!i$n+ The (eigh! assignmen! gi#es re"erence !$ !he
m$s! recen! $si!i$n+ The calcula!i$n $" !he curs$r (eigh!e' $si!i$n is &ase' $n !he use $" a #ec!$r $" O (eigh!s
genera!e' as "$ll$(s
;O<
I
>$r all 9 P i P 7, (here 7 is !he am$un! $" $si!i$ns !$ &e c$nsi'ere',

i
7eight
i
+
=
1
1
A3C
1+ ,! is necessar* s!an'ar'ize !he c$m$nen!s $" !he #ec!$r Q7eight iR !$ !he in!er#al ;9,1<

=
=
=
1
9
, 9
N 8
8
8
i
i
7eight
7eight
7eight N i
A4C
2+ The $si!i$ns are s!$re' in a #ec!$r, an' r$cee' !$ calcula!e !he ne/! $si!i$n (here !he curs$r
(ill &e l$ca!e'+

i
N i
i
9 x i 9 x
7eight movement osition :inal N S
1
9
C , A C , A
=
=
=
AOC
This (a* i! is manage' !$ $!imize !he m$#emen! $" !he curs$r gi#en a sm$$!h m$#emen! in !he screen+
()ecution of the left clic*" The ac!i$ns !$ ress an' !$ release !he le"! click are carrie' $u! &* !he user !hr$ugh
!he m$u!h $ening $r m$u!h cl$sing ac!i$n+ There"$re, an alg$ri!hm (as 'e#el$e' !ha! all$(s 'e!ec!ing !his
ac!i$n, as (ell as !he !ime !he m$u!h remains $en+ Then &* using =6, "unc!i$ns $" -in'$(sD is $ssi&le !$
e/ecu!e !he ac!i$ns require' &* !he user in !he s*s!em+ This r$cess is carrie' $u! &* means $" !he 'e!ec!i$n $"
!($ ellises A>igure OC, (hich reresen! !he $u!si'e &$r'er an' !he insi'e &$r'er $" !he lis, (hich are searche'
in in"eri$r hal" $" !he image re#i$usl* "il!ere' using a Gaussian sm$$!he' $era!i$n
;T<
+
>igure O+ 3e!ec!i$n $" !he $en m$u!h
The "$un' ellises mus! "ul"ill cer!ain c$n'i!i$ns as "ar as !heir 'imensi$ns, (i!h !he in!en!i$n $" n$! c$n"using
!he m$u!h (i!h an* $!her sha'e $r "$rm !ha! can &e (r$ngl* 'e!ec!e' &* !he alg$ri!hm+
-i!h !he in!en!i$n $" $!imizing !he e/ecu!i$n $" !he click $era!i$n, &e"$re making an* 'ecisi$n regar'ing !he
ac!i$n !$ make, !he alg$ri!hm all$(s #eri"*ing cer!ain s!a!es $" !he user.s m$u!h an' acc$r'ing !$ !his resul! i!
c$n'uc!s !he ac!i$n+ ATa&le 1C
+resent State +re'ious State &ouse Action
?en M$u!h Cl$se' M$u!h 6ress 2e"! Hu!!$n
?en M$u!h ?en M$u!h T$ $si!i$n Curs$r
?en M$u!h Cl$se' M$u!h T$ $si!i$n Curs$r
?en M$u!h ?en M$u!h Release 2e"! Hu!!$n
Ta&le 1+ @eri"ica!i$n $" s!a!es $" !he user.s m$u!h+
=''i!i$nall*, a !imer "$r !he ac!i$n $" !he click is use' !ha! ($rks $" !he "$ll$(ing (a*I
1+ -hen an $ene' m$u!h is 'e!ec!e', !he m$#emen! $" !he curs$r s!$s 'uring O h$!$gram, !his
(a* i! is $ssi&le !$ &e 'e!ermine' i" !he user (ishes !$ c$n'uc! !he clicking ac!i$n $r a''i!i$nall*
he (ishes !$ 'rag an $&)ec!+
2+ ," (hen assing !he O h$!$gram, !he m$u!h is cl$se', i! is c$me !$ release !he click, in !he
$riginal $si!i$n (here i! (as resse'+
3+ ," (hen "inalizing !he O h$!$gram !he m$u!h "$ll$(s $en, i! s!a*s resse' !he click, !$ e/ecu!e
!he 'ragging ac!i$n $" !he $&)ec! selec!e' &* !he user+
,. -esults
The 'e#el$e' alica!i$n is $" !*e -in'$(s GE,
;U<
, &ase' $n a "allbac4 !*e, !ha! su$r!s !he managemen!
$" messages sen! &* !he $era!ing s*s!em, (hich all$(s !$ !ake care $" !he requiremen!s $" !he user+ The
alica!i$n $""ers !he $ssi&ili!* $" m$#ing !he curs$r all $#er !he c$mu!er screen, !hr$ugh m$#emen!s $" !he
hea', as (ell as ressing $r releasing !he le"! click, (i!h a single m$u!h $ening $r cl$sing+ This (a* ac!i$n such
as $ening a r$gram $r 'ragging a "ile can &e ma'e+ The alica!i$n gi#es !he $ssi&ili!* $" !($ le#els $"
m$#emen! sensi!i#i!*, 'een'ing $n !he user.s caa&ili!ies+ =ls$ i! all$(s !he c$ns!an! m$ni!$ring $" !he ca!ure'
images an' r$cessing &* a (in'$( l$ca!e' in !he screen, (here !he ac!i#i!* $" !he user is sh$(n in real !ime
A>igure TC+
>igure T+ -in'$( $" m$ni!$ring
-in'$(s 56D $""ers 'i""eren! $!i$ns !$ imr$#e !he accessi&ili!* !$ !he s*s!em inclu'ing8 "$r e/amle, a
ke*&$ar' in screen+ These !$$ls al!$ge!her (i!h !he alica!i$n 'e#el$e' in !his ($rk all$( !he user !$ e/an'
!he lis! $" !asks !ha! can er"$rm, (i!h n$ nee' $" using !ra'i!i$nal inu! 'a!a eriherals+

.. Conclusions
=n al!erna!i#e !$ !ra'i!i$nal inu! 'a!a eriherals (as 'e#el$e' using ar!i"icial #isi$n+ The alica!i$n is
'irec!e' seciall* !$ in'i#i'uals !ha! resen!0'isla* s$me m$!$r 'isa&ili!* !ha! a""ec!s !he m$&ili!* $" !heir uer
mem&ers+
The alica!i$n c$un!s (i!h a simle in!er"ace, (hich '$es n$! 'eman' r$l$nge' !raining "$r i!s use+ =ls$, !he
s*s!em requires $nl* a camera -e& "$r i!s $era!i$n, reas$n (h* i! reresen!s an ec$n$mic al!erna!i#e+
The 'e#el$e' alg$ri!hm is a&le !$ r$cess u !$ 29 images er sec$n', (hich $""ers !$ !he user a see' $" use
near !$ a c$n#en!i$nal m$use+
The use $" !he me!h$' $" (eigh!e' $si!i$n a#erage, all$(e' a sm$$!h m$#emen! $" !he curs$r $n !he screen
making a m$re e/ac! m$#emen! $" !he curs$r, as (ell as $""ering !he $ssi&ili!* $" es!a&lishing !($ le#els $"
m$#emen! sensi!i#i!*, (hich "acili!a!es !he use $" !he alica!i$n+
-eferences
;1< =nz$la, R$sari$+ :&ndacin aso a aso; <Un m&ndo con acceso para todos=+ @enezuela+ 2999+
;2< Dapena P., Jos. Me!$'$l$gas ara 3esarr$ll$ en C$muni'a'+ ,@ Beminar* !he ?era!ing B*s!em 2inu/
, 2994+
;3< ,n!el+ >pen !o&rce "omp&ter $ision Librar9 ?e@erence Aan&al+ ,n!el C$r$ra!i$n+ E+B+=+ 2991+
;4< @i$la 6+ * V$nes M+ R$&us! Real%Time >ace 3e!ec!i$n. International Bo&rnal o@ "omp&ter $ision, + 13U%
1O4, 2993+
;O< C$ianiz T+ * T$rresani 2+ 23 3e"$rma&le M$'els "$r @isual Beech =nal*sis+ !peechreading b9 C&mans and
Aachines, + 3G1%3GW, 1GGT+
;T< 6a)ares G+, 'e la Cruz V+, M$lina V+, Cua'ra'$ V+ * 2ez =. Imgenes Digitales. rocesamiento prctico
con Bava+ =l"a$mega, MX/ic$+ 2993+
;U< Microsoft Corporation. 7indoDs EI+ E+B+=+ 299U+