A Visual Ontology-Driven Interface

for a Web Sign Language Dictionary

M. Felice EURAC
T. Di Mascio University of LAquila
R. Gennari Free University of Bozen
2/36
Outline
Context of use
Italian sign language (LIS)
The e-LIS project
The e-LIS ontology
Users and organisational requirements
Design solutions
Conclusions and future work
3/36
Italian sign language: definition
Context of use ->Italian sign language
Visual-gestural language developed in Italian
deaf communities
Signs are used instead of voice to convey
meaning
A sign is realised through hand(s), facial
expressions, and mouthings
Sign Languages (SLs) have a tempo-spatial
nature, they are difficult to write
4/36
Italian sign language: transcription
Context of use ->Italian sign language
Several transcription systems proposed in
the literature for SLs
The Stokoe transcription system can be
employed for notating single and
decontextualised signs
LIS -> Radutzky transcription system
5/36
Radutzky transcription system
Context of use ->Italian sign language
Four classes
Handshape
Palmorientation
Movement
Location
Non manual components are neglected!
6/36
The e-LIS project
Context of use ->The e-LIS project
A research project developed at the European
Academy (EURAC) of Bozen-Bolzano
Cooperation with
ALBA fromTurin, active in deaf studies
KRDB of Free University of Bozen-Bolzano
University of LAquila
Main goal
Create a bilingual website (Italian and LIS), offering a
dictionary, two search engines and a forum
7/36
The dictionary
Context of use ->The e-LIS project
It allows translations formItalian to LIS and fromLIS to
Italian
Not only the translation, but also lexicographic
information are shown
Information is shown both in Italian and LIS (digital
videos are used)
Word search and sign search
http://elisdiz.eurac.edu/diz
8/36
Sign search
Context of use ->The e-LIS project
9/36
Current advantages and limits
Context of use ->The e-LIS project
Advantages
Wizard-like interaction
Iconic representations of sign components
(transcription system is hidden)
Video of signing people are shown
Limits
Powerful undo tools are not present, thus the
decision-making process is not well supported
The interaction is too long (too many mouse clicks)
Users are not supported, thus they can make
mistakes
Users have to know the rules of sign composition
10/36
The e-LIS ontology
Context of use ->The e-LIS ontology
Ontology: a formal representation of a specific
knowledge, which encodes semantics
The e-LIS ontology represents a sign in LIS, and
it encodes the rules of sign composition (e.g.,
howthe Stokoe classes are linked, or howa sign
is realised)
Domain ontology (+ application ontology)
11/36
Overviewof the domain ontology
Context of use ->The e-LIS ontology
12/36
Taxonomy of Handshape
Context of use ->The e-LIS ontology
13/36
Taxonomy of PalmOrientation
Context of use ->The e-LIS ontology
14/36
Taxonomy of Location
Context of use ->The e-LIS ontology
15/36
Taxonomy of OneHandMovement
Context of use ->The e-LIS ontology
16/36
Taxonomy of RelationalMovement
Context of use ->The e-LIS ontology
17/36
Revised version of the ontology
Context of use ->The e-LIS ontology
Original ontology
Many concepts and relations
Intermediate abstract concepts (e.g. curved shapes)
Simplified ontology
Only the five concepts related to Stokoe classes
Focus on the taxonomy of such concepts
Stokoe classes are 0-level elements
Handshapes are grouped into more intuitive concepts
(0-finger, 1-finger, etc.)
Relational movement and one hand movement are
considered both children of movement Stokoe class
18/36
Outline
Context of use
Users and organisational requirements
Profiling of users
Tasks
Usability goals
Design solutions
Conclusions and future work
19/36
Profiling of users
Users and organisational requirements ->Profiling of users
Users
Deaf/hard-of-hearing or hearing people
LIS fluent or novice
Deaf people
cannot be assumed to be literate [1]
problems comprehending written language
can have problems coping with abstract concepts [2, 3]
larger spatial memory than hearing non-signers
visual learners
can cope with complex visuo-spatial structure
prefer pied interfaces: different shapes, different colours, etc.
20/36
Tasks
Users and organisational requirements ->Tasks
Browsing: specify the components of the sign
(e.g., handshape)
Querying: retrieve the sign
Result visualisation: browse the results
Currently: users can specify the sign
components by browsing the ontology -> query-
oriented browsing
21/36
Usability goals
Users and organisational requirements ->Usability goals
Effectiveness
users are supported, thus minimising mistakes
Efficiency
the interaction with the interface is fast and decision-
making process is well supported, thanks to efficient
undo tools
Users satisfaction
the interface is well organised and plain, thus
minimising cognitive effort and keeping users
attention alive
22/36
Outline
Context of use
Users and organisational requirements
Design solutions
Current prototype
Ontology visualisation
Conclusions and future work
23/36
The current prototype
Design solution ->Current prototype
Successive refinements of the initial idea
and mock-ups
It is an information visualisation system
Visual metaphor
Number of dimenions
Space-saving strategy
24/36
Visual metaphor
Design solution ->Ontology visualisation
We need to visualise a tree structure (the
taxonomy) -> Tree visual metaphor
TreeMap visual technique: the tree is shown in a
space-constrained layout, i.e. the tree is turned
into a planar space-filling map [4]
Each treemap is composed of several areas,
whose extents are proportional to the number of
child concepts the area represents
25/36
Visualisation of ontology concepts
Design solution ->Ontology visualisation
26/36
Dimension and space-saving strategy
Design solution ->Ontology visualisation
Number of dimensions
2.5 D
Simulated 3rd dimension
Space optimisation
Idea of link between concepts
Space-saving strategy
Focus + context
27/36
Focus + context strategy
Design solution ->Ontology visualisation
28/36
First step: 1-level concepts
Design solution ->Ontology visualisation
29/36
Temporary selection
Design solution ->Ontology visualisation
30/36
Second step: 2-level concepts
Design solution ->Ontology visualisation
31/36
Expert-based evaluation
Design solution ->Ontology visualisation
Effectiveness
Users are guided
Errors are minimised
Efficiency
Mouse clicks are minimised
Undo tools ->decision-making (users know the next step of the
search path)
Users satisfaction
Few graphic elements
Plain interface
Pied interface
32/36
Outline
Context of use
Users and organisational requirements
Design solutions
Conclusions and future work
33/36
Conclusions
Design solution ->Conclusions and future work
Advantages of the e-LIS ontology
Minimisation of errors
Creation of a dynamic interface
Non-expert users can use the dictionary
Advantages of proposed interface
Spatial-based, thus closer to deaf people wrt the
current interface
Decision-making support (two-stage selection)
Intuitive undo tools (the miniatures)
Less obtrusive interaction (mouse over)
34/36
Future work (1)
Design solution ->Conclusions and future work
Tests to evaluate:
Treemap choice
Use of colours
Interaction strategy
Cognitive effort
Experience curve (interface usage and sign
composition)
Usability goals
Overall usability of the prototype
35/36
Future work (2)
Design solution ->Conclusions and future work
Effective intuitive icons instead of text
labels have to be designed
Icons are usually more intuitive than text (if
well designed)
Deaf people have problem reading and
comprehending written Italian
36/36
References
[1] P. V. Paul. Literacy and Deafness: the Development
of Reading, Writing, and Literate Thought. Allyn &
Bacon. 1998
[2] C. Chesi. Inferenze Strutturali. Analisi sullUso degli
Elementi Funzionali nel Linguaggio Verbale dei Bambini
Sordi. Masters thesis, Siena U. 1999/2000
[3] D. Fabbretti. LItaliano Scritto dai Sordi: unIndagine
sulle Abilit di Scrittura dei Sordi Adulti Segnanti
[4] B. Shneiderman. Treemaps for Space-Constrained
Visualization of Hierarchies.
http://www.cs.umd.edu/hcil/treemap-history