Project Holoface: Cosmos

Design for Interactive Holography Environments

Victor Pardinho

Introduction
In the computerized world we live in, with the pervasiveness of computer technology
increasing, new ways to communicate, inform and interact with virtual contents start to
emerge. Today, with new technologies and practices of computing within reach of the
designer, we are able to communicate and inform virtual content in real physical spaces, as
well as interact with such content in a natural way, through our bodies and gestures. Such
possibilities are predicting the emergency of interactive holography projectors in a near
future, which will further broaden the potential of interactions and three-dimensional
visualizations.
Therefore, it's important that the new designers keep track of these technologies and its
new design possibilities. Through the study and practice tools of science / computer
engineering and design, this work of competition of course main goal is to create an
experimental simulation of an interactive holography environment, as well as their study
area, issues and questions regarding their designs, the human behavior when facing these
scenarios and their methods and project possibilities.
The project also involves the design of three-dimensional interfaces for direct interaction,
seeking to understand new and better ways to use elements of interface design for
applications that use augmented reality, mainly for the educational field. As proof of
concept for a possible use of such knowledge, we have been developing a study of
alternatives, production and usability tests for the design of an augmented reality interface
that brings the experience of reading a digital book close to an experience of reading books
on paper. Using technology such as direct manipulation and gestures recognition,
augmented reality and stereoscopic graphics, the users can manipulate virtual books with
their free hands and feel like they are floating in front of them. Also there's new
implementation of the framework and techniques for education in anatomy, astronomy,
health, and various project management activities.
This research is based on the concepts proposed by Tori et. al. (2006) on the use of
techniques and technologies of augmented reality for increased sense of presence in
Interactive 3D Atlas to be applied in distance learning courses, also following procedures of
software design (Mattson, 1996) and interfaces (Preece et. al., 2005)(Bowman et. al. 2004).
The project is in course in Centro Universitrio Senac, in the City of So Paulo, by Victor
Pardinho, student of Graphic Design.

The "3D Book" Experience

Today there's a wide range of applications for digital reading that attempt to simulate real
books, using shadows and textures taken from physical books, but they are usually limited
to a two-dimensional screen or tablet in wich we've to interact with the tip of our fingers,
making all those tridimensional effects look artificial in our hands. In this experience we
seek technological alternatives for digital reading using three-dimensional, augmented
reality and gestures reconigtion techniques.
Using tools of 3d modeling and the programing language Processing (Casey and Reas,
2007) we could build a prototype for a session of usability tests. In the software we build,
through the Kinect (Microsoft, 2009) sensor, the user could control the 3d Book without
the use of the mouse and keyboard, and with stereoscopic techniques and the use of
anaglyph glasses, it was possible for user to have the feeling that the 3d Book was floating
in front of him and not stuck in a two-dimensional screen. For the natural interaction we've
build it on a simple way, with the hand parallel to the screen the user can travel through
the borders of the pages all the way through the book, and with the hand perpendicular to
the screen and doing the fliping page gesture, they can turn the piece of pages to open
where they want to.
For the usability test, following methodology based on Preece et. al. (2005) and Chu et. al.
(2010), made with twelve participants: university students and professors among them, all
of those experienced with digital reading, we have built the prototype to display a
dictionary, in stereoscopic form and also without stereoscopic, we've built also a physical
version of the dictionary. Then we've asked the participants to look for three words, each
word was sought on a different medium, so we could record and analyze how the
participants behaved with each one of the mediums, giving us some interesting results:
Manipulation disturbance: it was reported by most participants that as the digital system
was new to them, being a system of natural interaction, without the tradicional mouse and
keyboard, they have to focus a lot on the manipulation, on how they would turn the pages
and were so afraid of making mistakes that they didn't pay much attention to the reading
content but mostly on the manipulation of the system, an issue that started to fade by the
third time that they were using the system and already knew how to easily manipulate it,
making us realize that a small learning curve becomes even more essencial for this kind of
system.
Instability: another issue that came up with the previous one is the instability problem.
Because of the sensitivity of the Kinect sensor, the participants were a little afraid of their
movements, sometimes when they were making some normal movements like scratching
or arranging their clothes, the sensor caught some movement making the book wiggle
slightly. This is a common problem of natural interactions and must be researched and
implemented very well. We're working on this issue for the next version of the prototype,
making keyframes and trying to make the software realize when there's a common
movement or a real gesture.
Physical book addicted: by the observation of the participants looking for the words on a
physical book we realized that, after a while, the physical book becomes addicted. Since the
words were sometimes the same per search, the book would open on the same pages. The

last participants began to find the words quite easily and stated that if they fliped through
the pages they ended up at the exact pages they were looking for, this is an interesting issue
that usually happens on physical books and won't happen on digital systems.
Techniques made by the users: one of the most interesting things we've achived on this test
was the techniques that the users had created to manipulate the digital system easily. One
of those was the technique of flipping a large piece of pages until the half and looking the
page on the back to see if it would turn on the right session of the book and if not, they just
got the piece back, this technique was extremelly used by all the users and was never
advised that this was possible, but because they do it a lot on physical books, everyone
learned easily how to do the same on the digital system.
Stereoscopy: the most interesting issue for Holoface project that we could achive with the
test was the stereoscopic one, it has become clear to us with a close analysis of the videos
and various statements made by the participants that they didn't act the same way with the
stereoscopic digital system and the conventional non-steorescopic. When they started to
use the stereoscopic system using anaglyph glasses they stated that the 3d Book was really
off the screen, making they forget about the two-dimensional screen and feeling it in front
of them, in the beginning we thought that this wasn't making any difference, but by the
time they began to interact with the stereoscopic 3d Book, everything changed, the position
of the hands and even the gesture of fliping the pages were different from the nonstereoscopic, the parallel hands to the screen had become a curved hand and when asked
why, some users stated that, as the book was now off the screen they really wanted to
"touch" the pages and no more interact by a distance, so they always tried to "catch" the
pages as if it was a real object, forgeting about the rules of parallel and perpendicular hands
to the screen that they had learned on the non-stereoscopic system.

Holoface: Cosmos
With this first test and results, the Holoface project gained strength and now we are
updating the prototype to work with astronomy (Cosmos), health (VIDA) training and
education. We now begin to realize that holographic environments with natural interaction
aren't like the two-dimensional screens we're used to, and the research for those kind of
environments to be used on educational fields are essential, we are entering on a universe
where the interaction will happen in three axis, where you can walk around virtual objects
and the class can learn and grow together, to build systems like that is a whole new
challenge for designers, engineer, architects and we are just on the tip of the iceberg.

References
1.
BOWMAN, D.; KRUIJFF, E.; LAVIOLA, J. J.; POUPYREV, I. 3D User Interfaces:
Theory and Practice. Addison-Wesley, 512 p. 2004.
2.
CHU, Y.; BAINBRIDGE, D.; JONES, M.; WITTEN, I. H. Realistic Books: A bizarre
homage to an obsolete medium?. 2010.
3.
MATTSON, M. ObjectOriented Frameworks A survey of methodological
issues, 128 f. Grau: Tese.(Doutorado em Cincia da Computao) Departament of
Computer Science and Business Administration, UCK, Ronneby. 1996
4.

Microsoft, Kinect Sensor, 2009.

5.
PREECE, J; ROGERS, Y.; SHARP, H. Design de Interao. Porto Alegre: Bookman,
550 p., 2005.
6.
REAS, C; FRY, B. Processing, a Programming Handbook for Visual Designers
and Artists. Cambridge: MIT Press, 2007.
7.
TORI, R.; KIRNER, C.; SISCOUTTO, R. Fundamentos e Tecnologia de Realidade
Virtual e Aumentada. In: Livro do Pr-Simpsio VIII Symposium Virtual Reality, BelmPA. Anais... , 2006.

Student: Victor Pardinho

Thesis supervisor: Romero Tori
Collaborators: Ligia Mie Jeon, Ian Muntoreanu
School: Centro Universitrio Senac Campus Santo Amaro: 2008 - 2011
Website: www.vpardinho.com
Birth: 05/07/1988 Londrina, Paran
Actual place: So Paulo, So Paulo