Page 1

SIXTH SENSE TECHNOLOGY

By:
AMIT KUMAR SINGH (08911502809)
GAURAV NIGAM (10611502809)
YOGESH KUMAR (09811502809)
RAJAT KUMAR (10711502809)
Page 2
Introduction
Components
Working
Applications
CONTENTS
Page 3
INTRODUCTION
When we come across something, we use our five senses viz.
hearing, sight, touch, smell, and taste in order to gather information about
that particular thing.

But most of the useful information that one wants cannot be obtained solely
by making use of the five senses since it mostly resides on the internet.

Although mankind has developed numerous devices such as computers,
laptops and the easy-to-carry mobile phones that makes the task of
information retrieval a lot easier but still it becomes a lot difficult in some
situation.

Page 4
EXAMPLE:
(For instance if you go to a book-store and find a book. Now you want to
know the reviews of this book, youll find it quite complex to search for the
book and read the reviews on your mobile phone.)


The Sixth Sense technology is a big step towards reducing this difficulty.
Sixth Sense frees information from its confines by seamlessly integrating it
with reality, and thus making the entire world your computer.


It narrows the gap between the digital world and the physical world as it
grants us access to the digital information simply by hand gestures.
Page 5


Sixth Sense is a wearable gestural interface that augments the
physical world around us with digital information and lets us
use natural hand gestures to interact with that information.

It was developed by Pranav Mistry, a PhD student in the Fluid
Interfaces Group at the MIT Media Lab.

Sixth Sense comprises a pocket projector, a mirror and a
camera. The hardware components are coupled in a pendant
like mobile wearable device.




Page 6
RELATED
TECHNOLOGY
Augmented reality

It is a term for a live direct or indirect view of a physical real world
environment whose elements are augmented by virtual computer generated
imagery. AR is used in European football to show the result (or an
advertisement) in the center circle. It could also be see showing the offside
situations.

Gesture recognition

It is a topic in computer science and language technology
with the goal of interpreting human gestures. Gestures can be used to
control interactions within video games to try and make the game player's
experience more interactive or immersive.


Page 7
Computer vision

It is the field concerned with the automated processing of images from
the real world to extract and interpret information on a real time basis.
Identification of a specific person's face or fingerprint and similar tasks
can be performed by this technology.

Radio Frequency Identification

It is basically an electronic tagging technology that allows the detection,
tracking of tags and consequently the objects that they are affixed to.
RFID is used for supply chain management in the fashion industry. The
RFID label is attached at the garment at production and is removed at the
point of sale.


Page 8
Camera
Projector
Mirror
Mobile Component
Colored Markers
COMPONENTS
The hardware components are coupled in a pendant like mobile wearable
device.
Page 9
Camera





The camera is the key input device of the Sixth Sense system. The camera acts as
a digital eye of the system.

It basically captures the scene the user is looking at. The video stream captured
by the camera is passed to the mobile computing device (which can be kept
inside the pocket) which does the appropriate computer vision computation.
Page 10
The major functions of the camera can be listed as:

Captures users hand movements and gestures (used in
reorganization of user gestures)

Captures the scene in front and objects the user is interacting with
(used in object reorganization and tracking)

Takes a photo of the scene in front when the user performs a
framing gesture

Captures the scene of projected interface (used to correct the
alignment, placement and look and feel of the projected interface
components.
Page 11
Projector








The projector projects visual information enabling surfaces and
physical objects to be used as interfaces

The project itself contains a battery inside, with 3 hours of battery life.

A tiny LED projector displays data sent from the smart phone on any
surface in viewobject, wall, or person.
Page 12
Mirror






The usage of the mirror is significant as the projector dangles pointing
downwards from the neck.
Smart Phone






A Web-enabled smart phone in the users pocket processes the video
data

Other software searches the Web and interprets the hand gestures

MIRROR
Page 13
Color Markers







It is at the tip of the users fingers .

Marking the users fingers with red, yellow, green, and blue tape helps
the webcam recognize gestures

The movements and arrangements of these makers are interpreted
into gestures that act as interaction instructions for the projected
application interfaces.

Page 14


HOW DOES IT WORKS
Page 15
Page 16
The hardware that makes Sixth Sense work is a pendant like mobile
wearable interface

It has a camera, a mirror and a projector and is connected wirelessly
to a bluetooth smart phone that can slip comfortably into ones
pocket

The camera recognizes individuals, images, pictures, gestures one
makes with their hands

Information is sent to the Smartphone for processing

The downward-facing projector projects the output image on to the
mirror

Mirror reflects image on to the desired surface

Thus, digital information is freed from its confines and placed in the
physical world



Cont
Page 17
APPLICATIONS

Page 18
Make a call

You can use the Sixth Sense to project a
keypad onto your hand, then use that virtual
keypad to make a call.
Call up a map

With the map application we can call up
the map of our choice and then use thumbs
and index fingers to navigate the map
Page 19
Check the time

Draw a circle on your wrist to get
a virtual watch that gives you the
correct time
Create multimedia reading
experiences

Sixth Sense can be programmed to project
related videos onto newspaper articles you
are reading
Page 20
Drawing application

The drawing application lets the user draw
on any surface by tracking the fingertip
movements of the users index finger
Zooming features

The user can zoom in or zoom out using
intuitive hand movements
Page 21
Get product information

Sixth Sense uses image recognition or
marker technology to recognize products
we pick up, then feeds us information on
those products
Get book information

The system can project Amazon ratings on
that book, as well as reviews and other
relevant information
Page 22
Get flight updates

The system will recognize your boarding
pass and let you know whether your flight
is on time and if the gate has changed.
Feed information on people

The system will project relevant information
about a person such as what they do, where they
work, and so on.
Page 23
Take pictures

If you fashion your index fingers
and thumbs into a square
("framing" gesture), the system
will snap a photo.





After taking the desired number of
photos, we can project them onto a
surface, and use gestures to sort
through the photos, and organize
and resize them.
Page 24





ADVANTAGES
Portable
Connectedness between world and information
Cost effective
Data access directly from machine in real time
Supports multi-touch and multi-user interaction
Page 25
Limitations

Hardware limitations of the devices, that we currently
carry around with us.

For example many phones will not allow the external
camera feed to be manipulated in real time.


Page 26
Sixth Sense recognizes the objects around us, displaying information
automatically and letting us to access it in any way we need


The Sixth Sense prototype implements several applications that demonstrate the
usefulness, viability and flexibility of the system











Allowing us to interact with this information via natural hand gestures


The potential of becoming the ultimate "transparent" user interface for
accessing information about everything around us
CONCLUSION
Page 27
THANK YOU



ANY QUERIES