SIXTH SENSE TECHNOLOGY

Sixth Sense Technology integrates digital information into the physical world and its objects, making the entire world your computer. It can turn any surface into a touch-screen for computing, controlled by simple hand gestures. It is not a technology which is aimed at changing human habits but causing computers and other machines to adapt to human needs. It also supports multi user and multi touch provisions. Sixth Sense device is a mini-projector coupled with a camera and a cell phone-which acts as the computer and your connection to the Cloud, all the information stored on the web. The current prototype costs around $350. The Sixth Sense prototype is used to implement several applications that have shown the usefulness, viability and flexibility of the system

Introduction of Sixth Sense Technology

'Sixth Sense' is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information the hardware components are coupled in a pendant like mobile wearable device. The Sixth Sense prototype is comprised of a pocket projector, a mirror,colored marker and a camera. The camera, mirror and projector is connected wirelessly to a blue tooth smart phone device that can easily fit into the user's pocket.A software then processes the data that is collected by the capturing device and produces analysis.The software that is used in sixth sense device is open source type

Gesture Recognition
It is a technology which is aimed at interpreting human gestures with the help of mathematical algorithms. Gesture recognition technique basically focuses on the emotion recognition from the face and hand gesture recognition. Gender recognition technique enables

humans to interact with computers in a more direct way without using any external interfacing devices. It can provide a much better alternative to text user interfaces and graphical user interface which requires the need of a keyboard or mouse to interact with the computer. Interfaces which solely depends on the gestures requires precise hand pose tracking. In the early versions of gesture recognition process special type of hand gloves which provide information about hand position orientation and flux of the fingers. In the SixthSense devices colored bands are used for this purpose. Once hand pose has been captured the gestures can be recognized using different technique's. Neural network approaches or statistical templates are the commonly used techniques used for the recognition purposes. This technique have an high accuracy usually showing accuracy of more than 95%. Time dependent neural network will also be used for real time recognition of the gestures

APPLICATION The sixth sense technology finds a lot of application in the modern world. The sixth sense devices bridge the gap by bringing the digital world into the real world and in that process allowing the users to interact with the information without the help of any machine interfaces. Prototypes of the sixth sense device have demonstrated viability, usefulness and flexibility of this new technology. According to the words of its developers the extend of use of this new device is only limited by the imagination of human beings

The sixth sense recognizes the objects around us and displays the information relating to those objects in a real time environment. The sixth sense technology allows the user to interact the information through hand gestures. This is a quiet efficient way compared to the text and graphic based user interface. It has the potential to form the transparent user interface for accessing the information around us

'SixthSense' is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information.

We've evolved over millions of years to sense the world around us. When we encounter something, someone or some place, we use our five natural senses to perceive information about it; that information helps us make decisions and chose the right actions to take. But arguably the most useful information that can help us make the right decision is not naturally perceivable with our five senses, namely the data, information and knowledge that mankind has accumulated about everything and which is increasingly all available online. Although the miniaturization of computing devices allows us to carry computers in our pockets, keeping us continually connected to the digital world, there is no link between our digital devices and our interactions with the physical world. Information is confined traditionally on paper or digitally on a screen. SixthSense bridges this gap, bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand gestures. SixthSense frees information from its confines by seamlessly integrating it with reality, and thus making the entire world your computer.

The SixthSense prototype is comprised of a pocket projector, a mirror and a camera. The hardware components are coupled in a pendant like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the users pocket. The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera recognizes and tracks user's hand gestures and physical objects using computer-vision based techniques. The software program processes the video stream data captured by the camera and tracks the locations of the colored markers (visual tracking fiducials) at the tip of the users fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction.

The SixthSense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system. The map application lets the user navigate a map displayed on a nearby surface using hand gestures, similar to gestures supported by Multi-Touch based systems, letting the user zoom in, zoom out or pan using intuitive hand movements. The drawing application lets the user draw on any surface by tracking the fingertip movements of the users index finger. SixthSense also recognizes users freehand gestures (postures). For example, the SixthSense system implements a gestural camera that takes photos of the scene the user is looking at by detecting the framing gesture. The user can stop by any surface or wall and flick through the photos he/she has taken. SixthSense also lets the user draw icons or symbols in the air using the movement of the index finger and recognizes those symbols as interaction instructions. For example, drawing a magnifying glass symbol takes the user to the map application or drawing an @ symbol lets the user check his mail. The SixthSense system also augments physical objects the user is interacting with by projecting more information about these objects projected on them. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper. The gesture of drawing a circle on the users wrist projects an analog watch.

Sixth sense is a working prototype of a multi functional, wearable, gestural interface device that has the potential to free the world from all limitations of the paper or screen, by turning any flat surface into a touch-screen display.

SIXTH SENSE TECHNOLOGY DEVELOPER

6th sense WUW (Wear UR World) gadget has been devised by PranavMistry, a P.hD. Scholar in the Fluid Interfaces Group at the Massachusetts Institute of Technology Media Lab.

SIXTH SENSE COMPONENTS

Portable Mini-projector Camera Mobile computing device Marker caps for fingers

WORKING OF SIXTH SENSE

The Camera and Projector are contained in a wearable form such as a pendant, and are further connected to the mobile computing device, that may be present in the user's pocket. The camera recognizes changes in the surroundings and tracks hand gestures along with gathering 'meta information', to then articulate it with the digital domain. The projector produces all images containing the information on any surface, turning it into a touch screen, eliminating the use of paper or screen. The mobile-computing device is connected to the Cloud carrying all information through the web. The colorful marker caps worn on the fingers help the camera in detecting hand gestures. The fingers can also be painted colorfully to add a stylish touch to it. The software program present in the mobile computing device processes the video stream data captured by the camera and tracks the tip of the user's fingers with the help of the colored markers.

COST OF SIXTH SENSE

Although WUW is still being developed and worked upon, its worth has been estimated at 350$. This price falls in the range in which many smart phones and computers are available today, hence making it relatively affordable. The software used in the WUW is already available as open source, enabling anyone to build their own sixth sense device.

APPLICATIONS OF SIXTH SENSE

Eliminating the use of a mouse, drawing is made easier and more fun with the use of fingers. A square frame made out of hands in the open air can replace the bulky camera, by highlighting the object that has to be captured into a photo.Also, a few movements made by fingers can perform the modern functions of editing and resizing the photo captured, along with e-mailing it to people. A phone call can be made, without using any mobile hardware. This technology enables the projection of a phone keypad on the user's hand, which can then be typed and the phone call made. It reads aloud a book for its user, and also helps him find out the ratings and other aspects of a book that the user may wish to find out, with the help of information on the web. It provides a video enabled newspaper experience to its users, through searching the web for the most appropriate video, related to the desired newspaper report or headline. It helps in checking the status and other details of the flight from the web, while off-board, through the simple action of placing the ticket in front of the projector. Through drawing a circle on his wrist, the device projects a digital clock face to its user. It can help a consumer in buying better, by providing him with immediate information about various products and services, leading to easy price comparisons and other details.

PURVIEW OF SIXTH SENSE

Sixth sense is a futuristic technology that connects the digital world with the physical world, eliminating hardware devices altogether. This technology, a decade ago, was only perceived in the fictional world. Its applications today can be closely matched with Tom Cruise's sci-fi movie 'Minority report' released in the year 2002 that shows Cruise drawing information from a glove-controlled interactive wall.

Also similarities can be made to the very popular 'Harry Potter' series , that show newspaper reports coming alive, similar to the video-streamed newspaper reading experience promised by the 6th sense technology. This technology has turned into reality a concept that could only be visualized in the fictional and imaginative world. WUW is a device that contains Microsoft Surface along with Google in a Portable Package. PranavMistry says "Sixth Sense frees information from its confines by seamlessly integrating it with reality, and thus making the entire world your computer." It would improve standards of living of the common man, and would help him in making decisions that are economic and efficient, providing him an ultra-modern lifestyle. Once fully developed, it would revolutionize the world by empowering every individual with the ultimate source of information through seamless connectivity to the web.
SixthSense is a wearable gestural interface device by PranavMistry, team leader in Samsung_Electronics's Silicon Valley office.

Construction and workings

The SixthSensetechnology contain a pocket projector, a mirror and a camera contained in a pendantlike, wearable device. Both the projector the camera and sensors are connected to a mobile computing device in the users pocket. The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera recognizes and [1] tracks users' hand gestures and physical objects using computer-vision based techniques. The software program processes the video stream data captured by the camera and tracks the locations of the colored markers (visual tracking fiducials) at the tips of the users fingers. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. SixthSense supports multi-touch and multi-user interaction. The SixthSense prototype contains a number of demonstration applications. The map application lets the user navigate a map displayed on a nearby surface using hand gestures to zoom and pan The drawing application lets the user draw on any surface by tracking the fingertip movements of the users index finger. SixthSense also implements Augmented reality; projecting information onto objects the user interacts with. For example a paper newspaper can be augmented with projected dynamic live information.

The system recognizes a user's free hand gestures as well as icons/symbols drawn in the air with the index finger, for example: A 'framing' gesture takes a picture of the scene. The user can stop by any surface or wall and flick through the photos he/she has taken. Drawing a magnifying glass symbol takes the user to the map application while an @ symbol lets the user check his mail. The gesture of drawing a circle on the users wrist projects an analog watch.

Cost and license

SixthSense prototypes cost approximately $350 to build (not including the computer), the main cost being the micro-projector. Mistry had announced in Nov 2009 that the source code will be [5] released with an open source licence. The open source code for the project can be found at SixthSense Google Code and SixthSenseGithub Repo. The source code is currently being ported into Java from C# to allow the developers to make mobile ports.
[2][3][4]

http://www.pranavmistry.com/projects/sixthsense/#PICTURES