Tele-Graffiti

This project is no longer active.

Head: Simon Baker and Jianbo Shi Contact:Simon Baker

Mailing address: Carnegie Mellon University Robotics Institute 5000 Forbes Avenue Pittsburgh, PA 15213

Associated center(s) / consortia:

Vision and Autonomous Systems Center (VASC)

Associated lab(s) / group(s):

Intelligent Desktop Group

Project Homepage

Overview Tele-Graffiti is a system allowing two or more users to communicate remotely via hand-drawn sketches. What one person writes at one site is captured using a video camera, transmitted to the other site(s), and displayed there using an LCD projector. With TeleGraffiti, the users write on regular pieces of paper that are tracked in real-time by the system. See the following movie for a demonstration of (1) the system hardware, (2) the TeleGraffiti tracking

algorithm, and (3) two users interacting using TeleGraffiti. Image shows video sequence. We have also built a handbased user interface and an automatic session summarization feature for Tele-Graffiti. Both of these features, as well as the basic system operation, are demonstrated in the following movie. Image shows video sequence. We have also analyzed the Tele-Graffiti feedback loop and derived a model for the Tele-Graffiti image formation process. We used this model to derive the final viewed image and the optimal gain with which to set up the

system to optimize the image quality. We also derived an algorithm to separate the final viewed images into the two actually drawn components (without imaging them directly). Two examples of this process are shown below.

and

See the unofficial webpage for more details.

One way to build a remote sketching system is to use a video camera to image what each user draws at their site, transmit the video to the other sites, and display it there using an LCD projector. Such camera-projector based remote sketching systems date back to Paul Wellner's (largely unimplemented) Xerox Double DigitalDesk. To make such a system usable, however, the users have to be able to move the paper on which they are drawing, they have to be able to interact with the system using a convenient interface, and sketching sessions must be stored in a compact format so that they can be replayed later. We have recently developed Tele-Graffiti, a remote sketching system with the following three features: (1) real-time paper tracking to allow the users to move their paper during system operation, (2) a hand based user interface, and (3) automatic session summarization and playback. In this paper, we describe the design, implementation, and performance of Tele-Graffiti. Many ideas are best communicated by drawing simple sketches. Good examples include the

explanation of computer algorithms to undergraduate students and the sketched preliminary designs of architecture. If two people are located remotely from each other a ``remote sketching system'' can help them communicate better. One good way to build a remote sketching system is to use a video camera to image what one user draws at one site and display the video at the second site using an LCD projector. Such camera-projector based remote sketching systems date back to the original proposal of Paul Wellner's Xerox Double DigitalDesk. To make such a system usable, the users have to be able to move the paper on which they are drawing during operation of the system, they have to be able to interact with the system in a straightforward manner, and the feedback through the two-way system has to be controlled. In view of this we recently developed Tele-Graffiti, a remote sketching system which provides solutions for these three problems. The advantages of Tele-Graffiti include: (1) real-time paper tracking to allow the users to move their paper during the operation of the system, (2) a hand based user interface, and (3) feedback control to maximize the quality of the shared images based on photometric analysis of the feedback loop. In this report we describe the design and implementation of Tele-Graffiti.

Keywords camera-projector system, remote sketching, paper tracking, communication, user interface, hand tracking, session summarization, feedback analysis, visual echoing, photometric, image separation

Tele Graffiti : Bringing the paper age back." Tele Graffiti : The use of paper has been abstained with computers gaining wider adoption. But as the
world stopped towards paperless office concept, a new technology called Tele Graffiti is giving paper a NEW LIFE. The technology integrates the pleasure of writing with information technology. It is a system that allows two or more users to communicate remotely through hand-drawn sketches .

How It Works : The system consists of a digital camera, a high resolution LCD projector to project the
received image, a mirror to reflect the project image onto a blank paper sheet, fixture to hold these devices in position and a writing pad to fix the sheet of paper on the top of the table at every participants station. You can write on a regular piece of paper with a pen at your work. When you impose your ideas or sketch on a paper, then your doodling is captured by a camera and the digitized image is transmitted to the receiving end through a data link. The image can be transmitted to LAN or Intranet or Internet. Data rate of more than 100 Mbps can by achieved through the LAN. Upto 15 frames of image can be transmitted per second with latencies of about 70 milliseconds. The time lag does not make the recipient scratch this head as working (writing process) and sketching are slow processes. If the users were to communicate through the global network, delayed will be considerable. Data will be transmitted as streaming video over the internet. Uncompressed data streams at 2 to 3 frames per second can be transmitted over direct Internet communications. The projector on other side handles the transmitted video. The projector beams the image on recipients paper. The sender and recipient can now interact as if they are both writing on the same piece of paper while sitting many miles apart. When the system is turned off, the user sees only his marks on the paper.

The Trick : You can place the writing pad any where on the desk, rotate it and even pass

around the table without the loss of paper orientation to allow several people to write on it at each side of the network. Inputs from both ends appear simultaneously at both ends. Now with the help of a tracking algorithms, the camera tracks the edges of the paper, so pen marks maintain the same relative position on the paper at every side.
As you move the pad, the camera follows it and projector angle changes. The computer ignores desktop clutter and works out a method on how to project what is read. The image is projected sharply and exactly at the same position as it was at sender end. In effect multiple users have the feeling of sharing the same piece of paper.

Uses :
(1) It gives graphic articles a global reach Architects can tem up with experts abroad to work on designs simultaneously. (2) Distance education will be benefited as students and teacher can interact in their own handwritings and get minute points and doubts classified. (3) It can be considered as a substitute for internet chat as en chat for many could be more interesting than key board exercise. News, information, alerts, warnings can be exchanged in an attractive way. (4) Forms can be filled u in your own hand writing without the need to download, scan and submit the same.

Future Prospects : The technology will be put to use commercially within two years. Researchers are
trying to add audio signals and face-tracking mechanism to develop a remote sketching teleconferencing system. Efforts are also going on to shrink the size of the system to the size of table lamp.