Massachusetts Institute of Technology

Media Lab
Projects | May 2010

MIT Media Lab

Buildings E14 / E15
75 Amherst Street
Cambridge, Massachusetts 02139-4307

communications@media.mit.edu
http://www.media.mit.edu
617 253-5960
Many of the MIT Media Lab research projects described in the following pages are conducted under the auspices of
sponsor-supported, interdisciplinary Media Lab centers, consortia, joint research programs, and initiatives. They are:

Autism & Communication Technology Initiative

The Autism & Communication Technology Initiative utilizes the unique features of the Media Lab to foster the development of
innovative technologies that can enhance and accelerate the pace of autism research and therapy. Researchers are especially
invested in creating technologies that promote communication and independent living by enabling non-autistic people to
understand the ways autistic people are trying to communicate; improving autistic people's ability to use receptive and
expressive language along with other means of functional, non-verbal expression; and providing telemetric support that reduces
reliance on caregivers' physical proximity, yet still enables enriching and natural connectivity as wanted and needed.

CE 2.0
Most of us are awash in consumer electronics (CE) devices: from cell phones, to TVs, to dishwashers. They provide us with
information, entertainment, and communications, and assist us in accomplishing our daily tasks. Unfortunately, most are not as
helpful as they could and should be; for the most part, they are dumb, unaware of us or our situations, and often difficult to use.
In addition, most CE devices cannot communicate with our other devices, even when such communication and collaboration
would be of great help. The Consumer Electronics 2.0 initiative (CE 2.0) is a collaboration between the Media Lab and its
sponsor companies to formulate the principles for a new generation of consumer electronics that are highly connected,
seamlessly interoperable, situation-aware, and radically simpler to use. Our goal is to show that as computing and
communication capability seep into more of our everyday devices, these devices do not have to become more confusing and
complex, but rather can become more intelligent in a cooperative and user-friendly way.

Center for Future Banking

This center explores how emerging technologies and insights into human behavior can transform the customers' experience and
elevate the role of the bank in their financial lives. Researchers seek to invent new ways to anticipate the needs and desires of
customers down to the level of the individual, to rethink the experience of bank-customer interactions as virtual and physical
reality become increasingly intertwined, and finally to leverage the unique position of a bank to make people's lives simpler and
more fulfilling. The Center brings together disciplines ranging from behavioral economics, to computer science, to urban design,
in order to take a truly holistic approach to imagining and realizing new possibilities in banking. Its research spans a wide range
of physical and social scales, from one-on-one interactions with customers, to new modes of global transactions.

Center for Future Civic Media

A joint effort between MIT's Media Lab and Comparative Media Studies Program, the Center for Future Civic Media creates
technical and social systems for sharing, prioritizing, organizing, and acting on information. The Center uses the term civic
media, rather than citizen journalism: civic media is any form of communication that strengthens the social bonds within a
community or creates a strong sense of civic engagement among its residents. Civic media goes beyond news gathering and
reporting: it ensures the diversity of inputs and mutual respect necessary for democratic deliberation. The Center for Future
Civic Media is funded by a four-year grant from the Knight Foundation.

Center for Future Storytelling

Storytelling is fundamental to being human: it's how we share our experiences, learn from our past, and imagine our future. With
the establishment of the Center for Future Storytelling, the Media Lab is rethinking what "storytelling" will be in the 21st century.
The Center will take a dynamic new approach to storytelling, developing new creative methods, technologies, and learning
programs that recognize and respond to the changing communications landscape. The Center will examine ways for
transforming storytelling into social experiences, creating expressive tools for the audience and enabling people from all walks of
life to embellish and integrate stories into their lives, making tomorrow's stories more interactive, creative, democratized, and
improvisational. It will seek to bridge the real and the virtual, creating tools for both adults and children that allow stories to
incorporate synthetic characters and actors, such as robots. It will also pioneer innovative imaging technologies, from new
systems for movement capture, to "morphable" movie studios that allow one physical space to represent a variety of settings.

The most current information about our research is available on the MIT Media Lab Web site, at
http://www.media.mit.edu/research/.

MIT Media Lab May 2010 Page i

Communications Futures Program
The Communications Futures Program conducts research on industry dynamics, technology opportunities, and regulatory
issues that form the basis for communications endeavors of all kinds, from telephony to RFID tags. The program operates
through a series of working groups led jointly by MIT researchers and industry collaborators. It is highly participatory, and its
agenda reflects the interests of member companies that include both traditional stakeholders and innovators. It is jointly directed
by Dave Clark (CSAIL), Charles Fine (Sloan School of Management), Andrew Lippman (Media Lab), and David P. Reed (Media
Lab).

Consumer Electronics Laboratory

The Consumer Electronics Laboratory provides a unique research environment to explore ideas, make things, and innovate in
new directions for consumer products and services. Research projects, which span the entire Media Lab and beyond, focus on:
innovative materials and design/fabrication methods for them; new power technologies; new sensors, actuators, and displays;
self-managing, incrementally and limitlessly scalable ecosystems of smart devices; cooperative wireless communications;
co-evolution of devices and content; and user experience. An overarching theme that runs through all the work is the
co-evolution of design principles and technological discoveries, resulting in simple, ubiquitous, easy- and delightful-to-use
devices that know a great deal about one another, the world, and the people in their proximity.

Digital Life
Digital Life consortium activities engage virtually the entire faculty of the Media Lab around the theme of "open innovation."
Researchers divide the topic into three areas: open communications, open knowledge, and open everything. The first explores
the design and scalability of agile, grassroots communications systems that incorporate a growing understanding of emergent
social behaviors in a digital world; the second considers a cognitive architecture that can support many features of "human
intelligent thinking" and its expressive and economic use; and the third extends the idea of inclusive design to immersive,
affective, and biological interfaces and actions.

Social Health Living Laboratory

Instead of building on a reactive health-care system centered around treating disease rather than preventing it, the Lab's Social
Health Living Laboratory is focused on developing a proactive, social health system: a network of organizations and tools to give
people the knowledge and support they need to maintain health, vitality, and happiness throughout their entire lives. This
involves developing devices such as mobile phones that record our daily patterns, and smart exercise equipment that knows our
personal patterns and life-style goals. This initiative integrates persuasive technologies, to help us make better decisions and
adopt better behaviors; personal sensing, to increase our awareness of our bodies; personal collective intelligence, to collect
knowledge from our peers; and socially aware computation and communication systems that are aware of us as social beings.

Things That Think

Things That Think, the Lab's largest consortium, is inventing the future of digitally augmented objects and environments. Toward
this end, Things That Think researchers are developing sophisticated sensing and computational architectures for networks of
everyday things; designing seamless interfaces that bridge the digital and physical worlds while meeting the human need for
creative expression; and creating an understanding of context and affect that helps things "think" at a much deeper level. Things
That Think projects under way at the Lab range from inventing the city car of the future to designing a prosthesis with the ability
to help a person or machine read social-emotional cues—research that will create the technologies and tools to redefine the
products and services of tomorrow.

Page ii May 2010 MIT Media Lab

 V. Michael Bove Jr.—Object-Based Media ............................................................................................................................ 1

1. Consumer Holo-Video ......................................................................................................................................................... 1

2. Everything Tells a Story ....................................................................................................................................................... 1
3. Guided-Wave Light Modulator ............................................................................................................................................. 1
4. ShakeOnIt ............................................................................................................................................................................ 1
5. SurroundVision .................................................................................................................................................................... 1
6. The "Bar of Soap": Grasp-Based Interfaces ........................................................................................................................ 1
7. uCom ................................................................................................................................................................................... 2
8. Vision-Based Interfaces for Mobile Devices ........................................................................................................................ 2

Ed Boyden—Synthetic Neurobiology .................................................................................................................................... 2

9. Cell-Type-Specific Optical Neuromodulation Interfaces ...................................................................................................... 2

10. Internomics .......................................................................................................................................................................... 2
11. Molecular Sensitizers for Optical Manipulation of Biological Systems ................................................................................. 3
12. Non-Invasive, Focal, and Portable Brain Stimulators .......................................................................................................... 3
13. Real-Time Data Mining ........................................................................................................................................................ 3
14. Funk2: Causal Reflective Programming .............................................................................................................................. 3
15. Gene Therapy Devices ........................................................................................................................................................ 3
16. Marketing and the Placebo Effect ........................................................................................................................................ 3
17. Moral Compass: A Model of Self-Conscious Learning ........................................................................................................ 3
18. Principles of Controlling Neural Circuits .............................................................................................................................. 3

Cynthia Breazeal—Personal Robots ...................................................................................................................................... 4

19. Crowdsourcing Human-Robot Interaction: Online Game to Study Collaborative Human Behavior ................................... 4
20. Illustrated Primer .................................................................................................................................................................. 4
21. MDS: Social Interaction Evaluation of Facial Expressions on Robots ................................................................................. 4
22. Robot Teams for Disaster Response ................................................................................................................................... 4
23. Squash-Stretch for a New Genre of Expressive Robots ...................................................................................................... 4
24. Trans-Fiction(xF) Characters ............................................................................................................................................... 5
25. AIDA: Affective Intelligent Driving Agent .............................................................................................................................. 5
26. Dynamics of Initial Trust and Cooperation .......................................................................................................................... 5
27. Huggable: A Robotic Companion for Long-Term Health Care, Education, and Communication ........................................ 5
28. MeBot .................................................................................................................................................................................. 5
29. Playing a Game with the MDS Robot .................................................................................................................................. 5
30. Storytelling in the Preschool of Future ................................................................................................................................. 5

Leah Buechley—High-Low Tech ............................................................................................................................................ 5

31. Fab FM ................................................................................................................................................................................ 5

32. LilyPond ............................................................................................................................................................................... 5
33. Living Wall ........................................................................................................................................................................... 6
34. Paper Computing ................................................................................................................................................................. 6
35. Textile Sensors .................................................................................................................................................................... 6
36. Tilt-Sensing Quilt ................................................................................................................................................................. 6
37. LilyPad Arduino .................................................................................................................................................................... 6
38. Teardrop: Paint, Paper, and Programs ................................................................................................................................ 6

Chris Csikszentmihályi—Computing Culture ........................................................................................................................ 6

39. Boycott Toolkit ..................................................................................................................................................................... 6

40. Exertion Music ..................................................................................................................................................................... 7
41. ExtrACT ............................................................................................................................................................................... 7
42. LostInBoston.org .................................................................................................................................................................. 7
43. RedInk ................................................................................................................................................................................. 7
44. Selectricity ........................................................................................................................................................................... 7
45. Ambient Addition .................................................................................................................................................................. 7
46. BrownBag Toolkit ................................................................................................................................................................. 7
47. Caché .................................................................................................................................................................................. 8
48. GoodApp .............................................................................................................................................................................. 8
49. GroundTruth ........................................................................................................................................................................ 8
50. Homeless Neighbors ........................................................................................................................................................... 8

MIT Media Lab May 2010 Page iii

51. Landman Report Card ......................................................................................................................................................... 8
52. News Positioning System .................................................................................................................................................... 8
53. Virtual Gaza ......................................................................................................................................................................... 8
54. VOIP Drupal ......................................................................................................................................................................... 8
55. Watching the Watchers ........................................................................................................................................................ 8
56. What's Up ............................................................................................................................................................................ 8

Hugh Herr—Biomechatronics ................................................................................................................................................. 8

57. Control of Muscle-Actuated Systems via Electrical Stimulation ........................................................................................... 8

58. Human Walking Models Applied to Prosthesis Control ....................................................................................................... 9
59. Human Walking Neuromechanical Models .......................................................................................................................... 9
60. Powered Ankle-Foot Prosthesis .......................................................................................................................................... 9
61. Artificial Gastrocnemius ....................................................................................................................................................... 9
62. Biomimetic Active Prosthesis for Above-Knee Amputees .................................................................................................. 9
63. Human Walking Model Predicts Joint Mechanics, Electromyography, and Mechanical Economy ...................................... 9

Henry Holtzman—Information Ecology ............................................................................................................................... 10

64. BiDi Screen ........................................................................................................................................................................ 10

65. Home Fabratory ................................................................................................................................................................. 10
66. Kairoscope: Social Time .................................................................................................................................................... 10
67. Konbit ................................................................................................................................................................................. 10
68. NeXtream: Social Television .............................................................................................................................................. 10
69. Tableau .............................................................................................................................................................................. 11
70. Takeover TV ...................................................................................................................................................................... 11
71. The Glass Infrastructure .................................................................................................................................................... 11
72. Twitter Weather ................................................................................................................................................................. 11
73. Audiograph: Superhero Hearing ........................................................................................................................................ 11
74. Constant Crit ...................................................................................................................................................................... 11
75. Daydar: Framework for Socially Motivated Goal Fulfillment .............................................................................................. 11
76. Glasses-Free HD3DTV ...................................................................................................................................................... 11
77. InfoSmell: Smell Your Data ................................................................................................................................................ 11
78. Proverbial Wallets .............................................................................................................................................................. 12
79. Salvage .............................................................................................................................................................................. 12
80. Shake4Action: Gestural Mobile Coordination .................................................................................................................... 12
81. Social Garden .................................................................................................................................................................... 12
82. Window Wallet ................................................................................................................................................................... 12

Hiroshi Ishii—Tangible Media ............................................................................................................................................... 12

83. Beyond: A Collapsible Input Device for 3D Direct Manipulation ........................................................................................ 12

84. MirrorFugue ....................................................................................................................................................................... 12
85. OnObject ............................................................................................................................................................................ 12
86. PingPongPlusPlus ............................................................................................................................................................. 12
87. Relief .................................................................................................................................................................................. 13
88. Sourcemap ........................................................................................................................................................................ 13
89. Video Play .......................................................................................................................................................................... 13
90. AFK Cookset ...................................................................................................................................................................... 13
91. Cost-Effective Wearable Sensor to Detect EMF ................................................................................................................ 13
92. g-stalt ................................................................................................................................................................................. 13
93. Gestural Interaction ........................................................................................................................................................... 13
94. IdeaGarden ........................................................................................................................................................................ 13
95. O-Link ................................................................................................................................................................................ 13
96. Picture This! ....................................................................................................................................................................... 13
97. Piezing ............................................................................................................................................................................... 13
98. Psychohaptics .................................................................................................................................................................... 13
99. Radical Atoms .................................................................................................................................................................... 13
100. SandScape ........................................................................................................................................................................ 13
101. Sensetable ......................................................................................................................................................................... 14
102. Stress OutSourced ............................................................................................................................................................ 14
103. Tangible Bits ...................................................................................................................................................................... 14
104. Topobo ............................................................................................................................................................................... 14
105. Trackmate .......................................................................................................................................................................... 14

Page iv May 2010 MIT Media Lab

106. Wetpaint ............................................................................................................................................................................. 14

Joseph M. Jacobson—Molecular Machines ........................................................................................................................ 14

107. GeneFab ............................................................................................................................................................................ 14

108. NanoFab ............................................................................................................................................................................ 14

Henry Lieberman—Software Agents .................................................................................................................................... 14

109. Human Goal Network ........................................................................................................................................................ 14

110. Luminoso: Understanding and Visualizing People's Opinions ........................................................................................... 15
111. Not-So-Common Sense ..................................................................................................................................................... 15
112. Raconteur: From Chat to Stories ....................................................................................................................................... 15
113. Reusing Code by Reasoning about Its Purpose ................................................................................................................ 15
114. Semantic Synesthesia ....................................................................................................................................................... 15
115. Agent-Application Communication ..................................................................................................................................... 15
116. Agents for Integrated Annotation and Retrieval of Images ................................................................................................ 16
117. Blending and AnalogySpace .............................................................................................................................................. 16
118. Collecting Common Sense ................................................................................................................................................ 16
119. Common-Sense Investing ................................................................................................................................................. 16
120. Common-Sense Reasoning for Interactive Applications ................................................................................................... 16
121. Common-Sense Recommendations .................................................................................................................................. 16
122. CommonConsensus: A Game for Collecting Commonsense Goals ................................................................................. 16
123. ConceptNet ........................................................................................................................................................................ 16
124. Divisi: Reasoning Over Semantic Relationships ................................................................................................................ 16
125. E-Commerce When Things Go Wrong .............................................................................................................................. 16
126. Emotus Ponens: Affective Story Understanding for Agents .............................................................................................. 16
127. Goal-Oriented Interfaces for Consumer Electronics .......................................................................................................... 16
128. Graphical Interfaces for Software Visualization and Debugging ........................................................................................ 16
129. Intelligent Technical Documentation .................................................................................................................................. 16
130. Learning Common Sense in a Second Language ............................................................................................................. 16
131. MARCO: Mutual Disambiguation of Recognition Errors in a Multimodal Navigational Agent ............................................ 16
132. Moral Compass: A Model of Self-Conscious Learning ...................................................................................................... 16
133. Multi-Lingual ConceptNet .................................................................................................................................................. 16
134. Multilingual Common Sense .............................................................................................................................................. 16
135. Navigating in Very Large Display Spaces .......................................................................................................................... 17
136. Open Mind Commons ........................................................................................................................................................ 17
137. Programming in Natural Language .................................................................................................................................... 17
138. Red Fish, Blue Fish ........................................................................................................................................................... 17
139. Relational Analogies in Semantic Networks ...................................................................................................................... 17
140. Storied Navigation ............................................................................................................................................................. 17
141. TwitterMap ......................................................................................................................................................................... 17

Andrew Lippman and David P. Reed—Viral Communications .......................................................................................... 17

142. Barter: A Market-Incented Wisdom Exchange ................................................................................................................... 17

143. Location Sharing in Large Indoor Environments ................................................................................................................ 17
144. Meal Time .......................................................................................................................................................................... 17
145. Pixel Infrastructures ........................................................................................................................................................... 18
146. The Glass Infrastructure .................................................................................................................................................... 18
147. Ego .................................................................................................................................................................................... 18
148. Open Spaces ..................................................................................................................................................................... 18
149. Social Menu ....................................................................................................................................................................... 18
150. Social Saver ....................................................................................................................................................................... 18

Tod Machover—Opera of the Future .................................................................................................................................... 18

151. Death and the Powers: Redefining Opera ......................................................................................................................... 18

152. Disembodied Performance ................................................................................................................................................ 19
153. Expressive Hand Rehab .................................................................................................................................................... 19
154. Gestural Media Framework ............................................................................................................................................... 19
155. Mobile Music Diagnostics: Targeting Alzheimer's Disease ................................................................................................ 19
156. Musical Furniture ............................................................................................................................................................... 19
157. Syncwalk ............................................................................................................................................................................ 20
158. The Chandelier .................................................................................................................................................................. 20

MIT Media Lab May 2010 Page v

159. Hyperinstruments ............................................................................................................................................................... 20
160. Hyperscore ........................................................................................................................................................................ 20
161. Music, Mind, and Health .................................................................................................................................................... 20
162. Musical Robotics ................................................................................................................................................................ 20
163. Personal Opera .................................................................................................................................................................. 20
164. Rumble .............................................................................................................................................................................. 20
165. Skellig: A "Surround" Opera .............................................................................................................................................. 20
166. Toy Symphony ................................................................................................................................................................... 20
167. Vocal Augmentation and Manipulation Prosthesis (VAMP) ............................................................................................... 20

Pattie Maes—Fluid Interfaces ............................................................................................................................................... 21

168. Cornucopia: Digital Gastronomy ........................................................................................................................................ 21

169. Defuse ............................................................................................................................................................................... 21
170. Face the Facts ................................................................................................................................................................... 21
171. InterPlay: Full-Body Interaction Platform ........................................................................................................................... 21
172. LuminAR ............................................................................................................................................................................ 21
173. ReflectOns: Mental Prostheses for Self-Reflection ............................................................................................................ 22
174. SixthSense ......................................................................................................................................................................... 22
175. Aeropticon .......................................................................................................................................................................... 22
176. Augmented Mirror .............................................................................................................................................................. 22
177. BitBrush ............................................................................................................................................................................. 22
178. Blossom ............................................................................................................................................................................. 22
179. Compact Contract: Commitments Made Easier ................................................................................................................ 22
180. Done: Reflective Personal Project Management ............................................................................................................... 22
181. EyeFlect: Low-Cost Multi-Flash Gaze Estimation .............................................................................................................. 22
182. Flexible Urban Display ....................................................................................................................................................... 22
183. GovLove: Simplifying Government Data Retrieval and Manipulation ............................................................................... 22
184. Inktuitive: An Intuitive Physical Design Workspace ........................................................................................................... 22
185. ioMaterials ......................................................................................................................................................................... 22
186. JotWatch: Instant Personal Note-Taking ........................................................................................................................... 22
187. Konbit ................................................................................................................................................................................. 23
188. Life in a Comic ................................................................................................................................................................... 23
189. Light Piping Solar Panels ................................................................................................................................................... 23
190. MemTable .......................................................................................................................................................................... 23
191. Midas ................................................................................................................................................................................. 23
192. Mouseless .......................................................................................................................................................................... 23
193. Moving Portraits ................................................................................................................................................................. 23
194. MTM "Little John" ............................................................................................................................................................... 23
195. Personas ............................................................................................................................................................................ 23
196. Pulp-Based Computing: A Framework for Building Computers Out of Paper ................................................................... 23
197. Quickies: Intelligent Sticky Notes ....................................................................................................................................... 23
198. ReachBand: An RFID Wristband ....................................................................................................................................... 23
199. Relational Pillow ................................................................................................................................................................ 23
200. Remnant: Handwriting Memory Card ................................................................................................................................ 23
201. Shutters: A Permeable Surface for Environmental Control and Communication .............................................................. 23
202. Siftables: Physical Interaction with Digital Media ............................................................................................................... 23
203. SoundForms ...................................................................................................................................................................... 23
204. SpaceMarks: Brain Offloading through Spatial Thinking ................................................................................................... 23
205. SpendTrend: Reflecting on Spending Habits ..................................................................................................................... 23
206. Spotlight ............................................................................................................................................................................. 23
207. Sprout I/O: A Texturally Rich Interface .............................................................................................................................. 23
208. subTextile: A Construction Kit for Computationally Enabled Textiles ................................................................................ 23
209. Surflex: A Shape-Changing Surface .................................................................................................................................. 24
210. TaPuMa: Tangible Public Map ........................................................................................................................................... 24
211. Theme Stream: Visualizing Complex Time-Based Information ......................................................................................... 24
212. thirdEye .............................................................................................................................................................................. 24
213. Transitive Materials: Towards an Integrated Approach to Material Technology ................................................................ 24
214. Watt Watcher ..................................................................................................................................................................... 24
215. What If the World Were Your n Facebook Friends? .......................................................................................................... 24
216. What Would They Think? .................................................................................................................................................. 24

Page vi May 2010 MIT Media Lab

 William J. Mitchell—Smart Cities .......................................................................................................................................... 24

217. CityCar ............................................................................................................................................................................... 24

218. GreenWheel Electric Bicycle ............................................................................................................................................. 25
219. Liberated Pixels ................................................................................................................................................................. 25
220. Mobility on Demand ........................................................................................................................................................... 25
221. Mobility on Demand: A Market Economy of Trips .............................................................................................................. 25
222. New Object Studio ............................................................................................................................................................. 26
223. RoboScooter with SYM and ITRI ....................................................................................................................................... 26
224. Wheel Robots .................................................................................................................................................................... 26
225. 3DprintedClock .................................................................................................................................................................. 26
226. Animated Playground Props .............................................................................................................................................. 26
227. Architectural Machines ...................................................................................................................................................... 26
228. Augmented Street Light ..................................................................................................................................................... 26
229. Building Blocks in the Mass-Customized Era .................................................................................................................... 26
230. Car in the City .................................................................................................................................................................... 26
231. Chameleon Guitar: Physical Heart in a Virtual Body ......................................................................................................... 26
232. CityCar Chassis ................................................................................................................................................................. 27
233. CityCar Folding Chassis ................................................................................................................................................... 27
234. Cornucopia: Digital Gastronomy ........................................................................................................................................ 27
235. Hiriko CityCar with Denokinn ............................................................................................................................................. 27
236. Light.Bodies ....................................................................................................................................................................... 27
237. plywoodServo .................................................................................................................................................................... 27

Frank Moss—New Media Medicine ....................................................................................................................................... 27

238. CollaboRhythm .................................................................................................................................................................. 27

239. Collective Discovery .......................................................................................................................................................... 27
240. ForgetAboutIT? .................................................................................................................................................................. 28
241. HealthMap ......................................................................................................................................................................... 28
242. HealthMap iPhone App: Outbreaks Near Me .................................................................................................................... 28
243. LAMsight: A Data-Driven Disease Community ................................................................................................................. 28
244. Oovit PT ............................................................................................................................................................................. 28
245. Awaken .............................................................................................................................................................................. 29
246. I'm Listening ....................................................................................................................................................................... 29
247. WeightMate ........................................................................................................................................................................ 29

Joseph Paradiso—Responsive Environments .................................................................................................................... 29

248. Boxie the Robot: Interactive Physical Agents for Story Gathering ..................................................................................... 29
249. DoppelLab ......................................................................................................................................................................... 29
250. Dynamic Video Layers for Privacy-Preserving "Opt-In" Media Services ........................................................................... 29
251. Feedback Controlled Solid State Lighting .......................................................................................................................... 30
252. Lab-Wide and Wearable Sensor and Video Network ........................................................................................................ 30
253. Moral Compass: A Model of Self-Conscious Learning ...................................................................................................... 30
254. Wearable, Wireless Sensor System for Sports Medicine and Interactive Media ............................................................... 30
255. Dense, Low-Power Environmental Monitoring for Smart Energy Profiling ........................................................................ 30
256. Funk2: Causal Reflective Programming ............................................................................................................................ 30
257. Interaction with Ubiquitous Dynamically Responsive Media .............................................................................................. 31
258. Ubiquitous Sensor Network Navigator and Media Explorer ............................................................................................... 31

Alex (Sandy) Pentland—Human Dynamics ......................................................................................................................... 31

259. Awaken .............................................................................................................................................................................. 31

260. Meeting Mediator ............................................................................................................................................................... 31
261. Reality Mining .................................................................................................................................................................... 31
262. Sensible Organizations ...................................................................................................................................................... 31
263. Social Evolution ................................................................................................................................................................. 32
264. WeightMate ........................................................................................................................................................................ 32
265. Economic Decision-Making in the Wild .............................................................................................................................. 32

MIT Media Lab May 2010 Page vii

 Rosalind W. Picard—Affective Computing .......................................................................................................................... 32

266. Affective-Cognitive Product Evaluation and Prediction of Customer Decisions ................................................................ 32

267. Cardiocam ......................................................................................................................................................................... 32
268. Emotion Communication in Autism .................................................................................................................................... 33
269. FaceSense: Affective-Cognitive State Inference from Facial Video .................................................................................. 33
270. iCalm (TM): Comfortable, Wearable, Wireless Bio-Sensing .............................................................................................. 33
271. Objective Self: Understanding Internal Responses ........................................................................................................... 33
272. Passive Wireless Heart-Rate Sensor ................................................................................................................................ 34
273. Affect as Index ................................................................................................................................................................... 34
274. Affect Valence Recognition From Facial Expressions ....................................................................................................... 34
275. Affective-Cognitive Framework for Machine Learning and Decision Making ..................................................................... 34
276. Auditory Desensitization Games ........................................................................................................................................ 34
277. Embedding Special Interests Into Computer-Mediated Interventions ................................................................................ 34
278. Emotional-Social Intelligence Toolkit ................................................................................................................................. 34
279. Evaluation Tool for Recognition of Social-Emotional Expressions from Facial-Head Movements .................................... 34
280. Externalization Toolkit ........................................................................................................................................................ 34
281. Frame It ............................................................................................................................................................................. 34
282. Gestural Control of Guitar Audio Effects ............................................................................................................................ 34
283. Girls Involved in Real-Life Sharing .................................................................................................................................... 34
284. Health Interventions Using Mobile Phones ........................................................................................................................ 34
285. Heartphones ...................................................................................................................................................................... 34
286. Infant Monitoring and Communication ............................................................................................................................... 34
287. Machine Learning and Pattern Recognition with Multiple Modalities ................................................................................. 34
288. Sensor-Enabled Measurement of Stereotypy and Arousal in Individuals with Autism ...................................................... 34

Ramesh Raskar—Camera Culture ........................................................................................................................................ 35

289. Bokode: Imperceptible Visual Tags for Camera-Based Interaction from a Distance ......................................................... 35
290. Coded Computational Photography ................................................................................................................................... 35
291. Femtosecond Transient Imaging ....................................................................................................................................... 35
292. Glasses-Free HD3DTV ...................................................................................................................................................... 35
293. LensChat: Sharing Photos with Strangers ......................................................................................................................... 35
294. Theory Unifying Ray and Wavefront Lightfield Propagation .............................................................................................. 36
295. Community Photography ................................................................................................................................................... 36
296. Second Skin: Optical Motion Capture with Actuated Feedback ........................................................................................ 36
297. Shield Field Imaging .......................................................................................................................................................... 36
298. Vision on Tap ..................................................................................................................................................................... 36

Mitchel Resnick—Lifelong Kindergarten ............................................................................................................................. 36

299. Computer Clubhouse ......................................................................................................................................................... 36

300. Drawdio .............................................................................................................................................................................. 36
301. Hook-Ups ........................................................................................................................................................................... 37
302. Scratch ............................................................................................................................................................................... 37
303. ScratchEd .......................................................................................................................................................................... 37
304. Singing Fingers .................................................................................................................................................................. 37
305. Color Code ......................................................................................................................................................................... 37
306. Computer Clubhouse Village ............................................................................................................................................. 37
307. Computer Crafting ............................................................................................................................................................. 37
308. DesignBlocks ..................................................................................................................................................................... 37
309. Glowdoodle ........................................................................................................................................................................ 37
310. Jots .................................................................................................................................................................................... 37
311. Mobile Scratch ................................................................................................................................................................... 38
312. Say What?! ........................................................................................................................................................................ 38
313. Scratch Board .................................................................................................................................................................... 38
314. Scratch Day ....................................................................................................................................................................... 38
315. Scratch for Computer Science ........................................................................................................................................... 38
316. Scratch Worlds .................................................................................................................................................................. 38
317. ScratchR ............................................................................................................................................................................ 38
318. Twinkle ............................................................................................................................................................................... 38
319. What's Up .......................................................................................................................................................................... 38

Page viii May 2010 MIT Media Lab

 Deb Roy—Cognitive Machines ............................................................................................................................................. 38

320. Behavior Capture from Thousands of People Online ........................................................................................................ 38

321. Connected Strangers: Manipulating Social Perceptions to Study Trust ............................................................................ 38
322. Gestalt Video Analyzer ...................................................................................................................................................... 38
323. Grounding Spatial Language for Video Retrieval and Robotic Direction Following ........................................................... 39
324. HouseFly: Immersive Video Browsing and Data Visualization .......................................................................................... 39
325. Speech Interaction Analysis for the Human Speechome Project ...................................................................................... 39
326. Speechome Recorder for the Study of Child Development Disorders ............................................................................... 39
327. Speechome Video for Retail Analysis ................................................................................................................................ 40
328. Study of Child Language Acquisition in the Human Speechome Project ......................................................................... 40
329. BlitzScribe: Speech Analysis for the Human Speechome Project ..................................................................................... 40
330. Collective Discovery .......................................................................................................................................................... 40
331. Data-Driven Architectural Design ...................................................................................................................................... 40
332. Human Speechome Project ............................................................................................................................................... 40
333. Internomics ........................................................................................................................................................................ 40
334. TrackMarks: Semi-Automatic Video Annotation ................................................................................................................ 40

Chris Schmandt—Speech + Mobility .................................................................................................................................... 40

335. Back-Talk ........................................................................................................................................................................... 40

336. Guiding Light ...................................................................................................................................................................... 41
337. Merry Miser ........................................................................................................................................................................ 41
338. Tin Can .............................................................................................................................................................................. 41
339. Wish You Were Here ......................................................................................................................................................... 41
340. Conch ................................................................................................................................................................................ 41
341. Flickr This .......................................................................................................................................................................... 41
342. Going My Way ................................................................................................................................................................... 41
343. My Second-Bike ................................................................................................................................................................. 41
344. Presentation Spaces .......................................................................................................................................................... 41
345. Sharemote: Collaborative TV ............................................................................................................................................. 42

David Small—Design Ecology .............................................................................................................................................. 42

346. Cartagen ............................................................................................................................................................................ 42

347. Face the Facts ................................................................................................................................................................... 42
348. Kaleido: Idiosyncractic Graphical Interfaces for Software Development ........................................................................... 42
349. What If the World Were Your n Facebook Friends? .......................................................................................................... 42
350. Daydar: Framework for Socially Motivated Goal Fulfillment .............................................................................................. 43
351. Grassroots Mapping with Balloons and Kites .................................................................................................................... 43

Barry Vercoe—Music, Mind and Machine ............................................................................................................................ 43

352. Musicpainter ...................................................................................................................................................................... 43

353. Musicscape ........................................................................................................................................................................ 43
354. Perceptual Sound Design .................................................................................................................................................. 43
355. Prediction and Intention in Network Music Performance ................................................................................................... 43
356. Radio-ish Media Player ...................................................................................................................................................... 44
357. Speaker Identification of Marine Mammals ........................................................................................................................ 44
358. Audio Spotlight ................................................................................................................................................................... 44

MIT Media Lab May 2010 Page ix

 V. Michael Bove Jr.—Object-Based Media
How to create communication systems that gain an understanding of the content they carry
and use it to make richer connections among users.

1. Consumer V. Michael Bove Jr., James Barabas, Daniel Smalley and Quinn Smithwick
Holo-Video
The goal of this project, building upon work begun by Stephen Benton and the Spatial
Imaging group, is to create an inexpensive desktop monitor for a PC or game console
CENTER FOR FUTURE
that displays holographic video images in real time, suitable for entertainment,
STORYTELLING
engineering, or medical imaging. To date, we have demonstrated the fast rendering of
holo-video images from OpenGL databases on off-the-shelf PC graphics cards; current
research addresses new optoelectronic architectures to reduce the size and
manufacturing cost of the display system.

2. Everything Tells a V. Michael Bove Jr., David Cranor and Edwina Portocarrero
Story
Following upon work begun in the Graspables project, we are exploring what happens
when a wide range of everyday consumer products can sense, interpret into human
CENTER FOR FUTURE
terms (using pattern recognition methods), and retain memories, such that users can
STORYTELLING
construct a narrative with the aid of the recollections of the "diaries" of their sporting
equipment, luggage, furniture, toys, and other items with which they interact.

3. Guided-Wave Light V. Michael Bove Jr., Daniel Smalley and Quinn Smithwick
Modulator
We are developing inexpensive, efficient, high-bandwidth light modulators based on
lithium niobate guided-wave technology. These modulators are suitable for demanding,
specialized applications such as holographic video displays, as well as other light
modulation uses such as compact video projectors.

4. ShakeOnIt V. Michael Bove Jr. and David Cranor

We are exploring ways to encode information exchange into preexisting natural

CENTER FOR FUTURE
STORYTELLING interaction patterns, both between people and between a single user and objects with
NEW LISTING which he or she interacts on a regular basis. Two devices are presented to provoke
thoughts regarding these information interchange modalities: a pair of gloves that
requires two users to complete a "secret handshake" in order to gain shared access to
restricted information, and a doorknob that recognizes the grasp of a user and becomes
operational if the person attempting to use it is authorized to do so.

5. SurroundVision V. Michael Bove Jr. and Santiago Alfaro

Adding augmented reality to the living room TV, we are exploring the technical and
CENTER FOR FUTURE
STORYTELLING creative implications of using a mobile phone or tablet (and possibly also dedicated
devices like toys) as a controllable "second screen" for enhancing television viewing.
Thus, a viewer could use the phone to look beyond the edges of the television to see
the audience for a studio-based program, to pan around a sporting event, to take
snapshots for a scavenger hunt, or to simulate binoculars to zoom in on a part of the
scene.

6. The "Bar of Soap": V. Michael Bove Jr. and Brandon Taylor

Grasp-Based
We have built several handheld devices that combine grasp and orientation sensing
Interfaces
with pattern recognition in order to provide highly intelligent user interfaces. The Bar of
Soap is a handheld device that senses the pattern of touch and orientation when it is
held, and reconfigures to become one of a variety of devices, such as phone, camera,
remote control, PDA, or game machine. Pattern-recognition techniques allow the device
to infer the user's intention based on grasp. Another example is a baseball that
determines a user's pitching style as an input to a video game.

MIT Media Lab May 2010 Page 1

7. uCom V. Michael Bove Jr. and James D. Barabas

uCom (the "u" stands for "ubiquitous") is a follow-on to our iCom (Media Lab/Media Lab
Europe, 1999) system for connecting architectural spaces to enable collaboration by
distributed groups. uCom takes advantage of input/output resources (e.g., displays,
cameras, speakers, sensors) already in place; it is not restricted to one "window" in one
location, but rather creates multiple video and audio portals between the spaces; it
scales in richness as the number of input and output devices increase; and it has both a
real-world and a virtual-world presence. The virtual-world model can be accessed by
users in other places, or for replay of past events. uCom also enables the aggregating
of sensor data to allow processes to draw higher-level inferences (e.g., for
understanding the actions of the community of users), monitoring individual users (e.g.,
for health or disability reasons), or augmenting the space with additional virtual
information.

Alumni Contributor: Ana Luisa Santos

Additional Object-Based Media projects

For full descriptions, visit: http://www.media.mit.edu/research/25

CENTER FOR FUTURE 8. Vision-Based Interfaces for Mobile Devices

STORYTELLING V. Michael Bove Jr. and Santiago Alfaro

Ed Boyden—Synthetic Neurobiology
How to engineer intelligent neurotechnologies to repair pathology, augment cognition, and
reveal insights into the human condition.

9. Cell-Type-Specific Ed Boyden, Brian Allen, Jacob Bernstein, Jeremy Chang, Alexander Guerra, Xue
Optical Han, Emily Ko, Ashutosh Singhal, Christian Wentz and Anthony Zorzos
Neuromodulation
Neural stimulation hardware has traditionally been either electrical or magnetic in
Interfaces nature. Our lab has recently developed optogenetic molecular methods for making
neurons able to be activated or silenced by multiple colors of light. We are engineering
optical hardware systems for targetedly stimulating and inactivating neurons precisely,
from one to many at a time, with complex spatiotemporal patterns, even in dense tissue
in the living brain. Our goal is to find ways to cure intractable psychiatric and
neurological disorders.

10. Internomics Ed Boyden, Dan Ariely, Deb Roy, Nathan Greenslit, Sheng-Ying (Aithne) Pao,
Coco Krumme, Deborah Egloff and James Barabas

How do high-level cognitive functions emerge from primitive neural computations to

mediate complex human behavior? We are developing precise, focal ways of
investigating phenomena such as trust and risk-taking, in order to understand how they
play roles in purchasing, decision-making, social interaction, and other real-world
scenarios.

Alumni Contributors: Barbara Barry, Marko Popovic, Jessica Schirmer and Rebecca L.
Waber

Page 2 May 2010 MIT Media Lab

11. Molecular Sensitizers Ed Boyden, Brian Chow, Amy Chuong, Alison Dobry, Xue Han, Nathan Klapoetke,
for Optical Tania Morimoto, Daniel Schmidt, Aimei Yang, Mingjie Li and Xiaofeng Qian
Manipulation of
We have engineered molecular sensitizers that make genetically specified neurons that
Biological Systems can be activated by pulses of blue light, and silenced by pulses of yellow light. This
revolutionary technology enables us to reprogram neural networks at the millisecond
timescale, opening up the systematic analysis and engineering of the brain, as well as
completely novel methods of therapy. We are now developing new and improved
molecules and pursuing pre-clinical translational testing.

12. Non-Invasive, Focal, Ed Boyden, Leah Acker, Mike Henninger, Gilberto Abram and Azadeh Moini
and Portable Brain
Despite use in treating depression, and promise in treating stroke, Parkinson's, tinnitus,
Stimulators
and other disorders, non-invasive brain stimulation technology is bulky, power-hungry,
non-focal, and requires precision alignment with neural structures. We are applying
modern engineering techniques to create a portable, focal, non-invasive brain
stimulator that will enable a new platform for therapeutic neuromodulation.

13. Real-Time Data Ed Boyden, Brian Allen and Doug Fritz

Mining
Complex data—such as neurophysiological recordings, or measures of human
behavior, Internet, and general network data—are extremely difficult to analyze
because of the dynamic nature of the high-dimensional set of interacting processes that
generates the data. Accordingly, traditional statistical and data analysis
methods—clustering, correlation, and so forth—can rarely create models sophisticated
enough to explain the data without trying to explain noise, demanding astronomically
sized datasets, or requiring enormous amounts of hand-tuning by insightful labor. We
propose to design and develop a system that continuously generates novel
data-modeling hypotheses and evaluates them in real time, testing models of
ever-increasing complexity on data as it comes in.

Alumni Contributor: Al Strelzoff

Additional Synthetic Neurobiology projects

For full descriptions, visit: http://www.media.mit.edu/research/2129

14. Funk2: Causal Reflective Programming

Marvin Minsky, Joe Paradiso and Bo Morgan

15. Gene Therapy Devices

Ed Boyden, Jacob Bernstein and Stephanie Chan

16. Marketing and the Placebo Effect

Dan Ariely and Nathan Greenslit

17. Moral Compass: A Model of Self-Conscious Learning

Ed Boyden, Henry Lieberman, Marvin Minsky, Joe Paradiso and Bo
Morgan

18. Principles of Controlling Neural Circuits

Ed Boyden, Leah Acker, Claire Ahn, Brian Allen, Rachel Bandler, Michael
Baratta, Jacob Bernstein, Gary Brenner, Giovanni Talei Franzesi, Vinay
Gidwaney, Alexander Guerra, Xue Han, Mike Henninger, Kyungman Kim,
Emily Ko, Sonya Makhni and Patrick Monahan

MIT Media Lab May 2010 Page 3

 Cynthia Breazeal—Personal Robots
How to build social robots that interact, collaborate, and learn with people as partners.

19. Crowdsourcing Cynthia Breazeal, Jason Alonso and Sonia Chernova

Human-Robot
Many new applications for robots require them to work alongside people as capable
Interaction: Online
members of human-robot teams. We have developed Mars Escape, a two-player online
Game to Study game designed to study how humans engage in teamwork, coordination, and
Collaborative Human interaction. Data gathered from hundreds of online games is being used to develop
Behavior computational models of human collaborative behavior in order to create an
autonomous robot capable of acting as a reliable human teammate. In the summer of
2010, we will recreate the Mars Escape game in real life at the Boston Museum of
NEW LISTING
Science and invite museum visitors to perform collaborative tasks together with the
autonomous MDS robot Nexi.

20. Illustrated Primer Cynthia Breazeal and Angela Chang

The Illustrated Primer is an interactive storybook for parents and children to explore and
CENTER FOR FUTURE
STORYTELLING read together. The Illustrated Primer is designed to allow emergent readers to explore
actively, via experiment, the abstract relationship between printed words and their
meanings, even before this relationship is properly understood.

21. MDS: Social Cynthia Breazeal, Jun Ki Lee and Jesse Gray
Interaction Evaluation
The fusion of "intelligence" and "gentleness" is the foundation of Toyota's partner robot
of Facial Expressions
project. Currently, Toyota is focused on enhancing the interactive partner robot
on Robots experience, for example, for patients in hospitals. "Social graces"—the ability of a
partner robot to interact with a person in a socially skillful and pleasant manner that is
likeable and engaging—is fundamental to the realization of human and partner robot
coexistence. To this end, Toyota has proposed a collaborative research initiative with
the Media Lab to examine how the expressive face and neck movements of a partner
robot contribute to a human's perception of the "social graces" of a robot using the
Media Lab's MDS platform.

Alumni Contributor: Michael Steven Siegel

22. Robot Teams for Cynthia Breazeal, Philipp Robbel and Kenton Williams
Disaster Response
We are demonstrating how a heterogeneous group of robots (MDS and helicopters)
can be used as a first disaster response before human teams enter the perimeter. The
goal is to have robots engage in the search for victims, build a map of the environment,
and report back through a wireless link to human operators at a safe distance.
Information collected by the robots is displayed on a remote operator interface in real
time so that human personnel can easily create new tasks that reflect the state of the
environment. In addition to this semi-autonomous implementation under human
supervision, we will demonstrate a completely autonomous search for victims using
both ground and air robots, incorporating novel tasking and planning algorithms.

23. Squash-Stretch for a Cynthia Breazeal and Ryan Wistort

New Genre of
Tofu is a robot designed to create what is known in the animation world as “the illusion
Expressive Robots
of life,” by leveraging design principles from 2D animation and cognitive psychology.
Through use of actuation and display technologies, these robot design principles have
been applied to create incredibly low-cost and expressive robot characters. To further
explore these robots as expressive characters, we are examining their use as a
storytelling medium. By combining elements of robot puppetry and autonomous robot
agents, we aim to create a storytelling medium that enables children to create engaging
stories within a constructivist-learning environment.

Page 4 May 2010 MIT Media Lab

24. Trans-Fiction(xF) Cynthia Breazeal and David Robert
Characters
Trans-Fiction (xF) characters promote new levels of bi-directional interactivity. This
project is an exploration of new ways both to bring fantasy into the real world, and vice
CENTER FOR FUTURE
versa.
STORYTELLING

Additional Personal Robots projects

For full descriptions, visit: http://www.media.mit.edu/research/53

25. AIDA: Affective Intelligent Driving Agent

Mikey Siegel and Cynthia Breazeal

26. Dynamics of Initial Trust and Cooperation

Cynthia Breazeal and Jin Joo Lee in conjunction with Northeastern
University and Cornell University

CENTER FOR FUTURE 27. Huggable: A Robotic Companion for Long-Term Health Care,
STORYTELLING Education, and Communication
Cynthia Breazeal, Walter Dan Stiehl, Robert Toscano, Jun Ki Lee, Heather
Knight, Sigurdur Orn Adalgeirsson, Jeff Lieberman and Jesse Gray

28. MeBot
Cynthia Breazeal, Sigurdur Orn Adalgeirsson and Nancy Foen

CENTER FOR FUTURE 29. Playing a Game with the MDS Robot
STORYTELLING Cynthia Breazeal, Jason Alonso, Sonia Chernova, Jesse Gray, Jin Joo Lee,
Philipp Robbel and Kenton Williams

CENTER FOR FUTURE 30. Storytelling in the Preschool of Future

STORYTELLING Ryan Wistort

Leah Buechley—High-Low Tech

How to engage diverse audiences in creating their own technology by situating
computation in new contexts and building tools to democratize engineering.

31. Fab FM David Mellis and Dana Gordon

Fab FM explores the possibilities for personal fabrication of consumer electronic

NEW LISTING
devices. It is a wood- and fabric-cased FM radio that can be manufactured in small
volumes by an individual with access to a laser cutter. Each radio can be customized
with materials (e.g., wood or fabric) provided by the customer. Because the radio can
be produced from its digital design files using minimal infrastructure, it offers a diverse
set of possible business models and distribution schemes. For example, radios could
be sold as kits to be assembled by the customer, or produced by individuals in many
different cities.

32. LilyPond Emily Lovell, Leah Buechley, Kanjun Qiu and Linda Delafuente

LilyPond is a budding e-textile Web community that fosters creative collaboration

NEW LISTING
through the sharing of personal projects. Home to a growing repository of skill- and
project-based tutorials, LilyPond provides support for young adults who want to design
and create soft, interactive circuits with the LilyPad Arduino toolkit.

MIT Media Lab May 2010 Page 5

33. Living Wall Leah Buechley, TungShen Chew, Bonifaz Kaufmann, Emily Lovell, David Mellis,
Hannah Perner-Wilson and Jie Qi

Run your hand across this wallpaper to turn on a lamp, play music, or control your
toaster. This project experiments with interactive wallpaper that can be programmed to
monitor its environment, control lighting and sound, and generally serve as a beautiful
and unobtrusive way to enrich environments with computational capabilities. The
wallpaper itself is flat, constructed entirely from paper and paint. The paper is paired
with magnetic electronic modules that serve as sensors, lamps, network interfaces, and
interactive decorations.

34. Paper Computing Leah Buechley

What interfaces might we build if we could sketch functional systems directly on paper?
What will circuits look like when they are painted or drawn instead of etched or
machined? To explore the potential of paper-based computing, we have developed a
construction kit of magnetic electronic modules that snap onto sketches made with
conductive and magnetic paints. We are currently using these materials to explore
sketch-able user interfaces, interactive architectural installations, and educational
opportunities in electronics and digital art.

35. Textile Sensors Leah Buechley and Hannah Perner-Wilson

We are exploring ways to build sensors using a variety of crafting and needlework
NEW LISTING
techniques, using affordable and available materials such as conductive threads, yarns,
fabrics, and paints. These materials are used to sew, knit, crochet, embroider, and
laminate, creating a range of textile-based sensors.

36. Tilt-Sensing Quilt Hannah Perner-Wilson

This shape-sensing quilt consists of a communicating grid of textile tilt sensors. It

showcases the wide variety of techniques we have developed for constructing
fabric-based sensors and circuits.

Additional High-Low Tech projects

For full descriptions, visit: http://www.media.mit.edu/research/2449

37. LilyPad Arduino

Leah Buechley

38. Teardrop: Paint, Paper, and Programs

Leah Buechley, TungShen Chew, Hannah Perner-Wilson, Emily Lovell,
David Mellis and Bonifaz Kaufmann

Chris Csikszentmihályi—Computing Culture

How artists and engineers can refigure technology for the full range of human experience.

39. Boycott Toolkit Chris Csikszentmihályi and Josh Levinger

The Boycott Toolkit provides tools for consumers to organize collective economic
NEW LISTING
action. Users can learn about the politics behind everyday companies and products,
pledge to join a campaign, and share information with their friends through social
networks.

Page 6 May 2010 MIT Media Lab

40. Exertion Music Noah Vawter

Are electronic instruments that generate their own power better than those that don't?
Can acoustic and electronic musical instruments be successfully merged? Can the
movement of the sound generation be tightly coupled to the power generation, as
opposed to merely modulating a large power reserve, as in traditional instruments?
What useful musical artifacts/affordances can be created through this technology?

41. ExtrACT Daniel Ring and Christina Xu

ExtrAct, a set of Internet-based databasing, mapping, and communications

technologies for communities impacted by natural gas development, is a novel platform
for community education and civic action. Its objective is to create and distribute
open-source, Web-based tools for mapping, analyzing, and intervening in this industry
based on supplementing data obtained from state and federal agencies with
user-generated reports, complaints, and experiences. All of these tools, though
accessible individually, will share information through a unified database. Because
these tools will serve both urban and rural populations, we are also developing
innovative paper and phone interfaces to the Web services. To develop these tools we
are working with a network of lawyers, citizen’s alliances, national activist organizations,
and environmental health experts in Colorado, New Mexico, Ohio, New York,
Pennsylvania, West Virginia, and Texas.

42. LostInBoston.org Chris Csikszentmihályi, Rick Borovoy and Christina Xu

LostInBoston.org is about helping Bostonians work together to make their

NEW LISTING
neighborhoods more visitor-friendly. Community groups are partnering with local
businesses and institutions to design signs that call out the key spots in their area.
Signs are placed on private land in public places.

43. RedInk Ryan O'Toole

RedInk is personal finance software with a social conscience. The Website provides
constituencies with tools to collectively measure the effect of their economic power as it
relates to specific industries and businesses, while maintaining privacy for individual
users. Up until now, accounting of this nature has been vague or unavailable. More
accurate spending data will be a valuable lever for organizations involved in collective
action, collective bargaining, and fundraising.

44. Selectricity Benjamin Mako Hill and Chris Csikszentmihályi

Selectricity (formerly HyperChad) is a Web-based voting system that supports

anonymous and voter-verifiable balloting, and includes an election-methods library that
implements a variety of election techniques, including several preferential systems.
Unlike most voting projects, Selectricity does not attempt to address the issues raised
in mainstream political elections. Instead, it provides a simple set of tools that small
groups and organizations can use to incorporate computationally complex
decision-making into new areas, and for purposes where they ordinarily would find such
decision-making prohibitively complex. By supporting a variety of election methods, it
provides a way for users to explore and compare the effects of different voting systems
and, ultimately, come to better decisions.

Alumni Contributor: Alyssa Wright

Additional Computing Culture projects

For full descriptions, visit: http://www.media.mit.edu/research/55

45. Ambient Addition

Noah Vawter

46. BrownBag Toolkit

Chris Csikszentmihályi, Rick Borovoy and Matthew Hockenberry

MIT Media Lab May 2010 Page 7

 47. Caché
Chris Csikszentmihályi and Nadya Peek

48. GoodApp
Matthew Hockenberry, Dan Ring, Pedro Brin and Chris Csikszentmihályi

49. GroundTruth
Josh Levinger

50. Homeless Neighbors

Chris Csikszentmihályi, Rick Borovoy, Matthew Hockenberry and Spare
Change News

51. Landman Report Card

Sara Wylie, Dan Ring and Christina Xu

52. News Positioning System

Sara Wylie, Dan Ring, Matthew Hockenberry and Christina Xu

53. Virtual Gaza

Josh Levinger and the Harvard Alliance for Justice in the Middle East

54. VOIP Drupal

Chris Csikszentmihályi and Leo Burd

55. Watching the Watchers

Chris Csikszentmihályi and Adam Whiton

56. What's Up
Chris Csikszentmihályi, Mitchel Resnick and Leo Burd

Hugh Herr—Biomechatronics
How technology can be used to enhance human physical capability.

57. Control of Waleed Farahat and Hugh Herr

Muscle-Actuated
Motivated by applications in rehabilitation and robotics, we are developing
Systems via
methodologies to control muscle-actuated systems via electrical stimulation. As a
Electrical Stimulation demonstration of such potential, we are developing centimeter-scale robotic systems
that utilize muscle for actuation and glucose as a primary source of fuel. This is an
interesting control problem because muscles: a) are mechanical state-dependent
actuators; b) exhibit strong nonlinearities; and c) have slow time-varying properties due
to fatigue-recuperation, growth-atrophy, and damage-healing cycles. We are
investigating a variety of adaptive and robust control techniques to enable us to achieve
trajectory tracking, as well as mechanical power-output control under sustained
oscillatory conditions. To implement and test our algorithms, we developed an
experimental capability that allows us to characterize and control muscle in real time,
while imposing a wide variety of dynamical boundary conditions.

Page 8 May 2010 MIT Media Lab

58. Human Walking Hugh Herr and Jared Markowitz
Models Applied to
Models of human walking can be used to improve the control of prostheses. Here we
Prosthesis Control
use motion capture, force plate, and EMG data to elucidate the behavior of the main leg
muscles throughout the gait cycle. To correctly determine muscle function, one must
NEW LISTING also understand the adjoining tendons, as they provide a series elasticity to each
muscle actuator. An optimization is implemented to determine the set of tendon
morphological parameters that minimizes metabolic energy consumption in walking.
Using these parameters, the activation, force, length, and velocity profiles of each
muscle may be determined and used to postulate the reflex arcs that control the gait
cycle. This postulated reflex architecture may then be used to provide feedback terms
in the controllers of robotic prostheses. We will test the performance of this data-based
reflex framework in coordinated control of knee and ankle prostheses, hoping to
observe adaptability across varying speeds and terrains.

59. Human Walking Hugh Herr, Ken Endo and Jared Markowitz
Neuromechanical
This research aims to extract a potentially small set of underlying principles that govern
Models
human movement and to apply that set of principles to biomimetic control systems.
Using a morphologically realistic human model and kinematic gait data, we find that
spin angular momentum in human walking is highly regulated, and that there exists a
nonlinear coupling between center of mass transverse forces, center of mass position,
and center of pressure location. Using an open loop optimization strategy, we show that
biologically realistic leg joint kinematics emerge through the minimization of spin
angular momentum and the sum of the joint torques squared. This suggests that both
angular momentum and energetic factors are important considerations for biomimetic
controllers.

60. Powered Ankle-Foot Samuel Au and Hugh Herr

Prosthesis
The human ankle provides a significant amount of net positive work during the stance
period of walking, especially at moderate to fast walking speeds. Conversely,
conventional ankle-foot prostheses are completely passive during stance, and
consequently, cannot provide net positive work. Clinical studies indicate that transtibial
amputees using conventional prostheses experience many problems during
locomotion, including a high gait metabolism, a low gait speed, and gait asymmetry.
Researchers believe the main cause for the observed locomotion is due to the inability
of conventional prostheses to provide net positive work during stance. The objective of
this project is to develop a powered ankle-foot prosthesis that is capable of providing
net positive work during the stance period of walking. To this end, we are investigating
the mechanical design and control system architectures for the prosthesis. We also
conduct a clinical evaluation of the proposed prosthesis on different amputee
participants.

Additional Biomechatronics projects

For full descriptions, visit: http://www.media.mit.edu/research/1106

61. Artificial Gastrocnemius

Hugh Herr and Ken Endo

62. Biomimetic Active Prosthesis for Above-Knee Amputees

Ernesto C. Martinez-Villalpando and Hugh Herr

63. Human Walking Model Predicts Joint Mechanics,

Electromyography, and Mechanical Economy
Hugh Herr and Ken Endo

MIT Media Lab May 2010 Page 9

 Henry Holtzman—Information Ecology
How to create seamless and pervasive connections between our physical environments
and information resources.

64. BiDi Screen Henry Holtzman, Matt Hirsch and Ramesh Raskar

The BiDi Screen is an example of a new type of thin I/O device that possesses the
CENTER FOR FUTURE
STORYTELLING ability both to capture images and display them. Scene depth can be derived from BiDi
Screen imagery, allowing for 3D gestural and 2D multi-touch interfaces. This
bidirectional screen extends the latest trend in LCD devices, which has seen the
incorporation of photo-diodes into every display pixel. Using a novel optical masking
technique developed at the Media Lab, the BiDi Screen can capture lightfield-like
quantities, unlocking a wide array of applications from 3D gesture and touch interaction
with CE devices, to seamless video communication.

65. Home Fabratory Henry Holtzman and David Carr

Using your personal fabratory, explore a world in which 3D printers cost as little as
NEW LISTING
today's inkjets and are found in every home. We've developed several sub-$100
machines that demonstrate the practicality of this future, and greatly expand the range
of items that can be created on your desktop. These new capabilities have far-reaching
implications for personalization of products, direct-to-consumer production, and the
creation of "information objects."

66. Kairoscope: Social ReeD Martin and Henry Holtzman

Time
If everyone says time is relative, why is it still so rigidly defined? There have been many
attempts to address the issue of coordinating schedules, but each of these attempts
runs into an issue of rigidity: in order to negotiate an event, a specific time must be
designated in advance. This model is inherently poor at accommodating life's
unpredictability. Kairoscope looks at time from a human perspective: allowing people to
coordinate events socially and on the fly, without worrying about precision. This project
evaluates the potential implications of a shared, malleable schedule, as well as the data
inputs and user interactions necessary to create such a system.

67. Konbit Greg Elliott, Aaron Zinman, Henry Holtzman and Pattie Maes

Konbit is a service that helps communities rebuild themselves after a crisis by indexing
NEW LISTING
the skillsets of local residents, allowing NGOs to find and employ them. Haitians, their
diaspora, and the international community can volunteer their skills via phone, SMS, or
Web. Skills can then be searched in real time and location by NGOs such as the
American Red Cross and Partners-in-Health. Konbit is language and medium neutral,
where Kreyol voice and text messages may be translated into other languages through
the Konbit phone, text, or Web interface.

68. NeXtream: Social Henry Holtzman, ReeD Martin and Mike Shafran
Television
Functionally, television content delivery has remained largely unchanged since the
introduction of television networks. NeXtream explores an experience where the role of
CENTER FOR FUTURE
the corporate network is replaced by a social network. User interests, communities, and
STORYTELLING
peers are leveraged to determine television content, combining sequences of short
videos to create a set of channels customized to each user. This project creates an
interface to explore television socially, connecting a user with a community through
content, with varying levels of interactivity: from passively consuming a series, to
actively crafting one's own television and social experience.

Alumni Contributor: Ana Luisa Santos

Page 10 May 2010 MIT Media Lab

69. Tableau Henry Holtzman and John Kestner

Remember when we made a connection by handing someone a photo? Now we fiddle

NEW LISTING
with too many cables, menus, and communication channels, and those individual
connections get drowned out. Can we return to physical experiences while retaining the
collective intelligence of the network? Tableau is a nightstand that stores and retrieves
memories. It may put friends' photo postcards in the drawer, or post mementos to your
online scrapbook. This is an example of task-centric computing, where the interface is
distributed across connected physical objects. Apps that run in the cloud can weave
available objects into environmental I/O, giving users computing experiences that fit into
the flow of life.

70. Takeover TV Henry Holtzman and Greg Elliott

Takeover TV heralds a new era of bar patronage where you and your like-minded
NEW LISTING
friends are in charge of the screens. When you check in at a location, your likes and
dislikes automatically influence what is being shown on local displays. If you want more
control, start a vote to pick a new show using your beer glass—or your iPhone. Create
season-premiere nights for your favorite shows, or work with friends to define the types
of shows that play at your local bars. Sick of watching sports? Assemble enough fans of
your favorite show at the local pub and take over the TV.

71. The Glass Henry Holtzman, Andrew Lippman, David Small, Greg Elliott, Jon Ferguson, Boris
Infrastructure Kizelshteyn, Polychronis Ypodimatopoulos, Chaki Ng and Rick Borovoy

This project builds an open, social information window into the Media Lab using 30
NEW LISTING
touch-sensitive screens strategically placed throughout both buildings in the complex.
The experience of using them is optimized for guests and visitors to collaboratively
explore and uncover the people, ideas, and connections behind the research of the
Lab. The system also makes suggestions about who to meet, where they may be, and
what project and people information—represented as "charms"—one ought to collect,
trade, and share. This is a model for an open IT system that can be used anywhere; it
is a framework for developing open-area and personally responsive access methods.

72. Twitter Weather Henry Holtzman, John Kestner and Stephanie Bian

The vast amounts of user-generated content on the Web produce information overload
as frequently as they provide enlightenment. Twitter Weather reduces large quantities
of text into meaningful data by gauging its emotional content. This Website visualizes
the prevailing mood about top Twitter topics by rendering a weather-report-style
display. Comment Weather is its counterpart for article comments, allowing you to
gauge sentiment without leaving the page. Supporting Twitter Weather is a user-trained
Web service that aggregates and visualizes attitudes on a topic.

Additional Information Ecology projects

For full descriptions, visit: http://www.media.mit.edu/research/919

73. Audiograph: Superhero Hearing

ReeD Martin and Henry Holtzman

74. Constant Crit

Henry Holtzman and Greg Elliott

75. Daydar: Framework for Socially Motivated Goal Fulfillment

Henry Holtzman, John Kestner and Richard The

76. Glasses-Free HD3DTV

Ramesh Raskar, Henry Holtzman, Douglas Lanman, Matt Hirsch and
Yunhee Kim

77. InfoSmell: Smell Your Data

ReeD Martin and Henry Holtzman

MIT Media Lab May 2010 Page 11

 78. Proverbial Wallets
Henry Holtzman, John Kestner, Daniel Leithinger, Danny Bankman, Emily
Tow and Jaekyung Jung

79. Salvage
Henry Holtzman and Greg Elliott

80. Shake4Action: Gestural Mobile Coordination

ReeD Martin and Henry Holtzman

81. Social Garden

Henry Holtzman and John Kestner

82. Window Wallet

ReeD Martin and Henry Holtzman

Hiroshi Ishii—Tangible Media

How to design seamless interfaces between humans, digital information, and the physical
environment.

83. Beyond: A Jinha Lee and Hiroshi Ishii

Collapsible Input
Beyond is a collapsible input device for direct 3D manipulation. When pressed against a
Device for 3D Direct
screen, Beyond collapses in the physical world and extends into the digital space of the
Manipulation screen, so that users have an illusion that they are inserting the tool into the virtual
space. Beyond allows users to interact directly with 3D media without having to wear
NEW LISTING special glasses, avoiding inconsistencies of input and output. Users can select, draw,
and sculpt in 3D virtual space, and seamlessly transition between 2D and 3D
manipulation.

84. MirrorFugue Xiao Xiao and Hiroshi Ishii

While modern technologies such as CDs, MP3s, and digital media players make
listening to music a portable activity, vital aspects of music such as learning,
rehearsing, and performing are still constrained by location. MirrorFugue is an interface
for the piano that bridges the gap of location in music playing by connecting pianists in
a virtual shared space reflected on the piano.

85. OnObject Keywon Chung, Michael Shilman, Chris Merrill and Hiroshi Ishii

OnObject transforms any object or surface into a gesture-triggered interactive toy. By

NEW LISTING
applying an RFID tag to an object, grabbing it wearing a sensing device, and making
hand gestures like shake, swing and tilt, you can trigger interactive audio output in
stuffed animals, a mug, your desk, or your body.

86. PingPongPlusPlus Hiroshi Ishii, Xiao Xiao, Michael Bernstein, Sean Follmer and Daniel Leithinger

PingPongPlusPlus builds on the PingPongPlus project (1998), a ping pong table that
NEW LISTING
sensed ball hit locations and displayed projected visualizations based on the hits. We
aim to publish and proliferate the design of our table. We are looking to set up a system
of distributed ping pong tables that records the history of hits in a centralized database.
From these records, we will explore ways to remix past games and present games to
teach, inform, and entertain.

87. Relief Hiroshi Ishii and Daniel Leithinger

Relief is an actuated tabletop display, able to render and animate 3D shapes with a
malleable surface. It allows users to experience and form digital models such as
geographical terrain in an intuitive manner. The tabletop surface is actuated by an array
of 120 motorized pins, which are controlled with a platform built upon open-source

Page 12 May 2010 MIT Media Lab

 hardware and software tools. Each pin can be addressed individually and senses user
input like pulling and pushing.

Alumni Contributor: Adam Kumpf

88. Sourcemap Leonardo Bonanni, Matthew Hockenberry, David Zwarg and Hiroshi Ishii

Sourcemap is a social network built around supply chains, enabling collective

engagement with where things come from and what they are made of. An open-source
project, Sourcemap provides resources for calculating the carbon footprint and
geographic spread of various products and services, including consumer electronics,
travel, and food. We are deploying Sourcemap through in-depth case studies with
designers and business owners in product design, hospitality, and food and drink. The
Sourcemap team is actively seeking collaborators and pilot study participants to
develop the tool for general use.

89. Video Play Sean Follmer, Hayes Raffle (Nokia Research Center) and Hiroshi Ishii

Long-distance families are increasingly staying connected with free video conferencing
NEW LISTING
tools. However, the tools themselves are not designed to accommodate children's or
families' needs. We explore how play can be a means for communication at a distance.
Our Video Play prototypes are simple video-conferencing applications built with play in
mind, creating opportunities for silliness and open-ended play between adults and
young children. They include simple games, such as Find It, but also shared activities
like book reading, where users' videos are displayed as characters in a story book.

Additional Tangible Media projects

For full descriptions, visit: http://www.media.mit.edu/research/39

90. AFK Cookset

Cati Vaucelle and Hiroshi Ishii

91. Cost-Effective Wearable Sensor to Detect EMF

Cati Vaucelle, Hiroshi Ishii and Joe Paradiso

CENTER FOR FUTURE 92. g-stalt

STORYTELLING Jamie Zigelbaum, Alan Lewis Browning, Daniel Leithinger, Olivier Bau,
Jinha Lee, Ryan Jackson and Hiroshi Ishii

93. Gestural Interaction

Jamie Zigelbaum, Daniel Leithinger, Jean-Baptiste Labrune, Alan
Browning, Olivier Bau and Hiroshi Ishii

94. IdeaGarden
Hiroshi Ishii and Jean-Baptiste Labrune

CENTER FOR FUTURE 95. O-Link

STORYTELLING Toshihiro Nakae and Hiroshi Ishii

CENTER FOR FUTURE 96. Picture This!

STORYTELLING Cati Vaucelle and Hiroshi Ishii

97. Piezing
Amanda Parkes, Adam Kumpf and Hiroshi Ishii

98. Psychohaptics
Cati Vaucelle, Leonardo Bonanni and Hiroshi Ishii

99. Radical Atoms

Hiroshi Ishii, Leonardo Bonanni, Keywon Chung, Sean Follmer,
Jean-Baptiste Labrune, Jinha Lee, Daniel Leithinger, Catherine Vaucelle,
Xiao Xiao and Jamie Zigelbaum

100. SandScape
Carlo Ratti, Assaf Biderman and Hiroshi Ishii

MIT Media Lab May 2010 Page 13

 101. Sensetable
Hiroshi Ishii

102. Stress OutSourced

Keywon Chung, Carnaven Chiu, Xiao Xiao, Peggy Pei-Yu Chi and Hiroshi
Ishii

103. Tangible Bits

Hiroshi Ishii, Keywon Chung, Sean Follmer, Jean-Baptiste Labrune, Jinha
Lee, Daniel Leithinger, Cati Vaucelle, Xiao Xiao and Jamie Zigelbaum

104. Topobo
Hayes Raffle, Amanda Parkes and Hiroshi Ishii

105. Trackmate
Adam Kumpf, Jean-Baptiste Labrune, Keywon Chung, Daniel Leithinger,
Jamie Zigelbaum and Hiroshi Ishii

106. Wetpaint
Leonardo Bonanni, Xiao Xiao, Bianca Costanzo, Andrew Shum, Matthew
Hockenberry, Maurizio Seracini and Hiroshi Ishii

Joseph M. Jacobson—Molecular Machines

How to engineer at the limits of complexity with molecular-scale parts.

107. GeneFab Bram Sterling, Kelly Chang, Joseph M. Jacobson, Peter Carr, Brian Chow, David
Sun Kong, Michael Oh and Sam Hwang

What would you like to "build with biology"? The goal of the GeneFab projects is to
develop technology for the rapid fabrication of large DNA molecules, with composition
specified directly by the user. Our intent is to facilitate the field of genetic engineering
as it moves from a focus on single genes to designing complete biochemical pathways,
genetic networks, and more complex systems. Sub-projects include: DNA error
correction, microfluidics for high throughput gene synthesis, and genome-scale
engineering (rE. coli).

Alumni Contributor: Chris Emig

108. NanoFab Jaebum Joo and Joseph M. Jacobson

We are developing techniques to fabricate nanostructures and logic devices using

nanoparticulate colloids of metals and semi-conductors as building blocks.

Henry Lieberman—Software Agents

How software can act as an assistant to the user rather than a tool, by learning from
interaction and by proactively anticipating the user's needs.

109. Human Goal Network Henry Lieberman and Dustin Smith

We examined a large corpus of people's goals and their descriptions of accomplishing

CENTER FOR FUTURE
STORYTELLING them. We asked the questions: which goals are most popular, most controversial,
easiest to accomplish? From this corpus we inferred a goal network that can be used
for plan recognition, story understanding, and building software agents that are aware
of their users' goals.

Page 14 May 2010 MIT Media Lab

110. Luminoso: Catherine Havasi and Robert H. Speer
Understanding and
Luminoso is a tool that uses common sense and blending to "read between the lines"
Visualizing People's
and better understand opinions and feedback expressed in free text such as customer
Opinions reviews. It creates a semantic space from the ideas in a set of documents, including
common-sense background information, and allows interactive exploration. This
interface can be used to discover trends in a text corpus, such as free-text responses to
a survey.

111. Not-So-Common Henry Lieberman, Catherine Havasi, Jayant Krishnamurthy and Robert H. Speer
Sense
We present a way of infusing any dataset with common sense. When domain-specific
data is combined with the general knowledge in ConceptNet, new ways of organizing,
visualizing, and reasoning over the data emerge. In domains such as consumer lifestyle
modeling, knowledge acquisition from free text, and personal financial management,
most information based on natural language can benefit from a little added common
sense.

112. Raconteur: From Henry Lieberman and Pei-Yu Chi

Chat to Stories
Raconteur is a story-editing system for conversational storytelling that provides
intelligent assistance in illustrating a story with photos and videos from an annotated
CENTER FOR FUTURE
media library. It performs natural language processing on a text chat between two or
STORYTELLING
more participants, and recommends appropriate items from a personal media library to
illustrate a story. A large common-sense knowledge base and a novel common-sense
inference technique are used to find relevant media materials to match the story intent
in a way that goes beyond keyword matching or word co-occurrence based techniques.
Common-sense inference can identify larger-scale story patterns such as expectation
violation or conflict and resolution, and helps a storyteller to chat and brainstorm his
personal stories with a friend.

113. Reusing Code by Henry Lieberman and Kenneth C. Arnold

Reasoning about Its
When programmers face unfamiliar or challenging tasks, code written by others could
Purpose
give them inspiration or reusable pieces. But how can they find appropriate code? We
believe the key to meaningful code search is to relate code (written in a programming
language) to the purpose for which the code was written (expressed in natural
language). Our interface enables searching for, reusing, and sharing code fragments
within the development environment by connecting code with a brief English description
of what it is for. To understand users' descriptions of the purpose of the code they have
written or want to find, we reason jointly over code structure features, code linguistic
features (e.g., variable and event names), and both domain-specific and
common-sense knowledge. We show how the system suggests possible purposes for
unannotated code and finds potentially applicable code given brief English purpose
statements.

114. Semantic Catherine Havasi, Robert Speer and Jason Alonso

Synesthesia
Semantic Synesthesia is a program that guesses a color to represent a given input
word or sentence, taking into account both physical descriptions of objects and
CENTER FOR FUTURE
emotional connotations. This novel application of artificial intelligence uses knowledge
STORYTELLING
NEW LISTING about the world to build a model of how people think about objects, emotions, and
colors, and uses this model to guess an appropriate color for a word. Colorizer works
over static text and real-time input, such as a speech recognition stream. It has
applications in games, arts, and Web page design.

Additional Software Agents projects

For full descriptions, visit: http://www.media.mit.edu/research/23

115. Agent-Application Communication

Henry Lieberman

MIT Media Lab May 2010 Page 15

 116. Agents for Integrated Annotation and Retrieval of Images
Henry Lieberman

117. Blending and AnalogySpace

Catherine Havasi, Robert Speer and Henry Lieberman

118. Collecting Common Sense

Henry Lieberman, Marvin Minsky, Jason Alonso, Kenneth Arnold,
Catherine Havasi, Jayant Krishnamurthy, Dustin Smith, Robert Speer and
Luis von Ahn

119. Common-Sense Investing

Henry Lieberman

120. Common-Sense Reasoning for Interactive Applications

Henry Lieberman

121. Common-Sense Recommendations

Henry Lieberman and Jayant Krishnamurthy

122. CommonConsensus: A Game for Collecting Commonsense

Goals
Henry Lieberman and Dustin Smith

123. ConceptNet
Catherine Havasi, Robert Speer, Kenneth Arnold, Henry Lieberman and
Marvin Minsky

124. Divisi: Reasoning Over Semantic Relationships

Henry Lieberman, Jason Alonso, Kenneth Arnold, Catherine Havasi and
Robert Speer

125. E-Commerce When Things Go Wrong

Henry Lieberman

126. Emotus Ponens: Affective Story Understanding for Agents

Henry Lieberman

127. Goal-Oriented Interfaces for Consumer Electronics

Henry Lieberman and Pei-Yu Chi

128. Graphical Interfaces for Software Visualization and

Debugging
Henry Lieberman

129. Intelligent Technical Documentation

Henry Lieberman

CENTER FOR FUTURE 130. Learning Common Sense in a Second Language

STORYTELLING Henry Lieberman, Ned Burns and Li Bian

131. MARCO: Mutual Disambiguation of Recognition Errors in a

Multimodal Navigational Agent
Henry Lieberman

132. Moral Compass: A Model of Self-Conscious Learning

Ed Boyden, Henry Lieberman, Marvin Minsky, Joe Paradiso and Bo
Morgan

133. Multi-Lingual ConceptNet

Hyemin Chung, Jaewoo Chung, Wonsik Kim, Sung Hyon Myaeng and
Walter Bender

134. Multilingual Common Sense

Aparecido Fabiano Pinatti de Carvalho, Jesus Savage Carmona, Marie
Tsutsumi, Junia Anacleto, Henry Lieberman, Jason Alonso, Kenneth
Arnold, Robert Speer, Vania Paula de Almeida and Veronica Arreola Rios

Page 16 May 2010 MIT Media Lab

 CENTER FOR FUTURE 135. Navigating in Very Large Display Spaces
STORYTELLING Henry Lieberman

136. Open Mind Commons

Henry Lieberman, Marvin Minsky, Jason Alonso, Kenneth Arnold, Robert
Speer, Catherine Havasi, James Pustejovsky and Junia Anacleto

137. Programming in Natural Language

Henry Lieberman and Moin Ahmad

138. Red Fish, Blue Fish

Robert Speer and Catherine Havasi

139. Relational Analogies in Semantic Networks

Henry Lieberman and Jayant Krishnamurthy

CENTER FOR FUTURE 140. Storied Navigation

STORYTELLING Henry Lieberman

141. TwitterMap
Catherine Havasi and Robert Speer

Andrew Lippman and David P. Reed—Viral Communications

How to construct agile, scalable, mobile ecosystems.

142. Barter: A Dawei Shen, Marshall Van Alstyne, Andrew Lippman and David P. Reed
Market-Incented
We are exploring the creation of Internal Knowledge Markets, with a goal of bringing an
Wisdom Exchange
electronic, distributed market (decentralized information creation) to an organization in
an effort both to measure the value of knowledge created and to link this to productivity
and/or profits. A key advantage of this study will be a direct appeal to information
economic theory to design an information market. In particular, we will appeal to
two-sided network theory, principles of information asymmetry, and price theory. We
seek to bring the rigor of this discipline to real-world applications, and measure the
results.

143. Location Sharing in Andrew Lippman, David P. Reed and Polychronis Ypodimatopoulos
Large Indoor
We leverage the ubiquity of Bluetooth-enabled devices and propose a decentralized,
Environments
Web-based architecture that allows users to share their location by following each other
in the style of Twitter. We demonstrate a prototype that operates in a large building and
NEW LISTING which generates a dataset of detected Bluetooth devices at a rate of ~30 new devices
per day, including the respective location where they were last detected. Users then
query the dataset using their unique Bluetooth ID and share their current locations with
their followers by means of unique Universal Resource Identifiers (URIs) that they
control. Our separation between producers (the building) and consumers (the users) of
Bluetooth device location data allows us to create socially aware applications that
respect user privacy while limiting the software necessary to run on mobile devices to
just a Web browser.

144. Meal Time Kwan Hong Lee, Yod Phumpong, Michael Plasmeier and Andrew Lippman

We are exploring the impact of mobile phones on the future of credit card usage and
NEW LISTING
payments. We are attempting to reinvent the payment process through mobile devices
by providing users with tools to help reflect on their finances and eating habits. We
have deployed the Meal Time system on the MIT campus, interlinking it with MIT's
TechCASH payment system. Students have been actively using it to share their
experiences and choose locations to eat.

MIT Media Lab May 2010 Page 17

145. Pixel Infrastructures Andrew Lippman and Grace Rusi Woo

We are building a wireless network that is faster, less expensive to implement, and has
location and orientation designed in. It is based on optical technologies that range from
LED stoplights to HDTV screens. We show this using an array of Samsung electronic
picture frames as transmitters and a simple Webcam as the receiver.

146. The Glass Henry Holtzman, Andrew Lippman, David Small, Greg Elliott, Jon Ferguson, Boris
Infrastructure Kizelshteyn, Polychronis Ypodimatopoulos, Chaki Ng and Rick Borovoy

This project builds an open, social information window into the Media Lab using 30
NEW LISTING
touch-sensitive screens strategically placed throughout both buildings in the complex.
The experience of using them is optimized for guests and visitors to collaboratively
explore and uncover the people, ideas, and connections behind the research of the
Lab. The system also makes suggestions about who to meet, where they may be, and
what project and people information—represented as "charms"—one ought to collect,
trade, and share. This is a model for an open IT system that can be used anywhere; it
is a framework for developing open-area and personally responsive access methods.

Additional Viral Communications projects

For full descriptions, visit: http://www.media.mit.edu/research/1045

147. Ego
Andrew Lippman, David P. Reed and Polychronis Ypodimatopoulos

148. Open Spaces

Viral Communications group

149. Social Menu

Kwan Hong Lee, Yod Phumpong, Michael Plasmeier and Andrew Lippman

150. Social Saver

Andrew Lippman, Kwan Lee, Michael Plasmeier and Phumpong
Watanaprakornkul

Tod Machover—Opera of the Future

How musical composition, performance, and instrumentation can lead to innovative forms
of expression, learning, and health.

151. Death and the Tod Machover, Andy Cavatorta, Wei Dong, Noah Feehan, Elena Jessop, Bob
Powers: Redefining Hsiung and Peter Torpey
Opera
"Death and the Powers" is a groundbreaking opera that brings a variety of
technological, conceptual, and aesthetic innovations to the theatrical world. Created by
CENTER FOR FUTURE Tod Machover (composer), Diane Paulus (director), and Alex McDowell (production
STORYTELLING
designer), the opera uses the techniques of tomorrow to address age-old human
concerns of life and legacy. The unique performance environment, including
autonomous robots, expressive scenery, new Hyperinstruments, and human actors, will
blur the line between animate and inanimate. The opera will open in Monte-Carlo in fall
2010 and then tour worldwide.

152. Disembodied Tod Machover, Peter Torpey and Elena Jessop

Performance
Early in the opera "Death and the Powers," main character Simon Powers is subsumed
into a technological environment of his own creation. The set comes alive through
robotic, visual, and sonic elements that allow the actor to extend his range and
influence across the stage in unique and dynamic ways. This environment must
assume the behavior and expression of the absent Simon; to distill the essence of this
character, we recover performance parameters in real time from physiological sensors,
voice, and vision systems. These gesture and performance parameters are then

Page 18 May 2010 MIT Media Lab

 mapped to a visual language that allows the off-stage actor to express emotion and
interact with others on stage. Additionally, we use these gestural performance
parameters for vocal manipulation. Our approach takes a new direction in augmented
performance, employing a non-representational abstraction of a human presence that
fully translates a character into an environment.

153. Expressive Hand Adam Boulanger

Rehab
We are developing the first multimodal interface for hand rehabilitation following stroke.
EMG forearm sensors read attempted finger presses in disordered limbs, and serve as
an input to an expressive feedback interface. Auditory, visual, and tactile cues are
presented to support rehabilitation of the representation of finger movements across
sensory domains. The multisensory feedback is embedded in a rich task, situated
between piano learning and expressive music performance. A user of this system will
rehabilitate finger movement while developing an expressive music performance.
Imagine a complete shift in the form and function of rehabilitation, towards something
empowering, where individuals strive in tandem with tailored interfaces, mapped to
push them forward at each step, and as part of fundamentally enriching expressive
tasks. Our rehabilitative health care environments can sculpt our minds, while changing
our lives, if we invent the right tools.

154. Gestural Media Tod Machover and Elena Jessop

Framework
Many performance artists and interaction designers use human gestures to drive,
manipulate, or generate digital media. However, the existing systems for developing
mappings between incoming data streams and output media have extremely low-level
concepts of “gesture,” forcing the user to focus on the particulars of input sensor or
video data, rather than on meaningful and expressive gestures. We are developing a
new framework for gestural control of media in performance, allowing users to easily
create clear, intuitive, and comprehensible mappings by using high-level descriptions of
gestures and of gestural qualities. This system currently is realized in a set of tools for
gestural media manipulation in performance and rehearsal, mapping gestural
vocabularies and qualities of movement to parameters of interactive visual applications.

155. Mobile Music Alzheimer's Association, Tod Machover, Adam Boulanger, Intel and McLean
Diagnostics: Geriatric Hospital
Targeting Alzheimer's
The scientific community is making marked progress in the area of Alzheimer's disease
Disease (AD) treatment: memory-related pharmaceuticals are available, the neurobiology of AD
is fairly well understood, and the genetic underpinnings of the disease continue to be
unraveled. However, despite these advances, it has been shown that individuals often
present the symptoms of AD years before they seek a diagnosis. The barrier to
treatment is the lack of structure with which to obtain a diagnosis or even predict the
onset of disease in a stigmatized environment. With technology, we can build clinically
valid assessment into the tools we use every day—the tools we care about. We are
developing music tools to detect cognitive performance in the memory domains at risk
of decline in the earliest stages of AD. These tools are mobile, longitudinal, and the
patient is the first point of feedback.

156. Musical Furniture Wei Dong and Paula Marie Countouris

Stage performances present many challenges and opportunities in the field of robotics.
CENTER FOR FUTURE
STORYTELLING Here we create a family of furniture robots that look and act like organic entities for the
production of Death and the Powers. Mei-Mei is a six-legged walking robot that is being
developed in the lab as a moving workbench. It can move forward, backward, and even
turn around with differential steering control. Di-Di is a modular robot that can transform
itself into a sofa onstage and walk with a rolling, lurching, and gliding motion. These two
robots will also be endowed with interactive behavior structured around scenes, beats,
and actions. The design of these robots not only incorporates technological,
conceptual, and aesthetic innovations, but also coordinates with narrative and musical
materials in the opera.

MIT Media Lab May 2010 Page 19

157. Syncwalk Tod Machover and Noah Feehan

Syncwalk is a platform for sound design in a geographic space: you place songs from
NEW LISTING
your music library onto a map, and then you go out into the world and, as you walk from
one place to another, you hear the songs you’ve placed along the way. The project
explores different ways to design and share experiences centered around, but not
wholly determined by, the spaces we transit and inhabit.

158. The Chandelier Tod Machover, Andy Cavatorta, Wei Dong, Paula Marie Countouris, Karen Hart
and Calvin Chung
CENTER FOR FUTURE
STORYTELLING The Chandelier is a large-scale robotic musical instrument that is being developed for
"Death and the Powers." Its 48 strings can be actuated both through powerful
electromagnets, and tactilely (plucked like a harp or bowed like a cello). With the strings
driven by electromagnets, the tactile player can also repeatedly damp strings or create
overtones by carefully touching the strings' anti-nodes, creating a new intimacy
between players, who play not just the same instrument, but the same strings. The
Chandelier is composed of many systems—logic for control of music and lighting,
networked servers, and playable interfaces—all built around an elegant, articulated
skeletal structure which allows changes to the length, angle, and tensions of the strings.
We are currently experimenting with playing it through new types of interfaces to take
advantage of its unusual tuning and sonorities.

Additional Opera of the Future projects

For full descriptions, visit: http://www.media.mit.edu/research/35

159. Hyperinstruments
Tod Machover

160. Hyperscore
Tod Machover

161. Music, Mind, and Health

Tod Machover and Adam Boulanger

CENTER FOR FUTURE 162. Musical Robotics

STORYTELLING Tod Machover, Wei Dong, Noah Feehan, Bob Hsiung, Jason Ku and Mike
Miller

163. Personal Opera

Tod Machover and Peter Torpey

164. Rumble
Tod Machover and Noah Feehan

CENTER FOR FUTURE 165. Skellig: A "Surround" Opera

STORYTELLING Tod Machover, Ben Bloomberg and Simone Ovsey

166. Toy Symphony

Tod Machover

167. Vocal Augmentation and Manipulation Prosthesis (VAMP)

Tod Machover and Elena Jessop

Page 20 May 2010 MIT Media Lab

 Pattie Maes—Fluid Interfaces
How to integrate the world of information and services more naturally into our daily
physical lives, enabling insight, inspiration, and interpersonal connections.

168. Cornucopia: Digital Marcelo Coelho, Amit Zoran, Pattie Maes and William J. Mitchell
Gastronomy
Cornucopia is a concept design for a personal food factory, bringing the versatility of
the digital world to the realm of cooking. In essence, it is a 3D printer for food that works
by storing, precisely mixing, depositing, and cooking layers of ingredients. Cornucopia's
cooking process starts with an array of food canisters that refrigerate and store a user's
favorite ingredients. These are piped into a mixer and extruder head that can accurately
deposit elaborate combinations of food; while this takes place, the food is heated or
cooled. This fabrication process not only allows for the creation of flavors and textures
that would be completely unimaginable through other cooking techniques, but it also
allows the user to have ultimate control over the origin, quality, nutritional value, and
taste of every meal.

169. Defuse Aaron Zinman, Judith Donath and Pattie Maes

Defuse is a commenting platform that rethinks the medium's basic interactions. In a

world where a single article in The New York Times can achieve 3,000 comments, the
original design of public asynchronous text systems has reached its limit; it needs more
than social convention. Defuse uses context to change the basics of navigation and
message posting. It uses a combination of machine learning, visualization, and
structural changes to achieve this goal.

170. Face the Facts Richard The, Doug Fritz, Pattie Maes and David Small

Face the Facts is an experiment in embodied data visualization. The goal is to foster
NEW LISTING
understanding of abstract information that is spatially or temporarily detached from us.
We hope to achieve a more personalized, evocative perception of information that
would be hard to to grasp otherwise. We are creating an art installation in the 3rd floor
atrium of the new Media Lab building that will augment the people wandering through it
with statistical information. As an example, we are using data produced by the Media
Lab, extracted from the student and faculty publications over the course of the Media
Lab's rich history.

171. InterPlay: Full-Body Pattie Maes, Seth Hunter and Pol Pla i Conesa
Interaction Platform
InterPlay is a platform for designers to create dynamic social simulations, which
transform public spaces into immersive environments where people become the central
NEW LISTING
agents. It uses computer vision and projection to facilitate full-body interaction with
digital content. The physical world is augmented to create shared experiences that
encourage active play, negotiation, and creative composition.

172. LuminAR Natan Linder and Pattie Maes

LuminAR reinvents the traditional incandescent bulb and desk lamp, evolving them into
NEW LISTING
a new category of robotic, digital information devices. The LuminAR Bulb combines a
Pico-projector, camera, and wireless computer in a compact form factor. This
self-contained system enables users with just-in-time projected information and a
gestural user interface, and it can be screwed into standard light fixtures everywhere.
The LuminAR Lamp is an articulated robotic arm, designed to interface with the
LuminAR Bulb. Both LuminAR form factors dynamically augment their environments
with media and information, while seamlessly connecting with laptops, mobile phones,
and other electronic devices. LuminAR transforms surfaces and objects into interactive
spaces that blend digital media and information with the physical space. The project
radically rethinks the design of traditional lighting objects, and explores how we can
endow them with novel augmented-reality interfaces.

MIT Media Lab May 2010 Page 21

173. ReflectOns: Mental Pattie Maes and Sajid Sadi
Prostheses for
ReflectOns are objects that help people think about their actions and change their
Self-Reflection
behavior based on subtle, ambient nudges delivered at the moment of action. Certain
tasks—such as figuring out the number of calories consumed, or amount of money
spent eating out—are generally difficult for the human mind to grapple with. By using
in-place sensing combined with gentle feedback and understanding of users' goals, we
can recognize behaviors and trends, and provide a reflection of their own actions
tailored to enable both better understanding of the repercussions of those actions, and
changes to their behaviors to help them better match their own goals.

174. SixthSense Pranav Mistry

Information is often confined to paper or computer screens. SixthSense frees data from
CENTER FOR FUTURE
STORYTELLING these confines and seamlessly integrates information and reality. With the
miniaturization of computing devices, we are always connected to the digital world, but
there is no link between our interactions with these digital devices and our interactions
with the physical world. SixthSense bridges this gap by augmenting the physical world
with digital information, bringing intangible information into the tangible world. Using a
projector and camera worn as a pendant around the neck, SixthSense sees what you
see and visually augments surfaces or objects with which you interact. It projects
information onto any surface or object, and allows users to interact with the information
through natural hand gestures, arm movements, or with the object itself. SixthSense
makes the entire world your computer.

Additional Fluid Interfaces projects

For full descriptions, visit: http://www.media.mit.edu/research/955

175. Aeropticon
Natan Linder

176. Augmented Mirror

Pattie Maes and Sajid Sadi

177. BitBrush
Pattie Maes and Marcelo Coelho

178. Blossom
Pattie Maes and Sajid Sadi

179. Compact Contract: Commitments Made Easier

Pattie Maes, Marcelo Coelho and Sajid Sadi

180. Done: Reflective Personal Project Management

Pattie Maes and Sajid Sadi

181. EyeFlect: Low-Cost Multi-Flash Gaze Estimation

Pattie Maes, Doug Fritz and Richard The

182. Flexible Urban Display

Pattie Maes, Pranav Mistry and Sajid Sadi

183. GovLove: Simplifying Government Data Retrieval and

Manipulation
Pattie Maes and Doug Fritz

184. Inktuitive: An Intuitive Physical Design Workspace

Pranav Mistry and Kayato Sekiya

185. ioMaterials
Pattie Maes, Sajid Sadi and Amir Mikhak

186. JotWatch: Instant Personal Note-Taking

Pattie Maes, Doug Fritz, Sajid Sadi, Eben Kunz and Melody Kuna

Page 22 May 2010 MIT Media Lab

 187. Konbit
Greg Elliott, Aaron Zinman, Henry Holtzman and Pattie Maes

188. Life in a Comic

Pattie Maes and Amit Zoran

189. Light Piping Solar Panels

Dale Joachim, Sandy Sener, Seth Hunter and Jean-Baptiste Labrune

CENTER FOR FUTURE 190. MemTable

STORYTELLING Pattie Maes, Seth Hunter, Alexandre Milouchev and Emily Zhao

191. Midas
Pattie Maes and Sajid Sadi

192. Mouseless
Pranav Mistry and Pattie Maes

193. Moving Portraits

Pattie Maes

194. MTM "Little John"

Natan Linder

195. Personas
Judith Donath and Aaron Zinman

196. Pulp-Based Computing: A Framework for Building

Computers Out of Paper
Marcelo Coelho, Pattie Maes, Joanna Berzowska and Lyndl Hall

197. Quickies: Intelligent Sticky Notes

Pranav Mistry and Pattie Maes

198. ReachBand: An RFID Wristband

Pattie Maes, Assaf Feldman and Sajid Sadi

199. Relational Pillow

Pattie Maes, Sajid Sadi and Amir Mikhak

200. Remnant: Handwriting Memory Card

Pattie Maes and Sajid Sadi

201. Shutters: A Permeable Surface for Environmental Control

and Communication
Marcelo Coelho and Pattie Maes

202. Siftables: Physical Interaction with Digital Media

David Merrill and Pattie Maes

203. SoundForms
Pattie Maes, Seth Hunter and Pol Pla i Conesa

204. SpaceMarks: Brain Offloading through Spatial Thinking

Pattie Maes and Doug Fritz

205. SpendTrend: Reflecting on Spending Habits

Pattie Maes and Sajid Sadi

206. Spotlight
Pattie Maes and Sajid Sadi

207. Sprout I/O: A Texturally Rich Interface

Marcelo Coelho and Pattie Maes

208. subTextile: A Construction Kit for Computationally Enabled

Textiles
Pattie Maes and Sajid Sadi

MIT Media Lab May 2010 Page 23

 209. Surflex: A Shape-Changing Surface
Marcelo Coelho and Pattie Maes

210. TaPuMa: Tangible Public Map

Pranav Mistry and Tsuyoshi Kuroki

211. Theme Stream: Visualizing Complex Time-Based Information

Pattie Maes and Doug Fritz

CENTER FOR FUTURE 212. thirdEye

STORYTELLING Pranav Mistry and Pattie Maes

213. Transitive Materials: Towards an Integrated Approach to

Material Technology
Pattie Maes, Marcelo Coelho, Neri Oxman, Sajid Sadi, Amit Zoran and Amir
Mikhak

214. Watt Watcher

Pattie Maes, Sajid Sadi and Eben Kunz

215. What If the World Were Your n Facebook Friends?

Richard The and Doug Fritz

216. What Would They Think?

Pattie Maes

William J. Mitchell—Smart Cities

How buildings and cities can become more intelligently responsive to the needs and
desires of their inhabitants.

217. CityCar William J. Mitchell, Ryan Chin, William Lark, Jr., Raul-David Poblano, Nicholas
Pennycooke and Praveen Subramani
CENTER FOR FUTURE
STORYTELLING The CityCar is a foldable, electric, two-passenger vehicle for crowded cities. It uses
Wheel Robots—fully modular in-wheel electric motors—that integrate drive motors,
suspension, braking, and steering inside the hub-space of the wheel. This drive-by-wire
system requires only data, power, and mechanical connection to the chassis of the
vehicle. Wheel Robots have over 120 degrees of steering freedom, allowing for a
zero-turn radius and 90-degree parking (sideways translation); they also enable the
CityCar to fold by eliminating the gasoline-powered engine and drive-train. Folded, the
CityCar is very compact (roughly 60” or 1500mm), with an on-street parking ratio of at
least 3:1 to traditional cars. It is also lightweight (1000lbs) and modular, and
automatically recharges when parked, reducing battery needs and excess weight. The
CityCar has two use models: private (traditional ownership), and shared (Mobility On
Demand, high-utilization, one-way shared systems like Paris’s Vélib' bicycle-sharing
program).

Alumni Contributors: Patrik Kunzler and Philip Liang

Page 24 May 2010 MIT Media Lab

218. GreenWheel Electric William J. Mitchell, Ryan C.C. Chin, Michael Chia-Liang Lin, Arthur Petron and
Bicycle MIT Mobile Experience Lab

GreenWheel is a modular, in-wheel electric motor that transforms any pedal-powered

CENTER FOR FUTURE
bicycle into an electrically assisted hybrid bicycle—an "e-bike." The patented design
STORYTELLING
integrates electric motor, batteries, and motor controllers inside the hub space without
any wires to the frame, allowing the GreenWheel to be retrofitted to any type of bicycle.
The GreenWheel is controlled by a pedal sensor which detects the amount of force on
the pedals and activates the electric motor. The GreenWheel can be driven
approximately 20 miles (30 kilometers) with motor assist and pedaling on one charge,
and it is capable of rapid charging in less than 25 minutes. The GreenWheel will enable
users to easily overcome inclines and to ride longer distances, thus opening up cycling
to a wider audience.

219. Liberated Pixels William J. Mitchell and Susanne Seitinger

We are experimenting with systems that blur the boundary between urban lighting and
digital displays in public spaces. These systems consist of liberated pixels, which are
not confined to rigid frames as are typical urban screens. Liberated pixels can be
applied to existing horizontal and vertical surfaces in any configuration, and
communicate with each other to enable a different repertoire of lighting and display
patterns. We are currently developing "urban pixels," a wireless infrastructure for
liberated pixels. Composed of autonomous, solar-powered units, the system presents a
programmable and distributed interface that is flexible and easy to deploy. Each unit
includes an on-board battery, solar cells, RF transceiver unit, and microprocessor.

220. Mobility on Demand William J. Mitchell, Ryan C.C. Chin, Chih-Chao Chuang, William Lark, Jr., Michael
Chia-Liang Lin, Raul-David Poblano, Andres Sevtsuk, Nicholas Pennycooke,
Praveen Subramani and Dimitris Papanikolaou
CENTER FOR FUTURE
STORYTELLING
Mobility-on-Demand (MoD) systems consist of a fleet of lightweight electric vehicles
placed at electrical charging stations that are strategically distributed throughout the
city. MoD systems solve the “first and last mile” problem that public transit systems do
not solve—providing mobility from the transit station to and from your home or
workplace. In a MoD system, users simply walk up to the closest station, swipe a
membership card, and are given access to vehicles. They are then allowed to drive to
any other station (one-way rental) closest to their desired destination. The Vélib' system
in Paris, consisting of over 20,000 shared bicycles, is the largest and most popular MoD
system in the world. We have designed and developed three MoD vehicles: the
CityCar, RoboScooter, and GreenWheel Bicycle. The team is also developing a
dynamic pricing structure to help redistribute the fleet.

Alumni Contributor: Philip Liang

221. Mobility on Demand: William J. Mitchell, Dimitris Papanikolaou and Ryan C.C. Chin
A Market Economy of
One-way vehicle sharing systems are decentralized urban mobility networks of vehicles
Trips
and parking stations; users can pick up a vehicle from any station and return it to any
other station. However, due to trip distribution asymmetries, station inventories become
unbalanced quickly, reducing system reliability. Existing solutions involve fleet
redistribution by trucks, which is complex, inefficient, and financially unsustainable.
Mobility on Demand (MoD) is a new, self-organized, one-way vehicle-sharing system
that uses dynamic pricing to incentivize users to redistribute the fleet and keep the
system in balance. Similar to a market, trip price adjusts to inventory needs in origin
and destination stations. We present a framework using system dynamics that explains
MoD system behavior and will be used to determine optimum pricing policy, number of
parking stations, and number of vehicles needed for a stable yet profitable system.

MIT Media Lab May 2010 Page 25

222. New Object Studio William J. Mitchell, Peter Schmitt, Susanne Seitinger and Amit Zoran

New Object Studio challenges traditional design paradigms by approaching old and
CENTER FOR FUTURE
STORYTELLING new design questions with innovative digital tools and fabrication processes. Using this
approach, [N][O] Studio focuses on creating new artifacts. These new products
combine mechanical and electronic components to challenge traditional notions of
manufactured objects through their integrated functional, visual, and narrative qualities.

223. RoboScooter with William J. Mitchell, Ryan C.C. Chin, Michael Chia-Liang Lin, Raul-David Poblano,
SYM and ITRI Andres Sevtsuk, Yaniv Fain, Dimitris Papanikolaou and Arthur Petron

The RoboScooter is an electric, foldable, sharable motorbike developed in collaboration

CENTER FOR FUTURE
with SYM and ITRI. The design tackles the biggest problems in major urban centers:
STORYTELLING
pollution, congestion, parking, and energy use. The RoboScooter system, part of the
Mobility on Demand project, allows users to pick up a bike from a scooter stack and
drop it off at any other stack. The bike uses scooter-sized Wheel Robots developed for
the CityCar project. The design team will develop innovative business and ownership
models to help implement the scooter through pilot programs developed jointly by
candidate cities.

224. Wheel Robots William J. Mitchell, Ryan C.C. Chin, William Lark, Jr., Michael Chia-Liang Lin,
Raul-David Poblano, Nicholas Pennycooke and Peter Schmitt

The mechanical components that make driving a vehicle possible—such as

acceleration, braking, steering, and springing—are located inside the space of a
hubless wheel, forming independent wheel robots and freeing the vehicular space of
these components. Connected to the chassis are simple mechanical, power, and data
connections, allowing for the wheel robots to plug in to a vehicle simply and quickly. A
CPU in the vehicle provides the input necessary for driving according to the vehicle's
dimensions or loading condition. The design of the wheel robots provides optimal
contact patch placement, lower unsprung and rotational mass, omnidirectional steering,
great space savings, and modularity, as the wheel robots can function appropriately on
vehicles of different dimensions and weight. By "putting the whole car in the wheel," it is
possible to separate production, service, and life-cycles of the mechanical components
of the car from those of its architectural components.

Alumni Contributors: Patrik Kunzler and Philip Liang

Additional Smart Cities projects

For full descriptions, visit: http://www.media.mit.edu/research/59

225. 3DprintedClock
William J. Mitchell, Peter Schmitt and Robert Swartz

226. Animated Playground Props

William J. Mitchell and Susanne Seitinger

227. Architectural Machines

William J. Mitchell and Peter Schmitt

228. Augmented Street Light

William J. Mitchell, Susanne Seitinger and Joshua Robles

229. Building Blocks in the Mass-Customized Era

William J. Mitchell and Ryan C.C. Chin

230. Car in the City

William J. Mitchell, William Lark, Jr., Ryan C.C. Chin and Polychronis
Ypodimatopoulos

CENTER FOR FUTURE 231. Chameleon Guitar: Physical Heart in a Virtual Body
STORYTELLING William J. Mitchell and Amit Zoran

Page 26 May 2010 MIT Media Lab

 232. CityCar Chassis
William J. Mitchell, Ryan C.C. Chin, William Lark, Jr., Raul-David Poblano,
Nicholas Pennycooke, Charles Guan and Tom Brown

233. CityCar Folding Chassis

William J. Mitchell, William Lark, Jr., Raul-David Poblano, Ryan C.C. Chin,
Charles Guan and Nicholas Pennycooke

234. Cornucopia: Digital Gastronomy

Marcelo Coelho, Amit Zoran, Pattie Maes and William J. Mitchell

235. Hiriko CityCar with Denokinn

William J. Mitchell, Ryan C.C. Chin, William Lark, Jr., Raul-David Poblano,
Nicholas Pennycooke, Praveen Subramani, Tom Brown and Charles Guan

236. Light.Bodies
William J. Mitchell, Susanne Seitinger, Alex S. Taylor and Microsoft
Research

237. plywoodServo
William J. Mitchell, Peter Schmitt and Susanne Seitinger

Frank Moss—New Media Medicine

How radical new collaborations between doctors, patients, and communities will catalyze a
revolution in human health.

238. CollaboRhythm John Moore MD and Frank Moss

The doctor-patient relationship is deteriorating. CollaboRhythm implements new

paradigms in doctor-patient interaction to improve health outcomes and the patient
experience. It uses ubiquitous connectivity, collaborative decision-making, and
compelling interfaces to educate patients, improve treatment adherence, and deliver
care seamlessly at any point in time or space. The foundation is a speech- and
touch-controlled hub for the office where doctor and patient make shared decisions and
where patients are encouraged to actively engage with their data. Patients also own
their data: everything they see in the doctor’s office is available at home, when they visit
another doctor, change jobs, or move across the world. Patients can contribute data
that are important to their health and lifestyles—information that today is invisible to the
doctor. By making patients active, informed participants in their own care, we believe
we can reduce health-care costs, increase quality, and improve health outcomes.

239. Collective Discovery Frank Moss, Deb Roy and Ian Eslick

The choices we make about diet, environment, medications, or alternative therapies

constitute a massive collection of "everyday experiments." These data are largely
unrecorded and underutilized by the traditional research establishment. Collective
Discovery aims to leverage the intuition and insight of patient communities to capture
and mine information about everyday experiences. Moving the community discourse
from anecdotes to data will lead to better decision-making, stronger self-advocacy,
identification of novel therapies, and inspiration of better hypotheses in traditional
research, accelerating the search for new drugs and treatments. The unique
characteristic of our Collective Discovery model is the use of knowledge representation
and natural language processing to mediate communal hypothesis generation and to
compensate for methodological errors and self-reporting bias. This model is being
deployed in a real-world context as part of a partnership with the LAM Treatment
Alliance and the greater LAM community.

MIT Media Lab May 2010 Page 27

240. ForgetAboutIT? John Moore MD and Frank Moss

Currently only 50% of patients with chronic diseases take their medications. The
problem is not simple forgetfulness; it is a complex combination of lack of
understanding, poor self-reflection, limited social support, and almost non-existent
communication between provider and patient. ForgetAboutIT? is a system to support
medication adherence which presupposes that patients engaged in tight, collaborative
communication with their providers through interactive interfaces would think it
preposterous not to take their medications. Technically, it is an awareness system that
employs ubiquitous connectivity on the patient side through cell phones, televisions,
and other interactive devices and a multi-modal collaborative workstation on the
provider side.

241. HealthMap Clark Freifeld, John Brownstein and Frank Moss

HealthMap is a multilingual, real-time disease outbreak tracking and visualization

system. Launched in fall 2006, the Web site collects over 300 reports per day in seven
languages, from both general news media and public health sources around the world.
Updated hourly, the system filters these reports to determine relevance, disease,
location, and duplication clustering by means of a series of custom-designed,
automated, text-processing algorithms. Relevant reports are then aggregated and
displayed on a freely available dashboard where users can tailor the view according to
date, disease, location, and source. HealthMap provides an overview of real-time
information on emerging infectious diseases, and has particular interest for public
health officials and international travelers.

242. HealthMap iPhone Clark Freifeld, John Brownstein and Frank Moss
App: Outbreaks Near
With HealthMap's Outbreaks Near Me application, you have all of HealthMap's latest
Me
real-time disease outbreak information at your fingertips. Open the app and see all
current outbreaks in your neighborhood, including news about H1N1 influenza ("swine
flu"). Search and browse outbreak reports on the interactive map, and set up the app to
alert you with a notice automatically whenever an outbreak is occurring in your area. If
you know of an outbreak not yet on the map, be the first to report it using the app's
unique outbreak reporting feature. You will be credited and your report will be featured
on the Website. With this iPhone app, we are launching an exploration of
crowd-sourced, user-generated, people-powered disease outbreak tracking and
collaboration.

243. LAMsight: A Frank Moss, Ian Eslick, Amy Farber and LAM Treatment Alliance
Data-Driven Disease
LAMsight is a practical experiment in creating new models for collaboration between
Community
researchers, clinicians, and patients. We are working with a rare-disease advocacy
organization to identify and implement collaboration modes that help accelerate
research on the rare disease LAM (Lymphangioleiomyomatosis), a multi-system, fatal
disease that typically strikes women in their child-bearing years.

244. Oovit PT Frank Moss, Clark Freifeld, John Moore, Sai T. Moturu, Neil Chao (MIT) and Stacy
Nemeroff
NEW LISTING
Patient adherence to physical therapy regimens is poor, and there is a lack of
quantitative data about patient performance, particularly at home. This project aims to
build an end-to-end virtual rehabilitation system for supporting patient adherence to
home exercise that addresses the multi-factorial nature of the problem. Using the
proposed system, the physical therapist and patient would make shared decisions
about appropriate exercises and goals and patients would use a sensor-enabled
gaming interface at home to perform exercises. Quantitative data is then fed back to the
therapist, who can properly adjust the regimen and give reinforcing feedback and
support.

Page 28 May 2010 MIT Media Lab

 Additional New Media Medicine projects

For full descriptions, visit: http://www.media.mit.edu/research/2346

245. Awaken
Frank Moss, Alex (Sandy) Pentland, Sai T. Moturu and Kimberly
Shellenberger

246. I'm Listening

John Moore MD, Henry Lieberman and Frank Moss

247. WeightMate
Kendra Markle, Lorin Wilde, Frank Moss, Alex (Sandy) Pentland, Sai T.
Moturu, Orestis Gartaganis and Todd Reid

Joseph Paradiso—Responsive Environments

How sensor networks augment and mediate human experience, interaction, and perception.

248. Boxie the Robot: Joe Paradiso and Alexander Reben

Interactive Physical
How do we leverage people to make systems more intelligent, efficient, and
Agents for Story
successful? Is it worthwhile to involve people heavily in the goals of a system? How
Gathering does a system most effectively coax stories out of people? To investigate these
questions, a robot was built that facilitated interaction and documentary gathering within
CENTER FOR FUTURE an ubiquitous media framework. We then let the robot roam freely, with the goal of
STORYTELLING capturing stories about its environment. This was done by leveraging human mobility
NEW LISTING
and intelligence, as the robot relied upon people to move long distances and achieve its
goals. The end products were a study of how people related to a robot asking for
assistance and interaction in various ways, and a set of movies showing the robot
navigating the resulting "thread" of a narrative.

249. DoppelLab Joe Paradiso, Gershon Dublon, Nan-Wei Gong, Mathew Laibowitz, Alexander
Reben, David Small and Laurel Smith Pardue
CENTER FOR FUTURE
STORYTELLING We are populating a detailed, 3D, graphic model of the Media Lab complex (buildings
E15 and E14) created by the Design Ecology group with graphics and audio driven by
sensor information derived from our Ubiquitous Sensor Portals system and other
embedded sensor/actuator infrastructures around our buildings. The goal of this work is
to explore fluid ways of visualizing information derived from a dense sensor network in
a cross-reality implementation that will always be running, so remote participants can
browse and interact with our physical facility from anywhere in the world.

250. Dynamic Video Joe Paradiso and Gershon Dublon

Layers for
We are developing an "opt-in" ubiquitous media system, in which users carrying
Privacy-Preserving
wearable tags are visible to the camera network and everyone else is invisible. Existing
"Opt-In" Media systems for configurable dynamic privacy are opt-out and catch-all; users desiring
Services privacy carry pre-registered tags that disable sensing and networked media services for
everyone in the room. To address these issues, we separate video into layers of flexible
NEW LISTING
sprites representing each person in the field of view, and transmit video of only those
who opt-in. Our system can also define groups of users who can be dialed in and out of
the video stream dynamically. For cross-reality applications, these dynamic layers
achieve a new level of video granularity, allowing users and groups to uncover
correspondences between their activities across spaces.

251. Feedback Controlled Joe Paradiso and Matt Aldrich

Solid State Lighting
At present, luminous efficacy and cost remain the greatest barriers to broad adoption of
LED lighting. However, it is anticipated that within several years, these challenges will
be overcome. While we may think our basic lighting needs have been met, this

MIT Media Lab May 2010 Page 29

 technology offers many more opportunities than just energy efficiency: this research
attempts to alter our expectations for lighting and cast aside our assumptions about
control and performance. We will introduce new, low-cost sensing modalities that are
attuned to human factors such as user context, circadian rhythms, or productivity, and
integrate these data with atypical environmental factors to move beyond traditional lux
measurements. To research and study these themes, we are focusing on the
development of superior color-rendering systems, new power topologies for LED
control, and low-cost multimodal sensor networks to monitor the lighting network as well
as the environment.

252. Lab-Wide and Joe Paradiso and Mathew Laibowitz

Wearable Sensor and
This is a suite of devices and protocols to support applications in wearable
Video Network
human/social sensing linked to a distributed camera and vision system. The current
system includes a sensate wristwatch with biological and gestural sensors, and a
CENTER FOR FUTURE lapel-pin device with motion and audio-affect sensing. These all communicate with
STORYTELLING
wall-mounted devices (Portals), each of which has a high-resolution camera,
environmental sensors, and a localization system for all devices in the network. All
devices record data and audio in sync with the recorded video. A full-spec Zigbee
network supports device synchronization and mesh networking. All devices have
enough on-board power to extract features from the data.

253. Moral Compass: A Ed Boyden, Henry Lieberman, Marvin Minsky, Joe Paradiso and Bo Morgan
Model of
Moral Compass is a model of how children learn in a problem-solving environment
Self-Conscious
where the child is learning to accomplish goals in the context of parents, strangers, and
Learning cultural knowledge. The child learns in multiple ways: playing alone, being told stories,
and being rewarded or punished. Our model aims to provide an explanation for
relatively complex reflective states of mind, such as desire, avoidance, focus,
ignorance, and personality traits. Our model also emphasizes different types of failure
in its reflective approach to learning, including surprise, disappointment, and guilt.
Possible applications include better understanding of the mental health of cognition in
social domains.

254. Wearable, Wireless Joe Paradiso, Michael Thomas Lapinski, Dr. Eric Berkson and MGH Sports
Sensor System for Medicine
Sports Medicine and
This project is a system of compact, wearable, wireless sensor nodes, equipped with
Interactive Media full six-degree-of-freedom inertial measurement units and node-to-node capacitive
proximity sensing. A high-bandwidth, channel-shared RF protocol has been developed
to acquire data from many (e.g., 25) of these sensors at 100 Hz full-state update rates,
and software is being developed to fuse this data into a compact set of descriptive
parameters in real time. A base station and central computer clock the network and
process received data. We aim to capture and analyze the physical movements of
multiple people in real time, using unobtrusive sensors worn on the body. Applications
abound in biomotion analysis, sports medicine, health monitoring, interactive exercise,
immersive gaming, and interactive dance ensemble performance.

Alumni Contributors: Ryan Aylward and Mathew Laibowitz

Additional Responsive Environments projects

For full descriptions, visit: http://www.media.mit.edu/research/42

255. Dense, Low-Power Environmental Monitoring for Smart

Energy Profiling
Nan-Wei Gong, Ashley Turza, David Way and Joe Paradiso with Phil
London, Gary Ware, Brett Leida and Tim Ren (Schneider Electric)

256. Funk2: Causal Reflective Programming

Marvin Minsky, Joe Paradiso and Bo Morgan

Page 30 May 2010 MIT Media Lab

 CENTER FOR FUTURE 257. Interaction with Ubiquitous Dynamically Responsive Media
STORYTELLING Joe Paradiso, Nan-Wei Gong, Mathew Laibowitz and Alexander Reben

CENTER FOR FUTURE 258. Ubiquitous Sensor Network Navigator and Media Explorer
STORYTELLING Joe Paradiso, Nan-Wei Gong, Mathew Laibowitz and Alexander Reben

Alex (Sandy) Pentland—Human Dynamics

How social networks can influence our lives in business, health, and governance, as well
as technology adoption and diffusion.

259. Awaken Frank Moss, Alex (Sandy) Pentland, Sai T. Moturu and Kimberly Shellenberger

Sleep problems such as insomnia have a significant impact on public health, affect the
NEW LISTING
quality of life and productivity of millions daily, present a substantial economic burden in
the billions yearly, and are strongly associated with multiple comorbid conditions.
Several factors affecting sleep are primarily behavioral and not always obvious. In this
project, we aim to detect the behaviors that affect sleep and use this knowledge to help
users improve sleep habits. While asleep, a wearable sensor headband is used to track
the quality of sleep gained by the users. While awake, smart phones are used to
capture behaviors that can impact sleep. Based on the data collected, the phones are
also used to provide context-sensitive suggestions and coaching elements borrowed
from cognitive behavioral therapy to improve awake behaviors and sleep habits, while
their communication capabilities are used to enhance social support from sleeping
partners and family members.

260. Meeting Mediator Taemie Kim

Meeting Mediator is a real-time, portable system that detects social interactions and
provides persuasive feedback to enhance group collaboration. Social interactions are
captured using sociometric badges, and are visualized on mobile phones to promote
behavioral change. Particularly in distributed collaborations, MM attempts to bridge the
gap among the distributed groups by detecting and communicating social signals.

261. Reality Mining Alex (Sandy) Pentland, Wen Dong, Anmol Madan and Ankur Mani

Every time you use your cell phone, you leave behind a few bits of information, and the
newest smart phones can record everything from users' physical activity to their
conversational cadences. People are—rightfully—nervous about trailing these sorts of
digital bread crumbs behind them. But the same information could help to solve
problems of identity theft and fraud by automatically determining security settings. More
significantly, cell-phone data can shed light on workplace dynamics and on the
well-being of communities. It could even help project the course of disease outbreaks
and provide clues about individuals' health.

Alumni Contributors: Sumit Basu, Tanzeem Choudhury, Brian Clarkson, Nathan Eagle,
Yuri Ivanov, Tony Jebara and Oliver Strimpel

262. Sensible Alex (Sandy) Pentland, Benjamin Waber, Daniel Olguin Olguin, Taemie Kim, Wen
Organizations Dong and Ankur Mani

Data mining of email has provided important insights into how organizations function
and what management practices lead to greater productivity. But important
communications are almost always face-to-face, so we are missing the greater part of
the picture. Today, however, people carry cell phones and wear RFID badges. These
body-worn sensor networks mean that we can potentially know who talks to whom, and
even how they talk to each other. Sensible Organizations investigates how these new
technologies for sensing human interaction can be used to reinvent organizations and
management.

MIT Media Lab May 2010 Page 31

263. Social Evolution Alex (Sandy) Pentland, Anmol Madan, Manuel Cebrian and Nadav Aharony

How do opinions and behaviors spread in face-to-face networks? In this project, we

measure the spread of political opinions, influenza and common colds, stress and
loneliness, and weight changes from 320,000 hours of automated sensor data. These
characteristic variations in individual behavior and network structure can be used to
accurately predict outcomes across various different contexts.

Alumni Contributors: Iolanthe Chronis and Luis Sarmenta

264. WeightMate Kendra Markle, Lorin Wilde, Frank Moss, Alex (Sandy) Pentland, Sai T. Moturu,
Orestis Gartaganis and Todd Reid
NEW LISTING
Nearly one-third of the population of the United States is obese, and another one-third
is overweight, resulting in significant health risks. Behavioral aspects including dietary
habits, emotional states, and lack of physical exercise are the primary contributors to
this phenomenon. In this project, we use smart phones to log dietary habits; track user
behaviors, social interactions and emotional states; and gather the context of their
actions. This information is then used to provide context-sensitive education based on
trend detection, and just-in-time persuasive feedback to improve eating habits, reduce
emotional eating, moderate exposure to unhealthy eating environments, and encourage
better choices including greater physical activity. Social reinforcement is used to further
motivate users.

Additional Human Dynamics projects

For full descriptions, visit: http://www.media.mit.edu/research/31

265. Economic Decision-Making in the Wild

Coco Krumme

Rosalind W. Picard—Affective Computing

How new technologies can help people better communicate, understand, and respond to
affective information.

266. Affective-Cognitive Rosalind W. Picard, Hyungil Ahn and Rana el Kaliouby

Product Evaluation
Companies would like more new products to be successful in the marketplace, but
and Prediction of
current evaluation methods such as focus groups do not accurately predict customer
Customer Decisions decisions. We are developing new technology-assisted methods to try to improve the
customer-evaluation process and better predict customer decisions. The new methods
CENTER FOR FUTURE involve multi-modal affective measures (such as facial expression and skin
STORYTELLING conductance) together with behavioral measures, anticipatory-motivational measures,
and self-report cognitive measures. These measures are combined into a novel
computational model, the form of which is motivated by findings in affective
neuroscience and human behavior. The model is being trained and tested with
customer product evaluations and marketplace outcomes from real product launches.

267. Cardiocam Ming-Zher Poh, Daniel McDuff and Rosalind W. Picard

Cardiocam is a low-cost, non-contact technology for measurement of physiological

NEW LISTING
signals using a basic digital imaging device such as a Webcam. The ability to perform
remote measurements of vital signs is promising for enhancing the delivery of primary
health care.

Page 32 May 2010 MIT Media Lab

268. Emotion Rosalind W. Picard, Matthew Goodwin, Jackie Lee, Rich Fletcher, Kyunghee Kim
Communication in and Robert Morris
Autism
People who have difficulty communicating verbally (such as many people with autism)
sometimes send nonverbal messages that do not match what is happening inside them.
For example, a child might appear calm and receptive to learning—but have a heart
rate over 120 bpm and be about to meltdown or shutdown. This mismatch can lead to
misunderstandings such as "he became aggressive for no reason." We are creating
new technologies to address this fundamental communication problem and enable the
first long-term, ultra-dense longitudinal data analysis of emotion-related physiological
signals. We hope to equip individuals with personalized tools to understand the
influences of their physiological state on their own behavior (e.g., "which state helps me
best maintain my attention and focus for learning?"). Data from daily life will also
advance basic scientific understanding of the role of autonomic nervous system
regulation in autism.

Alumni Contributor: Hoda Eydgahi

269. FaceSense: Daniel McDuff, Rana el Kaliouby, Abdelrahman Nasser Mahmoud, Youssef
Affective-Cognitive Kashef, M. Ehsan Hoque, Matthew Goodwin and Rosalind W. Picard
State Inference from
People express and communicate their mental states—such as emotions, thoughts,
Facial Video and desires—through facial expressions, vocal nuances, gestures, and other
non-verbal channels. We have developed a computational model that enables real-time
CENTER FOR FUTURE analysis, tagging, and inference of cognitive-affective mental states from facial video.
STORYTELLING This framework combines bottom-up, vision-based processing of the face (e.g., a head
nod or smile) with top-down predictions of mental-state models (e.g., interest and
confusion) to interpret the meaning underlying head and facial signals over time. Our
system tags facial expressions, head gestures, and affective-cognitive states at multiple
spatial and temporal granularities in real time and offline, in both natural human-human
and human-computer interaction contexts. The system is being made available on
multiple platforms, including portable devices. Applications range from measuring
people's experiences, to a training tool for autism spectrum disorders.

270. iCalm (TM): Rich Fletcher, Matthew Goodwin, Rosalind W. Picard, Micah Eckhardt, Elliott
Comfortable, Hedman and Ming-Zher Poh
Wearable, Wireless
We are developing a tiny, wearable, wireless sensor platform that allows comfortable,
Bio-Sensing long-term sensing of physiological information, coupled with low-cost connectivity to
consumer devices including mobile phones and the XO laptop. This platform has many
applications, including health monitoring for outpatients or the elderly, communication of
affective information for people who are non-speaking or otherwise interested in sharing
this information, education for individuals who want to learn about their own internal
physiological changes during daily life, and customer-experience data gathering in
mobile situations.

Alumni Contributors: Hoda Eydgahi and Clayton Williams

271. Objective Self: Rosalind W. Picard, Matthew Goodwin, Elliott Hedman and SPD Foundation
Understanding
How can technology help us understand ourselves better? In order to measure the
Internal Responses
physiological arousal of children with sensory challenges such as ASD and ADHD,
tools were developed to help children understand and control what makes them
overexcited. Using iCalm hardware, children in therapy sessions measured their
arousal while eating, throwing tantrums, playing in ball pits, and making challenging
choices. Beyond progressive findings in the field of occupational therapy, this research
is a basis for bio-information technology: tools to help children, their parents, and their
teachers better understand what is going on in their bodies in a comfortable, affordable,
and adaptable way. With future work, technology will be developed to help children
understand and control their own internal states. In addition, this project will go beyond
children’s therapy—helping adults in various settings including business and home life.

MIT Media Lab May 2010 Page 33

272. Passive Wireless Rich Fletcher and Sarang Kulkarni
Heart-Rate Sensor
We have developed a low-cost device that can wirelessly detect a beating heart over a
short distance (1m) and does not require any sensor placed on the person's body. This
device can be used for wireless medical/health applications as well as security and
safety applications, such as automobile/truck drivers as well as ATM machines. We
have also created a small battery-powered version of this sensor that can be worn on a
person's clothing but does not require touching the person's skin.

Additional Affective Computing projects

For full descriptions, visit: http://www.media.mit.edu/research/30

273. Affect as Index

Shaundra Bryant Daily and Rosalind W. Picard

274. Affect Valence Recognition From Facial Expressions

Rana el Kaliouby, Rosalind W. Picard and Daniel McDuff

275. Affective-Cognitive Framework for Machine Learning and

Decision Making
Hyungil Ahn and Rosalind W. Picard
276. Auditory Desensitization Games
Rosalind W. Picard, Matthew Goodwin and Robert Morris

277. Embedding Special Interests Into Computer-Mediated

Interventions
Rosalind W. Picard and Robert Morris

278. Emotional-Social Intelligence Toolkit

Rosalind W. Picard, Rana el Kaliouby, Matthew Goodwin, Mish Madsen,
Micah Eckhardt and M. Ehsan Hoque

279. Evaluation Tool for Recognition of Social-Emotional

Expressions from Facial-Head Movements
Rosalind W. Picard

280. Externalization Toolkit

Rosalind W. Picard, Matthew Goodwin and Jackie Chia-Hsun Lee

281. Frame It
Rosalind W. Picard and Micah Eckhardt

282. Gestural Control of Guitar Audio Effects

Rosalind W. Picard, Robert Morris and Tod Machover

283. Girls Involved in Real-Life Sharing

Rosalind W. Picard and Shaundra Bryant Daily

284. Health Interventions Using Mobile Phones

Rich Fletcher, Rosalind Picard, Sharon Tam and Micah Ekhardt

285. Heartphones
Rosalind W. Picard and Ming-Zher Poh

286. Infant Monitoring and Communication

Rana el Kaliouby, Rich Fletcher, Matthew Goodwin and Rosalind W. Picard

287. Machine Learning and Pattern Recognition with Multiple

Modalities
Hyungil Ahn and Rosalind W. Picard

288. Sensor-Enabled Measurement of Stereotypy and Arousal in

Individuals with Autism
Matthew Goodwin, Clark Freifeld and Sophia Yuditskaya

Page 34 May 2010 MIT Media Lab

 Ramesh Raskar—Camera Culture
How to create new ways to capture and share visual information.

289. Bokode: Ramesh Raskar, Ankit Mohan, Grace Woo, Shinsaku Hiura and Quinn Smithwick
Imperceptible Visual
With over a billion people carrying camera-phones worldwide, we have a new
Tags for
opportunity to upgrade the classic bar code to encourage a flexible interface between
Camera-Based the machine world and the human world. Current bar codes must be read within a short
Interaction from a range and the codes occupy valuable space on products. We present a new, low-cost,
Distance passive optical design so that bar codes can be shrunk to fewer than 3mm and can be
read by unmodified ordinary cameras several meters away.

290. Coded Computational Jaewon Kim, Ahmed Kirmani, Ankit Mohan and Ramesh Raskar
Photography
Computational photography is an emerging multi-disciplinary field that is at the
intersection of optics, signal processing, computer graphics and vision, electronics, art,
CENTER FOR FUTURE
and online sharing in social networks. The first phase of computational photography
STORYTELLING
was about building a super-camera that has enhanced performance in terms of the
traditional parameters, such as dynamic range, field of view, or depth of field. We call
this 'Epsilon Photography.' The next phase of computational photography is building
tools that go beyond the capabilities of this super-camera. We call this 'Coded
Photography.' We can code exposure, aperture, motion, wavelength, and illumination.
By blocking light over time or space, we can preserve more details about the scene in
the recorded single photograph.

291. Femtosecond Ramesh Raskar, Ahmed Kirmani and James Davis

Transient Imaging
Our goal is to exploit the finite speed of light to improve image capture and scene
understanding. New theoretical analysis, coupled with emerging ultra-high-speed
CENTER FOR FUTURE
imaging techniques, can lead to a new source of computational visual perception. We
STORYTELLING
are developing the theoretical foundation for sensing and reasoning using transient light
transport, and experimenting with scenarios in which transient reasoning exposes
scene properties that are beyond the reach of traditional machine vision.

292. Glasses-Free Ramesh Raskar, Henry Holtzman, Douglas Lanman, Matt Hirsch and Yunhee Kim
HD3DTV
For 3D displays to be successful, they must be bright enough to compete with 2D
displays and not diminish display resolution in plane. To date, stacked-LCD displays
NEW LISTING
have employed parallax barriers, which use pinhole or bar patterns to provide
view-dependent imagery. We show a prototype that adapts the imagery on both layers
to multi-view 3D content, increasing brightness while maintaining in-plane resolution.
This promises a future of devices with sharp 2D screens and 3D displays with full
horizontal and vertical parallax.

293. LensChat: Sharing Ramesh Raskar, Rob Gens and Wei-Chao Chen
Photos with
With networked cameras in everyone's pockets, we are exploring the practical and
Strangers
creative possibilities of public imaging. LensChat allows cameras to communicate with
each other using trusted optical communications, allowing users to share photos with a
CENTER FOR FUTURE friend by taking pictures of each other, or borrow the perspective and abilities of many
STORYTELLING
cameras.

MIT Media Lab May 2010 Page 35

294. Theory Unifying Ray Ramesh Raskar, George Barbastathis, Tom Cuypers and Se Baek Oh
and Wavefront
This work focuses on bringing powerful concepts from wave optics to the creation of
Lightfield
new algorithms and applications for computer vision and graphics. Specifically,
Propagation ray-based, 4D lightfield representation, based on simple 3D geometric principles, has
led to a range of new applications that include digital refocusing, depth estimation,
synthetic aperture, and glare reduction within a camera or using an array of cameras.
The lightfield representation, however, is inadequate to describe interactions with
diffractive or phase-sensitive optical elements. Therefore we use Fourier optics
principles to represent wavefronts with additional phase information. We introduce a
key modification to the ray-based model to support modeling of wave phenomena. The
two key ideas are "negative radiance" and a "virtual light projector." This involves
exploiting higher dimensional representation of light transport.

Additional Camera Culture projects

For full descriptions, visit: http://www.media.mit.edu/research/2356

CENTER FOR FUTURE 295. Community Photography

STORYTELLING Ramesh Raskar, Kevin Chiu and Gabriel Taubin

CENTER FOR FUTURE 296. Second Skin: Optical Motion Capture with Actuated
STORYTELLING Feedback
Ramesh Raskar and Dennis Miaw

297. Shield Field Imaging

Jaewon Kim

298. Vision on Tap

Ramesh Raskar and Kevin Chiu

Mitchel Resnick—Lifelong Kindergarten

How to engage people in creative learning experiences.

299. Computer Clubhouse Mitchel Resnick, Natalie Rusk, Amon Millner, Chris Garrity and Robbie Berg

At Computer Clubhouse after-school centers, young people (ages 10-18) from

low-income commmunities learn to express themselves creatively with new
technologies. Clubhouse members work on projects based on their own interests, with
support from adult mentors. By creating their own animations, interactive stories, music
videos, and robotic constructions, Clubhouse members become more capable,
confident, and creative learners. The first Computer Clubhouse was established in
1993, as a collaboration between the Lifelong Kindergarten group and The Computer
Museum (now part of the Boston Museum of Science). With financial support from Intel
Corporation, the network has expanded to more than 20 countries, serving more than
20,000 young people. The Lifelong Kindergarten group continues to develop new
technologies, introduce new educational approaches, and lead
professional-development workshops for Clubhouses around the world.

Alumni Contributors: Leo Burd, Robbin Chapman, Rachel Garber, Tim Gorton, Michelle
Hlubinka and Elisabeth Sylvan

300. Drawdio Jay Silver and Mitchel Resnick

Drawdio is a pencil that draws music. You can sketch musical instruments on paper
and play them with your finger. Touch your drawings to bring them to life—or
collaborate through skin-to-skin contact. Drawdio works by creating electrical circuits
with graphite and the human body.

Page 36 May 2010 MIT Media Lab

301. Hook-Ups Amon Millner and Mitchel Resnick

The Hook-Ups system is a set of technologies and activities that enables young people
to create interactive experiences by programming connections between physical and
digital media. With the Hook-Ups system, young people integrate sensors with a myriad
of materials to create their own tangible interfaces. These interfaces control digital
images and sounds in computer programs (such as games or responsive art pieces)
the young people write. For example, a 10-year-old created a paper-plate-based flying
saucer, added a sensor, then wrote a program to control an animation of a flying saucer
on her computer screen.

302. Scratch Mitchel Resnick, John Maloney, Andres Monroy-Hernandez, Natalie Rusk, Evelyn
Eastmond, Karen Brennan, Amon Millner, Eric Rosenbaum, Jay Silver, Rita Chen,
Amos Blanton and Brian Silverman
CENTER FOR FUTURE
STORYTELLING
Scratch is a programming language and online community that makes it easy to create
interactive stories, games, animations, and simulations—and share those creations
online. Scratch is designed to enhance the technological fluency of young people (ages
8 and up), helping them learn to express themselves creatively with new technologies.
As they create and share Scratch projects, young people learn to think creatively,
reason systematically, and work collaboratively—while also learning important
mathematical and computational ideas.

Alumni Contributors: Margarita Dekoli and Tamara Stern

303. ScratchEd Karen Brennan and Mitchel Resnick

As Scratch proliferates through the world, there is a growing need to support learners.
CENTER FOR FUTURE
STORYTELLING But for teachers, educators, and others who are primarily concerned with enabling
Scratch learning, there is a disconnect between their needs and the resources that are
presently available through the Scratch Web site. ScratchEd is an online environment
for Scratch educators to share stories, exchange resources, ask questions, and find
people.

304. Singing Fingers Eric Rosenbaum, Jay Silver and Mitchel Resnick

Singing Fingers allows children to fingerpaint with sound. Users paint by touching a
NEW LISTING
screen with a finger, but color only emerges if a sound is made at the same time. By
touching the painting again, users can play back the sound. This creates a new level of
accessibility for recording, playback, and remixing of sound.

Additional Lifelong Kindergarten projects

For full descriptions, visit: http://www.media.mit.edu/research/33

305. Color Code

Mitchel Resnick, Eric Rosenbaum and Jay Silver

306. Computer Clubhouse Village

Chris Garrity, Natalie Rusk and Mitchel Resnick

CENTER FOR FUTURE 307. Computer Crafting

STORYTELLING Jay Silver, Karen Brennan and Mitchel Resnick

308. DesignBlocks
Mitchel Resnick, Evelyn Eastmond, Eric Rosenbaum, Brian Silverman and
Paula Bonta

309. Glowdoodle
Eric Rosenbaum

CENTER FOR FUTURE 310. Jots

STORYTELLING Eric Rosenbaum and Mitchel Resnick

MIT Media Lab May 2010 Page 37

 311. Mobile Scratch
John Maloney, Jay Silver, Karen Brennan, Andres Monroy-Hernandez and
Mitchel Resnick

CENTER FOR FUTURE 312. Say What?!

STORYTELLING Karen Brennan, Shaundra Bryant Daily and Mitchel Resnick

313. Scratch Board

Amon Millner, Robbie Berg, John Maloney and Mitchel Resnick

CENTER FOR FUTURE 314. Scratch Day

STORYTELLING Karen Brennan and Mitchel Resnick

315. Scratch for Computer Science

John Maloney and Mitchel Resnick

316. Scratch Worlds

Eric Rosenbaum and Mitchel Resnick

317. ScratchR
Andres Monroy-Hernandez and Mitchel Resnick

318. Twinkle
Jay Silver, Eric Rosenbaum and Mitchel Resnick

319. What's Up
Chris Csikszentmihályi, Mitchel Resnick and Leo Burd

Deb Roy—Cognitive Machines

How to build machines that learn to use language in human-like ways, and develop tools
and models to better understand how children learn to communicate.

320. Behavior Capture Jeff Orkin and Deb Roy

from Thousands of
The Restaurant Game is a multiplayer simulation that captures the behavior and
People Online
language of thousands of people playing the roles of wait staff and customers. We are
developing machine-learning algorithms that mine game-play logs to acquire generative
models of human language, behavior, and social roles. These models will power
synthetic conversational characters that interact with humans in training simulations,
games, and other virtual worlds.

321. Connected Strangers: Deb Roy and Sheng-Ying (Aithne) Pao

Manipulating Social
Many important decisions and interactions in everyday life are made with strangers. In
Perceptions to Study
these interactions, perceived trust is an important factor. As virtual interactions
Trust increase, there is less to gauge this perceived trust upon. Decisions with strangers
make for a higher uncertainty in the entry stage. Such interactions introduce a need for
a new way to make decisions involving privacy, ownership, and sharing. The question
we ask is: How can we leverage the uncertainty of interactions between strangers to
motivate trust?

322. Gestalt Video Deb Roy and Leo Tsourides

Analyzer
This project is an effort to develop computer vision methodologies to perform behavioral
evaluation on dyadic interactions grounded in longitudinal video recordings. The
NEW LISTING
proposed system can be applied on any surveillance output, to analyze or recover
event dynamics hidden on multi-agent interactions. Applications include analyzing
word-learning dynamics in young children, new kinds of video retail analysis aimed at
customer-associate interactions, determining dominant agents in a group, and
establishing longitudinal diagnostic means for child developmental disorders.

Page 38 May 2010 MIT Media Lab

323. Grounding Spatial Deb Roy, Stefanie Tellex, Nicholas Roy and Thomas Kollar
Language for Video
Understanding spatial language is a challenging problem that requires the ability to map
Retrieval and Robotic
between language and real-world situations. We are building a spatial language
Direction Following understanding system that bridges this representational gap by computationally
modeling the semantics of spatial prepositions. Our model enables a system to retrieve
CENTER FOR FUTURE video clips that match natural language queries such as "Show me people going across
STORYTELLING the kitchen." We are also applying it to build robots that can follow natural-language
directions such as "Go through the door near the elevators." By using corpus-based
machine learning techniques, our model is robust to real-world noise and linguistic
variation. Exploring the connection between language and the real world in concrete
domains enables us to make progress towards computers that understand language in
human-like ways.

324. HouseFly: Immersive Deb Roy, Philip DeCamp, Rony Kubat and George Shaw
Video Browsing and
HouseFly combines audio-video recordings from multiple cameras and microphones to
Data Visualization
generate an interactive, 3D reconstruction of recorded events. Developed for use with
the longitudinal recordings collected by the Human Speechome Project, this software
CENTER FOR FUTURE enables the user to move freely throughout a virtual model of a home and to play back
STORYTELLING
events at any time or speed. In addition to audio and video, the project explores how
different kinds of data may be visualized in a virtual space, including speech transcripts,
person tracking data, and retail transactions.

325. Speech Interaction Deb Roy, Brandon Roy and Michael Frank
Analysis for the
The Speechome Corpus is the largest corpus of a single child learning language in a
Human Speechome
naturalistic setting. We have now transcribed significant amounts of the speech to
Project support new kinds of language analysis. We are currently focusing on the child's lexical
development, pinpointing "word births" and relating them to caregiver language use.
Our initial results show child vocabulary growth at an unprecedented temporal
resolution, as well as a detailed picture of other measures of linguistic development.
The results suggest individual caregivers "tune" their spoken interactions to the child's
linguistic ability with far more precision than expected, helping to scaffold language
development. To perform these analyses, new tools have been developed for
interactive data annotation and exploration.

326. Speechome Recorder Sophia Yuditskaya, Kleovoulos Tsourides, Philip DeCamp, George Shaw,
for the Study of Child Matthew Goodwin and Deb Roy
Development
Collection and analysis of dense, longitudinal observational data of child behavior in
Disorders natural, ecologically valid, non-laboratory settings holds significant benefits for
advancing the understanding of autism and other developmental disorders. We have
developed the Speechome Recorder—a portable version of the embedded audio/video
recording technology originally developed for the Human Speechome Project—to
facilitate swift, cost-effective deployment in special-needs clinics and homes. Recording
child behavior daily in these settings will enable us to study developmental trajectories
of autistic children from infancy through early childhood, as well as atypical dynamics of
social interaction as they evolve on a day-to-day basis. Its portability makes possible
potentially large-scale comparative study of developmental milestones in both
neurotypical and autistic children. Data-analysis tools developed in this research aim to
reveal new insights toward early detection, provide more accurate assessments of
context-specific behaviors for individualized treatment, and shed light on the enduring
mysteries of autism.

MIT Media Lab May 2010 Page 39

327. Speechome Video for George Shaw, Rony Kubat, Philip DeCamp, Kenneth Jackowitz (BOA) and Deb
Retail Analysis Roy

We are adapting the video data collection and analysis technology derived from the
Human Speechome Project for the retail sector through real-world deployments. We
are developing strategies and tools for the analysis of dense, longitudinal video data to
study behavior of and interaction between customers and employees in commercial
retail settings. One key question in our study is how the architecture of a retail space
affects customer activity and satisfaction, and what parameters in the design of a space
are operant in this causal relationship.

328. Study of Child Deb Roy and Soroush Vosoughi

Language Acquisition
What is the relationship between the input children hear and the words they acquire?
in the Human
We are investigating the role of variables such as input word frequency and prosody in
Speechome Project one child's lexical acquisition using the Human Speechome Project corpus. We are
analyzing data from ages nine to 24 months, including the child's first productive use of
language at about 11 months, ending at the child’s active use of a vocabulary with more
than 500 words.

Additional Cognitive Machines projects

For full descriptions, visit: http://www.media.mit.edu/research/27

329. BlitzScribe: Speech Analysis for the Human Speechome

Project
Brandon Roy and Deb Roy

330. Collective Discovery

Frank Moss, Deb Roy and Ian Eslick

331. Data-Driven Architectural Design

Rony Kubat, Kenneth Jackowitz (BOA) and Deb Roy

332. Human Speechome Project

Deb Roy, Philip DeCamp, Brandon Roy, Jethran Guinness, Rony Kubat,
Stefanie Tellex and George Shaw

333. Internomics
Ed Boyden, Dan Ariely, Deb Roy, Nathan Greenslit, Sheng-Ying (Aithne)
Pao, Coco Krumme, Deborah Egloff and James Barabas

334. TrackMarks: Semi-Automatic Video Annotation

Philip DeCamp and Deb Roy

Chris Schmandt—Speech + Mobility

How speech technologies and portable devices can enhance communication.

335. Back-Talk Chris Schmandt and Andrea Colaco

The living room is the heart of social and communal interactions in a home. Often
present in this space is a screen: the television. When in use, this communal gathering
space brings together people and their interests, and their varying needs for company,
devices, and content. This project focuses on using personal devices such as mobile
phones with the television; the phone serves as a controller and social interface by
offering a backchannel to convey engagement, laughter, and viewer comments, and to
create remote co-presence.

Page 40 May 2010 MIT Media Lab

336. Guiding Light Chris Schmandt, Jaewoo Chung, Ig-Jae Kim and Kuang Xu

Guiding Light is a navigation-based application that provides directions by projecting

them onto physical spaces both indoors and outdoors. It enables a user to get relevant
spatial information by using a mini projector in a cell phone. The core metaphor
involved in this design is that of a flashlight which reveals objects in and information
about the space it illuminates. For indoor navigation, Guiding Light uses a combination
of e-compass, accelerometer, proximity sensors, and tags to place information
appropriately. In contrast to existing heads-up displays that push information into the
user's field of view, Guiding Light works on a pull principle, relying entirely on users'
requests and control of information.

337. Merry Miser Chris Schmandt and Charlie DeTar

Merry Miser is a mobile application which persuades people to spend less money, and
think more about their spending. By combining users' real financial transaction
information, their location, and personal assessments of spending, the application
presents deeply personalized and compelling interventions at the time and place when
they are near an opportunity to shop. The interventions help to reinforce choices that
are in the users' better long-term self interest, against short-term impulses.

338. Tin Can Chris Schmandt, Matthew Donahoe and Drew Harry

Distributed meetings present a set of challenges to staying engaged and involved.

NEW LISTING
Because one person is talking at a time, it is easy (particularly for remote participants)
to disengage from the meeting undetected. But non-speaking roles in a meeting can be
as important as speaking ones, and if we could give non-speaking participants ways to
participate, we could help support better-run meetings. The Tin Can project collects
these background tasks—taking notes, managing agendas, sharing relevant content,
tracking to-dos—in a distributed interface that uses meeting participants' phones and
laptops as input devices and represents current meeting activities, by both local and
remote participants. These activities are represented on a tablet computing device in
the center of the table in each meeting location. By publicly representing background
processes, meeting attendees have new ways to participate and be recognized for their
non-verbal participation.

339. Wish You Were Here Chris Schmandt and Matthew Donahoe

Postcards are a visually pleasing way to share our location with friends and family.
NEW LISTING
Traditionally we send postcards only when we go someplace special, like on vacation.
What if we sent postcards everywhere we went? Wish You Were Here replaces paper
postcards with a digital photo frame that automatically updates its photo as the user
travels. To friends and family members, these pictures can be very meaningful because
they have some expectations about where you are, while strangers can glean only very
general information from them.

Additional Speech + Mobility projects

For full descriptions, visit: http://www.media.mit.edu/research/26

340. Conch
Chris Schmandt, Drew Harry and Judith Donath

341. Flickr This

Chris Schmandt and Dori Lin

342. Going My Way

Chris Schmandt and Jaewoo Chung

343. My Second-Bike
Chris Schmandt, Jaewoo Chung, Andrea Colaco and Kuang Xu

344. Presentation Spaces

Drew Harry, Jordan Slott and Nicole Yankelovich

MIT Media Lab May 2010 Page 41

 345. Sharemote: Collaborative TV
Chris Schmandt and Matthew Donahoe

David Small—Design Ecology

How to enhance understanding, enable creativity, and ease our interactions with the
technological environment.

346. Cartagen Jeffrey Warren and David Small

Cartagen is a set of tools for mapping, enabling users to view and configure live
streams of geographic data in a dynamic, personally relevant way. Today's mapping
software is largely based on static data sets, and neither incorporates the time
dimension in its display nor provides for real-time data streams. Cartagen, built in
HTML5, and viewable on mobile devices such as the iPhone and Android platforms,
helps users to analyze and view shared geodata from multiple sources. Cartagen is a
dynamic map renderer which employs Geographic Style Sheets (GSS), a cascading
stylesheet specification for geospatial information—a decision which leverages literacy
in CSS to make map styling more accessible. However, GSS is a scripting language as
well, making Cartagen an ideal framework for mapping dynamic data. Applications
include mapping real-time air pollution, citizen reporting, and disaster response.

347. Face the Facts Richard The, Doug Fritz, Pattie Maes and David Small

Face the Facts is an experiment in embodied data visualization. The goal is to foster
NEW LISTING
understanding of abstract information that is spatially or temporarily detached from us.
We hope to achieve a more personalized, evocative perception of information that
would be hard to to grasp otherwise. We are creating an art installation in the 3rd floor
atrium of the new Media Lab building that will augment the people wandering through it
with statistical information. As an example, we are using data produced by the Media
Lab, extracted from the student and faculty publications over the course of the Media
Lab's rich history.

348. Kaleido: Agnes Chang and David Small

Idiosyncractic
One of the major barriers encountered by designers and artists when programming
Graphical Interfaces
digital media is difficulty translating mental models of interactive creations into a format
for Software and language that can be interpreted by computers. This problem arises because
Development current software-development environments demand a sequential format for code. In
contrast, Kaleido proposes a new interface that enables a user-defined, conceptual,
visiospatial representation of computation that complements the traditional text-based
perspective. Kaleido is a tool designed to help visual thinkers program; users can use
Kaleido to create personally meaningful visuals for their code. Kaleido allows
individuals to plan, organize, and navigate code in the idiosyncratic way we each think.

349. What If the World Richard The and Doug Fritz

Were Your n
Information that is spatially or temporarily detached from us is hard to relate to our own
Facebook Friends?
personal lives. In addition, we are confronted with more and more information, and we
need tools to simplify and abstract this vast amount of data. Using Facebook as an
NEW LISTING example, we have created a tool for personalized data visualization. Your Facebook
friends and acquaintances become stand-ins for the entire world population, and serve
as a more easily comprehensible representation for this vast data set. It generates a
sense of empathy and connection to what is usually a bodiless and abstract set of
numbers. This project is part of the Face the Facts series, which experiments with
different ways to make abstract statistical data more tangible using unusual,
personalized data-visualization techniques.

Page 42 May 2010 MIT Media Lab

 Additional Design Ecology projects

For full descriptions, visit: http://www.media.mit.edu/research/2447

350. Daydar: Framework for Socially Motivated Goal Fulfillment

Henry Holtzman, John Kestner and Richard The

351. Grassroots Mapping with Balloons and Kites

Jeffrey Warren

Barry Vercoe—Music, Mind and Machine

How to build intelligent music systems out of interacting audio-processing agents.

352. Musicpainter Barry Vercoe and Wu-Hsi Li

Musicpainter is a networked, graphical composing environment that encourages

sharing and collaboration within the composing process. It provides a social
environment where users can gather and learn from each other. The approach is based
on sharing and managing music creation in small and large scales. At the small scale,
users are encouraged to begin composing by conceiving small musical ideas, such as
melodic or rhythmic fragments, all of which are collected and made available to all
users as a shared composing resource. The collection provides a dynamic source of
composing material that is inspiring and reusable. At the large scale, users can access
full compositions that are shared as open projects. Users can listen to and change any
piece. The system generates an attribution list on the edited piece, allowing users to
trace how it evolves in the environment.

353. Musicscape Barry Vercoe and Wu-Hsi Li

Musicscape is a two-dimensional, spatial music navigation interface designed for

browsing large-scale sound archives. It simulates a 2D sound field by applying
Head-Related Transfer Function (HRTF), and enables users to virtually walk around the
sound space with a computer mouse.

354. Perceptual Sound Barry Vercoe and Mihir Sarkar

Design
Sound designers, audio professionals, and musicians often spend time and energy
looking for the right sound for a particular piece of music or sonic environment. Current
sound synthesizers either contain numerous sound presets that are laborious to parse,
or batteries of parameters to tweak without straightforward connections to one's
intuitive expectation. We propose a sound retrieval and modification engine based on
everyday words like "bright," warm," and "fat." The perceptual sound synthesis engine
is informed by a survey of musicians and listeners worldwide and can also be
customized. This system allows dynamic tagging of sound material from online libraries,
and "sound sculpting" based on common verbal descriptors instead of obscure
numerical parameters.

355. Prediction and Barry Vercoe and Mihir Sarkar

Intention in Network
A live music event where musicians are located remotely from each other is possible
Music Performance
through the Internet but highly constrained by network latency. This is especially true
with rhythmic music that requires tight synchrony, or in situations where musicians are
separated by long distances. To overcome time delays, we propose an intelligent
system that listens to the audio input at one end and synthesizes a predicted audio
output at the other. In this context, we study how our musical exposure gives rise to
musical anticipation. Moreover, as we admit that such prediction cannot be error-free,
we aim to model the musical intentions of the performers.

MIT Media Lab May 2010 Page 43

356. Radio-ish Media Barry Vercoe and Wu-Hsi Li
Player
How many decisions does it take before you hear a desired piece of music on your
iPod? First, you are asked to pick a genre, then an artist, then an album, and finally a
song. The more songs you own, the tougher the choices are. To resolve the issues, we
turn the modern music player into an old analog radio tuner, the Radio-ish Media
Player. No LCDs, no favorite channels, all you have is a knob that will help you surf
through channel after channel accompanied by synthesized noise. Radio-ish is our
attempt to revive the lost art of channel surfing in the old analog radio tuner. Let music
find you: your ears will tell you if the music is right. This project is not only a
retrospective design, but also our reflection on lost simplicity in the process of
digitalization. A mobile phone version is also available for demo.

357. Speaker Identification Judith Brown

of Marine Mammals
Speaker identification techniques (such as a Gaussian Mixture Model (GMM) and a
Hidden Markov Model, using cepstral coefficients as features) have been applied to
determine the question of whether individual marine mammals can be identified by their
vocalizations alone. With a dataset of four killer whales uttering sounds previously
classified as belonging to call type n2, and over 10 sounds from each individual, we
have found a very high success rate of 80 to 100% correct for the six pairwise
comparisons and around 78% correct for identification among all four individuals. The
ability to identify marine mammals from their vocalizations alone, in addition to the
theoretical interest for production mechanisms, is extremely valuable in the ability to
track these mammals from remote locations where visual information is not present.

Additional Music, Mind and Machine projects

For full descriptions, visit: http://www.media.mit.edu/research/20

358. Audio Spotlight

Barry Vercoe

Page 44 May 2010 MIT Media Lab