Charisma Cam

prototyping the socio-emotional interface

Future Interfaces#2: Ambient Intelligence European Vision & Innovation 2003-03-03 @ TechGate Vienna

Michael Bechinie | Christian Bechinie

The machines story

is largely about...

...buttons, keyboard, files, folder and mouse.

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 2

The co-evolution of
systems and interfaces
Mainframes PCs Distributed systems of Information Appliances "AmI"

Complexity of Systems ("Users View")

Command Line User Interface

Graphical User Interface Human Centeredness of Interface Technology

time Yesterday Now Tomorrow

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 3

The human story

is about...

...a stone age mind in a modern skull.

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 4

A closer look
at the nature of human behaviour
social matter 22,50%

hard facts 7,50%

non verbal 70,00%

behaviour Language
[Dunbar '96]

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 5

The co-evolution of
systems and interfaces
Mainframes PCs Distributed systems of Information Appliances appliances "AmI" Socio-emotional User Interface (SEI)

Complexity of Systems ("Users View")

Multimodal User Interface Command Line User Interface Graphical User Interface Human Centeredness of Interface Technology time Yesterday Now Tomorrow

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 6

Future interfaces
are about designing...

...the socio-emotional man machine interface

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 7

digitalMankind
Company
core competence computer supported analysis and simulation of human behaviour. long term goal improve and augment humancomputer interaction by introducing virtual humans as a socially capable interface (Advanced Interfaces). short term goal first generation of virtual humans are available as interactive game characters (PC-Games).
FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface digitalMankind | p. 8

Virtual Humans
are a social counterpart for the user if...
show a distinct personality act autonomously react context specific read user's behaviour show humanlike behaviour

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 9

digitalMankind
Research & Technology

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 10

Basic Research
Ludwig Boltzmann Institute for Urban Ethology, Vienna

director Karl Grammer empiric data of human behaviour scientific computing Digital Image & Video Analysis Neural Networks Simulation modern communication theory Gestalt-free approach of observation meaning encoded in movement quality emotions as vectors

Attractiveness Mate selection Communication Emotion Interaction Language Synthetic Social Behavior
digitalMankind | p. 11

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

Biomimetics
imitating nature...

Analysis

Simulation

Lotus Real

self purification Social Behaviour

Galalpagos shark Behavioural Synthetic

Data flow resistance Social Behaviour
digitalMankind | p. 12

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

Technology
Virtual Character Engine
USER SIMULATION GOAL

Content

AIM

ACOS

NAT
Processing

iDSO
Body language
FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

BOLIB
Data
digitalMankind | p. 13

Charisma Cam
showcase for the iDSO

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 14

Charisma Cam
what is it?
real-time behavioural analysis interprets body language "dance to your own beat" gives back personality profile Extroversion Sociability Emotional stability Conscientiousness Openness

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 15

Charisma Cam
how does it work?
video signal capturing mimics the human eye digital video camera motion energy detection mimics low level vision processing changes in body contours calculating "difference pictures" deriving motion energy motion energy analysis mimics human appraisal process categorisation, pattern recognition presentation of data to neural networks results for "Big5" personality traits
FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface digitalMankind | p. 16

Charisma Cam
how was the system built?
100 students were filmed dancing for 30 seconds mean age 22 years students filled in questionnaire NEO-FFI personality assessment analysis of questionnaire data motion energy detection first derivative over time training of neural networks Time Delayed Neural Networks 10 networks (5 personality parameters, 2
sexes)

real time classification system

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 17

Charisma Cam
what's the theory behind?
categorical communication insufficient information processing approach signals = movement-Gestalten one signal in many qualities comm. as a dynamic non linear process continuous flow of information co-regulated activities inferential comm. key concept internal states are probably encoded in

comm. is context dependent

movement quality mirror (F5) neurons as enabling brain structures [Fadiga et al.'95, Gallese et al. '96] mental simulation, from "doing" to "communicating" the shared manifold [Gallese '01]

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 18

Charisma Cam
what is it good for?
iDSO for virtual humans information about the user personality, mood etc. Virtual Character Engine behavioural input mirror matching system generation of feedback loop adaptive system substantial module for SEI

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 19

Charisma Cam
Print
.copy falter e!trend The Sunday Times managerSeminare Die Zeit Wirtschaftswoche Focus com!online

what's the buzz?

Online
science.orf.at yahoo.de t-online.de

TV
BBC Tomorrows World Arte Galileo Nano Modern Times Future Trends FitforFun

Radio
OE3 FM4

Events
Int. Money Trends 2001 BBC TWLE 2001 Lange Nacht d. Museen 2002

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 20

Thank you for your attention

michael@digitalmankind.com

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 21

Charisma Cam
Video TWLE 2001 @ London

FI#2, 2003-03-03 | Charisma Cam - Prototyping the socio-emotional Interface

digitalMankind | p. 22