Вы находитесь на странице: 1из 7

Ego-Jections: A “Theatrical” Reactive System of

Kinetic Structures for Augmented Multimedia

Medea Electronique, Athens, Greece, medea_electronique@yahoo.com


Installation Interactive Robotics Multimedia Immersion

System Kinetic Artificial Life Man Machine Sensing

Abstract — This is a paper describing EgoJections, an Raising questions about the evolutionary course of
ongoing project, dealing with the research and development humans, their social structure and formations, in the
of a series of kinetic structures for augmented multimedia contexts of art whilst using the means and media of the
performance practices, in immersive, reactive environments. time, is being done by research groups and individuals for
The structures are interconnected in a network forming a several decades. Specifically robotics are being seen as a
system of globally controlled, sensorial input processing and
actuating event processes, proposing a complex animated
new medium helping artists pushing the limits in these
artwork. The use of video, sound and light reinforce the pursuits. Especially the last 20 years artists like Simon
notion of an immersive environment, in which the presence Penny, Bill Vorn, Norman White, Stelios Arcadiou (the
of the human agents merge with the machine agent network famous Greek/Australian Stelarc), Robotlab and many
in a cybernetic loop, presenting a unified system, able to more have been exploring these fields by using dynamic
express artistic objectives. The implementation of the first virtual/artificial architectures to simulate organic
machine entity of the system, called “EgoJections: Try once functions, bringing closer the living and the artificial the
for better experience”, is the presented showcase here, organic and the machine.
supporting the description of a network as such. “Try once
for better experience” took place in Electromedia Works ‘08 Keeping as a basic framework the production of
Athens festival. The design strategies, protocol, actuating, artwork, the pursuit is being affected by advances in
and sensing technology and works, involved in presenting research concerning Artificial Life, intelligence, genetic
the above system are explained. The autonomous behavior algorithms, neural networks, system theory, emergence
of a single machine entity justifies it as a separate ‘wearable’ and adaptive behaviors. Investigating at the same time
installation, capable of forming a smaller cybernetic loop
concepts such as connectionism, reactivity, behavioral
between the augmented man/machine entity and the human
observers, resulting in emergent behavioral tactics and tactics (local and emergent) and immersive environments,
formations. in order to produce aesthetics using the machine medium.
During this quest there are some basic questions that
I.INTRODUCTION arise, while, providing answers to these issues, formulates
Robotic art seems an emerging domain where research very specific design strategies and architectures. One of
and practice are connected, created from a mixture of these questions has to do with the elements that make a
several fields and disciplines such as electronics, machine diversify from its own function and form,
programming, mechanics. All these disciplines combined attaining at the same time characteristics similar to those
with some by definition, immersive media, like sound of a living being.
and light, facilitate the creation of environments capable Is it a mimetic approach to form or functionality of the
of expressing artistic purpose. The sensorial stimulation structure (anthropomorphic form, familiar characteristics,
of evolutionary convergence of human perception, speech, humanoid movement etc.)? Is it a more abstract
facilitated from the use of diverse means and evolving approach like intelligence and behavior? And if is
animated forms and functions, support the merging of the possible to approach a simulation of a living organism by
human agents with the machine environment into an formulating abstract characteristics and impose it on a
evolved ego-space creature, forming in larger scale, a structure, how is it possible for these characteristics to
“social”/systemic network. Conditioning the network become complex enough to really refer to an organic
renders it able to express artistic purposes and at the same living entity? Finally even if this happens, are the above
time raising and process/elaborate fundamental arguments enough to ensure that the resulting outcome is
sociological issues/questions dealing with the possible to facilitate meaningful artistic purpose?
man/machine relationship and the evolutionary race of
human beings in the two last centuries. Supporting in this way the term ‘theatrical’ (this term
is borrowed from artist/researcher Bill Vorn), referring to
machine/robotic agents integrating multimedia objects to machine/man creature and the environment, including the
their functionalities (metabolic processes). rest of the human agents being present, on another.
II. DESIGN PROCESS Given the fact that this augmentation was implemented
by the technological extension/intervention, it was
The design process was based on two parallel sensible to consider the relationship between human
procedures. One being an aesthetic approach to the actual beings in a sociological manner and the relationship each
appearance and form of a particular machine agent, human being develops (self image) towards its self,
proposed by artist and Medea Electronique founding through technological advancements and means. The
member Christos Laskaris, the other being a continuous projection of the latter in the social boundaries and
dynamic collective process taking place in Medea interactions, affects the processes taking place in the
Electronique’s regular brainstorming meetings, with the former. Thus the name EgoJection was formed,
purpose of answering the questions mentioned above. suggesting a network of technological enhanced
Christos came with the idea of a short of “wearable” organisms developing interactions inside specific artistic
installation, initially to be suspended from the ceiling, context.
functioning as a ‘head projectile’ for human agents. In This relationship made obvious the presence of
this way an intimate, peculiar, connection between the possible emergent phenomena concerning the
environment/space and the human user was established. functionality of a sole machine itself. At this stage it was
also realized that however complex the movement and
functionality of one man/machine entity would be,
additional machine agents would have to be added, in
order to create a complex system enabling the research
team to analyze and control emergent behaviors, thus
creating a unpredictable but still at some level
controllable, immersive environment.
Taking into account the above discussion two different
scenarios were discussed, these scenarios are described
along with the current status of the first machine agent
created for presentation at Electromedia Works
(emw’08) festival, at the User Interaction chapter. The
next chapter describes the actual structure and the
technology used for its implementation.
Figure 1. Initial drawing
This first machine agent is a simple, interactive,
robotic structure, comprised from a sensing system, an
actuating system and a control system, functioning in
essence as an autonomous nervous system. The structure
uses video, sound and movement (current version one
degree of freedom only), to express its own status and its
understanding of the surrounding environment status.

It displays its function through two different states,

depending on whether the cybernetic loop is closed,
through the presence of a human agent at its main input
The main components of the structure are:
A.Sensing Subsystem
Figure 2. Initial drawing evolved State control of the installation is achieved through
processing the information of the sensing device in front
The above photos show the initial ‘blueprints’. The of the ‘cockpit’ of the machine. Presence or not of a
final form of the installation is very similar to these participant is detected and the desired control state is
proposals. initiated.
Having these initial ideas to work with it was obvious Sensing devices are also placed on the body of the
that the form of the installation itself proposed a installation giving it the ability of determine whether
modified, augmented relationship between the user and there are potential users in the perimeter around it.
the physical space in one level and the augmented
The boards are connected on a local Ethernet network
with a 2 GHz Intel double core MacBook, running Mac
OS X version 10.4.8, controlling the actuating devices
and processing sensorial inputs, a desktop pc running
Windows Xp controlling and synchronizing the state of
the machine with the video output, and a macmini
controlling the sound parameters. This network of
computers is used for parallel processing of information
and control of the different subsystems simultaneously
and in real time. It is the global control of the installation
and able to process information coming from a system
with much bigger complexity.
C.Actuating Subsystem
The basic actuator of the machine is a 1Hp AC motor
Figure 3. Sensors arrangment made by Bonfiglioli mounted on the main structure with
handmade steel/copper mechanical joints.
In total 8 infrared sensors are used in the current An AC motor was used to provide sufficient direction
version. and speed control and still being able to handle heavy
loads and provide smooth silent movement. Other
technologies like servo or stepper motors were not used
B.Control/communication Subsystem since accuracy of positioning was not of an issue for this

D.Sound sources
As sound sources 4 loudspeakers (passive) are used
two inside the main body of the machine and 2 outside.
The pairs of speakers do not perform simultaneously, but
in relation to the current state of the system. This
arrangement is implemented using the Kyma sound
Figure 4. Microcontroller board processing platform.
E.Electrical Panel
Communication between the computer and all the
An electrical panel is used to convert PWM signals
different sensors and actuators was implemented using a
and digital signals, from the board’s digital outputs, to a
Make Controller board. The Make Controller
drive signal controlling an AC, 1Hp, Bonfiglioli motor.
The main component is a Telemecaique Altivar 31
CTRL/) is built around an Atmel AVR family architecture
inverter, processing the signals coming from the
RISC Flash memory microcontroller, the
MakingThings board.
AT91SAM7X256. The controller’s board provides
control, computation, along with CAN and Ethernet

Figure 5. Analog inputs digital outputs

The application board supports connectivity through

USB, CAN interface and Ethernet. It provides 8 analog
inputs, 8 digital outputs and 4 servo outputs. These input Figure 6. Adding components to the electrical panel the last minute is
and output ports are used to read values from the sensors always a drag
and control all the actuating devices.
In general the installation’s control panel (electrical Electronique members, using milling machines and metal
panel along with the microcontroller board) is a ‘multi- process equipment.
converter’ of OSC and MIDI messages to voltages
capable of operating different electrical, pneumatic,
mechanical components (motors, solenoids, pneumatic
pistons etc.) and assuring electrical safety during the
F.Main Wooden Frame and Mechanical Joints
The body of the installation is made of wood and
paper. Stainless steel joints have been used in order to
transmit the motion of the actuator to the rest of the
structure. The front part of the installation is a projection
screen, made from a PVC panel.
A wooden boxlike frame, 2700x1400x1200 mm, has
been constructed to shape the structure and to support the
mounted projection and camera. Paper cartons have been
used to seal the frame to provide an immersive, solitary Figure 9. Assembling the steel structure
space for the main participant.

Figure 7. Wooden frame

Figure 10. Frame, steel joints, actuator

Figure 8. Wooden frame sideways

Figure 11. Dressed with paper carton

The whole frame is supported by stainless steel leg-

like structures and joined using steel and copper
components allowing it to move in the vertical direction.
All the mechanics and frames were tailor made by Medea
A web cam mounted on the frame and a video
projector also mounted inside the structure, provided the
footage and allowed a short of communication to the
environment of the ‘internal’ structure’s processes.

Interfaces between the sensors and actuators and the
microcontroller interface device (Making Controller)
have been developed for the following systems:
1.Processing: Java based programming environment
for real time animated graphics synthesis and
information processing. http://www.processing.org
2.Kyma: Graphic programming environment for real-
time sound synthesis and control.
3.VVVV: Real time modular/nodal programming
Figure 12. Stainless steel kinematic system environment for video and animated graphics
processing. http://www.vvvv.org
These interfaces are implemented on the basis of a
RTOS (Real Time Operating System). This software
activates all the useful hardware components for
computation and communication. The firmware is
programmed in C programming language and includes
files to:
­Provide network functionality
­Provide basic operating system functions
­Provide OSC and USB functionality
Protocols and functions supported on the firmware are:
­TCP/IP, used to provide network operations over the
Figure 13. Ready for test internet. Both UDP and TCP are supported
­OSC, is used as an option to connect all the
subsystems of the board
G.Video/Projection subsystem
A video screen in the front of the construction, was ­Controller library functions, giving access to the
used to project live video input from the installation’s controllers subsystems, for example the CAN interface or
‘cockpit’, seen both from the user ‘inside’ the installation PWM devices
and those from the outside. ­Application Board library functions, providing
access to additional subsystems for hardware control. For
example the AnalogIn() functions help read any of the 8
inputs or the Motor() functions provide DC motor
The RTOS uploaded on the microcontroller
communicates with Processing. Processing performs all
the parameter computations and controls the flow of
information between the software and hardware
The Processing engine is also used to create a simple
virtual model of the structure and its sensing devices,
providing visual feedback in the virtual plane on the
status of these devices and in effect the status of the
Figure 14. ‘Cockpit’ with mounted projector and camera
interaction between human and machine agents. This is
achieved by a simple 2d representation of the structure
and sensing devices range, along with their state (active human-agents around it. The essential information are
or not), using a color code simple system to represent it provided from the seven sensors attached to its perimeter.
(red color equals active area equals presence in that
space). This allows control of the machine’s behavior
according to environmental input information processing.

Figure 16. Transition from active to idle state

In the second state the machine ”recognizes” an active

participant in front of its predefined drive position, and
starts performing its routine, always affected from the
Figure 15. Simple 2d virtual status representation diagram information of human presence around it. This
information, is given by the sensor attached at the
‘cockpit’, which is the predefined drive position.
A uniform protocol was used on the basis of UDP the
OSC (Open Sound Control) protocol. This architecture Multimedia themes are presented in accordance to the
gives a uniform “transparency” to the different software machine-man interaction status supporting the ongoing
and hardware platforms. performance.
OSC messages are exchanged throughout the networks
components and platforms allowing the parallel
processing of information and essentially unified control.
Messages are exchanged from the microcontroller board
to Processing, from Processing to VVVV, from VVVV to
Kyma and the microcontroller board, controlling the state
of the installation, in a continuous mode.
The development goals were directed towards creating
an autonomous robotic agent, still able to perform and
function as a subcomponent inside a superimposed more
complex system if necessary. Since a system as such is
yet to be implemented, the characteristics of a single Figure 17. Active state
agent are going to be presented in this chapter.

A. Behavior:
The behavioral tactics of the machine-entity depends
mainly on the environmental information provided by the
sensing devices all around it. There are two main states,
including several sub-states in each one, reflecting the
basic behavior of the system towards its human
The first is an idle state, where there is no human
agent, completing the cybernetic loop with the machine,
in front of its “drive” position. In this state the machine
performs ‘by itself’, inviting the participants to join. It
shows elements of “life” through periodical animation of Figure 18. Active state multimedia
its structure, always depending on the presence or not of
Additional software development and investigation of
communication protocols, allowing the remote exchange
of data (music, video) is required. The project presented
is the first in a series of installations employing the same
structural design elements able to form a larger scale,
more complex systems presenting emergent properties
and characteristics. Larger networks are going to enable
us to explore swarm networking between remote places,
connected through Internet, exchanging cultural and
environmental elements from different
For the completion of different installations the
Figure 19. Transitions between active substates utilization of various OSC to voltage converters is
necessary, providing flexibility during the design process.
It is also considered essential the development of
different sensor devices, for the implementation of
different interaction scenarios. Diverse designs and
technologies allow the implementation of sensors,
varying from noticeable, personal, wearable, (single
agent monitoring), to invisible global ones, for crowd
behavior and environmental monitoring.
Many thanks to “Constrinox SA”, D.Giannoukakis and
A. Giannoukakis for providing means and materials used
in the installation, D.Tomaras and G.Tsotsos, helping in
the realization of the installation with their experience in
mechanical engineering and electronics.
Figure 20. P.Tsangarakis programming vvvv
Assigning character to the machine agent is of course
an illusive attribute implemented through the
connection/relation between the agents and the
performance of each subsystem. It is the artificial
organism’s performance, utilized with machine motion
and producing sound and light while functioning, that
generates the notion of behavioral structures.
The resultant behaviors in a system of machine/man
organisms as such, can vary, from peaceful to aggressive,
linear to chaotic, attractive to repulsive, individual to
flocking, according to the systems emergent
Appropriate design and architecture, even in small
scale interaction systems can allow the creation of
complex, meta-systems able to perform structured,
identifiable behavioral strategies, through simple
interactions between the system components.
Permitting and conditioning agents to operate in
unison, allows the system to generate an integrated
output. There are key points in architecture morphology
of such systems, to facilitate multi-agent interaction. The
focus while designing these systems is not their
appearance, but the flow and processing of information.
Conditioning the unified sub-systems enables us to
express specific design and functional scenarios,
including the pursuit of valid artistic purpose.

Похожие интересы