Echology: An Interactive Spatial Sound and Video Artwork

Meghan Deutscher1, Reynald Hoskinson1, Sachiyo Takashashi2 and Sidney Fels1,2

1 2
Dept. of ECE, UBC Media and Graphics Interdisciplinary
Vancouver, BC, Canada Centre(MAGIC), UBC
+1 604 822-5338 Vancouver, BC, Canada
{deutscher, hoskinson, ssfels}@ece.ubc.ca +1 604 822-8990
taksaci@telus.net

ABSTRACT provides an effective means to bring the visual elements of the

We present a novel way of manipulating a spatial soundscape, one
that encourages collaboration and exploration. Through a table-top
display surrounded by speakers and lights, participants are invited to
engage in peaceful play with Beluga whales shown through a live
web camera feed from the Vancouver Aquarium in Canada. Eight
softly glowing buttons and a simple interface encourage
collaboration with others who are also enjoying the swirling Beluga
sounds overhead.
Beluga whales into the piece as well as driving the sound space
using video processing. The amount of motion resulting from
processing the video is mapped to the amplitudes of several sound
sources. The sounds of the Beluga whales, recorded over 3 years
using underwater microphones, are remixed to provide the base
sounds that are spatialized interactively. Enjoyment, understanding
and engagement are key elements of the expressive meaning of the
interactive installation.

Categories and Subject Descriptors

J.5 ARTS AND HUMANITIES

General Terms
Design, Experimentation, Human Factors

Keywords
Mediascape, sound spatialization, interactive art, Beluga whale

1. INTRODUCTION Beluga Webcam Interaction Space

Concept
Artists Rendition of
Atrium Space
The playful, graceful motions of Beluga whales swimming in water
create a mesmerizing motion space to watch. These highly
communicative creatures use a variety of vocalizations, physical Figure 1: Beluga webcam, sketch of interaction
expressions and physical contact in their navigation, social space, atrium for installation
interaction and survival. Beluga whales have fatty structures on the
top of their heads; it is hypothesized that these structures, called
While our table-top interactive installation was created as an
melons, act as acoustical lenses for focusing and directing sounds
artwork, the exploration of the table-top interaction space
for echolocation. We have created an interactive sound and video
contributes to the understanding of how people work together and
installation for participants to play with directional sounds initiated
communicate in a shared interactive space. We reflect upon our
by the playful movement of Beluga whales in the water captured
observations about how our design decisions facilitated social
with a single live webcam provided by the Vancouver Aquarium
interaction using imagery linked to a spatial sound space. We
(www.vanaqua.org/belugacam/index.html). The piece allows
noticed that people developed a sense of ownership of part of the
participants to experience a sonic aquarium while feeling linked to
shared interface when they were standing around it. They used this
the live, organic and mesmerizing movements of the Beluga whales.
territory to collaborate with other people around the installation to
Our artistic motivations stem from creating an interactive
play with the sounds. The blend of nature-oriented video and
soundscape that provides a representation of the Beluga’s space
spatialized sound used in the installation helped people standing
they inhabit. Figure 1 depicts our initial concept of bringing the
around the exhibit to overcome awkward moments when there was
Belugas into an atrium space to create an interactive soundscape.
silence during discussion around the table. The awkward moments
The single webcam feed provided by the Vancouver Aquarium
were avoided by providing an unobtrusive, pleasing media
experience when participants focus attention on the work, while not
Permission to make digital or hard copies of all or part of this work for being distracting while they are in conversation. These observations
personal or classroom use is granted without fee provided that copies are are related to research in human-computer interaction (HCI) and
not made or distributed for profit or commercial advantage and that copies computer-supported cooperative work (CSCW) using shared
bear this notice and the full citation on the first page. To copy otherwise, or displays.
republish, to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee. We discuss our work in the context of other table-top interactive
MM’05, November 6–11, 2005, Singapore. works before describing details of how the installation works. We
Copyright 2005 ACM 1-59593-044-2/05/0011…$5.00.

937
exhibited the work from May 26-28, 2005 at the New Interfaces for
Musical Expression interactive installation program (NIME’05,
www.nime.org) and report on our observations of participants’
experiences.
2. RELATED WORK
Our interactive installation uses a combination of a table-top
interaction environment, web camera and spatialized musical
sounds as its main technical elements and whales as our main
theme. Each element by itself has formed the basis of other
interactive art installations, however, the integration of the different
media in our piece provides a unique expressive experience.
Artworks based on wildlife are very popular in traditional forms
such as painting, sculpture, music, movies, computer animation and
performance. Human’s attraction to animals is exploited in
computer interfaces such as Microsoft’s Bob, the dog icon used in
Softwork’s Fetch program and many games such as Nintendo’s
Nintendog™ (http://www.nintendog.com/) and Sunstorm
Interactive’s Deer Hunter™. Interactive artworks that use wildlife
forms are still relatively new, but have appeared in works such as
Alpha Wolf [1] and A-Volve [2]. In Alpha Wolf, participants interact
with a synthetic wolf that is simulated using an artificial intelligence
engine. The representations and processes involved in the
formation of hierarchical social structures provides the basis of the
interactive experience. A-Volve uses the simulation of genetic
processes to drive the simulation of swimming creatures who live
Figure 2: System Diagram showing data flow and the relationship between components.
938
and die in a virtual fish-tank. Participants create the genetic codes
that are converted to a unique creature.
Table-top interaction surfaces have formed the basis of interactive
artworks. Toshio Iwai’s Composition on the Table [3] provided
inspiration artistically and technically for the sound and interaction
method that we use. Iwai’s work uses a large horizontal projection
surface with a light grid on the display. At each node of the grid a
button allows the player to change the direction of the arrow
associated with the node. A colored ball of light moves along the
lines of the grid and then follows a path decided upon by the
direction of the arrow at each node it encounters. When a node is
hit, a specific MIDI note plays. There are four lights moving at
different speeds. Thus, with careful selection of the direction of the
arrows, complex loops and rhythms can be created. The table is
large enough for multiple people to change the patterns. Working
with multiple players allows very complex sounds that cannot be
made as effectively by a single person, since there are too many
lights active at the same time. In Echology, we use four circular
graphics to represent four different sound tracks that are reflecting
spatially when they hit a node rather than triggering a MIDI note, as
described below. Furthermore, the movements of Beluga whales
control our sounds. Other table-top sound installations include:
Augmented Groove [4], Jamodrum [5], and Jamoworld [6].
Blaine and Fels [7] have identified many of the qualities to organize
multi-person interactive sound installations and guidelines based on
these qualities to make the sound interaction successful.
Accordingly, we place our work in their classification system along
with integrating their guidelines into the design of the work. The
relevant qualities and the recommendations we followed in our
design (from [7]) are:
• capacity – multiplayer, single interface; use turn-taking
protocols as well as having clear relationships between
action and sound with a multi-person, single interface.
• aptitude – novice; prioritize that the sound be engaging at
the expense of virtuosity.
• media – sound and video; use video to strengthen the
relationship between action and sound without being too
distracting.
• player interaction – participants use 8 buttons arranged in
a circle to reflect sounds; each player has the same type of
interaction so that they can learn from each other.
• musical range – players control spatial parameters of
music; restrict the set of sound controls to help create an
engaging, satisfying experience for novices.
• physical interface – buttons; provide Norman-style
affordances [8] to engage novices and make it easy for
them to join the group.
• directed interaction – low; attendants explicitly direct
novices so they understand how the interaction works.
• learning curve – fast; use a direct mapping between
gesture and sound to speed up learning.
• path to expert – no; compromise virtuosity for ease-of-use
as the sound installation is intended for novices.
• levels of physicality – high; encourage high levels of
physical interaction that “lays the foundation for
developing intimate personal connections with other
players and their instruments over relatively short periods
of time, which can also lead to a sense of community.”
Spatial sound has been the topic of much research and commercial
application. If not using headphones, two speakers (stereo) can be
used to provide a sound source that moves along a path between the
two speakers through adjusting the sound mix levels. This has been
extended to quadraphonic (4 speakers) and octophonic systems (8
speakers). Dolby 5.1™ surround sound systems use 5 mid-range
speakers and a sub-woofer to provide spatial sound effects. It takes
advantage of the fact that human perception does not localize bass
sounds very well so uses only one sub-woofer. The mid-range
speakers are spatially arranged around the listerner. Works such as
InTheMix [8] have used virtual spatial sounds in their work based
on techniques such as the Crystal River Engineering’s
Convolvotron™. These works are well suited for single person
sound spatialization in virtual environments. de Nijs and van der
Heide [10] present a unique way to incorporate spatial sound in
their work. They use a speaker on a counter-balanced arm to
physically move the sound source around at different speeds so that
participants see and hear the sound moving. Echology uses spatial
sound as one of its core elements also. We use an octophonic
spatial sound system that allows all participants to experience the
movement of sounds in a plane parallel to the floor formed above
their heads. As Echology does not have a strong bass component,
we do not use a subwoofer as would be done with a Dolby 5.1™
surround sound system.
939
3. THE ECHOLOGY SYSTEM
The installation consists of an interaction table, 8 surround speakers,
a PC running Max/MSP/Jitter [11], 8 lights and a light controller, a
live webcam at the Vancouver Aquarium, and the atrium projection
space. The system diagram in Figure 2 shows the paths between
inputs, the Max/MSP/Jitter software modules, and outputs.
Input to the Echology system consists of the Vancouver Aquarium
webcam [12] that is maintained by the aquarium, and the 8
interaction table buttons. These large arcade style buttons represent
the reflection points on the edge of the soundscape (i.e. the eight
loudspeakers). The 8 buttons are arranged symmetrically around the
perimeter of the table as shown in Figure 7. These buttons are used
to control the movement of the sounds in the sonic aquarium.
The interaction table also houses a monitor to display our
visualization of the Beluga webcam and sound space for the
participants. The display gives visual feedback to the participants
so they can see where the sounds are and where they will go. This
same display is also projected on an overhead rear projection screen
mounted in the truss for the spectators. The screen can be seen in at
the top of Figure 3. The sound space is created using 8 speakers
distributed symmetrically just above head level. Eight lights with
blue filters are mounted below each speaker and light up
momentarily to indicate when a sound passes by a speaker to
enhance the spatialized sound.
All controls between the inputs and outputs are written in the
Max/MSP/Jitter graphical programming language by Cycling ’74
[11]. Below is a brief overview of the software components:
• The video and motion module processes the live webcam feed.
Motions in the webcam view are mapped to the amplitude of 4
sound sources managed by the sound controller.
• The sound controller sends 4 sound signals to the panner object
for output to the 8 speakers.
• The panner can take up to 8 sound signals as inputs. It
spatializes the sound by controlling the amplitudes of 8 output
channels connected to the speakers. Our spatialization model is
of a two-dimensional plane surrounded by 8 “reflection points”.
Sound signals (represented by coloured circles in our model)
can travel a path from one reflection point to any other dictated
by the buttons. The panning controller manages the position of
each sound signal in the two-dimensional space. The
coordinates are sent to the panner to do the spatialization.
• The paths of the sounds in the sound space are determined by
the directions of the reflection points. The direction of
reflection from any reflection point can be to any other of the 7
points, or to itself. In the latter case the sound doesn’t move.
The interaction table buttons control the directions of reflection
points. Button presses are monitored by the panning controller
through a data acquisition interface called a phidget [13]
discussed below.
• The panning controller also switches the 8 lights on and off via
the phidgets interface.
• The graphics controller generates the output display. It blends
the live webcam feed with iconic representations of the
reflection points and coloured circles for the sounds.

Figure 3: A photo taken from above of the
Echology installation as presented for NIME
2005. The overhead truss has 8 speakers attached
and a rear projection screen. The participants
are standing around the white interaction table.
Echology was produced as an interactive multimedia installation
piece for the Open Media Environment (OME) situated in the
atrium of a building at the University of British Columbia. The
OME incorporates a center stage area, a movable, suspended,
circular theatrical truss that has eight attached speakers and can
support various lighting attachments. The circular truss has a rear
projection screen stretched inside it. The complete installation is
shown in Figure 3.
The following sections discuss the component details as they were
implemented for the OME installation of Echology.
Beluga Video and Motion Control
The Vancouver Aquarium has six Belugas in the tank shown
through the webcam. From the webcam, a variety of different
Beluga activities can be observed. They have various swimming
patterns, sometimes play with bubble rings, do headstands, bob their
tails up in the air and perform trained movements during the
aquarium show times.
The Echology installation uses Jitter to process this live webcam
feed. Motions of whales change given the different areas of the
camera’s view. For example, the top portion of the view is of the
Beluga tank surface. The Belugas spend most of their time near this
surface and are often bobbing around in one place; their activity is
particularly high during the training shows that occur every 1.5 to 2
hours during opening hours. We used this idea of different areas to
divide the view into four “layers”, the motion detected in each
controls one sound signal. These four layers are evenly spaced
from top to bottom as shown in Figure 4. The layers were given
labels of Splash, Play, Swim, and Deep for layers 1 to 4
respectively. The 4 sound signals associated with these four layers
are composed to be aesthetically and conceptually characteristic of
Beluga motions within the given layer. For example, we have a
playful motif for the sound associated with the Play layer. The
nature of the sounds will be discussed further in the section on
Echology’s musical elements.
Beluga motion in each layer is mapped to the amplitude of the
associated sound signals. Motion is calculated by subtracting
grayscale pixel values in consecutive frames of the Beluga webcam
feed. If the difference between 2 frames is greater than a threshold
value, the intensity of motion (or magnitude of difference) is used to
940
modify the sound’s amplitude. Motion values are averaged over a
sliding window of 10 frames; the averages are then normalized and
sent to the sound controller.
SPLASH
PLAY
SWIM
DEEP
Figure 4: Beluga webcam view catching a whale swimming
close to the camera divided into 4 horizontal slices or “layers”
(close up image courtesy of Vancouver Aquarium website).
Sound Controller
The sound controller manages the Echology audio media. It uses
the MSP portion of the Cycling environment to open, play and loop
prerecorded samples.
The amplitudes of the 4 sound signals are controlled by the
normalized motion values, and the resulting audio signals are sent to
the panner for output. Often the motion of Belugas in the webcam
view is only for a short duration of time. This makes sounds
disappear too quickly to interact with after entering the soundscape.
To remedy this we add a 10 second sustain to keep the sounds
playing after they pass a threshold volume level.
Musical Elements
As already discussed, whale activity captured with the webcam
coupled with the participants’ interaction directs a spatialized
soundscape. To enrich this soundscape musically, we carefully
selected various sound sources, including actual Beluga sounds and
composed segments. Although each sound source had to be
elementally simple enough to achieve successful interaction and
spatial perception, we also wanted the soundscape to be rich enough
to give the audience the feel of playing with the Belugas underwater.
We use two categories of sounds in Echology. The first consists of
short samples of Beluga whale voices received from the Vancouver
aquarium research team. The second category is a number of
composed synthesizer tracks meant to create an aquatic atmosphere,
and provide continuity and context to the soundscape.
The sound signals of the four layers in Echology's soundscape are as
follows:
1. Splash contains a synthesizer motif creating an aquatic
atmosphere when a whale moves across the top of the
aquarium.
2. Play loops several short samples of Beluga voices that are
randomly triggered by the Beluga movements.
3. Swim consists of additional Beluga voice samples
accompanied with the drone of synthesizer sound.
4. Deep contains low frequency mass to create a drone.
We selected high frequency Beluga voices since they provide better
cues for spatial perception. Atmospheric background synthesizer
sounds are designed to give continuity to the entire soundscape. In
general, there is enough variation in the soundscape to be engaging
and interesting because of the variety of motion of whales in the
aquarium.
Spatialized Sound
The four sound signals are spatialized around the installation space
using a spatial sound engine based on the Max/MSP plugin for
Vector-Based Amplitude Panning (VBAP), developed by Ville
Pukki [15][16]. VBAP provides sound spatialization using an
arbitrary specification of speaker position in two or three
dimensions. This is an essential feature for installations such as
ours that require a high degree of control over sound positions, and
are set up in disparate environments that we have little advance
control over.
Amplitude panning involves sending the same sound signal to a
number of different loudspeakers, with amplitudes based on the
loudspeaker positions and the location of the intended sound signal.
Using VBAP allows us to easily fit the loudspeaker setup to the
space our installation will inhabit. With the VBAP plugin, the
speaker placements can be changed on the fly in a modular fashion
that does not affect any of our other software components.
B C
A D
H E
G F
Figure 5: Sound signals travelling between reflection
points A, C, and E. Pressing the buttons rotates the
reflection points to direct the sounds to other locations.
This method of spatialization is used in the 8-channel panner that
we use. The panner accepts up to eight channels of input sound
signals that can each exist in a unique soundscape position at the
same time. We use four of these, one for each sound signal. The
panner accepts x and y coordinates for each sound signal.
The panning controller calculates and maintains the coordinates for
each of the sound signals. These coordinates are sent to the panner,
as well as to the graphics and lighting controllers discussed below.
The panning controller also accepts button press inputs from the
interaction table via a phidgets interface. When a participant
presses a button on the interaction table, the panning controller
adjusts the direction of reflection for the associated reflection point.
The new direction is shown on the display by the graphics
controller.
For example, consider the configuration of the reflection points as
shown in Figure 5. When the sound hits reflection point A it
reflects towards reflection point C. When it hits reflection point C,
it reflects to point E. When it reaches point E, it returns to point A.
941
It will continue in this triangle pattern until a participant presses one
of the buttons associated with reflection points A, C or E which will
change the direction of the reflection.
Graphics / Interaction
Graphical visualization of the reflection directions and sound signal
paths are created as feedback for participants. The graphics are
intentionally kept simple so as not to detract from the live Beluga
webcam feed in the background. All graphics are generated with
Jitter’s OpenGL capabilities.
Four circles of different colours move about the interaction table
display. Each circle directly represents the location and amplitude
of the sound signal associated with a “layer”. The colour mapping
is as follows: Splash is red, Play is yellow, Swim is orange and
Deep is green. The amplitude is mapped onto the alpha value of the
circles, so that they fade in and out of the background as the level of
activity of the Belugas increases or decreases in the corresponding
layer of the tank. Figure 6 shows a view of the table display with all
4 sound channels at peak amplitude
Figure 6: Screen capture of interaction screen, showing
the sounds (circles) orbiting the reflection points (pies),
and a Beluga whale in the background.
Eight blue circles resembling a pie with a slice removed represent
the reflection points. The sound reflects in the direction of the pie
slice. Each press of the associated button rotates the pie slice so that
the sound reflects to the next point. For example, in figure 5,
pressing the button associated with point A changes the sound
direction from going to point C to point D.
Reflection points cycle through 8 positions, the default scenario
being each pointing to its neighbouring reflection point so that
sounds travel in a circle around the soundspace. Each press of the
button rotates the reflection point direction by one in a counter-
clockwise direction including pointing to itself. When this happens,
the pie becomes a torus, indicating that it will now ‘catch’ sounds
that hit it. To release the sounds, the participant hits the button
again, letting caught sounds go on their path towards the next
reflection point.
The blue lights suspended from the truss turn on when a sound hits
its associated reflection point. We use the phidget interface to
control the lights as described below.

Figure 7: The interaction table, showing multiple
participants jointly creating the sound-scape.
We placed the buttons, speakers, suspended lights and reflection
points on radially collinear axes distributed symmetrically around
the space centred on the table. This placement makes it easy for
participants to know where a sound is and where it will be reflected.
Interaction Table
The functions of the interaction table buttons and display have been
discussed in the previous sections, but it is important to quickly note
the design of the table itself. In [14] the importance of the
aesthetics of technological art installations is stressed. No matter
how novel the underlying technology, appearance is paramount.
Aesthetics aid in drawing people into participating with the piece.
Also, installations are set in public places, so surroundings must be
taken into account during aesthetics design.
As shown in Figure 3 and 7, the table-top is circular: allowing a
larger number of participants to comfortably stand around it and see
what is going on. The large glowing buttons were chosen to draw
people towards the table and entice pressing. The buttons,
purchased from Happ Controls Inc., are light sea blue, 3 inches in
diameter and illuminated inside with a small DC lamp. When a
button is pressed, the light turns off to provide immediate, direct,
visual feedback. The large buttons also fulfilled the design
requirement of having an interface robust enough to withstand
sustained use by the general public. We chose to cover the table
base and top in a fluffy white fabric to appear white like a Beluga’s
skin. The table-top itself has a clear plastic cover to feel like a
Beluga’s skin and protect the monitor surface. The table-top sits on
a metal barrel. The Phidget interface boards and cabling are inside
the barrel to protect them and hide them from view.
Phidgets Interface
A Phidget interface [13] is used to detect button presses and to send
output to the lighting controller. Phidgets are inexpensive digital
acquisition (DAQ) boards that provide simple mechanisms to
connect and control various sensors and actuators to a computer via
USB. The Phidgets 8/8/8 board allows for 8 analog inputs and 8
analog outputs.
To access these input and output channels, a Max/MSP external
Phidget object was written. A button press causes the object to send
a signal in the panner controller software module, changing the
942
direction of a reflection point. This same software module sends a
signal to the Phidget outputs to turn on lights when a sound hits a
reflection point.
Lighting
The Phidget interface output is connected to a lighting controller
board that allows eight 110V light bulb sockets to be switched
under computer control. The lighting controller is made from a
Light-o-Rama 8 Channel Triac Board (MC-TB08).
4. INTERACTION
Participants can play with the spatialized sounds initiated by Beluga
whale motion. The Beluga web feed appears in the background of
the graphic display on the table as well as on the video projection
overhead. Using the buttons, participants control the path of the
sound overhead, or if desired, they can just watch and listen to the
sounds move along their predetermined course. The melon of the
Beluga (the fleshy part on the top of the head) inspires this use of
redirected sound. We intentionally made the interaction simple, so
that participants need only focus on the spatial aspects of the sound
rather than the tonal qualities. The imagery of Beluga whales
swimming blends with the sound spatialization and visualization to
make a rich, playful and enjoyable mediascape. The whales’ play
results in movement patterns that are fairly repeatable, but
unpredictable and organic. The audience member (or participant)
can listen and watch the mediascape on the overhead speaker truss
and video projection. However, the “sweet-spot” is at the
interaction table located in the centre of the raised platform of the
atrium.
Using a webcam feed of live Belugas raises interesting ethical
questions we have tried to address in our piece. Though they are in
an aquarium environment, the Belugas are wild animals that cannot
fully be controlled. They are there for us to learn from them, but
demand proper respect. For this reason, we gave participants only
partial control over the sounds. Sounds fade in and out according to
the activity of the Belugas in each layer; if there is no Beluga
activity at all for a certain amount of time, the installation will
become silent until the Belugas return. Only when there is a
significant level of activity in all layers will all the sounds be
present simultaneously. This aspect of the interaction encourages
participants to reflect on the Beluga whales as living creatures seen
in real time. As such, participants are not able to demand
performance from the whales, but must patiently wait for them to
swim into view.
Experience Goals
There were three main experience goals we kept in mind when
designing Echology: simplicity, approachability, and collaboration.
Mimicking the gentle nature of Beluga whales, interaction with
Echology was designed to be simple and playful. For this reason,
we attempted to make the mapping between the Beluga webcam
video, soundscape, and control through the interaction table as
straightforward as possible. Each button only controls one
reflection direction, and each reflection is only in one of 8 states.
This makes the control of sound spatialization simple and
predictable.
To create an inviting, approachable space, we constructed the
interaction table to appear fun and exciting. We did this through
our choice of covering it in a fluffy white fabric and using large,
lighted, colorful buttons to entice spectators over to the table . Our
design includes a projection of the Beluga webcam feed so that
those not at the table could observe and appreciate the mapping
between Belugas and sound.
The last experience goal of our piece was to allow for collaboration
among participants at the interaction table. Our circular table-top is
large enough so that up to 8 people may comfortably stand around
the table at a time. Given the large table size, it is difficult for one
person to control all of the buttons at once because of the distance
they must reach over the table. Complex control over the movement
of the sounds is only achievable from the coordinated efforts of
multiple participants.
5. OBSERVATIONS
The installation was exhibited for the first time in a public setting
during the 5th International Conference on New Interfaces for
Musical Expression (NIME05) in Vancouver, Canada. It was
running for one day, and during that time we received a substantial
amount of feedback from conference attendees and members of the
general public. More than one hundred people were observed
interacting with our exhibit.
Our observations reflect the goals presented in the previous section
and how well they were accomplished. Many of our observations
are also found in other collaborative musical experiences [7].
We achieved our goal of approachability by creating an inviting
space. Participants would immediately come to the interaction table
upon walking into the installation space. They did not hesitate to try
pressing the buttons.
When the Belugas were inactive, people who had not yet
experienced the soundscape would wait around patiently. It was
quite similar to a guest at the aquarium quietly waiting for an
aquatic creature to swim into a more visible area.
Moments of silence when there were no whales did not feel
awkward amongst participants at the table. Participants were
comfortable being silent for a few minutes or spent time conversing
about the Belugas to easily break the silence when desired. This
allowed some participants to stand around the table and chat for
lengths of time up to 20 or 30 minutes. Thus, the piece works very
well as a “calm” technology [20].
For some, the connection between the Beluga and the mode of
interaction was not well understood. The abstraction between how
Belugas communicate, their actions, and what participants could do
was not obvious to them. Usually an attendant had to explicitly
state that the Beluga webcam images were indeed live and not
prerecorded. It is difficult to determine the importance of the
audience understanding that the Belugas are live and how much that
understanding changes the metaphor. Because of the live nature of
the Beluga camera, some people wanted to take the next step – to be
able to communicate directly with the Belugas. Many people asked,
“Can the Belugas hear the music too?”
There was a prominent pattern for a participant’s visit in the
installation space. The participant walked up to the table, glanced at
the display, and then looked at the overhead lights and speakers. He
or she would then focus on the display and either press a button to
receive a reaction, or watch what other people were doing. Most
people asked questions about the installation, how it worked, how it
was made, etc. After discussing the installation itself, the
conversation often opened up to other topics, other times
participants would spend a few quiet moments by themselves,
enjoying the Belugas and soundscape.
943
6. DISCUSSION
Some of the most interesting observations made in the installation
space were of the behaviour between the people around the table. A
substantial amount of literature has been developed in the area of
table-top displays for computer-supported collaborative work, such
as [17,18]. This research sheds light on some of our observations of
the people interacting with each other while experiencing the work.
Researchers have consistently seen that the area immediately in
front of a participant is naturally considered their own. This notion
is encapsulated by idea of territoriality, which can be defined as
“an interlocking system of attitudes, sentiments, and
behaviours that are specific to a particular, usually
delimited, site or location which, in the context of
individuals, or a small group as a whole, reflect and
reinforce for those individuals or group some degree of
excludability of use, responsibility for, and control over
activities in these specific sites. [19]
In Echology, without explicit instruction, participants chose a button
to stand in front of, and subsequent use of that button was solely by
that participant. That button became their territory. It was often the
case, however, that not all of the buttons had people in front of
them. Because of the control each button had on the overall path of
the sounds, these unmanned buttons became collaborative territory.
Another effect of territoriality, described by Taylor, is that it helps
people mediate their social interaction through the very act of laying
claim to a space [19]. Unmanned buttons, and even the occasional
co-opting of manned buttons by other enthusiastic patrons wanting
to achieve a specific effect, created dynamic changes in territory
that encouraged conversation and cooperation.
Because of these dynamics, we found Echology was successful at
supporting social interaction between participants. The availability
of territory, collaborative activities, and the draw of the spatial
sound, graphics and lights all helped to create a novel experience
that people felt comfortable interacting around.
7. FUTURE WORK
The installation has proven very successful in its current form;
however there are a number of changes we are considering to
further improve user experience and better achieve our artistic
objectives.
Changes to the physical installation involve a reworking of the look
of the interaction table. It is currently constructed from a custom
made circular table-top that sits on a metal barrel. Both are covered
with white cotton and the table-top itself is wrapped in a heavy-duty
transparent plastic cladding to protect the cotton and provide a
Beluga-like skin feeling. A different material covering the barrel
might prove more robust for long-term installations where there are
many children. The eight dynamically-changing lights hanging
from the speakers surrounding the installation also add to the sense
of immersion for participants, and we are currently experimenting
with different coloured filters to enhance their feel.
Changes to the Echology Max/MSP components may include
making the graphics less abstract; for instance, more like Belugas.
We may integrate them more with the background webcam video
by applying dynamic textures. There are also interesting
opportunities to experiment with spatialization techniques.
Currently, the speakers are all arranged in a 2-D plane, but VBAP
also supports 3D soundscapes. For example, it might be interesting
to arrange the eight speakers in a hemisphere. These experiments
will present challenges, as any changes made to Echology must be
balanced against the current simplicity, which in itself was a design
goal.
The next iteration of Echology will also include improvements in
the video processing and sound control modules based on further
experimentation. Overall people generally felt a relationship
between the soundscape and the Beluga motion but we feel this
relationship could more tightly coupled. We believe that the lack of
tight coupling in the mapping is largely due to variations in the
webcam video quality. Video conditions vary greatly throughout the
day and according to weather, making it impossible to have one
video processing setting. For example, the best conditions for
processing the video are in the mid afternoon on a cloudy day.
However, if there is direct sun, it causes ripples on the water’s
surface to glare brightly. Their motion is picked up by the current
video processing module, causing sound to be played even with no
whales in the scene. The current system also has no way to keep
track of the number of Belugas in the scene at one time. The
addition of this functionality is desirable, but again difficult to
implement due to the colour and quality of the webcam feed.
Once changes are made to the installation we hope to conduct a
more extensive, formal user study. The study will be based on
video observation and questionnaires to answer questions like the
following:
- Did participants find the spatialization control simple to
understand? Could they hear the spatialization?
- Did participants find the space at all intimidating or
unapproachable?
- What patterns of spatialization are most commonly attempted
with the interaction table?
- How does perception of the installation differ between someone
who knows the beluga feed is live compared to someone who
does not?
- How long is the average time spent at the interaction table?
- How do people feel about socializing around the table?
- How do they feel after visiting the table alone?
- Would people visit the installation more than once?
In addition to our artistic motivations, through our creation of
Echology and our observations of participants, we hope to
contribute to the pool of knowledge concerning the use of table-top
displays in interactive arts. As well, since this work provides an
effective means to facilitate social interaction it could be used in
applications to help groups communicate better such as team
building applications.
8. CONCLUSIONS
Overall, Echology was well liked by participants. Many spent
considerable time watching, playing or just being close to it while in
discussion. People didn’t always recognize the relationship
between the coloured balls moving around and the spatialized
sound. However, the exhibit was engaging and enjoyable and
facilitated social interaction. Thus, at this stage of creation we
believe that the piece works well as an interactive artwork and
meets our artistic objectives.
9. ACKNOWLEDGEMENTS
Our gratitude to Valeria Vergara for providing Beluga sounds, and
to the Vancouver Aquarium for their support. This project received
funding from Natural Sciences and Engineering Research Council
944
(NSERC), the Media and Graphics Interdisciplinary Centre
(MAGIC) and the Institute for Computing, Information and
Cognitive Systems (ICICS).
10. REFERENCES
[1] Tomlinson, B., Downie, M., Berlin, M., Gray, J., Lyons, D.,
Cochran, J., and Blumberg, B. , Leashing the AlphaWolves:
Mixing User Direction with Autonomous Emotion in a Pack of
Semi-Autonomous Virtual Characters., Proceedings of the
2002 ACM SIGGRAPH Symposium on Computer Animation.
San Antonio, TX.
[2] Sommerer, C. and Mignonneau, L., A-Volve: a real-time
interactive environment,
http://www.iamas.ac.jp/~christa/WORKS/FRAMES/TOPFRAME
S/A-VolveTop.html, 1994.
[3] Iwai, Toshio. Composition on the table. In ACM SIGGRAPH
’99 Electronic art and animation catalog, Los Angeles, CA,
pg. 10, 1999.
[4] Poupyrev, I., Berry, R., Billinghurst, M., Kato, H., Nakao, K.,
Baldwin, L., & Kurumisawa, J., Augmented Reality Interface
for Electronic Music Performance, Proceedings of the 9th
International Conference on Human-Computer Interaction
(HCI International 2001), August, New Orleans, LA, USA.
805-808, 2001.
[5] Blaine, T. & Perkis, T., The Jam-O-Drum Interactive Music
System: A Study in Interaction Design, DIS2000 Conference
Proceedings, New York, NY, August, 165-173, 2000.
[6] Blaine, T. & Forlines, C., JAM-O-WORLD: Evolution of the
Jam-O-Drum into the Jam-O-Whirl Gaming Interface,
Proceedings of the 2nd International Conference on New
Interfaces for Musical Expression(NIME02), Dublin, Ireland,
May 24-26, 17-22, 2002.
[7] Blaine, T. and Fels, S.S., Collaborative Musical Experiences
for Novices. Journal of New Music Research. Volume 32. No.
4. Pages 411-428. Dec. 2003.
[8] Chapin, W., InTheMix, ACM SIGGRAPH 2000, Conference
Abstracts and Applications, 2000.
[9] Norman, D., The Design of Everyday Things, New York:
Currency/Doubleday. 79-80, 1990.
[10] de Nijs, M. and van der Heide, E., Spatial Sounds 100 dB/100
km/u, sound/kinetic installation. 2000, 2001.
[11] http://www.cycling74.com/
[12] http://www.vanaqua.org/belugacam/
[13] http://www.phidgets.com/
[14] Omojola, O., Post, E. R., Hancher, M. D., Maguire, Y., Pappu,
R., Schoner, B., Russo, P. R., Fletcher, R., and Gershenfeld, N.
2000. An installation of interactive furniture. IBM Syst. J. 39,
3-4 (Jul. 2000), 861-879.
[15] Pulkki, V. Virtual Sound Source Positioning Using Vector
Base Amplitude Panning. Journal of the Audio Engineering
Society, vol 45, no. 6 1997.
[16] Pulkki, V. Generic panning tools for MAX/MSP. Proceedings
of International Computer Music Conference 2000. pp. 304-
307. Berlin, Germany, August, 2000.
[17] Scott, S. D., Sheelagh, M., Carpendale, T., and Inkpen, K. M.
2004. Territoriality in collaborative table-top workspaces. In
 Proceedings of the 2004 ACM Conference on Computer Group Territorial Cognitions, Behaviors, and Consequences.
Supported Cooperative Work (Chicago, Illinois, USA, New York: Cambridge University Press.
November 06 - 10, 2004). CSCW '04. ACM Press, New York, [20] Weiser, M. and Brown, J. S., Designing calm technology.
NY, 294-303 PowerGrid Journal, Vol. 1, No. 1, 1996.
[18] Tang, J. C., Findings from observational studies of
collaborative work, International Journal of Man Machine
Studies 1991, Vol. 34 No. 2, 143-160
[19] Taylor, R. B. (1988) Human Territorial Functioning: An
Empirical Evolutionary Perspective on Individual and Small

945