Overture

byRandallPackerandKenJordan
fromhttp://www.w2vr.com/overture/prologue.html
Multimediaisemergingasthedefiningmediumofthe21stcentury.The
WorldWideWeb,CDROMs,virtualrealityarcadegames,andinteractive
installationsonlyhintattheformsofmultimediatocome.Yettheconceptof
integrated,interactivemediahasitsownlonghistory,anevolutionthatspans
over150years.Remarkably,thishasbeenalargelyuntoldstory.Discussions
ofthedevelopmentofthepersonalcomputerandtheInternettendtofocuson
afewhighlysuccessfulentrepreneurs,neglectingthelessknownworkofthe
engineersandartistswhofirstsoughttocraftamediumthatwouldappealtoallthesenses
simultaneouslyamediumthatwouldmimicandenhancethecreativecapacitiesofthehumanmind.
Here,then,isa"secrethistory"ofmultimedia:anarrativethatincludesthepioneeringactivitiesofa
diversegroupofartists,scientists,poets,musicians,andtheoristsfromRichardWagnertoIvan
Sutherland,fromVannevarBushtoBillViola.
IntheyearssinceWorldWarII,scientistshavepursuedpersonalcomputing
andhumancomputerinteractivityasvehiclesfortransforming
consciousness,extendingmemory,increasingknowledge,amplifyingthe
intellect,andenhancingcreativity.Theidealisticandideologicalaspirations
ofbothgroupshaveresultedinanewmediumthatemphasizesindividual
choice,freeassociation,andpersonalexpression.

Newtechnologyhasalwaysbeenusedtomakemedia.GeorgeLucasshothis
latestStarWarsepicwithdigitalcameras,thoughtheaudienceexperience
wasnodifferentthanifithadbeenshotoncelluloid.Butwhilenotall
computerbasedmediaismultimedia,today'smultimediastartswiththe
computer,andtakesthegreatestadvantageofthecomputer'scapabilityfor
personalexpression.
Digitalcomputerswere
fullyelectronic
duringWorldWarIIto
Computersthenwere
vacuumtubes,and
scientificresearch.Only
personalcomputingfor

initiallydesignedascalculatingmachines.Thefirst
computer,theENIAC,wasbuiltbytheU.S.military
produceballisticstablesforartilleryinbattle.
clumsy,hulkingdevicestheENIAChad18,000
measured50x30feetthatdidcalculationsfor
ahandfulofscientistsconsideredthepossibilityof
creativepurposesbynonspecialists.

ThefirstscientisttothinkseriouslyofthispotentialwasVannevarBush.In
his1945article"AsWeMayThink,"heoutlined"afuturedevicefor
individualuse,whichisasortofmechanizedprivatefileandlibrary."Before
theENIACwascompleted,Bushwasalreadycontemplatinghowinformation
technologycouldenhancetheindividual'scapabilityforcreativethought.
"Thehumanmind...operatesbyassociation,"Bushobserved.Thedevicethat
heproposed,whichhenamedtheMemex,enabledtheassociativeindexingof
information,sothatthereader'strailofassociationwouldbesavedinsidethemachine,availablefor
referenceatalaterdate.Thisprefiguredthenotionofthehyperlink.WhileBushneveractuallybuilt
theMemex,andwhilehisdescriptionofitreliedontechnologythatpredateddigitalinformation
storage,hisideashadaprofoundinfluenceontheevolutionofthepersonalcomputer.

IntheyearsimmediatelyaftertheWar,undertheshadowoftheatomicbomb,
thescientificestablishmentmadeaconcertedefforttoapplyrecent
advancementsintechnologytohumanitarianpurposes.Inthisclimate,
NorbertWienercompletedhisgroundbreakingtheoryoncybernetics.While
Wienerdidnotlivetoseethebirthofthepersonalcomputer,hisbook,The
HumanUseofHumanBeings,hasbecomederigeurforanyoneinvestigating
thepsychologicalandsocioculturalimplicationsofhumanmachine
interaction.Wienerunderstoodthatthequalityofourcommunicationwithmachineseffectsthequality
ofourinnerlives.Hisapproachprovidedtheconceptualbasisforhumancomputerinteractivityandfor
ourstudyofthesocialimpactofelectronicmedia.
BushandWienerestablishedafoundationonwhichanumberofcomputer
scientistsassociatedwiththeAdvancedResearchProjectsAgency(ARPA)a
U.S.governmentfundedprogramtosupportdefenserelatedresearchin
the1960sbegantobuild.LeadingARPA'sefforttopromotetheuseof
computersindefensewastheMITpsychologistandcomputerscientistJ.C.R.
Licklider,authoroftheinfluentialarticle"ManComputerSymbiosis."
Defyingtheconventionalwisdomthatcomputerswouldeventuallyrival
humanintelligence,ratherthanenhancingit,Lickliderproposedthatthecomputerbedevelopedasa
creativecollaborator,atoolthatcouldextendhumanintellectualcapabilityandimproveaperson's
abilitytoworkefficiently.
WhileatARPA,Lickliderputsignificantresourcestowardsthepursuitofhis
vision.AmongthescientistshesupportedwasDouglasEngelbart,whosince
themid1950shadbeenseekingsupportforthedevelopmentofadigital
informationretrievalsysteminspiredbyBush'sMemex.APRAfunding
enabledEngelbarttoassembleateamofcomputerscientistsand
psychologistsattheStanfordResearchInstitutetocreatea"toolkit"that
would,ashephrasedit,"augmenthumanintellect."DubbedtheoNLine
System(NLS),itspublicdebutin1968attheFallJointComputerConferenceinSanFranciscowasa
landmarkeventinthehistoryofcomputing.EngelbartunveiledtheNLSbeforearoomof3,000
computerscientists,whosatinraptattentionfornearlytwohourswhilehedemonstratedsomeofhis
majorinnovations,includingthemouse,windowsfortextediting,andelectronicmail.Engelbartwas
makingitpossible,forthefirsttime,toreachvirtuallythroughacomputer'sinterfacetomanipulate
information.Eachofhisinnovationswasakeysteptowardsaninterfacethatallowedforintuitive
interactivitybyanonspecialist.Attheendofhispresentation,hereceivedastandingovation.
However,thecontributionsoftheNLSwentbeyondinnovationregardingthecomputerinterface.
Engelbartandhiscolleaguesalsoproposedthatcreativitycouldbeenhancedbythesharingofideas
andinformationthroughcomputersusedascommunicationsdevices.TheoNLineSystemhadits
computerswiredintoalocalnetwork,whichenabledthemtobeusedformeaningfulcollaboration
betweencoworkers.Engelbartunderstoodthatthepersonalcomputerwouldnotonlyaugment
intelligence,butaugmentcommunicationaswell.In1969hisresearchinonlinenetworkingcameto
fruitionwiththecreationoftheInternet.
Engelbart'sNLSpioneeredsomeoftheessentialcomponentsnecessaryfor
thepersonalcomputer,butitwouldbeuptoanewgenerationofengineersto
advancecomputingsoitcouldembracemultimedia.Asagraduatestudentin
thelate1960s,AlanKaywroteahighlyinfluentialPh.D.thesisproposinga
personalinformationmanagementdevicethat,inmanyways,prefiguredthe
laptop.In1970,asresearchininformationsciencewasshiftingfromEast
Coastuniversitiesandmilitaryinstitutionstoprivatedigitalcompaniesin
SiliconValley,KaywasinvitedtojointhenewXeroxPARCinPaloAlto.PARC'smandatewasno
lessthantocreate"thearchitectureofinformationforthefuture."
AtPARC,AlanKayconceivedtheideaoftheDynabookanotebooksizedcomputerthatenabled
hyperlinking,wasfullyinteractive,andintegratedallmedia.WiththeDynabook,digitalmultimedia

cameintobeing.EchoingLicklider,EngelbartandcolleaguesatPARC,Kaydeclaredthepersonal
computeramediuminitsownright,.Itwasa"metamedium,"ashedescribeditinhis1977essay
"PersonalDynamicMedia,"capableofbeing"allothermedia."WhiletheDynabookremaineda
prototypethatwasneverbuilt,theworkthatcamefromitsdevelopment,includingtheinventionofthe
GraphicalUserInterface(GUI)andsubsequentbreakthroughsindynamiccomputing,was
incorporatedintothefirsttruemultimediacomputer,theXeroxAlto.

ByproposingthattheDynabookbea"metamedium"thatunifiesallmedia
withinasingleinteractiveinterface,AlanKayhadglimpsedintothefuture.
Buthemaynothaverealizedthathisproposalhadrootsinthetheoriesof
the19thcenturyGermanoperacomposer,RichardWagner.In1849,Wagner
introducedtheconceptoftheGesamtkunstwerk,orTotalArtwork,inanessay
called"TheArtworkoftheFuture."Itwouldbedifficulttooverstatethe
powerofthisidea,oritsinfluence.Wagner'sdescriptionofthe
Gesamtkunstwerkisoneofthefirstattemptsinmodernarttoestablishapractical,theoreticalsystem
forthecomprehensiveintegrationofthearts.Wagnersoughttheidealizedunionofalltheartsthrough
the"totalizing,"orsynthesizing,effectofmusicdramatheunificationofmusic,song,dance,poetry,
visualarts,andstagecraft.Hisdrivetoembracethefullrangeofhumanexperience,andtoreflectitin
hisoperas,ledhimtogiveequalattentiontoeveryaspectofthefinalproduction.Hewasconvinced
thatonlythroughthisintegrationcouldheattaintheexpressivepowershedesiredtotransformmusic
dramaintoavehiclecapableofaffectingGermanculture.
Twentiethcenturyartistshavecontinuedtheefforttoheightentheviewer's
experienceofartbyintegratingtraditionallyseparatedisciplinesintosingle
works.Modernexperience,manyoftheseartistsbelieved,couldonlybe
evokedthroughanartthatcontainedwithinitselfthecompleterangeof
perception."Oldfashioned"formslimitedtowordsonapage,paintona
canvas,ormusicfromaninstrument,wereconsideredinadequatefor
capturingthespeed,energyandcontradictionsofcontemporarylife.Intheir
1916manifesto"TheFuturistCinema,"F.T.Marinettiandhisrevolutionarycohortsdeclaredfilmtobe
thesupremeartbecauseitembracedallotherartformsthroughtheuseof(then)newmedia
technology.Onlycinema,theyclaimed,hada"totalizing"effectonhumanconsciousness.

experience.

Lessthanadecadelater,inhis1924essaydescribingthetheaterofthe
Bauhaus,"Theater,Circus,Variety,"LszlMoholyNagycalledforatheater
ofabstractionthatshiftedtheemphasisawayfromtheactorandthewritten
text,andbroughttotheforeeveryotheraspectofthetheatricalexperience.
MoholyNagydeclaredthatonlythesynthesisofthetheater'sessentialformal
componentsspace,composition,motion,sound,movement,andlight
intoanorganicwholecouldgiveexpressiontothefullrangeofhuman

TheperformanceworkofJohnCagewasasignificantcatalystinthe
continuingbreakdownoftraditionalboundariesbetweenartisticdisciplines
afterWorldWarII.Inthelate1940s,duringaresidencyatBlackMountain
CollegeinNorthCarolina,Cageorganizedaseriesofeventsthatcombined
hisinterestincollaborativeperformancewithhisuseofindeterminacyand
chanceoperationsinmusicalcomposition.Togetherwithchoreographer
MerceCunninghamandartistsRobertRauschenbergandJasperJohns,Cage
devisedtheatricalexperimentsthatfurtheredthedissolutionofbordersbetweenthearts.Hewas
particularlyattractedtoaestheticmethodsthatopenedthedoortogreaterparticipationoftheaudience,
especiallyifthesemethodsencouragedaheightenedawarenessofsubjectiveexperience.Cage'suseof
indeterminacyandchancerelatedtechniqueshiftedresponsibilityfortheoutcomeoftheworkaway
fromtheartist,andweakenedyetanothertraditionalboundary,thedividebetweenartworkand
audience.

Cage'sworkprovedtobeextremelyinfluentialonthegenerationofartiststhatcameofageinthelate
1950s.AllanKaprow,DickHigginsandNamJunePaikwereamongthemostprominentoftheartists
who,inspiredbyCage,developednontraditionalperformancetechniquesthatchallengedaccepted
notionsofform,categorization,andcomposition,leadingtotheemergenceofgenressuchasthe
Happenings,electronictheater,performanceart,andinteractiveinstallations.
AllanKaprow,whocoinedtheterm"Happening,"wasparticularlyinterested
inblurringthedistinctionbetweenartworkandaudience.Theultimate
integratedart,hereasoned,wouldbewithoutanaudience,becauseevery
participantwouldbeanintegralpartofthework.Ashewroteinhis1966
primer,"UntitledGuidelinesforHappenings,""Thelinebetweenartandlife
shouldbekeptasfluid,andperhapsindistinct,aspossible."Thisapproachled
toaperformancestylethatpioneereddeliberate,aestheticallyconceived
groupinteractivityinacomposedenvironment.Happeningsartistsdevisedformalelementsthat
allowedparticipantsthefreedomtomakepersonalchoicesandcollectivedecisionsthatwouldaffect
theperformance.
Inthisclimate,artistsbecameincreasinglyinterestedinintegrating
technologyintotheirwork.Whiletechnologyclearlyplayedasignificantrole
in20thcenturyarts(suchasphotography,film,andvideo,aswellasvarious
fineartsgenres),itwasnotuntilBellLabsscientistBillyKlverplacedthe
potentialofadvancedengineeringintothehandsofartistsinNewYorkthat
integratedworksofartandtechnologybegantoflourish.Klverconceived
thenotionofequalcollaborationbetweenartistandengineer.Hepioneered
formsofartandtechnologythatwouldhavebeenunimaginabletotheartistwithouttheengineer's
cooperationandcreativeinvolvement.WithRobertRauschenberg,Klvercreatedseveralofthe
earliestartworkstointegrateelectronicmediaandtoencourageaparticipatoryrolefortheaudience,
includingOracle(196365)andSoundings(1968).
In1966KlvercofoundedE.A.T.(ExperimentsinArtandTechnology)tobringartistsandengineers
togethertocreatenewworks.E.A.T.'smostambitiousproductionwasthePepsiPavilion,designedfor
theOsakaExpo'70inJapanatremendouslyambitiouscollaborative,multimediaprojectthat
involvedover75artistsandengineers.AsKlverexplained,audienceparticipationwasattheheartof
theirinterests:"TheinitialconcernoftheartistswhodesignedthePavilionwasthatthequalityofthe
experienceofthevisitorshouldinvolvechoice,responsibility,freedom,andparticipation.ThePavilion
wouldnottellastoryorguidethevisitorthroughadidactic,authoritarianexperience.Thevisitor
wouldbeencouragedasanindividualtoexploretheenvironmentandcomposehisownexperience."
Duringthisperiod,theBritishartistandtheoristRoyAscottbegantoexplore
theuseofcomputersinartisticexpression.Oneofthefirsttheoretical
attemptstointegratetheemergingfieldsofhumancomputerinteractivityand
cyberneticswithartisticpracticeisAscott'sarticle,"BehavioralArtandthe
CyberneticVision,"from196667.Ascottnotedthatthecomputerwas"the
supremetoolthat...technologyhasproduced.Usedinconjunctionwith
syntheticmaterialsitcanbeexpectedtoopenuppathsofradicalchangein
art."Ascottsawthathumancomputerinteractionwouldprofoundlyaffectaesthetics,leadingartiststo
embracecollaborativeandinteractivemodesofexperience.
When Alan Kay arrived in Xerox PARC in 1970, the foundation was in place
for a multimedia that synthesized all the existing art forms, and presented
them in an environment that allowed for meaningful interactivity. With the
Dynabook, the interactive Gesamkunstwerk was brought into the digital
realm, and put on-line.

Thefantasyofbeingtransportedintoanotherworld,tobetakenwhollyinto

animaginaryrealm,isaprimaldesire.Withcomputerbasedmultimedia,encounterswithimmersive,
virtualworldswillsoonbecomecommonplace.Virtualreality,afterall,isalogicalextensionofthe
integrationofthearts.Itisalsoanidealenvironmentforapplyingourknowledgeofhumancomputer
interactivity.

Thereisanevocativeechoinvirtualenvironmentsoftheearliestknownform
ofhumanexpressiontheprehistoriccavepaintingsfoundatsuchsitesasthe
cavesofLascauxinthesouthofFrance.Theseimmersiveenvironments,
datingfrom15,000BC,arethoughtbyscholarstohavebeentheatersforthe
performanceofritualsthatintegratedallformsofmediaandengagedallthe
senses.Onthewallswerepaintedanimalimagesandshamanistscrawls,but
theenvironmentinwhichthepaintingsappearedwassurelyassignificantas
thepaintingsthemselves.AsJosephCampbelldescribedit,"thesemagicalspotsoccurfarfromthe
naturalentrancesofthegrottos,deepwithinthedark,wanderingchillcorridorsandvastchambers,so
thatbeforereachingthemonehastoexperiencethefullforceofthemysteryofthecaveitself."To
encounterthecavepaintingswastoimmersetheselfinanotherworldlydomain,whichwouldheighten
consciousnessandtriggeralteredstatesofperception.
Therearemanyexamplesofimmersioninthehistoryofart.TheDyonisian
ritualsofGreekTheater,andtheconstructionofthegreatcathedralsof
Europe,aretwoobviousexamples.RichardWagner'sGesamtkunstwerkwas
drivenbyavisionoftheaterinwhichtheaudiencelosesitselfintheveracity
ofthedrama,creatinganimmersiveexperience.Ashewrotein"TheArtwork
oftheFuture,""thespectatortransplantshimselfuponthestage,bymeansof
allhisvisualandauralfaculties."Tofacilitatehisvision,Wagnerreinvented
theconventionsoftheoperahouse,andin1876openedtheFestpielhausTheaterinBayreuth,
Germany,withthefirstcompleteproductionofTheRingcycle.TheFestpielhaus,withitsemployment
ofGreekamphitheatricalseating,surroundsoundacoustics,thedarkeningofthehouse,andthe
placementofmusiciansinanorchestrapit,focusedtheaudience'sundividedattentiononthedramatic
action.Hisintentwastomaximizethesuspensionofdisbelief,todrawtheviewerintoanillusionary
worldstagedwithintheprosceniumarch.
Inthe1950s,asimilarvisioninspiredtheAmericancinematographerMorton
Heiligtoproposea"cinemaofthefuture"thatwouldsurroundtheaudience
withfacsimilesoflifesoconvincingtheywouldbelievethemselvestobe
transportedtoanotherdomain.Suchacinema,hewrote,"wouldfaithfully
reproduceman'souterworldasperceivedinhisconsciousness,itwill
eventuallylearntocreatetotallynewsensematerialsforeachofthesenses...
[that]theyhaveneverknownbefore,andtoarrangethemintoformsof
consciousnessneverbeforeexperiencedbymaninhiscontactwiththeouterworld."
WhileHeiligdevisedatheoreticalframeworkthatappliedthetechnologiesof
hisdaytowardtheachievementofvirtualexperience,itwasonlyasa
consequenceofadvancesincomputersciencethattheimmersionhiswork
suggestedbecamepossible.By1965,IvanSutherlandhadalreadyachieved
legendarystatusamongcomputerscientistsastheinventorofSketchpad,the
firstinteractivegraphicssoftware.Inashortpaperpublishedthatyear,
Sutherlandmusedovertheoptionsavailabletotheengineertodisplay
computerdata,tocreatea"alookingglass"intowhathedescribedasa"mathematicalwonderland."It
seemedreasonableforhimtosuggestthat"TheUltimateDisplay"(asthepaperwastitled)would
representthisdatain3dimensionalform,allowingtheconstructionofentirelybelievable3
dimensional,computercontrolled,virtualworlds.However,likeHeiligbeforehim,Sutherlandtook
thissuggestiononestepfurther."Theultimatedisplay,"hewrote,"wouldbearoomwithinwhichthe
computercancontroltheexistenceofmatter."Hesuggestedthatsuchadisplaycouldpresentrealities

heretoforeonlyimagined,asifseenthroughAlice'slookingglass.Sutherland'sproposalwasstartling,
butitlaunchedanentirefieldofscientificinquiry.
Italsofueledtheimaginationofagenerationofartists.Oneofthefirstto
considerthepossibilitiesofdigitallyconstructedvirtualexperienceswas
MyronKrueger.Intheearly1970s,Kruegercreatedthepioneeringworks
MetaplayandVideoplacetoexplorethepotentialofcomputermediated
interactivity.Theseworkswereinteractiveartisticenvironments,influenced
byHappenings,designedtogiveparticipantsfreedomofchoiceand
opportunitiesforpersonalexpression.Videoplacealsoconnectedparticipants
indifferentlocationsthroughnetworkedtechnologies,creatingtheillusionofsharedspace.AsKrueger
laterwroteaboutthepiece,"ourteleconferencecreatedaplacethatconsistedoftheinformationwe
bothshared...aworldinwhichfullphysicalparticipationwouldbepossible.Thisworldisan'artificial
reality.'"
Duringthe1970sand1980s,severalengineeringprojectspursuedvirtual
environmentdisplaysystemsthatcouldrepresentsuchan"artificialreality.
Perhapsthemostsignificantoftheseinthemid1980swasledbyScottFisher
attheNASAAmesResearchCenter.Fisher'sintentwastoengagetheentire
nervoussysteminamultisensorypresentationofvirtualspaceextending
multimediabeyondthescreen.TheAmesVIEWsystem(anacronymfor
VirtualInterfaceEnvironmentalWorkstation)includedaheadsetwithtwo
smallliquidcrystaldisplayscreens,amicrophoneforspeechrecognition,earphonesforsurround
soundeffects,aheadtrackingdevice,anddataglovetorecognizetheuser'sgesturesandplacethem
withinthevirtualenvironment.Thedirectionthisworkpointedinwasclear.AsFisherwroteinhis
1989article"VirtualInterfaceEnvironments,""withfullbodytrackingcapability,itwouldalsobe
possibleforuserstoberepresentedinthis[virtual]spacebylifesizevirtualrepresentationsof
themselvesinwhateverformtheychoose."Immersiveenvironments,Fisherobserved,couldgivebirth
toanewformofparticipatory,interactiveelectronictheater.
Thepossibilityofsuchatheaterhadalreadytakenholdofthepublic's
imagination.InNeuromancer,hiswidelyreadnovelfrom1984,William
Gibsondescribedinpalpabledetailafutureinwhichvirtualrealitywasafact
oflife.EchoingMyronKrueger'snotionthatteleconferencingcreateda
"place"thatconsistedofsharedinformation,Gibson'scharactersinhabiteda
virtualenvironmentmadepossiblebythenetworkingofcomputers,whichhe
named"cyberspace."Gibson'scyberspaceprovidedthefirstliterarydefinition
forthecomputers,hubs,servers,anddatabasesthatmakeupthematricesofthenetwork.His
discussionofcyberspacewassotangibleandseductive,withitssuggestionthatanycomputerhacker
could"jackintothematrix"withanencounterwithasexyavataritbecameatouchstoneforevery
engineer,artistandtheoristworkinginthefield.
MarcusNovaktookGibson'sdescriptionofvirtualenvironmentsasthe
startingpointforhisowntheoreticalandartisticexplorations.Inhisessay
from1991,"LiquidArchitectureinCyberspace,"hefollowsthepioneering
workofSutherland,Fisher,andGibson,etal,toitslogicalconclusion,and
notesitsprofoundimplicationsforarchitecture,ournotionsofspace,andour
attitudestowardstheorganizationofinformation.Henotesthatincyberspace,
sinceallstructureisprogrammable,allenvironmentscanbefluid.Theartist
whodesignstheseimmersivedigitalhabitatswillbeabletotranscendthelawsofthephysicalworld.
Asaconsequence,architecturalformsbuiltincyberspacecanrespondtotheviewer,encouraging
provocativeandilluminatinginteractions.Incyberspace,architecturebecomesaformofpoetry.
Whilemostresearchinvirtualrealityaimstoprojecttheviewerintoadigital
environmentbymeansofaheadmounteddisplay,someengineershavetaken
analternativeapproach.Intheearly1990s,DanielSandinandThomas
DeFanticonceivedofavirtualrealitysystemthatplacesthehumanbody

directlyinsideacomputergeneratedenvironment.Theydescribetheirsystem,calledtheCAVE(an
acronymforCaveAutomaticVirtualEnvironment)intheirarticle"RoomWithaView":"Unlikeusers
ofthevideoarcadetypeofvirtualrealitysystem,CAVE'dwellers'donotneedtowearhelmets,which
wouldlimittheirviewofandmobilityintherealworld...toexperiencevirtualreality."Instead,
participantsintheCAVEaresurroundedbyanimmersive,digital"cavepainting"whichbringsthe
evolutionofimmersionfullcircle,backtotheprehistoriccavesofLascaux,andhumankind'searliest
effortsatpersonalexpression.

ItwasVannevarBushwho,in1945,determinedthechiefnarrative
characteristicofmultimediabyproposingamechanicaldevicethatoperated
literally"aswemaythink."Thechallenge,ashesawit,wastocreatea
machinethatsupportedthemind'sprocessoffreeassociation.Bushnoted
howideastendtoevolveinanonlinear,idiosyncraticfashion.HisMemex
wouldbeatoolthatcouldsupplementthisaspectofhumancreativityby
organizingitsmediaelementstoreflectthedynamicsofthemindatplay.
DouglasEngelbartexpandedonBush'spremise.Hisquestto"augment
humanintelligence,"asheaptlyphrasedit,wasbasedontheinsightthatthe
openflowofideasandinformationbetweencollaboratorswasasimportantto
creativityasprivatefreeassociation.Thepersonalcomputer,asheenvisioned
it,wouldnotonlyallowforthearrangementofdatainidiosyncratic,non
linearformats.Byconnectingworkstationstoadatasharingnetworkand
turningthemintocommunicationsdevices,Engelbart'soNLineSystem
allowedforaqualitativeleapinthecollaborationbetweenindividualsalmostasifcolleaguescould
peerintooneanother'smindsaspartofthecreativeprocess.Intheearly1960s,experimentswith
networkedpersonalcomputingpromisedthenonlinearorganizationofinformationonagrandscale.
Whilefewrecognizedthispossibilityatthetime,itinspiredaseriesof
influentialtheoreticalwritingsbytheroguephilosopherTedNelson.Working
outsideoftheacademicandcommercialestablishments,followinghisown
stronglyheldconvictions,Nelsondevisedanelaboratesystemforthesharing
ofinformationacrosscomputernetworks.CalledXanadu,thissystemwould
maximizeacomputer'screativepotential.CentraltoNelson'sapproachwas
the"hyperlink,"atermhecoinedin1963,inspiredbyBush'snotionofthe
Memex'sassociativetrails.Hyperlinks,heproposed,couldconnectdiscretetextsinnonlinear
sequences.Usinghyperlinks,Nelsonrealized,writerscouldcreate"hypertexts,"whichhedescribedas
"nonsequentialwriting"thatletthereadermakedecisionsabouthowthetextcouldbereadinother
thanlinearfashion.Asheobservedinhislandmarkbookfrom1974,ComputerLib/DreamMachines,
"thestructuresofideasarenotsequential."Withhypertext,anditsmultimediacounterpart,
"hypermedia,"writersandartistscouldcreateworksthatencouragedtheusertoleapfromoneideato
thenextinaseriesofprovocativejuxtapositionsthatpresentedalternativestoconventionalhierarchies.
Nelson'sinsightswereparalleledbyexperimentsintheliteraryavantgarde
thatchallengedtraditionalnotionsoflinearnarrative.Inhisbook,herefersto
VladimirNabokov'sPaleFireandJulioCortazar'sHopscotchastwonovels
thatuseunconventionalbranchingstructurestoencouragethereader'sactive
collaborationintheconstructionofthestory.Aswehavealreadyseen,
experimentalperformancesinspiredbyJohnCageincludingHappenings,
interactiveinstallations,andperformanceartalsogaverisetoavarietyof
nonlinearnarrativestrategies.Butperhapsthemostprescientexplorerofthisterrainwasthenovelist
WilliamS.Burroughs.
LikeTedNelson,Burroughswasdeeplysuspiciousofestablishedhierarchies.Hewasespecially
interestedinwritingtechniquesthatsuggestthespontaneous,momentbymomentmovementofthe
mind,andhownonlinearwritingmightexpandthereader'sperceptionofreality.Throughhisuseof
thecutupandfoldintechniques,whichhedescribedinhis1964essay,"TheFutureoftheNovel,"

Burroughstreatedthereadingexperienceasoneofenteringintoamultidirectionalwebofdifferent
voices,ideas,perceptions,andperiodsoftime.Hesawthecutupasatoolthatletthewriterdiscover
previouslyundetectedconnectionsbetweenthings,withpotentiallyenlighteningandsubversiveresults.
Withthecutup,Burroughsprefiguredtheessentialnarrativestrategyofhypertextanditsabilityto
allowreaderstoleapacrossboundariesintimeandspace.
SincetheinventionoftheelectrictelegraphbySamuelMorseinthe1830s,
commentatorshavebeennotingthetransformationofourconceptsofspace
andtimebywiredtechnology.Fromthetelegraphtothetelephoneto
televisiontosatellitecommunications,moderntelecommunicationshas
eradicatedgeographicborders,andmadespeedacentralfactorinmodernlife.
Thiseffectwascommonlyacknowledgedaslongagoas1868,when,ata
banquetheldinhonorofMorse'slifeachievement,hewastoastedforhaving
"annihilatedbothspaceandtimeinthetransmissionofintelligence.ThebreadthoftheAtlantic,with
allitswaves,isasnothing."
Artistshavegrappledwiththeimplicationsofthistechnologysinceits
inception;thenarrativeexperimentsofliteraryauthorsreflectsthiscurrentin
modernart.TedNelson'sconceptofhypertextrepresentedaprofoundeffort
toputthistechnologytowardtheserviceofpersonal,idiosyncratic
expression.Nelsonbecameanevangelistforhypertext,publishingarticles,
speakingatconferences,spreadingthegospelwhereverhecould.Oneof
thoseplaceswasBrownUniversity,whichduringthe1980sbecameahotbed
ofliteraryexplorationsoftheform.AtBrown,theliterarycriticGeorgeLandowandhiscolleagues
developedhypertexttools,suchasIntermedia,whichallowedauthorswithlittleexperiencein
programmingtoinventnewgenresofcreativewriting.Inhisownwork,Landowappliedatrained
criticaleyetotheformalaspectsofhypertext,makingconnectionstothepoststructuraltextual
analysisofcriticslikeRolandBarthesandJacquesDerrida.Justasacademictheoreticaldiscoursewas
questioningthecentralityoftheauthorintheproductionoftexts,hypermediasuggestedthat,ina
futureofnetworkeddigitalmedia,responsibilitywouldshiftfromauthortoreader,actively
encouraging
Duringthisperiod,mediaartistswhoserootslayinperformanceandvideo
alsobeganinvestigatinghypermediaasameansofexploringnewformsfor
tellingstories.ArtistssuchasLynnHershmanandBillViolaweredrawnto
thecomputer'sabilitytobreakdownlinearnarrativestructures.Viola
approachedthemediumasarepositoryforevocativeimagesthatcouldbe
projectedonscreensininstallations,"withtheviewerwanderingthrough
somethreedimensional,possiblylifesizedfieldofprerecordedorsimulated
scenesevolvingintime,"ashedescribedit.
LynnHershmanwasamongthefirsttocreatedigitalartworksusing
interactivemedia,insuchpiecesasDeepContact,from1989.Sheintroduced
interactivityintoherworktocombatthelossofintimacyandcontrolbrought
aboutbythedominanceofmediasuchasradioandtelevision.Heruseof
hypermediaallowedtheviewertochoosedirectionsinsidetheartwork's
complexbranchingstructure,andshapeapersonalexperienceofit.
Bythelate1980s,
multimedia,whichhadbeenatthefringeofthearts
andsciences,reached
criticalmassandwentmainstream.MarcCanter,
whodevelopedthefirst
commercialmultimediaauthoringsystems,wasa
chiefcatalyst.Canter
pioneeredsoftwaretoolsthatartistsanddesigners
usedtocreate
multimediaontheirpersonalcomputers.His
authoringsystems
synthesizedtext,images,animation,videoand
soundintoasingle
integratedwork,usinghyperlinksandother
hypermediatechniquestoconnectitsvariouselements.

In1974,TedNelsonhaddeclaredthat"Therealdreamisfor'everything'to
beinthehypertext."ItwasaproposalthatechoedMarshallMcLuhan's
influentialobservationthat,"aftermorethanacenturyofelectrictechnology,
wehaveextendedourcentralnervoussystemitselfinaglobalembrace,
abolishingbothspaceandtimeasfarasourplanetisconcerned."Butdespite
hisbestefforts,andmanyadvancesinthefieldsofhypermediaandcomputer
basedtelecommunications,theglobalhypermedialibraryheenvisioned
remainedmoredreamthanreality.Mostinnovationsinhypermediafocusedonclosedsystems,suchas
theCDROMandinteractiveinstallations,ratherthanonopensystemsusingacomputernetwork.
In1989TimBernersLee,ayoungBritishengineerworkingatCERN,the
particlephysicslaboratoryinGeneva,Switzerland,circulatedaproposalfor
aninhouseonlinedocumentsharingsystemwhichhedescribedmodestlyas
"a'web'ofnoteswithlinks."Aftergettingagrudginggoaheadfromhis
superiors,BernersLeedubbedthissystemtheWorldWideWeb.TheWeb,as
hedesignedit,combinedthecommunicationslanguageoftheInternetwith
Nelson'shypertextandhypermedia,enablinglinksbetweenfilestoextend
acrossaglobalnetwork.Itbecamepossibletolinkeverydocument,soundfileorgraphicontheWeb
inaninfinitevarietyofnonlinearpathsthroughthenetwork.Andinsteadofbeingcreatedbyasingle
author,linkscouldbewrittenbyanyoneparticipatinginthesystem.Notonlydidtheopennatureofthe
Weblenditselftoawidearrayofinteractive,multimediaexperiences,butbyhewingtoanon
hierarchicalstructureandopenprotocols,BernersLee'sinventionbecameenormouslypopular,andled
toanexplosioninthecreationofmultimedia.By1993theWebhadtrulybecomeaninternational
phenomenon.
ThesuccessoftheWebseemedtoconfirmtheintuitionofartistsengagingin
digitalmediathatinthefuture,aglobalmediadatabasewouldinspirenew
formsofexpression.RoyAscott,forexample,hadalreadybeenexploringthe
creativepossibilitiesofnetworkingsincethe1980s.Hewasinterestedinthe
notionof"dataspace,"aterritoryofinformationinwhichalldataexistsina
continualpresentoutsidethetraditionaldefinitionsoftimeandspace
availableforuseinendlessjuxtapositions.Ascottconsidersdataspaceanew
typeofGesamtkunstwerk,oraGesamtdatenwerkashecallsit,inwhichnetworkedinformationis
integratedintotheartwork.Insuchanenvironment,Ascottwrote,"meaningisnotsomethingcreated
bytheartist,distributedthroughthenetwork,andreceivedbytheobserver.Meaningistheproductof
interactionbetweentheobserverandthesystem,thecontentofwhichisinastateofflux,ofendless
changeandtransformation."
Thisnotionoftheartworkasaterritoryforinteraction,asalocusof
communicationsforacommunity,echoestheHappeningsofaprevious
generation.Onlineroleplayinggameshavebecomelaboratoriesfor
exploringthisformofinteractivity.AsthesocialtheoristSherryTurklehas
pointedout,onlinecommunities,suchasMultiUserDungeons(MUD),"are
anewgenreofcollaborativewriting,withthingsincommonwith
performanceart,streettheater,improvisationtheater,Commediadell'Arte,
andscriptwriting."PavelCurtiscreatedoneoftheearliestMUDs,LambdaMOO,in1990atXerox
PARC.Thoughitconsistedonlyoftext,itsinteractivequality,madepossiblethroughintricate
storytellingdevicesviatheInternet,gaveparticipantstheillusionofimmersioninavirtual
environment.Interactionintheonlineenvironment,Curtisclaimed,createsakindofsocialbehavior
which"insomewaysitisadirectmirrorofbehaviorinreallife."
Throughouthistory,arthasoftenbeenreferredtoasamirroroflife.Butbybuildingupontheconcepts
ofassociationandcollaboration,computerbasedmultimediamaywellbecomemorethanamirrorof
life.Alreadywehaveseenhowmultimediablurstheboundariesbetweenlifeandart,thepersonaland
themediated,therealandthevirtual.Theimplicationsofthesetendenciesweareonlynowbeginning
tograsp.

The breadth and potential of multimedia lends itself to utopian proposals. The
French media theorist Pierre Levy describes multimedia as belonging to a
trajectory of planetary evolution that runs from DNA to cyberspace an arc
that follows pure information as it reaches towards its most evolved form of
expression. He proposes that today's global networks will usher in an era of
"collective intelligence," and suggests that "cyberspace constitutes a vast,
unlimited field... designed to interconnect and provide interface for the
various methods of creation, recording, communication and simulation." His enthusiastic perspective is
full of intriguing possibilities.
Atthesametime,weareallawareofthedystopianqualitiesofthe24/7
infotainmentjuggernautthatisbeingdeliveredacrosstheglobethroughan
evermoresophisticatedtelecommunicationsnetwork.Wereaddailyaboutthe
newmedia'sencroachmentonprivacy,itsopportunityforabuse,andthe
specterofcentralizedcontrolthatitmightmakepossible.Thesedangersare
real.ThereisatensionbetweenopposingattheheartoftheInternet
betweenthosewhoprizeitspotentialforanopen,freewheelingexchangeof
artandideas,andthosewhoseeitspervasivenessasanopportunitytoexpanduponthemarketing
drivenbroadcastmodelof20thcenturymediaanditisnotatallclearwhetherutopianordystopian
visionswillultimatelyprevail.
Thisprojectservesasapoignantreminderoftheintentionsofmultimedia's
pioneers.Theirwords,typicallywrittenduringtheheatofinvention,conveya
passionateinvolvementwithhigherideals.Toaremarkabledegree,these
scientists,artists,andtheoristsshareacommitmenttoformsofmediaand
communicationsthatarenonhierarchical,open,collaborative,andreflective
ofthefreemovementofthemindatplay.Itis,insum,anextraordinary
vision.Butwhetherwewillachieveitisanunresolvedquestion.