19/11/2016

Gestalttheory:VisualandSonicGestalt

Gestalttheory:VisualandSonicGestalt
AlexMcLean,MScArtsComputing
1stNovember2005

Introduction
Gestalttheoryputsforwardtheprimacyofthewholeinperception,notasanaccumulationofperceptionsofitspartsbut
insomethingmore.Ourperceptionsaresubconsciouslyconstructedtowardstheendofcreatingaperceptionasstable,or
as"good"aspossible.Ourconsciousmindisthenpresentedwiththisorganisedperceptofthewhole.
Gestalttheoryisweakwhenitattemptstojustifyitselfthroughphysiologicalexplanation.Wesimplydonotyetknow
enoughaboutthebraintoexplainperceptioninthisway,andsotogetatthetruthweisolatefigureswhichevokeGestalt
percepts,thatisperceptsshowingfeaturesnotinthepartsbutinourperceptualconstructionofthewhole.
ThefocusofGestalttheoryhaslargelybeenuponvisualphenomena,butgoodresearchisbeingdoneonsonicandmusical
Gestalttheorytoo.TheessaywillexploretheprinciplesofGestalttheoryusingbothvisualandauralexamples.Itwillthen
lookatsomeoftheapplicationsofGestalttheorywithinthestudyofmusic,andconcludewithananalysisofsomeofthe
criticismsofthetheory.

CommonalitiesbetweenVisualandSonicGestalt
Fromoneperspective,visualandsonicperceptioncouldnotbemoredifferent,oneisspatialperception,theother
temporal.Howeveritiseasytoseehoweachcontainselementsoftheothervisualperceptionalsochangesovertime,
whenwelookatmovingorchangingforms,evenwhenweseeastaticimageoureyesmoveacrossitinmeaningful
patterns.Further,ourtwoearsallowustodetectdistanceanddirection,andourmusicalsensibilitiesperceivemovement
inaspacedefinedbysuchdimensionsastimbre,pitch,duration,distortion,resonanceandsoon.
GjerdingenappliestheGrossbergRuddneuralmodelofapparentmovementinvisiontomusicwithfavourableresults.He
goesontohighlightstrikinganalogsbetweenvisualandauralperceptionsuchasluminance/amplitudeand
colour/timbre.Butyettheoverallexperienceofseeingandhearingseemtobesuchdifferentexperiences.Gjerdingenoffers
acompellingexplanationforthis,thatwhilehighlevelcognitionofvisionandsoundmaybeanalogouslyweak,theirlow
levelneuralprocessesshowstrikingsimilarities.Thatis,eventhoughsoundandlightareverydifferentmediums,the
brainmayprocesstheminverysimilarways.
Withthislowlevelcommonalityinmind,IhaveexploredGestaltprinciplesbyconstructingbothvisualandaudio
examplesinthefollowingsection.

SonicvsVisualGestaltexamples
Thefollowingexamplesaimtoillustrate,inastraightforwardmanner,presenceofgestaltprinciplesacrossvisualand
auralperception.Notethattheaudioexamplesareofmyowndevising,andhavenotmetwithrigorousscientifictesting.
TheprinciplesexemplifiedarebasedonthoseseeninlecturesbyProfessorLeymarie,GoldsmithsCollege,2005.
Whenviewedwithajavaenabledbrowser,thevisualexampleswillbeclickabletorevealorhighlightagestalteffect.They
weremadeusingtheprocessingjavaenvironment.TheaudioexampleswererenderedusingPerl,insomecasesusingmy
feedback.plenvironmenttriggeringsynthesisbymydatadirtsoftware.Sourcecodeforallexamplesislinked.

Proximity/Contiguity

figure1.Proximity

Intheaudioexamplewehearidenticalsoundsplayed,butgroupthem
intosetsoffourandthreesoundsbecauseoftheirproximityonthe
musicalsurfaceofthetimeline.Thisisanalogoustothegroupingof
visualpropertiesbyproximity,forexampleinthetwodimensionalspace
offigure1.
audio:(ogg|mp3|sourcecode)
http://doc.gold.ac.uk/~ma503am/essays/gestalt/

Staticversion(javaunavailable)
1/6

19/11/2016

Gestalttheory:VisualandSonicGestalt

Similarity

figure2.Similarity

Thisaudiosampledemonstrateshowwegroupsoundsbysimilarity.The
soundsarethesamedistanceaparttemporally,butarepitched
differently.Weseeananalogouseffectusingshadinginfigure2.
audio:(ogg|mp3|sourcecode)
Proximityandsimilarityinconflict

Staticversion(javaunavailable)

Inthefollowingaudiosampleyoucanheardifferencesinpitchsuggestinggroupingsinconflictwiththosesuggestedby
differenceintemporaldistance.Youcanseethevisualequivalenteffectbyclickinguponfigure2.
audio:(ogg|mp3|sourcecode)
Gestaltprinciplesleadustowardscertaingroupingjudgements,butaswehavejustseenandheard,theyoftenconflict.
Thisleadstoanuncertainjudgement,orinsomeextremecasesanunstableone,whereweflipbetweentwojudgements(as
infigure6).However,usuallyoneprincipleoverrulestheother,andtheserelativestrengthsmaybemeasuredthrough
controlledtesting.SomeofthetestsperformedinthestudyofLerdahl&Jackendoff'sGroupingPreferenceRules(GPR)are
exploredlaterinthisessay.

Closure/Goodcontinuation

figure3.Closure

Thisaudioexamplecontainsanumberofevenlyspacedsoundsoffixed
duration,withsilenceinbetween.However,togetherthesoundsarelikely
tobeperceivedasacontinuousmovement.Theeffectisstillpresentwhen
listeningonclosedbackedheadphones,whichrulesoutenvironmental
reverbasafactorinjoiningthesounds.
audio:(ogg|mp3|sourcecode)

Staticversion(javaunavailable)
Figures3and4showequivalentvisualeffects,showingadistinction
betweenclosurewherelinesarejoinedtogethertofindthesimplestform,
figure4.Continuation
andcontinuationwherewepreferthesimplestpathwhenperceivinglines.
Afterclickingonfigure3werealisethattheperceivedwhitetriangleis
invisible,andclickingonfigure4revealsanalternate,lesspreferable
groupingofthelinesthanwewouldnormallysee.
Figure3isavisualillusionthatfoolsusintoperceivingwhatisnotthere.
Sostrongarethecluestoasuperimposedwhitetrianglethatwearelikely
toseeaphantom,illusionaryshadingaroundit.Equivalentsonic
Staticversion(javaunavailable)
illusionsexisttooDianaDeutschresearchesintheareaofsonicand
musicalillusion,findingarrangementsthatlistenerstendtoperceiveasdifferentfromtheirreality.Forexample,her
"octaveillusion"causespeopletoheardifferenttonesindifferentears.Whatishearddiffersfrompersontopersonbut
rarelyreflectstheactualpitchandpanningofthesounds.

Area/smallness

figure5.Area

Figure5showthatareahasaroletoplayinperception,wetendto
perceivethesmallerareaasfigureandthebiggeroneasground.Figure
beinganobject,groundbeingthebackgroundonwhichitisplaced.
Iattempttoreproducethevisualeffectoffigure5inaudiobycontrasting
burstsofwhitenoisewithburstsofsilence(ifyoucanhaveaburstof
silence).Ichosethiscontrastbecausebeingdevoidofanyfeatures,white
noiseisstructurallysimilartosilence.Indeedinmanysituationsthe
braininterpretsnoiseassilence,forexamplethebackgroundhissofa
radioortaperecording.[1]

Staticversion(javaunavailable)

Thedurationofthenoiseandsilenceisvariedasananalogtovaryingareainfigure5.Isuggestitiseasiertointerpretthe
shortersilenceasthefigureagainstthegroundofthenoiseinaudioexampleone,andtheshorternoiseasfigureagainst
thegroundofsilenceinexampletwo.
http://doc.gold.ac.uk/~ma503am/essays/gestalt/

2/6

19/11/2016

Gestalttheory:VisualandSonicGestalt

Example1:
audio:(ogg|mp3|sourcecode)
Example2:
audio:(ogg|mp3|sourcecode)

Figureandground

figure6.Figureground

Wehavealreadyencounteredfigureandgroundinthepreviousexample.
Suchinterrelationshipsandoverlapsarefoundbetweenmanyofthe
Gestaltprinciples.Indeed,areacouldalsohavebeenusedtoillustratethe
similarityrule,commonlysizedshapesbeingmorelikelytobeperceived
asagroup.
Figure6showsanunstablefiguregroundrelationship.Wecaneithersee
twosymmetricalfacesoravase,butprobablynotbothatthesametime.
Fortheaudioexample,Ichoseadifferenttechniquetoexplorefigureand
groundrelationshipspolyrhythm.

Staticversion(javaunavailable)

audio:(ogg|mp3|sourcecode)
Thispolyrhythmhasfourrepeatingsounds,eachspaceddifferently.Onesoundplayseverythirdtimeunit,oneevery
fourth,oneeverytenthandoneeveryfourteenth.Thiscreatesapatternthatrepeatsevery840thtimeunit,butitwould
seemlikelythatwewouldperceivealoopingsoonerthanthat.Iperceiveitrepeatingeverytwelfthunit.Thisperceived
looppointcouldbeconsideredthegroundoverwhichtheotherfeaturescreatecombinationsasvaryingfigures.Ourchoice
ofgroundandfigurecouldbeaproductoffeaturesofthesoundsthemselvessuchasvolumeandattacktime,aswellasthe
mathematicalinteractionsbetweenthedifferentspacings.

MusicalGestalt
WesternmusicaltheoryconsidersGestaltTheoryintwomainareasgrouping/segmentationandexpectation,whichI
shallexploreseparately.

GroupingandSegmentation
In"AGenerativeTheoryofTonalMusic",FredLerdahlandRayJackendoffdefinesetsofrulesforanalysingmusic.Forthe
purposesofthisessay,wefocusonthegroupingrules,whichareseparatedintotwosets,firstthefiveGroupingWell
FormednessRules(GWFR)andsecond,thesevenGroupingPreferenceRules(GPR).TheGWFRsimplydefinesome
structuralconstraints,simplyput:
GWFR1:Agroupisacontiguoussequenceofsoundevents.
GWFR2:Amusicalpieceisagroup.
GWFR3:Agroupmaycontainsmallergroups.
GWFR4:Agroupmaynotbelongtomorethanonegroup.
GWFR5:Agroupmustbeexhaustivelypartitionedintogroupsifitispartitionedatall.
Sowhatweareleftwithisasimpletreestructure,wheregroupscontaingroups,thebiggestgroupbeingawholemusical
piece,andthesmallestgroupbeinganoteordrumbeatthatcannotbepartitionedfurtherwithoutgoingbeyondthelimits
ofclassicalmusicalnotation.Betweenthoseliegroupingssuchasmotives,themes,phrases,periods,themegroupsand
sections.
TheirGPRaretherulesforhowthesegroupsmightbechosen.Theynameseven,whichIsummarise
GPR1:Stronglyavoidgroupscontainingasingleevent.Avoidanalyseswithverysmallgroupsthesmaller,the
lesspreferable.
GPR2:Proximitytendtowardsmarkingboundarieswhereagapisheard,measuredeitherastimebetweentheend
ofonenoteandthebeginningofthenext(therest)orasthetimebetweentheirattackpoints(theinteronset
intervalorIOI).
GPR3:Changeinregister,dynamics,articulation,length,timbreorinstrumentation.
GPR4:IntensificationwhereeffectsofgroupingsformedbyGPR2and3becomemorepronounced,thegroups
maybesplitintotwohigherlevelgroups.
GPR5:Symmetrytendtowardsgroupingtwopartsofequallength.
GPR6:Parallelismwheresegmentsofmusicaresimilar,tendtowardsgroupingtheminthesameway.
GPR7:TimeSpanandProlongationalStabilitytendtowardslargescalegroupingsthatallowthegreatest
stabilityofthegroupingswithinit.
http://doc.gold.ac.uk/~ma503am/essays/gestalt/

3/6

19/11/2016

Gestalttheory:VisualandSonicGestalt

TherelativestrengthsoftheirrulesarelargelyunspecifiedanduntestedbyLerdahl&Jackendoff,buttheyexplicitlyinvite
otherstoresearchfurther.Herefollowsanexplorationofsomeofthosewhoacceptedthisinvitation.
Frankland&CohenidentifiedtheGPR2proximityruleandtheGPR3changeruleasbeing"localgroupingrules"most
fundamentaltolowlevelgroupingstructurewithintheGenerativeTheoryofTonalMusic.Theytestedandquantifiedthe
RestandAttackPointaspectsofGPR2andtheRegisterandLengthaspectsofGPR3.Theresultsbroadlysupportedthe
overalltheory,butfoundtheAttackPointruletobebyfarthestrongestofthosetheymeasured,andsuggesteduseofthe
otherrulestobetemperedinlightofthis.
ThoughhisstudyoftheLocalBoundaryDetectionModel(LBDM),EmiliosCambouropoulosextendsGPR3tosuggest
thatchangeisnotalwaysongroupingboundaries,butbetweenthetwoeventsthatproceedaboundary.Inexperiments
withexpertperformancesoftheworkofChopin,hefoundthatwhilelessthan50%ofnotesonboundarieswere
lengthened,92%ofnotesproceedingthemwerelengthened.ThesefigurescouldbearrivedatbecauseCambouropoulos
chosea20barsectionofapiecewhereboundarieswereunambiguousanduncontentious.
Cambouropoulosproposesonehypothesisthatmightexplainthistendencyforthepenultimatenoteinagrouptobe
emphasised"WhenanoteIOIislonginrelationtoitssurroundingnotes,furtherlengtheningshouldbequitesignificant
inabsolutetermsforittobeperceptiblewhereasamuchsmallerlengtheningofaprecedingshortnote)delayoflongnote)
ismoreeffective."Aninterestingideathatrequiresfurtherresearchandtesting.
ThetestsofDelige[1987]backedupaninterestingassertionbyLerdahl&Jackendoffthatgroupingsaremoreeasily
identifiedbymoreexperiencedlisteners,whichDeligeclarifiedastrainedmusicians.Howevershefoundnonmusicians
alsomostlymadesegmentationsinlinewiththerules,andsoconcludesthatthesegroupingpreferencerulesmaybe
appliedbroadly.
DeligeandFrankland&CohenwereinagreementinsuggestingthattheGPRareincomplete.TheresultsofDelige'stests
suggestaneedforarulesbasedonchangesinharmony,andshegoesontosuggestadditionalrulesmightbebasedon
sounddensity.

Expectancy
TheothermajorareaofinfluenceofGestaltovermusictheoryisonexpectation.
WertheimerandKoffkapointedoutthatperceptionisnotmerelyaproductoftheenvironment.Althoughourinfluence
overwhatweperceiveislargelyunconscious,itisnonethelessactive.Whatwehaveperceivedbeforehasinfluenceover
whatwewillseeagain.SupportingevidenceispopularlyfoundwithDalmatianasphotographedbyR.C.James(figure7).
Thishelpsexplainexpectationasaculturalphenomenon.Welearnto
expectthingsbasedonwhatwehaveexperiencedbefore.Itismucheasier
toparsethedalmatianimagehavingalreadyseenit,evenwhenmany
yearshavepassedinthemeantimeourmemoryinsomewaytellsus
whattoexpect.Butwhathappenswhenthatexpectationisunfulfilled?

figure7.Veridicalexpectation

Figure8showsanattempttoscramblemanyoftheidentifyingelements
thatformtheperceptofthedalmatian.Havingseenhersomanytimes,
weexpecttodosoagaininthisalteredimage.Wequicklyrealisethatshe
hasgone,butnonethelesstheexpectationofherpresenceremainsinher
place.[2].
Thisphenomenonisobservableintheworldofmusictoo.Eveninawell
knownandlovedpiece,unfulfilledexpectationremainsanimportant
featureofthemusic.Howcanwecontinuetoexpectsomethingthatwe
knowissimplynotthere?
Dalmatian,byR.C.James
Theanswermaybefoundbydividingexpectationintotwotypes,
schematicandveridical.Schematicexpectationisbasedoncultural
figure8.Schematicexpectation
experiencesoftheworldofmusicandveridicalexpectationisbasedon
previouslistensofthepieceinquestion.Schematicexpectationiswhatwe
areparticularlyinterestedinhere,anautomaticexpectationbasedon
culturalexperienceofmusicingeneral,anunsuppressableexpectation
evenwhenweknowitwillgounfulfilled.Acertainsoundisstillimplied
bythesoundsthatprecededit,eventhoughweknowfromprevious
listensthatthatitwillnotoccur.
http://doc.gold.ac.uk/~ma503am/essays/gestalt/

4/6

19/11/2016

Gestalttheory:VisualandSonicGestalt

Whatkindofexpectanciesariseinmusic?Conventionalknowledgein
westernmusicsaysthataskip,orextendedpause,givesexpectancyofa
reversal,orchangeindirection.ThiseffectisattributedbyEugene
Narmour(1989)totwoprinciplesregistralreturnandregistraldirection.
AccordingtoNarmour'sfirstprincipleofregistralreturn,thelistener
expectsanintervaltobeoppositeindirection,butsimilarinsizetothe
onethatprecededit.NarmourattributesthisexpectationtotheGestalt
ruleofsimilarity.
Narmour'ssecondprincipleofregistraldirectionisgovernedbythesizeof
intervalsbetweennotes.Itpredictsthatasmallintervalcreatesthe
expectationofmotioninthesamedirection,andalongerintervalcreates
expectationofachangeindirection.Theseexpectationscanbeattributed
totheGestaltrulesofgoodcontinuationandsymmetryrespectively.

NoDalmatian,originallybyR.C.James,
scrambledbyAlexMcLean

Bharucha(1994)alsowroteofhisresearchinexpectationinmusic,but
tookadifferentapproachtotestingthetheory.Insteadofusinghumansubjects,heconstructedartificialneuralnetworks.
Heputsforwardtheideathatifyoucangetsuchamodeltodemonstratelearningofexpectations,youcansuggestthat
humansensesmightfollowasimilarmodel.
Bharucha'sapproachseemsfrustratingononelevelwhatevertheresults,itonlyallowsastartingpointforsuggestinga
theory.Wedonotknowagreatdealaboutthebrain,butcanbesurethatartificialneuralnetworksarenotanaccurate
modelofitsfunctioning.SoBharuchadoesnotproveordisproveanythingbeyondthesuccessofanartificialmodel.
Nonetheless,Bharucharelatesthathisnetworkisindeedabletolearnamusicalschemaandexhibitschematic
expectation,aslongaswedonottaketheneuralnetworktooliterally,theresultsareofinteresttous.

Conclusion
Likeanytheory,Gestalthasitscritics.ItisoftenarguedthatthelargenumberofGestaltprinciplesaresoflexiblethatthey
couldfitanypatternoftestresults.Pomerantz(1986)arguesthat"thelaws...aredistinctlydisorganized...thereis
considerablevariationinthenamesthelawsaregiven,theirdescription,howtheyaregrouped,andhowmanyofthem
thereare."Healsopointsoutthatthenumberofgestaltlawsvaryfrombetween114toasingle,allencompassinglaw.
However,Pomerantzthengoesontoproposea"neoGestalt"psychologythataddressesmanyoftheseconcerns,withgood
referencetothespiritoftheoriginalGestaltclaims.SohisargumentisnotwiththespiritofGestalttheory,butwithits
originalimplementation.
Withthiskindofcriticisminminditiscertainlyworthbeingcarefulwhileresearchingaroundthissubject.Thetendency
shouldbetowardsscientificanalysis,usingGestalttheoryasastartingpointtowardsamorerigorouscorpusofrules.Any
vaguenessinthetheorycanthereforebecounteredwithtesting,measuringandrefinement.Partofthisprocessmay
involvedeemphasisingpartsofthetheory.AsNarmoursuggestswithreferencetoPomerantz,perhapsweshouldfocuson
thebottomupGestaltprinciplessuchassimilarityandproximity,principlesthatare"measurable,formalizableandthus
opentoempiricaltesting."
Howevertodiscardprinciplessimplybecausethey'reunformalisablewouldbeamistake.Inthepracticeofcomposing,
formalconsiderationsmayonlybeastartingpoint,andsubjectivejudgementsmaybecomecentral.PaulKlee(tr.1953),a
painterwhoisknowntohavefollowedGestalttheoryputsitthisway"Alreadyattheverybeginningoftheproductiveact,
shortlyaftertheinitialmotiontocreate,occursthefirstcountermotion,theinitialmovementofreceptivity.Thismeans:
thecreatorcontrolswhetherwhathehasproducedsofarisgood."Thisimpliesthatbetweencreativeactsontheirworkin
progress,theartistconstructsthe"goodness."Klee'schoiceofthewordcontrolhereisperhapsrevealing,itimpliesthatin
themidstofacreativeact,responsibilityfordefining"good"istakenawayfromtheoryandplacedinthehandsofthe
artist.Ifthisisthecase,thatwouldexplainwhyGestaltprinciplescanbevague,thedetailofthetheoryisremadeduring
theconstructionofeveryartwork.
WhateverKleemeant,it'sclearthatweshouldn'tattempttooverconstrainartisticpractice,andlikewiseshouldn't
encouragewild,ungroundedscientificresearch.Butthen,perhapsGestalttheoryasawholecanprovidelanguagetoallow
bothsidestomeet,converseandconverge.

Notes
1.AccordingtoJohnCage,truesilenceisimpossibletoachieve.Evenwhensittinginananechoicchamber,Cage
heardsounds,whichheattributedtothelowpitchedsoundofbloodrushingthroughhisveinsandthehigh
http://doc.gold.ac.uk/~ma503am/essays/gestalt/

5/6

19/11/2016

Gestalttheory:VisualandSonicGestalt

pitchedsoundofhisnervoussystem(perhapsthatwasjusttinnitus,though).
2.Althoughitisperhapsinterestingtonotethathavingstaredatthescrambledimageforawhile,itseemstobecome
easiertonotseethedalmatianintheoriginalimage.

Bibliography
F.LerdahlandR.Jackendoff.AGenerativeTheoryofTonalMusic.MITPress,Cambridge,MA,1983.
I.Delige.Groupingconditionsinlisteningtomusic:AnapproachtoLerdahlandJackendoff'sgroupingpreference
rules.MusicPerception,4(4):325360,1987.
B.W.FranklandandA.J.Cohen.Parsingofmelody:Quantificationandtestingofthelocalgroupingrulesof
LerdahlandJackendoff'sAGenerativeTheoryofTonalMusic.MusicPerception,21(4):499543,2004.
R.O.Gjerdingen.Apparentmotioninmusic?InN.GriffithandP.M.Todd,editors,MusicalNetworks:Parallel
DistributedPerceptionandPerformance,pages141173.MITPress/BradfordBooks,Cambridge,MA,1999.
JamesRPomerantz,VisualFormPerception:AnOverview,PatternRecognitionbyHumansandMachinesVol2,
Ed.ShwabandNusbaum,1986.
J.J.Bharucha.Tonalityandexpectation.InR.Aiello,editor,MusicalPerceptions,pages213239.Oxford
UniversityPress,Oxford,1993.
J.Cage,Silence,WesleyanUniversityPress,1973.
D.Deutsch,Anauditoryillusion.Nature,1974,251,307309.
D.Deutsch,MusicalIllusionsandParadoxes(CDsleevenotes),2003.
E.Narmour.Thegeneticcodeofmelody:Cognitivestructuresgeneratedbytheimplicationrealizationmodel.
ContemporaryMusicReview,4:4563,1989.
P.Klee,(tr.MoholyNagy)PedagogicalSketchbook,1953,Faber&Faber

http://doc.gold.ac.uk/~ma503am/essays/gestalt/

6/6