World, Environment, Umwelt and Innerworld

Jan Koenderink

D E C LOOTCRANS P RESS
World, Environment, Umwelt
and Innerworld
Jan Koenderink

D E C LOOTCRANS P RESS , MMXII

Front cover: Frank Frazettas (19282010) Golden Girl (of 1977) illustrates a sin-
gle geographical location and many distinct Umwelts. Even the Umwelts of the mam-
mals (in the image only human, feline, and canine), the vertebrates (here mammal,
avian, and amphibian) are very different, those of the invertebrates (in this case Lep-
idoptera) perhaps completely alien to man. (Plants also have Umwelts, of course. I
skip those.) Frank Frazetta is mainly known for his work in comic books, paperback
book covers, paintings, posters, LP record album covers, and so forth. Frazetta was
no doubt one of the Great in this genre.
Foreword
This short EBook is intended as preliminary reading material for a sum-
mer course in visual perception. It is one of a series of short introductions.
The book was prepared in PDFLATEX, using the movie15 and hyperref
packages. In order to make use of its structure use the (free!) Adobe Reader.
Make sure that all options are enabled! Video and sound clips will start by
clicking (or double clicking) the image, they are embedded in the file.
Make sure you are online. Clicking the light blue text will get you to Inter-
net sites with additional material (try this page!). Be sure to read some of that,
if good material is available elsewhere Ill skip it in the text. Most references
occur only once, saving both you and me time. However, it means that you
may have to backtrack at times.
All links were active when I checked last. I will check them again with the
next reprint.

De Clootcrans Press Utrecht, march 5, 2013 Jan Koenderink

Utrecht The Netherlands
jan.koenderink@telfort.nl

Copyright 2012 by Jan Koenderink

All rights reserved. Please do not redistribute this file in any form without my
express permission. Thank you!
Jan Koenderink
Katholieke Universiteit Leuven
pax / jan koenderink Laboratory of Experimental Psychology
Tiensestraat 102 bus 3711
3000 Leuven
Belgium
First edition, 2012 jan.koenderink@ppw.kuleuven.be
Personal page
10 9 8 7 6 5 4 3 2 1

i

What is this?
The contents of this eBook are the slides of an invited talk held
by me at San Francisco CA at the IS&T/SPIE Electronic Imaging
(Conference 8651 Human Vision and Electronic Imaging XVIII)
meeting, february 2013. The talk was scheduled for an hour, hence
the many slides.
Judging from the responses (of course, most people were nice to
me!) the larger part of the audience failed to get the message. Many
expressed surprise that I apparently didnt believe in reality, a sure
indication of mis-communication. Most of the audience found the
talk entertaining though, as I hope you will. Nevertheless, my aims
(though not my expectations) were set higher. I seriously believe
this to be important stuff! Perhaps people are right in considering
me an old fool.
fallacy to believe that Homo Sapiens is special in the larger scheme of things,
except from being dear (that is: understandable) to us.
I fight such ideas by attacking the Gods Eye View, that most unfor-
tunately still pervades current scientific(?) thinking.
This eBook contains the slides only. They will be hard to follow without
me (the speaker) providing the glue. For those who attended the talk this
eBook might be of some use. Others I would suggest to use the slides for
meditation. When read through fast, the sequence of slides will not make
much sense, and will seem trivial. But I touch on important conceptual issues
that (as you may notice yourself) are not easily exhausted. At least, they are
likely to keep me busy for a lifetime. This only works if you find your own
problems, of course.
For this eBook I added Internet references to the slides. They are indicated
by a yellow icon with a black question mark. Click them (make sure you
are online!) to obtain additional information. At this moment (march 2013)
the free Adobe Reader seems to be the only application that handles dynamic
PDF well though. It is available on most common platforms.

The topic

The topic of the talk involves the relations between awareness, mind, and
the scientific description of the world. I argue that awareness is best under-
stood as an interface with the physical world. Of course, this implies that the
physical world cannot be known, the reason being that interfaces screen the
user from unnecessary complexity. Here the necessity can be interpreted in
terms of biological fitness.
Your reality is made up of interface elements, not replicas of physical
entities. Thus the qualia and meanings of your awareness are like the (arbi-
trary!) icons on your computer desktop: these dont reveal anything of interest
concerning the underlying electronics of the machine.
Another important topic is that different beings have different interfaces,
reflecting their life styles. Even the simplest beings (say an amoeba) have
complete, and (for their life styles) perfect interfaces. It is an anthropocentric

1
World, Environment, Umwelt and Innerworld
a biological perspective on visual awareness

Jan Koenderink

1
Jakob von Uexkll
(1864 - 1944) was a
Baltic biologist who may
be considered the
father of ethology

his idea of Umwelt is

more sophisticated than
the superficially similar
notions of James Gibson

Here if follow up some

ideas in modern context

2
 fuzzy concepts:
physical world
geographical location
environment
ecological niche
Umwelt

3
 Frank Frazetta

although inhabiting the same

physical environment, these
sentient beings (and more!)
live in very different Umwelts

4
the sense world of
- bats is ultrasonic
- dogs is olfactory
- sharks is electric
- humans is visual

5
- rattlers inject poison
- chameleons change color
- electric eels shock
- man displaces mass
the action worlds differ

6
environment Umwelt
action: erect spines
(just in case)

sea urchin

action: flee
(if dynamic)
scallop

action: visit
(among more)
honey bee

7
 8

the oak tree is different for the forester and little girl
both in their sense worlds and in their action worlds

von Uexkll describes the oak tree as experienced by widely

different animals (fox, owl, woodpecker, squirrel, beetle, )
Jakob von Uexkll on UMWELTS:

the space peculiar to each animal, wherever that animal may be,
can be compared to a soap bubble which completely surrounds the
creature at a greater or lesser distance

the extended soap bubble constitutes the limit of what is finite

for the animal, and therewith the limit of its world

what lies behind that is hidden in infinity

the Umwelt is the limit of an organisms world - not only spatially

it contains only what exists in its sense and action worlds
therefore every element has a functional tone (quality, meaning)

the complexity of the Umwelt equals the complexity of the

animals performances - being & Umwelt are not distinguishable 9
 identical
behaviors in the
ants UMWELT

Aphid-ranching ants
10
 the ant & aphid largely share their ecological niches
their Umwelts are almost disjunct - but meshed

11
 12

two Umwelt bubbles:

a fly and a spider

the spiders web accurately

reflects the geometry and
mechanics of the fly
although different bubbles,
they sing a duet together
some magical actions correspond to properties outside the scope of
the current Umwelt! (When & to whom are actions meaningful?)
e.g., pea weevil larvae dig escape tunnels for the (yet to be!) beetles

according to von Uexkll although each actor simply plays its own role,
the result is a play that suggests a concerted action
- this is the harmony of nature (the totality of beings & objects) 13
Indras net illustrates Leibnizs monadology: the monads have no windows,
but each reflects all others in pre-established harmony
von Uexklls Umwelts also mesh - harmony is not pre-established by a
Creator, but von Uexkll does perceive overall structure, not a chaos
where Darwins evolution describes the vertical structure, von Uexkll
describes the horizontal structure of the organic world 14
 a famous example due to
von Uexklls researches

the oldest known fossile

ticks are 90 million years old

ticks once fed on dinosaurs

the (female) tick is a remarkably successful animal

it feeds on mammalian blood, thus it needs a mammal finder
its Umwelt is rather restricted, for the ticks are blind and deaf
15
in the ticks definition a mammal is the conjunction of

- the smell of 1-Propanecarboxylic acid (sweat), and

- a feeling of warmth (about 38C)

according to the American Heritage Science Dictionary 2005

a mammal is

any of various warm-blooded vertebrate animals of the class

Mammalia, whose young feed on milk that is produced by the
mother's mammary glands. Unlike other vertebrates, mammals have a
diaphragm that separates the heart and lungs from the other
internal organs, red blood cells that lack a nucleus, and usually hair or
fur. All mammals but the monotremes bear live young. Mammals
include rodents, cats, dogs, ungulates, cetaceans, and apes

WHICH DEFINITION IS THE MORE EFFECTIVE? 16

the tick has NO SPACE (it hangs motionless at a fixed place),
and it does have NO TIME either (it may hang there for
twenty years before it meets its mammal - from one event to
the next, that is a moment)

it is often held that the backbone of visual awareness is a

pre-established (container-like) SPACE-TIME
apparently such an idea is WRONG! (-> The Gods Eye notion)

without living subject there is neither SPACE, nor TIME

sentient beings vary widely in this respect

each Umwelt has its own space-time
17
 Rizostoma pulmo has only a single reflex arc,
which is repeated systolically

its Umwelt is hardly more than a point

Echinus melo is a reflex republic, it has no CNS,

its Umwelt might be incoherent, like a foam

Octopus vulgaris, is the most intelligent invertebrate

it has a highly structured Umwelt

due to its lack of a jointed skeleton its space must be

very different from ours - the octopus is an alien being
18
sea squirt larvae
are like chordates

after affixing
itself to a
rock the
mature animal
eats its own
brain

the Umwelt of the larvae has space, but the adult doesnt need one:
a sentient being may lose or gain space during the course of its life! 19
 Konrad Lorenz

Niko Tinbergen

the Fixed Action Patterns (FAPs) and Releasers

identified by the founding fathers of ethology reveal

perception to be an idiosyncratic user interface

20
 excellent visual acuity!

obsessive humping of beer bottles endangers survival

of the species of this Australian jewel beetle

even fishes & (fellow) birds get

mixed up - does the bird think
it is feeding chicks?

a breakdown of size constancy? 21

main stream vision research
blindly puts its trust in the
existence of the
Gods Eye View,
the idea being that

there is a unique way things

should look (as seen by Him!)
it is independent of the
observer (objective!)
physics is our best way to
discover this objective reality
limited anatomy/physiology &
mental capacity yields illusion/
error
modern Western man comes
close to seeing things as they
are
22
modern ethology (heritage of von
Uexkll) suggests that all these
notions are nonsense

reality-as-it-is-in-itself is an
utterly incoherent concept -
all claims to reality can be
reduced to intuition

verum factum - Giambattista Vico

the same insight is found with Kant,

Schopenhauer & later

23
 back to basics

the human condition

(biology of awareness
from an evolutionary
perspective)

24
human newborns share a core system with all vertebrates
25

it may be considered the default interface of a vertebrate

the core system includes (at least)

- inanimate manipulable objects

- animate agents
- numbers
- geometrical shapes & space
- social partners
(e.g., Elizabeth Spelke in humans, Giorgio Vallortigara in fish, freshly
hatched chicks, ) before being human, we are vertebrates!
 26
touch hearing sight smell taste

the
human
sense
world

Hans Makart
blind guy

touch was historically held

to calibrate vision

but phenomenologically,
the senses co-exist
harmoniously, no single
sense taking the lead

27
blind monks investigating an elephant
each gains
a different
perspective

28 Itcho Hanabusa
they perceive the elephant as being similar to
a wall
a tree trunk
a snake
a rope
a fan
a spear
their data is not conflicting, but neatly complementary

of course, any awareness is perspectival 29

an easy task:

how many (one or two?) black

blobs are there in each box?

a hard task:

how many (one or two?)

black blobs are there
in each box?

30
an easy task:

how many (one or two?) black

blobs are there in each box?

a hard task:

how many (one or two?)

black blobs are there
in each box?

31
 an easy task:

how many (one or two?) black

blobs are there in each box?

a hard task:

how many (one or two?)

black blobs are there
in each box?

all senses are limited through narrow bottlenecks

of structural complexity - tunnel vision is typical! 32
 visual awareness HAPPENS TO
YOU - you dont DO IT

it does so instantaneously.

reflective thought knows that

awareness is not veridical,
but visual awareness is not
corrected by it

sense and ratio are two

different things, sense is
largely autonomous

Walter Gerbino
33
Grfin Vera von Lehndorff-Steinort (1939-present) famously uses
body painting to create scenes where immediate visual awareness
overrides cognition or even reflective thought 34
visual awareness (sense) might
be called proto-rational

it creates reality!

good interfaces promote fitness,

rather than veridicality 35
 36

humans see faces everywhere

Schiaparelli discovered the Martian canali in 1877
(the back of the envelope drawing dates from 1890)

the crowned face on Mars was discovered by Greg Orme in 2000,

on a NASA image (MOC image M0203051)
37
 Frank Miller

hallucination can be very useful! figure-ground segmentation

is largely automatic and it involves no cognitive factors 38
Chico Hayasaki

Peter Jeroense

contour or figure are handled

gracefully, yet automatically 39
possible renderings of a
common shape are infinite
but no problem! 40
surprisingly, almost any type of rendering reads spontaneously!
there is a creativity in drawing and a creativity in observing, and
there appear to be no limits on either of the two
41
 von Uexklls functional loop is what we
now call the perception-action cycle

the sense world (Merkwelt) and action

world (Wirkwelt) are connected through
the body and the environment

the object appears as the counterpart

(Gegengefge) of an aspect of the
Innenwelt (inner world)

the world in the agents awareness

(inner world) is the counter world or
mirror world to the system of sense-
action relations

42
 von Uexklls functional loop is what we
now call the perception-action cycle

the sense world (Merkwelt) and action

world (Wirkwelt) are connected through
the body and the environment

the object appears as the counterpart

(Gegengefge) of an aspect of the
Innenwelt (inner world)

the world in the agents awareness

(inner world) is the counter world or
mirror world to the system of sense-
action relations

43
 von Uexklls functional loop is what we
now call the perception-action cycle

the sense world (Merkwelt) and action

world (Wirkwelt) are connected through
the body and the environment

the object appears as the counterpart

(Gegengefge) of an aspect of the
Innenwelt (inner world)

the world in the agents awareness

(inner world) is the counter world or
mirror world to the system of sense-
action relations

44
 von Uexklls functional loop is what we
now call the perception-action cycle

the sense world (Merkwelt) and action

world (Wirkwelt) are connected through
the body and the environment

the object appears as the counterpart

(Gegengefge) of an aspect of the
Innenwelt (inner world)

the world in the agents awareness

(inner world) is the counter world or
mirror world to the system of sense-
action relations

45
 46

von Uexkll introduces a new cycle (Neuer Kreis), which is nothing but
von Holst and Mittelstaedts Reafference Principle (of 1950!)

in this figure the object is vaguely marked as cue bearer (Merkmalstrger)

it has become the unreachable Ding an sich of Kant

according to von Uexkll the inner world in terms of qualia & meanings is not subject
to objective, scientific research, it is essentially subjective
 47

von Uexkll introduces a new cycle (Neuer Kreis), which is nothing but
von Holst and Mittelstaedts Reafference Principle (of 1950!)

in this figure the object is vaguely marked as cue bearer (Merkmalstrger)

it has become the unreachable Ding an sich of Kant

according to von Uexkll the inner world in terms of qualia & meanings is not subject
to objective, scientific research, it is essentially subjective
 48

20th c. progress
seeks to explain
the inner world as
a representation
of the physical
world (inverse
optics)

von Uexkll
explains
properties of
objects as
deriving from
their role in the
interface
(counter world)

Fritz Kahn
 in psychology inverse optics
implies behaviorism

(speech is the movement of

air molecules)

objectivist accounts stop at

enactive vision, discounting
visual awareness as
epiphenomenal

visual awareness and brain activity are phenomena

on different ontological levels
there are no causal connections possible
the proper setting for the study of brain
processes is physiology
the proper setting for the study of visual
awareness is experimental phenomenology

49
are humans any different from generic vertebrates?

yes, in (at least) two major respects

- a high degree of intra-species empathy (all primates)
- a well developed linguistic system (sense + ratio)

this enables experimental phenomenology

it can be a science because intersubjectivity may replace

formal objectivity - the alternative is mere behaviorism

thus first person reports are acceptable (visual proofs!)

this is how qualia and perceptual meanings can be the topics
of scientific interest

50
von Uexkll speculates that the counterparts of invariants of the perception-action
cycle will gain a functional tone based on continual experience

this is similar to Gibsons affordances except that Gibson seeks affordance in

physical objects (the stone is throwable by itself) - whereas to von Uexkll a stone
becomes throwable only at the moment you intend to throw it

Erwin Schrdingers in Mind and Matter

proposes a psychophysical linking hypothesis
that connects the functional tones to
meanings and qualities:

if an expectation is falsified in perception,

you meet nature - it is a moment of learning:
it discharges a spark of awareness

51
 52

the contents of the Alexius Meinong (1853-1920):

inner world in man
everything whatever whether
thinkable or not, possible or not,
Meinongs jungle
complete or not, even perhaps
paradoxical or not is an object

very many objects do not exist; and in
many cases they do not exist in any
way at all, or have any form of being
whatsoever

not only is the oft-quoted golden
mountain golden but the round square
too, is as surely round as it is square
in what
sense do
these
objects all objects in visual awareness
exist, their possible world is
exist?
the awareness

all objects in visual cognition

exist, their possible world is
the mind

qualia and perceptual meanings

are more real than mere
abstractions such as electrons

a chair exists (in your mind),

even though it has no meaning
in terms of physics (an odd
collection of molecules
perhaps?)

53
thus we arrive at an interface theory of perception (Im
indeed very sympathetic to Don Hoffmans ideas!)

what is important from a phenomenological viewpoint is that:

- interfaces screen the user from what they dont need to
know, they dont yield veridical knowledge
- qualia and meanings are part of the interface, they have
little to do with the way the world is
- different interfaces yield different Umwelts
- there is no notion of a Gods Eye View that might fit the
perception as user interface concept
- the interface model is the only interpretation of qualia and
perceptual meanings that makes some sense it is essentially
von Uexklls functional tone in Schrdingers notion of the
microgenesis of awareness
54
thank you for your attention
JanKoenderink@ppw.kuleuven.be

55
 The key argument is
OTHER E B OOKS FROM T HE C LOOTCRANS P RESS :
de cloten sullen uyt haer selven een eeuwich roersel maken, twelck
1. Awareness (2012) valsch is.
2. MultipleWorlds (2012)
3. ChronoGeometry (2012) Simon Stevin was a Dutch genius, not only a mathematician, but also an
4. Graph Spaces (2012) engineer with remarkable horse sense. I consider his clootcrans bewijs one
5. Pictorial Shape (2012) of the jewels of sixteenth century science. It is natural philosophy at its
6. Shadows of Shape (2012) best.
7. Through the Looking Glass: on Viewing Aids (2012)
8. Painting to Marble (2012)
9. Experimental Phenomenology: Art & Science (2012)
10. World, Environment, Umwelt and Innerworld (2013).

(Available for download here.)

A BOUT T HE C LOOTCRANS P RESS

The Clootcrans Press is a selfpublishing initiative of Jan Koenderink. No-

tice that the publisher takes no responsibility for the contents, except that he
gave it an honest tryas he always does. Since the books are free you should
have no reason to complain.

T HE C LOOTCRANS appears on the front page of Simon Stevins

(Brugge, 15481620, Den Haag) De Beghinselen der Weeghconst, published
1586 at Christoffel Plantijns Press at Leyden in one volume with De Weegh-
daet, De Beghinselen des Waterwichts, and a Anhang. In 1605 there appeared
a supplement Byvough der Weeghconst in the Wisconstige Gedachtenissen.
The text reads Wonder en is gheen wonder. The figure gives an intuitive
eye measure proof of the parallelogram of forces.