Depth Perception 0

Depth Perception

Suggested reading:!
!  Stephen E. Palmer (1999) Vision Science.
!  Read JCA (2005) Early computational processing in binocular vision and
depth perception. Progress in Biophysics and Molecular Biology 87: 77-108

Depth Perception 1

Depth Perception!
Contents:
Contents:
•  The problem of depth perception
•  •  Was
Depth
is lerning?
cues
•  •  Panum’sMechanismus
Zellulärer fusional area der LTP
•  •  Unüberwachtes
Binocular disparity
Lernen
•  •  Random dotLernen
Überwachtes stereograms
•  •  Delta-Lernregel
The correspondence problem
•  •  Early models
Fehlerfunktion
•  •  Gradientenabstiegsverfahren
Disparity sensitive cells in V1
•  •  Disparity tuning
Reinforcement curve
Lernen
•  Binocular engergy model
•  The role of higher visual areas
 Depth Perception 2

The problem of depth perception!

Our 3 dimensional environment gets reduced to two-dimensional images
on the two retinae.

Viewing Retinal
geometry image

The “lost” dimension is commonly called depth.

As depth is crucial for almost all of our interaction with our environment
and we perceive the 2D images as three dimensional the brain must try to
estimate the depth from the retinal images as good as it can with the help
of different cues and mechanisms.

Depth Perception 3

Depth cues - monocular!

Texture Accretion/Deletion
Accommodation
(monocular, optical)
(monocular, ocular)

Motion Parallax Convergence of Parallels

(monocular, optical) (monocular, optical)
 Depth Perception 4

Depth cues - monocular !

Position relative to Horizon Familiar Size
(monocular, optical) (monocular, optical)

Relative Size Texture Gradients

(monocular, optical) (monocular, optical)

Depth Perception 5

Depth cues - monocular !

!  Edge Interpretation
(monocular, optical)

!  Shading and Shadows

(monocular, optical)

! Aerial Perspective
(monocular, optical)
 Depth Perception 6

Depth cues – binocular!

Convergence Binocular Disparity
(binocular, ocular) (binocular, optical)

As the eyes are horizontally displaced we see the world from two slightly different
viewpoints.

The visual fields of the two eyes overlap in the central region of vision but points
that are not on the so-called horopter fall onto different retinal positions.

This lateral displacement which is relative to the fixation point is called binocular
disparity.

Depth Perception 7

Panum's fusional area!

If we see different views of the world with both eyes why don't we
experience double vision?

Points in an area near the horopter (called Panum's fusional area) are
fused perceptually into a single experienced image.
 Depth Perception 8

Panum's fusional area!

You can easily check this perceptual fact for yourself

Hold one forefinger in front of you and

the other forefinger 30cm behind it. If you
focus on one finger you will see the other
finger twice because it is not within the
limits of Panum's fusional region.

Depth Perception 9

Binocular disparity!

The point that we fixate on (here P) by definition has

no binocular disparity as it falls directly onto the fovea
at the same retinal position in both eyes.

Fixating a point P the image of points

like C that are closer to the viewer
than P are displaced outwardly on
the two retinae in a so-called
crossed disparity, whereas farther points
as F are displaced inwardly in a
so-called uncrossed disparity.

The direction and amount of binocular

disparity therefore specify relative depth
information in regard to the horopter.
 Depth Perception 10

Binocular disparity!
Example of the use of binocular disparity for the experience of depth -
Anaglyphs

Nothing new, discovered in the 1850s by Joseph D’Almeida and Louis Du

Hauron,
William Friese-Green created the first 3D anaglyphic motion pictures in 1889.

Depth Perception 11

Random dot stereograms!

Until about 40 years ago, a viable theory was that the visual system operates on
the two retinal images separately, performing scene segmentation and object
recognition, and then finally, for each object in the scene, compares where its
images fall in the two retinae.

However, in the 1960s, Bela Julesz drew attention to the fact that stereopsis
works perfectly well with random dot stereograms.
 Depth Perception 12

Random dot stereograms!

Julesz,
1971
1. two identical images with random dots
2. cut out a shape that you want to appear in a different depth plane and
slide it to the right in one image – the amount will determine the depth
3. fill the empty hole that has appeared through the sliding with random dots
4. put the original image left to the newly created one – that’s it!

Random dot stereograms show that binocular disparity is such a strong

depth cue that it does not need any monocular cue for depth perception.

Depth Perception 13

The correspondence problem!

Something is missing – the correspondence problem!

Until now we have talked about how the direction and amount of disparity
between corresponding image features in the two eyes can encode depth
information but the problem we have excluded until now is how to determine

which features in one retinal image correspond to which features in the other?

Computer Vision:
match contents of image parts
using global cost functions.
 Depth Perception 14

The brains solution to correspondence

problem!

Depth Perception 15

Early models!
First Marr-Poggio Algorithm (MIT 1977)
Approach: matching individual pixels in the left and right image, inverse
projection
Constraints
1. Color constancy:
corresponding points have the
same color

2. Surface opacity: most surfaces

in the world are opaque so only
the nearest can be seen !
If A10 is a correct match, then
B10, C10, D10 etc. cannot be
correct.

3. Surface continuity: surfaces in

the world tend to be locally
continuous in depth !
excitatory connections between
close matches

Note! The constraints are

heuristic assumptions that are
 Depth Perception 16

Early models!
First Marr-Poggio Algorithm – dynamic behavior of the network

Depth Perception 17

Disparity sensitive cells in V1!

In the human visual system the inputs of the two eyes converge upon
single cells in V1.

Position disparity Phase disparity

From: Qian, Neuron, 1997.

Recent studies indicate that most disparity sensitive cells are hybrid, showing
both position and phase disparity (Ohzawa, Freeman 1997/1999 – Livingstone,
Tsao 1999 – Prince, Cumming, Parker 2002).
 Depth Perception 18

Disparity tuning curve!

Schematic Drawing of a Disparity
Tuning Curve for a Binocular Cell:

Three representative retinal stimulus

configurations correspond to a point in
space that is nearer than, at, or further away
from the fixation point.

Depth Perception 19

Disparity tuning curve!

Idealized tuning curves after characterization by Poggio & Fischer (1977)

Disparity tuning: tuned-excitatory, near, tuned-inhibitory, and far cells.

In each graph, the horizontal

axis is disparity and the
vertical axis is response rate.

The curves are Gabor

functions (the product of a
cosine and a Gaussian); the
angle represents the phase of
the cosine component.
 Depth Perception 20

Binocular Energy model!

Ohzawa et al., 1990; Qian, 1994, Read and Cumming 2007

A complex cell Cx receives input from binocular simple cells (BS). Receptive
fields of two BS that differ in their phase by 90° get combined to a complex cell
Cx.

BS =

Cx = BS + BS

Depth Perception 21

Binocular Energy model !

Image value at
position (x,y)

Receptive field of
equivalent left eye simple cell

With the Gabor filter

preferred preferred
position phase
disparity disparity
with standard and
deviation
 Depth Perception 22

Solution to the correspondence problem!

V1 neurons are sensitive to local matches, not to global solutions of the
correspondence problem.

Anti-correlated stereograms

With the discovery of V1 disparity sensitive cells the question arose if the
correspondence problem was solved in V1 already or in a higher brain region.

One approach to test for this were anti-correlated stereograms. These consist
of normal stereograms but the brightness information in one of the images is
reversed in comparison to the other image. Because of this there is no global-
match solution to the stereo correspondence problem in anti-correlated
stereograms.

Nevertheless the disparity sensitive cells in V1 respond to the local false

matches in anti-correlated stereograms which is a strong indication that the
correspondence problem does not get solved in V1.

Depth Perception 23

The role of higher visual areas!

V2: relative disparity and
disparity-defined edges
(Thomas et al. 2002, von
der Heydt et al., 2000, Qiu
& von der Heydt, 2005)
V5/MT: Microstimulation can bias
the monkey’s judgement in
a far-near stereo task that
relies on absolute disparity
(DeAngelis et al. 1998), but
no strong support for other
aspects of stereovision.

V4: sensitive to the orientation of disparity defined planes (Hinkle & Connor,
2005).
 Depth Perception 24

Vergence Control!

Depth Perception 25

Additional reading:!
•  Qian N. (1997) Binocular disparity and the perception of depth. Neuron, 18:359-68.!
•  Qian N, Zhu Y. (1997) Physiological computation of binocular disparity. Vision Res,
37:1811-27. !
 Depth Perception 26

References:!
•  Hinkle, D.A. and Connor, C.E. (2005) Quantitative characterization of disparity tuning in
ventral pathway area V4. Journal of Neurophysiology, 94:2726-37.
•  Ohzawa I, DeAngelis GC, and Freeman RD (1990) Stereoscopic depth discrimination in the
visual cortex: neurons ideally suited as disparity detectors. Science 249:1037-1041.
•  Poggio GF, Fischer B. (1977) Binocular interaction and depth sensitivity in striate and
prestriate cortex of behaving rhesus monkey. J Neurophysiol, 40:1392-405.
•  Qian, N (1994) Computing stereo disparity and motion with known binocular cell properties,
Neural Computation, 6: 390-404.!
•  Qiu, F.T. and von der Heydt, R. (2005) Figure and ground in the visual cortex: V2 combines
stereoscopic cues with Gestalt rules. Neuron, 47:155-166.!
•  Read JCA, Cumming BG (2007) Sensors for impossible stimuli may solve the stereo
correspondence problem. Nature Neuroscience, 10:1322-1328. !
•  Thomas OM, Cumming BG, and Parker AJ (2002) A specialization for relative disparity in V2.
Nat Neurosci 5: 472–478.!
•  von der Heydt, R., Zhou, H. and Friedman, H.S. (2000) Representation of stereoscopic
edges in monkey visual cortex. Vision Research, 40:1955-1967.!