Cogs 101A

Midterm 2
Review
Lecture 11 - 18

General Information
Test on Monday during lecture
Lecture 11 - 18
Materials from after midterm 1
Still review key concepts from midterm 1
Marrs levels of analysis
Experience-dependent plasticity, top-down processing
Hierarchical information processing
How receptive fields change after convergence

Visual Processing Streams

Object discrimination task &

landmark discrimination task with
monkeys
Isolation of visual processing
streams in human
Length estimate task - what
pathway
Grasping task - where pathway

What pathway

Where/how
pathway

Ventral stream;
projection to
temporal lobe

Dorsal stream;
projection to
parietal lobe

Target
identificati
on

Using
landmarks
& spatial

Types of Evidence
Ablation studies with monkeys
Training -> Ablation of specific brain
parts -> Retrain -> Test
Object discrimination task (scores
lower when temporal lobe is
ablated)
Landmark discrimination task
(scores lower when parietal
lobe is ablated)
Line length task
Length estimate task - what

Concept: Single and Double

Disassociation
Single dissociation: Two functions/abilities seem to involve different brain mechanisms
Double dissociation: Two functions/abilities involve different brain mechanisms and
operate independently
Need at least two people to have double dissociation
Eg. Stroke study

Patient A:
Temporal
lobe damage
(ventral
stream)

Naming
Objects

Determin
ing
Objects
Location

NO

YES

Types of Sensory Codes

Specificity Coding
A single neuron represents each object
Grandma cell
What if we lose the neuron?
Will lose the object with one cell
death & not as efficient

Distributed Coding
Representations are instantiated in the
brain by firing patterns of many groups
of neurons
Allows fewer neurons to represent more
stimuli more efficient
More resistant to damage

Versions of Distributed Coding

Sparse coding
place constraints on the number
of neuron groups involved
with a representation fewer
groups are needed
Extreme case: specificity code

Population coding
a larger, more expansive set of
neurons are involved in
representation more
groups of neurons in
representation

Models of Object
Perception/Representation
Domain specificity models
certain brain regions are dedicated to
specific categories (domains) of objects

Nativist view - more modularity, less

plasticity
Generally expect to find sparse codes

Eg. Infero-temporal (IT) neurons in

monkeys & facial recognition
In human, under IT: Fusiform gyrus/
Fusiform face area

Property-based models
object knowledge is linked to sensory and
motor attributes of the item beyond
physical features

Constructivist/embodied cognition view less modularity, more plasticity

Generally expect to find population codes
Eg. Experience dependent plasticity in humans
Fusiform gyrus (FFA) neurons & Greebles
training tunes fusiform gyrus (FFA)
neurons to respond to novel or
non-face sti & identify objects of
certain category (expertise)

Representing Objects
Prosopagnosia - inability to recognize/distinguish faces
Damage to FFA, a visual deficit
can occur when you have damage to other areas

Pareidolia - perceiving a vague stimulus as something of importance

The visual system may be biased to create (recognize) a perception

Other Suggested Modules in

Temporal Cortex

Parahip
pocam
pal
gyrus
(PPA)

Res
pons
e
best
to
spati
al
layo
ut/
spati
al
relati
onsh
ip of
phys
ical

Binocular Rivalry
Each picture shown to one eye at the same time
Can only perceive one at a time
Parahippocampal place area
Fusiform face area
Demonstrates how perception and recognition correlate
with specific brain activity
Can be trained to see and attend specific stimuli
Same distal & proximal stimulus
Top down information changes experience

Obstacles of Object Perception

Inverse projection problem: An image on the retina can be caused by an
infinite number of objects
Different distal stimuli can create similar proximal stimulus
Occlusion: Objects can be hidden or blurred.
Objects look different from alternative viewpoints
Viewpoint invariance: the ability to recognize an object regardless of the
viewpoint/perspective

Object Perception Models

Structuralism: perceptions are
created by combining
elements called
sensations.

Gestalt Principles

Hierarchical, well-ordered
approach to perception.
Strongly bottom-up (an
algorithmic account)

8 principles

Could not explain apparent

movement.

Strong top-down influence

Heuristic, going through steps
(algorithmic)

Gestalt Principles
Prag
nanz
(goo
d
figur
e/sim
plicity
)

Proxi
mity

Simil
arity

Unifo
rm
conn
ected
ness

Figure-Ground Segregation
Determining what part of the environment is the figure and which is the
background
A highly spatial task, involved with use/location (dorsal stream)
Figure: more thinglike (i.e. can be acted upon) and more memorable than
ground
In front of the ground (symmetrical, small, oriented vertically, meaning)
Border ownership: The contour separating figure from ground belongs to
the figure
Ground: more uniform (e.g. one color or texture) and extends behind the figure

Recognition-by-components
theory
Address the issues of identifying occluded objects or seeing objects from
different viewpoints
Geons: objects that are recognized by volumetric features
Properties:
View-invariant properties: aspects of the object that remain visible from
(most) different viewpoints
Non-accidental properties: properties of edges in the retinal image that
correspond with the 3-D environment
Discriminability: the ability to distinguish geons from one another
Principle of componential recovery: ability to recognize an object if we

Perceiving Objects & Scenes

Scene - contains background elements & objects arranged in a meaningful
way
Note: a scene is acted within while an object is acted upon (both related to
dorsal pathway)

Gist: a quick understanding and recognition of major elements in a complex

picture (250ms; 27-500ms)

Learned Regularities
Physical regularities
Oblique effect: vertical and horizontals are easier to identify
Uniform connectedness: objects defined by areas of same color/texture
Light-from-above heuristic: light in natural environment comes from
above us
Semantic regularities
characteristics associated with the functions of scenes

Contextual Modulation
Neurons in V1 respond to Gestalt principles
Contextual modulation: stimuli outside of a neurons (classical) receptive
field can affect neural firing rates.
When a stimulus is presented with a congruent background, cells in V1 will
be activated
Feedback from dorsal pathway

Four Ways to Perceive Motion

Real motion
Illusory motion - no real motion present, but its perceived
Apparent movement
Induced motion - real movement of one object results in the perception of
movement in another object
Motion after-effect
The waterfall illusion

Real Motion & Reichardt

Detectors
Reichardt detectors: neurons that
fire to real movement in a
specific direction
Perception of motion is done
through one-way delayed
activation
Key = timing & direction

Aperture Problem
Observing a small portion of a larger stimulus may lead to misleading
(ambiguous) information about the direction of movement
Complex cortical cells has a small receptive field & cannot see the
motion of whole object
Solution: Information of V1 gets pooled together vias hierarchical
convergence in middle temporal (MT) cortex

Evidence
1. Firing and coherence experiment on by Newsome et al.
As coherence of dot movement increased, so did the firing of the MT
neurons and the judgment of movement accuracy
2. Microstimulation experiment by Movshon and Newsome
Judgment to the artificially stimulated direction is shifted when
experimenter used microstimulation to activate different direction
sensitive neurons in MT
1

Motion After-Effect &

Physiological Account
If coherent motion of stimuli is associated with cells in MT, perhaps they can
become fatigued through selective adaptation
The waterfall illusion
Perception = vector sum of neural activities

Spiral Motion After-effects

MT cells converge on cells in the medial superior temporal dorsal (MSTd)
area.
Respond to optic flow the relative motion of you going through your
environment

Corollary Discharge Theory

Movement perception depends on three signals:
Motor signal (MS) - signal sent to eyes to move eye muscles
Corollary discharge signal (CDS) identical copy of the motor signal
Image displacement signal (IDS) - movement of image stimulating
receptors across the retina
Movement is perceived when comparator receives wither CDS or IDS

Physiological Evidence for Corollary

Discharge Theory
1.Real-movement Neurons
a.Found in monkeys
b.These neurons only responded when the stimulus (a bar)
moved across the retina (IDS only)
i. If the eye moved across a stationary bar (IDS and CDS),
there was no neural activity for these real-movement
neurons

Physiological Evidence for Corollary

Discharge Theory
2. Damage to the Medial Superior Temporal (MST)
area
a.Can cause the perception of motion when the
environment is stationary but the eyes are moving
b.In corollary discharge theory this would be a loss of the
CDS signal

Physiological Evidence for Corollary

Discharge Theory
3. Damage to the MT area
Damage here can lead to loss of the ability to
see motion
This is called akinestopia or motion blindness

Biological Motion
Emily Gromssman et al (2005)
The setup:

Presented the point-light walker stimulus to subjects

They were asked to determine if the stimulus was
biological or scrambled
TMS or transcranial magnetic stimulation was applied
over the STS (superior temporal sulcus)

Biological Motion Cont.

Emily Gromssman et al (2005)
The Results:
TMS over the STS caused a decreased ability to
detect biological motion
However, TMS over the MT area had no effect
This suggested that STS was necessary for
biological motion
(Other neurological studies have shown that the FFA
is also involved in processing biological motion.)

Depth Perception
1.Oculomotor depth cues
a.Convergence and accommodation

2.Monocular depth cues

a.Pictorial Cues
b.Movement produced cues

Why/How do we use these cues?

Experience Dependent Plasticity
Through repeated exposure over our lifetimes, we have
learned to associate these cues with depth
(Essentially changes in brain connectivity)

Overtime, these associations become automatic

Oculomotor Cues
These are based on the position of
the eyes and muscle tension
Two major Cues
1) Convergence
Inward movement of the eyes
when you focus on nearby objects

2) Accommodation
Change in the shape of the lens
when we focus on objects at
different distances

Monocular Pictorial Cues

Monocular:
Cues from one eye
Pictorial:
The depth information is coming from 2-D images
E.g. photographs or paintings

Monocular Pictorial Cues

Occlusion
When one object partially covers the other

Monocular Pictorial Cues

Relative Height

Objects below the horizon that are higher in the field of

vision are more distant
(Which man seems more distant?)

Objects above the horizon that are lower in the visual

field are more distant
(Which cloud seems more distant?

Monocular Pictorial Cues

Relative Size
When objects are of equal size, the closer one will take
up more of your visual field

Monocular Pictorial Cues

Perspective
Convergence
Parallel lines
appear to come
together in the
distance

Monocular Pictorial Cues

Atmospheric perspective
Distant objects are fuzzy and have a blue tint

Monocular Pictorial Cues

Familiar Size
Distance information based on our knowledge of object
size

Monocular Pictorial Cues

Texture gradient
Equally spaced elements are more closely
packed as distance increases

Monocular Pictorial Cues

Shadows
Indicate where objects are located, relative to a light
source
Enhances the 3-D nature of objects

Monocular Motion-Produced Cues

Previous: Monocular Pictorial (stationary) cues
Now: Motion-produced cues

Monocular Motion-Produced Cues

Motion Parallax:
Close objects in
direction of
movement glide
rapidly past, but
objects in the
distance appear to
move slowly

Monocular Motion-Produced Cues

Deletion and accretion:
Objects are covered and uncovered as we move
relative to them
Covering an object is deletion
Uncovering an object is accretion

In the picture below, the object is the blue

grid

What cues do we use and when?

We combine these cues to optimize our depth
perception at different distances

Sound Perception

TWO DEFINITIONS OF SOUND

1) Perceptual Definition
Sound as the experience we have when we listen to
sound

2) Physical Definition
Sound as changes in air pressure (can also happen in
other mediums)

SOUND IS A MECHANICAL PROCESS

The transmission of FORCE
force is applied to a local area, then transmitted to
neighboring areas

It is a pattern of changes in pressure across a space

CLARIFICATION
Sound waves are longitudinal waves
This essentially means that :
Sound waves are areas of condensation and rarefaction

The top part of this picture just shows how we

represent these changes in pressure with sine waves

condensation

rarefaction

PHYSICAL DIMENSIONS OF SOUND (1)

1) Amplitude

Describes the differences in pressure between the high and low peaks
Associated with perceived quality of loudness
Units: Decibel (dB)
The decibel describes the physical changes in pressure
But it can be used to relate this physical change to our
perception/psychological experience of sound

PHYSICAL DIMENSIONS OF SOUND (2)

2) Frequency
The number of cycles within a period of time
Perception:
affects the pitch we perceive (high vs. low pitch)
When the frequency is increased we perceive an increase in
pitch this experience is called tone height

Units: Hertz (Hz)

1 hertz = 1 cycle per second

RANGE OF HUMAN HEARING

PHYSICAL DIMENSIONS OF SOUND (3)

3) Complexity
1. Pure tones (single frequency)
2. Complex tones: multiple frequencies present

FORMING A COMPLEX TONE

Additive Synthesis:
Start with the fundamental frequency (also called: first
harmonic)
Then you add more harmonics that are multiple of the
first harmonic

Decomposing a Complex Tone

If instead you have a complex tone and you want
to decompose it into its components
(harmonics), you use a Fourier analysis
The result is a series of sine waves

PERCEPTUAL ELEMENTS OF SOUND

We already covered:
1. Amplitude (loudness)
2. Frequency (pitch)

The 3rd perceptual element is Timbre

. Timbre covers any other perceptual characteristic of
sound that is not loudness, pitch or duration
. Associated with the physical element of complexity

TIMBRE
Timbre:
Also known as the quality of the sound, is closely related to the
harmonics of a tone
It helps differentiates two sounds that are the same note
E.g. Different instruments playing the same note/tone sound
different

Timbre is also affected by attack and decay

Attack:
The build up of the sound intensity before the start of the tone

Decay:
The decrease as the tone fades

COMPLEXITY AND TIMBRE

The effect of the missing fundamental:
The loss of a harmonic, even if it is the fundamental frequency, still
maintains the periodicity pitch
In other words, we perceive the same pitch, but with a different timbre

Why?
These complex tones were made up of MULTIPLES of the fundamental frequency
The spacing of the harmonics carries the information of the missing
fundamental frequency

COMPLEXITY AND TIMBRE

Picture from lecture

HUMAN HEARING

Intensity/
loudness

Range of human hearing: 20 to 20,000 Hz

HUMAN HEARING
1. The Audibility
Curve (green)
The threshold related to
each frequency
The lower the curve
goes, the lower the
intensity of the sound
has to be for us to be
able to hear it
Notice the 2,0004,000Hz area

HUMAN HEARING
2. Auditory
Response Area
The area between the
audibility curve (green,
bottom) and the threshold
for feeling (top, blue)
The entire range over
which we can hear
Range for both frequency
and intensity

HUMAN HEARING
3. Equal loudness curves:
Pick a 1000 Hz standard tone
The you compare a second
frequency to the standard

At what decibel must the 2nd tone

be played in order to be perceived
as the same loudness(dB) as the
1st?

Example:

A 100Hz tone would need to be

played at 60 decibels to be
perceived as the same loudness
as a 1000Hz tone played at only
40dB

EAR ANATOMY

OUTER EAR
The Pinna
Sound location
Does some amplification

The auditory canal

Protects the tympanic membrane (found at the end of the canal)
The canal amplifies frequencies between 1,000-5,000Hz
This a great range for amplifying voices

The tympanic membrane (the eardrum)

It responds to pressure waves from the air
The motion of the membrane will drive the motion of the ossicles in
the middle ear

OUTER EAR

MIDDLE EAR
The 3 Ossicles:
Malleus
Vibration caused by
movement of the
tympanic membrane

Incus
Transmits the malleus
vibrations

Stapes
Transmits vibrations to
the oval window of the
inner ear/cochlea

IMPORTANCE OF THE MIDDLE EAR

From the middle ear to the inner ear, there is a change in
medium
Air liquid
Pressure changes do not transmit well into the liquid medium
Ossicles amplify the vibrations to allow them to be
sufficiently transmitted to the fluid

THE ACOUSTIC REFLEX

The ossicles can also be used to REDUCE
transmission!
When there is a loud external noise, or a loud internal
noise (e.g. chewing), the middle ear muscles tense and
stop the stapes from transmitting all the pressure wave
This protects the inner ear
Called the acoustic reflex

INNER EAR
Cochlea
Fluid filled
Has snail-like shape
The place where sound waves
get transduced into neural
signals
These neural signals leave the
cochlea via the auditory nerve

Within the Cochlea

The organ of corti
Within the cochlear partition

THE ORGAN OF CORTI

The organ of corti sits
on the basilar
membrane
The basilar
membrane
vibrates in response to
sounds

Tectorial membrane
extends over the hair cells

TRANSDUCTION
Sound waves neural signals
How:
Movement of the cilia on inner hair cells in one direction
opens ion channels
These channels release neurotransmitters
The neurotransmitters bind to the auditory nerve fiber(s)
This causes them to depolarize, and triggers an action potential

Note:
Movement in the other direction will close the channels
Outer hair cells amplify this activity

PLACE THEORY
Place Theory
Frequency of sound is indicated by the location (hair cells) along the
cochlea that has the highest firing rate

The basilar membrane:

Base:
Very narrow, and much more stiff than the apex
High frequencies activate hair cells near the base
Apex:
Much wider, and floppier
Low tones activate toward the apex

Envelope of the Travelling

Wave
Most of the basilar membrane will vibrate in response
to a travelling wave, but the displacement will be
greatest at one point

So the wave shows the entire range of displacement

Plus the area of maximum displacement (the peak)

At this peak, hair cells are stimulated the strongest

The peak is a function of frequency

PLACE THEORY
Electrical recordings of the cochlea have supported the
organization of the cochlea based on frequencies
(Implementational level evidence)
Was called a tonotopic map

MORE EVIDENCE FOR PLACE THEORY

Physiological:
Neural frequency tuning curves
1st used pure tones to determine the threshold of ONE
neuron for different frequencies
2nd played the different frequencies and recorded the # of
action potentials
3rd Plotted the thresholds for each frequency tuning curves
The frequency at which the cochlear neuron was the MOST
sensitive = the characteristic frequency

MORE EVIDENCE
Psychophysical support: Auditory Masking experiments (on humans)
The threshold for multiple frequencies for that person were determined
Then a masking noise was presented at the same time as one of those
frequencies
The masking effect is determined by how much the threshold is affected
(how much more intense must that frequency be played before it can be detected
again)

Note:
Masking effects (raised threshold) spreads more to higher frequencies than lower
ones
This matched the predicted envelopes of the traveling waves

Cochlear Amplifier
Bekesy had used basilar membranes isolated
from cadavers in his experiments
These didnt show a difference in response for close
frequencies

Newer research:
Showed that the outer hair cells respond to sound by
tilting and changing in length
This why they are called the cochlear amplifier

Outer hair cells

When we selectively destroy
outer hair cells (chemical
ablation) we see a decrease in
the firing rate of the inner hair
cell at the characteristic
frequency
Proof that the outer hair cells
work as amplifiers

Complex Tones
For complex tones, the
basilar membrane will show
peaks in its vibration that
correspond to each harmonic
Each peak is associated with
the frequency of the
harmonic