KNOWLEDGE REPRESENTATION

WHAT ARE MENTAL

REPRESENTATIONS?

Mental representations are

the FORM for what you
KNOW (inside your mind)
about things, ideas, events,
etc…

… that exist outside of your mind

Mental representations are the mental units

that we use to STORE and PROCESS
information.

Cognitive activities involve the MANIPULATION

and TRANSFORMATION of these mental
representations.
 WHAT FORM(S) DO OUR MENTAL
REPRESENTATIONS TAKE?

DUAL-CODE THEORY (Allan Paivio, 1969)

We use both IMAGE and VERBAL codes for

representing information.

MENTAL IMAGES are ANALOGUE codes

 ANALOGUES are representations of

specific objects that preserve the
features and properties of the real-
world object

 May represent things that have never been

observed by your senses at any time (even
things that do not exist at all outside your
mind!)
VERBAL information is represented in
SYMBOLIC code

 SYMBOLIC representations are

arbitrary: there is nothing
about the symbol that
depicts the actual
properties of the object
it represents

 Use of symbols needs to be governed by

rules

 Information associated with words is abstract

and general and may apply to classes or
categories

Experiments have demonstrated that we do

process verbal information differently from
imaginal information!

Brooks, 1968
 a task involving visual perception interfered
with a visual imaging task
 response involving verbal expression
interfered with a verbal processing task
 PROPOSITIONAL THEORY:
One Universal Format

Knowledge is mentally
represented in terms of their
underlying meanings, or as
abstract propositions.

Images and words are mere by-

products of more basic
cognitive processes involving
propositions.

A PROPOSITION

 is the smallest possible meaningful assertion

about the world that can be evaluated as
true or false (Boole again!)

 uses shorthand “predicate calculus” to

express meanings, stripping away superficial
differences in how we express ideas

(relationship between elements) ([subject] [object])

Where did this idea come from?

The mind-as-computer metaphor:

Thinking IS computation!

Computers can process many different kinds

of information using a universal binary
format…

… as the human mind is an even more

intelligent information-processing machine,
then at its deepest level of processing, we
must also be using a basic, universal mental
format

Evidence that propositional codes may shape

or override imaginal codes:

 Carmichael, Hogan & Walter (1932)

 Chambers & Reisberg (1985)

 MENTAL IMAGERY

I see meaning as the blue-grey tip of a kind of

scoop, which has a bit of yellow above it
(probably a part of the handle), and which is
just digging into a dark mass of what appears
to be plastic material. I was educated on
classical lines; and it is conceivable that this
picture is an echo of the oft-repeated
admonition to “dig out the meaning” of some
passage of Greek or Latin. I do not know; but I
am sure of the image. And I am sure that
others have similar images. (Edward
Titchener)

What does a person mean when he closes his

eyes…and says, “I see the house where I was
born…I can even see my mother as she tucks
me in…” Touching, of course, but sheer bunk.
We are merely dramatizing. The behaviorist
finds no proof of imagery in all this. We have
put these things in words long, long ago and
we constantly rehearse those scenes verbally
whenever the occasion arises. (John Watson)
My particular ability does not lie in
mathematical calculation, but rather in
visualizing effects, possibilities, and
consequences. (Albert Einstein)
What is a mental image?

Are mental images like “pictures” in our mind?

Where are these pictures located?

Who is “looking at” these

pictures? (the physical
eye cannot be turned
inward!)

What do we DO with
mental images?

Is thinking in images more like computing with

graphics or more like computing with a database of
propositions?

Are images different from other forms of

thought/representation?

FUNCTIONAL EQUIVALENCE Hypothesis:

Mental imagery and visual perception use the
same operations
The same brain processes and parts (the visual
cortex) are involved in visual perception and
mental imagery:

 interference studies
 brain damage studies
 analogous effects when images are used in
place of sensory input

Images are ANALOGUE

representations:

Every part of the image

corresponds to parts of the
actual stimulus

Capture the geometry of the stimulus, not just its

meaning
 Functional Properties of Mental Images

We possess a well-stocked mental toolbox of

graphic operations, such as zooming, shrinking,
panning, scanning, tracing, and coloring!

Mental Rotations

Shepard & Metzler, 1971

Findings are functionally equivalent to what

we might expect if the participants had been
rotating physical objects in space

 rotating objects at larger angles of rotation

takes longer, whether clockwise,
counterclockwise, or 3D
Image Scanning

Kosslyn, Ball & Reiser, 1978

The relative distances in

physical stimuli are
replicated in the mental
image.

 There is a linear relationship between the

relative distances of objects on the map and
the amount of time for participants to “scan”
or travel from one point to another
Image Scaling

Kosslyn, 1975

On our “mental screen” for visual images, our

“screen resolution” is more detailed for
objects that are larger.

 we “zoom in” to see parts of

objects imaged to be smaller
(and blurrier), and thus take
longer to respond

 more quickly verify non-distinct

properties of animals if large-scaled (does a
cat have knees?), versus distinct properties
of animals that are small-scaled (does a cat
have claws?)
Imagery is a wonderful faculty, BUT…

 Images are fragmented

 Images are slaves to the organization of

memory and conceptual knowledge

 Images cannot serve to represent our

concepts

 Images are ambiguous

Arguments against mental imagery as a
form of representation:

Imagery is a by-product of symbolically

encoded propositions and meanings

Imagery is cognitively penetrable: it is at the

mercy of propositions and is NOT a default
mechanism of the cognitive system

Thus, images cannot contribute to our

knowledge of how the mind works!

Kosslyn argues back…

Useless to argue whether

images are an irreducible
form of mental representation

Simply, mental images have properties that

distinguish it as a distinctive and functional
form of representation!
 Propositional Representations of Underlying Meanings

Type of Representation Propositional Imaginal

Relationshi in Words Representation Representati
p on
Actions A mouse bit a cat. Bite [action] (mouse [agent of
action], cat [object of action])

Attributes Mice are furry. [external surface

characteristic] (furry
[attribute], mouse [object])

Spatial A cat is under the [vertically higher position]

positions table. (table, cat)

Class A cat is an animal. [category membership]

membership (animal [category], cat
[member])