READING AND AUDITORY-VISUAL

EQUIVALENCES

MURRAY SIDMAN

Massachusetts General Hospital, Boston, Massachusetts

A retarded boy, unable to read printed words orally or with comprehension, could
match spoken words to pictures and could name pictures. After being taught to match
spoken to printed words, he was then capable of reading comprehension (matching
the printed words to pictures) and oral reading (naming the printed words aloud).

Reading may be regarded broadly as a type of stimulus-response relation in

which the controlling stimuli are visual words-written or printed text. Within
this general type of stimulus-response relation, several subcategories may be
identified. One is oral reading. A simple example: If we show a child the word,
bay, and he says boy, he indicates that he can read the word orally. Oral read-
ing may or may not involve comprehension; for example, one can read words in
a foreign language aloud without understanding them. Oral reading may, in
fact, be more appropriately called "oral naming of words." As such, it may be
no different than the oral naming of objects, or the pictures of objects. A com-
mon observation, however, is that children generally learn to name objects or
pictures aloud before they learn to name the corresponding printed words.
To demonstrate reading comprehension, we require a different kind of stim-
ulus-response relation. A simple example: If we show a child the printed word,
bay, and he is then able to select a picture of a boy out of several other pictures,
we say that he understands the word. One simple kind of reading comprehen-
sion, then, may be demonstrated by the child's accurate matching of printed
words to pictures. Defined this way, reading comprehension is a purely visual
task. Note that one may be capable of this kind of reading comprehension with-
out being able to read the words orally.
A third stimulus-response relation, rarely discussed explicitly, might be
termed, auditory-receptive reading. For example, we say the word "bay," to a
child, and he is then to select the word boy out of several other printed words.
This differers from oral reading in that the word is spoken to, and not by, the
child. Nevertheless, discussions of the role of auditory-visual equivalences in
reading often confuse oral reading and auditory-receptive reading under some
such common heading as word recognition.
Like oral reading, auditory-receptive reading may or may not involve com-
prehension, either of the auditory (spoken) or the visual (printed) word. As

Downloaded From: http://jslhr.pubs.asha.org/ by a La Trobe Univ User on 02/08/2016

Terms of Use: http://pubs.asha.org/ss/rights_and_permissions.aspx
 noted before, simple visual comprehension can be tested by a visual word-pic-
ture matching task. Similarly, simple auditory comprehension can be tested by
matching auditory words to visual pictures: We say the word boy, to a child,
and he is then to select the picture of a boy out of several other pictures.
Several lines of converging evidence and theory have led many writers to
postulate that reading comprehension, a visual task, evolves from the previous
learning of auditory-visual equivalences (Birch, 1962; Geschwind, in press;
Wepman, 1962). First, there are certain common observations of normal devel-
opmental sequences: ( 1 ) Children normally understand words they hear before
they learn to read with comprehension; auditory comprehension of words usu-
ally preceds visual comprehension. (2) Children usually name objects, or pic-
tures of objects, before they learn to name the printed or written words that
correspond to those objects; object naming precedes word naming (oral read-
ing). Most children break through the "sound barrier" in the first or second
grade, and learn to understand not just words they hear, but words they see.
They also learn not just to name pictures, but to read words orally. A large
group of retarded children and dyslexic children, however, have not made the
transfer from auditory comprehension and picture naming to visual reading
comprehension and oral reading. It is likely that this transfer marks a critical
point in the development of behavior and the central nervous system.
A second indication that auditory-visual equivalences and reading are closely
linked comes from correlational studies by Birch and his coworkers (Birch and
Belmont, 1964, 1965; Kahn and Birch, 1968). Using a test of auditory-visual
integration (matching sound patterns to visual patterns), they found positive
correlations between scores on this test and scores on standard reading achieve-
ment tests.
A third set of considerations comes from the neuroanatomical theories of
Geschwind (1965), who takes as his starting point the observations, like those
noted above and others, that cross-modal equivalences and language are closely
linked. He has proposed that cross-modal equivalences, particularly auditory-
visual, actually make language possible. Furthermore, he has suggested that
the evolution of the angular gyrus region, strategically located at the junction
of auditory, visual, and somesthetic association cortexes, makes that region the
prime candidate as the central-nervous-system site for the mediation of cross-
modal equivalences. As a consequence, the angular gyrus is held to be critical
for language in general and for reading in particular. Geschwind has proposed
that developmental dyslexia may be correlated with the slow maturation of the
angular gyrus bilaterally, or perhaps even with its failure to develop.
In spite of these empirical and theoretical considerations, and in spite of the
educational practices (e.g., the "look-say" method of teaching reading) that are
based on them, the question of whether auditory-visual learning is indeed a
necessary or even a suflqeient prerequisite for the development of oral reading
or reading comprehension seems not to have been studied experimentally. The
experiment to be described now will demonstrate that certain learned auditory-
visual equivalences are indeed sufficient prerequisites for the emergence of

6 Journal of Speech and Hearing Research 14 5-13 1971

Downloaded From: http://jslhr.pubs.asha.org/ by a La Trobe Univ User on 02/08/2016

Terms of Use: http://pubs.asha.org/ss/rights_and_permissions.aspx
 reading comprehension, even without explicitly teaching reading comprehen-
sion. Although the data raise a number of unanswered questions, the major
finding is sufficiently provocative, and relevant to both theory and teaching
practice, to warrant this report in advance of more extended studies.

METHOD

Subiect
The subject of the experiment was an institutionalized 17-year-old boy,
microcephalic and severely retarded. During the past two years, he had ex-
tensive experience with the apparatus and matching-to-sample procedures de-
scribed below. The following findings are a relevant background for the present
experiment: He was able to match pictures, colors, and printed numbers to
picture names, color names, and number names that were spoken aloud to him.
But he was unable to do the matching correctly when the names were presented
to him visually rather than spoken. Also, he could name the pictures aloud, but
not the corresponding printed words. Therefore, he showed good auditory com-
prehension and picture naming, but little if any reading comprehension or oral
reading. He could not write.

Apparatus and Procedures

The subject sat before a panel of 9 translucent windows, each 2 inches square,
arranged in a 3 3 matrix. Visual stimuli Were projected from the rear onto the
windows (Rosenberger et al., 1968). Each trial began by presentation of a
sample stimulus. Visual word or picture samples appeared on the center win-
dow of the matrix; auditory word samples, repeated at 2-second intervals, were
dictated from tapes over a speaker (Figure 1, left column).
In matching tests, the subject pressed the center window to bring choice
stimuli, always visual, onto the outer windows of the matrix. Schematic exam-
ples of the displays are in the second column of Figure 1. On each trial, one
choice, the correct one, corresponded to the sample; the other seven choices
did not. The subject selected and pressed one of the choice windows. His cor-
rect choices were rewarded by chimes ringing and delivery of a candy and a
penny. No rewards followed incorrect choices. The stimuli disappeared after
each choice, and 1.5 seconds later a new sample began the next trial.
In oral naming tests, the subject had simply to name the sample picture or
word aloud. Reward procedures were the same as in the matching tests.
Each test had 20 trials. The sample and choice stimuli, taken from a list of
20 pictures, or the printed (lower case) or spoken names of the pictures were:
axe, bed, bee, box, boy, bug, car, cat, cow, dog, ear, hat, hen, hut, ho~, man,
pie, pig, saw, zoo.
Preliminary tests evaluated the subject's proficiency at simple comprehension
and naming tasks; then, he was taught to match spoken to printed words; final

SIDMAN: Reading and Auditory-Visual Equivalences 7

Downloaded From: http://jslhr.pubs.asha.org/ by a La Trobe Univ User on 02/08/2016

Terms of Use: http://pubs.asha.org/ss/rights_and_permissions.aspx
 SAMPLE RESPONSE
(AUDITORY ( N A M E OR
OR V I S U A L ) BASELINE POST-
MATC H) CONTROL TEACHING TEACHING
TESTS TESTS
100-
oar ear key B
80-

-I
~CAT"
dot eat 60-
(spoken t_.o 40-
subject) bet kod eow 20- A
0., , , I
, 0 , 0 <) , 0 , 0
100- 9 .~ ~&
t'q~p NM
80- Iq. k~ t~ t ~ l ~
"CAT"

I
b0-
eat
(spoken by 40-
subject) 20-
0.... 9
100-
80-

I
eat
60-
40-

~) t 0 0 -
,mr omr hol *- 80-
v)
-a ~ ,,t ~ 6o-
i- 40-
hut hod eow ~ 20-
. nn
m 0-' ' *
100-

~C AT m 60-
(spoken b_y 40-
subject) 2o-
O.s i !

[
100-

~' C A T "
(spoken to
subject)

*O1~ q-
qr qr , 0 t~ t~

FIGUItE 1. In the two left columns are examples of the sample stimuli and re-
sponses that comprised each type of test. Choice stimuli and correct window posi-
tion in the matching tests varied from trial to trial. The three columns of bars rep-
resent scores in each depicted test during the three phases of the experiment.
Absenco of a bar means no test on the indicated date. Letters identi~ the six
auditory-visual word matching sets (uppermost row).

8 1ournal of Speech and Hearing Research 14 5-13 1971

Downloaded From: http://jslhr.pubs.asha.org/ by a La Trobe Univ User on 02/08/2016

Terms of Use: http://pubs.asha.org/ss/rights_and_permissions.aspx
 tests evaluated the effects of this teaching on his reading comprehension and
word naming.

RESULTS
Each row of bar graphs shows the subject's test scores on the task depicted
at the left.

Baseline Control Tests

The results of preliminary tests are in the left column of bar graphs in
Figure 1. Bars at the lower left show the subject's scores in tests that required
him to match spoken word samples to picture choices. In four tests, adminis-
tered from April 1967, to July 1969, he scored from 60 to 95% correct, demon-
strating a fair proficiency at this type of auditory comprehension. He also scored
85% in naming the pictures (second row from bottom).
In reading-all tests that involved printed words-the subject scored poorly.
Continuing up the left column, these tests were: Matching picture samples to
printed word choices; matching printed word samples to picture choices (3 tests
over 2 years); naming printed words; and matching spoken word samples to
printed word choices.
The possibility that the subject could not distinguish the printed words from
each other was ruled out by his score of 95% in matching printed word samples
to printed word choices (not shown in Figure 1). The words in this test were
the same 20 that comprised the other tests. Also, the two types of control tests
at the bottom of Figure 1 (matching spoken words to pictures, and picture
naming) show that the subject could already distinguish the pictures from each
other, and the auditory words from each other, and that he could say the
words aloud. His difficulties were neither with the discrimination of the stimuli
used here, nor with the oral responses, but were specifically with the stimulus-
response relations that operationally define simple reading comprehension, oral
reading, and auditory-receptive reading.

Teaching Auditory-Receptive Reading

Teaching the subject to match auditory to visual words was the critical ex-
perimental operation. Figure 2 will illustrate the logic of the experiment and
will serve as a basis for later discussion. The three boxes at the left and center
of Figure 2 represent the three types of stimuli, and the arrows represent stimu-
lus equivalences as defined by the matching performance. The arrows connect-
ing the center boxes to the right hand box represent the two naming perform-
ances, picture naming and oral reading.
The subject came to the experiment knowing the equivalence of spoken words
to pictures (Equivalence I). Would teaching him auditory-visual (Equivalence
II), spoken words to visual words, suffice to establish reading comprehension,

SIDMAN.' Reading and Auditory-Visual Equivalences 9

Downloaded From: http://jslhr.pubs.asha.org/ by a La Trobe Univ User on 02/08/2016

Terms of Use: http://pubs.asha.org/ss/rights_and_permissions.aspx
 VISUAL
PICTURES

I
I
AUDITORY I ORAL
WOR D S 1 NAMING
III llV
(SPOKEN TO I
# (SPOKEN B,Y
I I
SUBJECT) I I SUBJECT)
I I

"~[
I
ii\ , I
, /V I
J
VISUAL i/
o
WORDS

FZCURE2. Schematic summary of the experiment. Of the stim-

ulus equivalences, I-IV, the subject came to the experiment
knowing I. Of the naming tasks, V and VI, he could do V. After
being taught equivalence II, he could then do III, IV, and VI.

the purely visual equivalence of printed words to pictures (Equivalences III

and IV)? He also came to the experiment able to name the pictures (V); given
this ability, would teaching him auditory-visual word matching suffice for oral
reading (VI) to emerge?
In the teaching procedure, sample stimuli were words spoken to the subject;
choices were printed words (Figure 1, top row). Teaching differed from testing
in several ways. (1) A correction procedure was used; when the subject chose
a wrong printed word, the display remained unchanged until he pressed the
correct window. (2) Errors had different consequences; if the child made one
or more errors on a given trial, the chimes rang when he finally pressed the
correct window, but he did not receive candy or a penny. (3) Each phase of
the teaching procedure started with only two trials (sample-choice combina-
tions); the two being repeated until the subject's first choices on both were
correct. Then a third trial was added; when his first choices on all three were
correct, a fourth was added. This progressive enlargement of the set continued
as the subject attained each criterion of mastery, until his first choices were
correct on the full set of twenty trials; (4) Six versions or sets of auditory-
visual word matching materials were used. Each set presented the same 20
sample words in different trial sequences, and displayed a different combination
of seven wrong words along with each correct word.
Set A was used for the preliminary control test. Then the subject was taught
Set B until he scored 100%, and was tested on Set C. His low score on Set C
(Figure 1, center section, first bar), suggested that his learning of Set B had
been specific to the particular sequence of correct window positions and to the
particular wrong words displayed along with each correct word. The subject
then learned Set C, reviewed Set B to the same 100% criterion, and was tested
on Set D. The process of learning, reviewing, and testing on a new set con-

10 ]oumal of Speech and Hearing Research 14 5-13 1971

Downloaded From: http://jslhr.pubs.asha.org/ by a La Trobe Univ User on 02/08/2016

Terms of Use: http://pubs.asha.org/ss/rights_and_permissions.aspx
 tinued through Set F, and the center section of Figure 1 shows the gradual
improvement on each new test. (The teaching process itself is not shown; only
the test scores on each new set.) Finally, the subject was retested on Set A,
which he had not seen since the preliminary test. The change from 20 to 8 0 ~
correct on Set A, one month after the preliminary test, demonstrated his new
proficiency at the task.

Post-Teaching Tests
After the teaching, all comprehension and oral naming tests were adminis-
tered once more. Scores are in the right column of Figure 1. The subject main-
tained his good performances on the first auditory-visual word matching set
he had learned (upper right), in matching spoken words to pictures, and in
picture naming (lower right).
Of major interest are the subject's reading comprehension and oral reading
tests (visual word-picture and picture-word matching; word naming). These
improved greatly. Having learned to match spoken word samples to printed
word choices, he was then able, without additional teaching, to match picture
samples to the printed word choices, to match printed word samples to pic-
ture choices and to name printed words.
Given the subject's initial ability to match spoken words to pictures, and to
name the pictures, teaching him the second auditory-visual equivalence, spoken
to printed words, sufficed for the emergency of purely visual reading compre-
hension and oral reading.

DISCUSSION
The findings will be discussed with reference to Figure 2. A simple connec-
tionistic interpretation of the emergence of reading comprehension might
be that the visual words and pictures became equivalent to each other (III,
IV) because each, independently, had become equivalent to the same audi-
tory words (I, II). This would be entirely consistent with the theoretical sup-
positions of Geschwind (1965), particularly with respect to the integrating
functions of the angular gyrus region. It is not clear from this experiment,
however, whether equivalences I and II need be cross-modal. Suppose, for
example, that visual nonsense syllables were substituted for the auditory words,
and that arbitrary equivalences between these visual symbols and the words
and pictures were taught to the subject. Would the words and pictures then
emerge as equivalent to each other, even with a common intramodal, rather
than a cross-modal linkage? That deaf children learn to read suggests an a~rm-
ative answer. There is no need, however, to assume only a single mechanism for
reading comprehension.
The emergence of visual-word naming, or oral reading (VI)I complicates
the simple connectionist view, since the equivalence of visual words to pic-

SIDMAN': Reading and Auditory-Visual Equivalences 11

Downloaded From: http://jslhr.pubs.asha.org/ by a La Trobe Univ User on 02/08/2016

Terms of Use: http://pubs.asha.org/ss/rights_and_permissions.aspx
 tures may have been mediated by naming (V, VI) rather than by auditory
words. This, too, is testable. It should be emphasized, however, that even if the
emergence of word naming permitted reading comprehension to develop, it
did not do so through the auditory channel. The subject did not name the words
or pictures aloud during the reading comprehension tests; the only auditory
stimuli were the words spoken to him in previous tests and teaching sessions.
Furthermore, whatever proves to be the role, if any, of the subject's ability to
read orally in mediating the transfer from the cross-modal to the purely visual
equivalences, the experiment has demonstrated that matching auditory words
to pictures and to printed words are sufficient prerequisites for the emergence
of both types of stimulus-response relation, reading comprehension, and oral
reading.
The identification of these sufficient prerequisites for reading comprehension
suggests a most important practical consequence. Both auditory-visual equiva-
lences (I, II) can be taught completely without the intervention of a teacher.
Reading comprehension is usually taught by way of oral naming, and this does
require that a teacher participate actively. Automated programs to teach read-
ing comprehension via purely receptive auditory-visual training (Equivalences
I and II) would permit a far larger number of children to be reached than is
now possible. Furthermore, the technique provides a rapid method for deter-
mining whether a child who has not yet made the transfer from the auditory to
the visual comprehension of words is actually incapable of passing through this
apparently critical developmental stage.
Independently of comprehension, one may ask whether oral reading (VI) will
always emerge, in a child capable of speech, after the child has learned audi-
tory-visual word matching. Guess (1969) has shown that receptive language
training need not facilitate the learning of productive speech. The auditory
stimuli Guess used were singular and plural object names, and he taught the
children to match these spoken names to singular and plural objects, analogous
to our Equivalence I. The children were unable, then, to use the correct singu-
lar and plural forms in naming the objects, analogous to our oral naming Task
V. In addition to the differences in stimulus materials and responses (simple
nouns versus singular and plural nouns), a likely reason for the discrepancy
between Guess' experiment and the present experiment is that our subject was
taught Equivalence II and tested on Task VI (word naming) after he had
demonstrated his ability to do Task V (picture naming). If receptive training
is to facilitate oral speech, it may be necessary for the child already to have the
words in his own speech repertoire..

ACKNOWLEDGMENT
This research was supported by Grant NS 03535 from the National Institute of Neurolog-
ical Diseases and Stroke, and by the Joseph P. Kennedy, Jr., MemorialLaboratories, Neurol-
ogy Service, Massachusetts General Hospital. I thank Osborne Cresson, Martha Wilson, and
James Sidman for technical assistance.

12 Journal of Speech and Hearing Research 14 5-13 1971

Downloaded From: http://jslhr.pubs.asha.org/ by a La Trobe Univ User on 02/08/2016

Terms of Use: http://pubs.asha.org/ss/rights_and_permissions.aspx
 REFERENCES
BIaCH, H. G., Dyslexia and and the maturation of visual function. Ch. 10 in J. Money (Ed.),
Reading Disability, Baltimore: Johns Hopkins (1962).
BmCH, H. G., and BELMONT, L., Auditory-visual integration in normal and retarded readers.
Amer. 1. Orthopsychiat., 34, 852-861 (1964).
BllaCH, H. G., and BELMONT, L., Auditory-visual integration, intelligence, and reading
ability in school children. Percept. Motor Skills, 20, 295-305 (1965).
GESCHWIND, N., Aphasias and related disturbances. In R. W. Wilkins (Ed.), Textbook of
Medicine, Boston: Little, Brown (in press).
GESCHWXND, N., Disconnexion syndromes in animals and man: Part I. Brain, 88, 237-293
(1965).
GuEss, D., A functional analysis of receptive language and productive speech: Acquisition
of the plural morpheme. 1. Appl. Behav. Analysis, 2, 55-64 (1969).
KAHN, D., and BmCH, H. G., Development of auditory-visual integration and reading
achievement. Percept. Motor Skills, 27, 459-468 (1968).
ROSENnEnCEa, P. B., MOHR, J. P., STODDAP,D, L. T., and SIDMAN,M., Inter- and intramodality
matching deficits in a dysphasic youth. Arch. Neurol., 18, 549-562 (1968).
WEPMAN, J. M. Dyslexia: Its relationship to language acquisition and concept formation.
Ch. 12 in J. Money (Ed.), Reading Disability, Baltimore: Johns Hopkins (1962).

Received January 2, 1970.

SIDMAN: Reading and Auditory-Visual Equivalences 13

Downloaded From: http://jslhr.pubs.asha.org/ by a La Trobe Univ User on 02/08/2016

Terms of Use: http://pubs.asha.org/ss/rights_and_permissions.aspx