Book review

Manning, W. H. 1 987: Development of doze-elide tests of

English as a second language. TOEFL Research Report 23, April
1987, Princeton, New Jersey: Educational Testing Service.
This study reports a programme of research aimed at developing
'evidence pertinent t o the construct validity of cloze-elide tests of
English as a second language' and at providing 'evidence relevant t o
evaluating the potential usefulness of cloze-elide testing in meeting
existing or emerging needs of the clienteles served by the TOEFL
program' (p. 4). As its name implies, the cloze-elide test belongs to the
cloze family, but whereas in the classic cloze words are systematically
deleted from a text and the testee is required to replace them, the cloze-
elide test requires the testee to detect and eliminate - by drawing a line
through - incorrect, randomly irlserted words, e.g.:
There are laws that protect rare animals pilot that are in danger run of being
destroyed.
Clearly student errors may take the form of omission (neglecting t o
elide an inserted word) or commission (eliding a non-inserted, legiti-
mate word). The scoring procedure therefore takes account of both.
It is suggested that the ability to perform this task involves the appli-
cation of background knowledge together with mastery of a variety
of linguistic and cognitive strategies necessary for acquiring and
evaluating new knowledge. While the author considers the procedure
to be 'an on-line measure of reading comprehension that uses the
error-detection task paradigm' (p. 9), he considers that the test taps
'more general cognitive processes that underlie language behavior at
deeper, or more fundamental levels' and would 'expect performance
on cloze-elide tasks to be related to a variety of communication skills,
including listening and speaking as well as writing and reading'
(P. 11).
Before the main study, a preliminary investigation - The Elgin
Study - is reported. This appears to have consisted primarily of corre-
lations between the performance of Hispanic, Laotian and native
English-speaking schoolchildren at three grade-levels on cloze-elide
tests and on a variety of other measures of linguistic ability, including
self-ratings and teacher appraisals, and also of a number of factor
Book review 251
analyses. Despite the severely restricted number of score points on
some of the measures, leading one t o expect rather depressed corre-
lations in these cases, correlations were in fact surprisingly encourag-
ing. Details of the cloze-elide passages - topics, length, number of
inserted words, etc. - are not supplied, although this is clearly impor-
tant information.
Turning now to the main study, the question of construct validity is
again addressed first in terms of whether correlations between the
cloze-elide test and a variety of criterion measures suggest that the test
is indeed a measure of language proficiency and, if so, whether the test
assesses reading, writing, listening or speaking, or some mixture of
these; and then through factor analysis.
A second topic of investigation is whether cloze elide adds usefully
to the accuracy of predictions of various criteria of student success
that can be obtained from other tests, including TOEFL.
Thirdly, it is said to be 'relevant' 'to look carefully at the relation-
ships between cloze-elide test performance and various student
characteristics and interests. How are the cloze-elide test scores related
to students' academic and career interests, to students' assessments of
their own English language skills . . .?' Relevant to what? It is of
course important to know whether a test format is biased for or
against a particular student group, but given the well-known iaaccu-
racy and unreliability of students' assessment of their own linguistic
abilities (for confirmation, see comments on p. 62) I am unsure of the
usefulness of students' linguistic self assessments in establishing either
the validity or the practicality of the test.
Fourthly, and unaccountably just before last, the author says it is
important to estimate the reliability of cloze-elide because such tests
'might at some point become a basis for making decisions by or about
students'. More serious even than this undeniably important reason
for investigating reliability is that without it the cloze-elide technique
can have no validity of any kind! So the need to estimate reliability
should have been placed not fourth but first!
And finally, it is proposed to look at the reactions of students and
teachers to this test format.
To cut a long story short, and despite a number of administrative
hiccups (e.g. giving students twelve instead of twenty minutes for the
standard cloze task) the results are extremely promising in each area of
investigation, beginning with a reliability (internal consistency
between 'macro-items', i.e. whole texts treated as items) of .89.
Among the 8 criteria this coefficient is exceeded only by TOEFL Part 3
and TOEFL Total. It should be borne in mind, however, that the
fewer the macro-items - and here there were only three as a result of
another hiccup - the greater the inflationary effect on the coefficient
252 Book review
n
of - in the reliability formula.
n - 1
Overall, cloze-elide correlates more highly than other experimental
measures with the individual sections of TOEFL and with the total
score, and is outdone only by TOEFL sections 1, 2 and Total in corre-
lations with teacher ratings. Moreover, the correlations of cloze-elide
tests are systematically higher than those of the other experimental
tests in predicting the common factors shown by the factor analysis
to underlie TOEFL. Also, stepwise multiple regression showed
TOEFL Part 1 (Listening) and cloze-elide together as the most
efficient predictors of teacher ratings of skills needed in classroom
participation - reading, speaking, listening and understanding lec-
tures, and also of formal and informal communication in and out of
class.
Finally, both students and teachers found the tasks of some interest,
but also a little difficult.
Given this extremely positive result, one would expect further
research and development on this technique.
University of Reading Don Porter