B.

Two common misinterpretation of correlations

For instance, it is not reasonable to interpret a low
correlation between two tests as an indication that the
two tests are measuring different skills and also that
neither of them is reliable (or that one of them is not
reliable). Since reliability is prerequisite to validity, a
given statistic cannot be taken as an indication of low
reliability and high validity (yet this is sometimes
suggested in the literature as we will see bellow).
The observed low correlation could result if both tests
were in fact measures of the same basic factor but were
both relatively unreliable measures of that factor.
B. Two common misinterpretation of correlations
A low correlation between a Grammar test and a
vocabulary test might well be the product of poor
tests rather than an independence of hypothesized
components of proficiency.

In an important sense, knowing a word is knowing how

to use it in a meaningful context that is subject to the
normal syntactic (and other) constraints of a particular
language.
B. Two common misinterpretation of correlations
Is reading comprehension subtest more of a measure of
reading ability that it is of writing ability or grammar
knowledge or vocabulary or mere test-taking ability or
general proficiency factor or intelligence?
The fact that the reliability coefficients are higher than
correlations between different part scores is no proof
that the tests are measuring different kinds of
knowledge.
In fact they may be measuring the same kinds of
knowledge and their low intercorrelations may indicate
merely that they are not doing as they could.
B. Two common misinterpretation of correlations
In any event, it is axiomatic that validity cannot exceed
reliability indeed the general rule of thumb is that
validity coefficients are not expected to exceed the
square of the reliabilities of the intercorrelated test
(Tate, 1965).
If a certain test has some error variance in it and a
certain other test also has some error variance in it, the
error is apt to be compounded in their intercorrelation.
Therefore, the correlation between two tests can hardly
be expected to exceed their separate reliabilities.
It can equal them only in the very special case that the
tests are measuring exactly the same thing.
B. Two common misinterpretation of correlations
Surprisingly high correlations have been observed
between a wide variety of testing techniques with a
whole family of procedures under the general rubric of
cloze testing, dictation, elicited imitation, essay writing,
and oral interview.
Populations have ranged from children and adult
second language learners, to children and adult tested in
their native language.
What then can be made of such high correlations?
B. Two common misinterpretation of correlations
Two interpretations have been offered:
1. One of them argues that the strong correlations
previously observed between cloze and dictation,
for instance, are merely indications of the
reliability of both procedures and proof in fact
that they are both measuring basically the same
thing
2. A second interpretation is that the high
correlations between diverse tests must be taken
as evidence not only of reliability but also of
substantial test validity.