Educational Evaluation and Policy Analysis

Winter 1988, Vol. 10, No. 4, pp. 301-313

Symbolic Validation: The Case of State-Mandated, High-Stakes

Testing

Peter W. Airasian
Boston College

Because the wisdom and likely effects of educational innovations rarely are established before
their adoption, the innovations must seek their legitimacy in nonempirical sources. This
paper uses the example of the rapid rise of state-mandated, high-stakes testing programs to
suggest that proposed innovations find their way into practice by virtue of their potency as
symbols of prevalent value orientations in the wider culture. Thus, innovations undergo a
type of consensual, symbolic validation that determines their acceptability. State-mandated,
high-stakes testing programs have gained widespread popular support in the quest for
heightened educational standards because they symbolize order and control, desired educa-
tional outcomes, and traditional moral values.

Most educational innovations are adopted
with high levels of technical uncertainty; the
wisdom of their adoption and the range of
their likely effects rarely are known in ad-
vance of their implementation (Rowan,
1982; Wise, 1979). Although many thought-
ful individuals have reminded us of this fact
and urged a more systematic, evidence-
based approach to innovation (Campbell,
1969; Fullan, 1982; Suchman, 1967), their
advice has gone largely unheeded.
The past 25 years have seen scant empir-
ical evidence to justify the implementation
of a wide range of educational programs
including, but by no means limited to, Title
(Chapter) I, Public Law 94-142, and state-
mandated high school graduation tests.
There is little disagreement that reform was
needed in each of these areas, and empirical
evidence was brought forth to document the
severity of the problems in each area. Data
showing test score differences between dif-
ferent racial and ethnic groups were a key
justification for the need for compensatory
education programs of the 1960s. Evidence
of the failure to provide services to special
needs pupils and the inadequacies of many
services that were provided argued for re-
form in special education in the 1970s. Var-
ious forms of test score data showing de-
clines in pupil performance were offered in
the early 1980s to support the contention
that our nation was at risk and in need of
higher educational standards.
Each of these types of evidence provided
legitimacy to the importance of a particular
problem area and to the need for reform in
that area. The evidence did not, however,
provide legitimacy to the Title I programs,
Public Law 94-142 statutes, and competency
testing programs that were ultimately se-
lected to remedy the problems. There is an
important difference between evidence that
documents the need for intervention and
evidence that documents the efficacy of a
particular strategy of intervention. Typi-
cally, once a need for intervention is docu-
mented empirically, the specific policy, pro-
gram, or approach advanced to meet that
need must seek its own sources of legitimi-
zation and validation. These sources rarely
include trial testing and collection of empir-
ical evidence.
Several reasons explain why empirical evi-

301
Peter W. Airasian
dence does not and cannot play an impor-
tant role in the legitimization of innovations.
In the first place, innovation almost always
is conceived of in response to some social
problem or crisis that demands urgent solu-
tion. Whether the problem is disparities in
school performance between racial/ethnic
groups, the large number of latchkey chil-
dren, or declining school standards, the ur-
gency to alleviate the problem—to "do
something now" lest the problem be exac-
erbated—generally limits both the opportu-
nity and the will to carry out empirical prea-
doption trials of proposed solutions (Aira-
sian, 1983).
Second, the political benefit that accrues
to those who promulgate, mandate, and en-
act reform programs is itself often a prime
and sufficient motivation for unexamined
implementations (Moynihan, 1970). The
possibility that pilot studies will identify de-
ficiencies in an innovation or delay imple-
mentation by provoking debate over study
findings can eliminate the political will to
carry out pretrial studies before widespread
adoption (Timpane, 1976).
Finally, a general lack of technical knowl-
edge about how to remedy identified edu-
cational problems (Airasian, 1987b; Wise,
1979) forces policymakers to frame innova-
tions in such general terms that pretrial stud-
ies are difficult to plan and carry out. As
Elmore and McLaughlin (1982) point out,
some policies are explicit and tell us all we
need to know to do what is correct (e.g.,
"Don't exceed 55 miles per hour"), whereas
others do not contain sufficient information
or the knowledge necessary to carry them
out (e.g., "Improve the educational achieve-
ment of disadvantaged children"). The
stated intent of most educational innova-
tions fits into this latter group. For these
three reasons, there are pressures to intro-
duce unproven remedies once a problem or
need is documented.
Nevertheless, when pressed to "do some-
thing now," policymakers typically have a
choice of many actions that can be taken;
there are many ways to attack identified
educational or social problems. A decline in
levels of academic performance among
302
school children, for example, may be at-
tacked by extending the length of the school
day, adding required courses to the school
curriculum, mandating exit tests to certify
pupils' competence, instituting remedial ed-
ucation programs, raising grade to grade pro-
motion requirements, developing incentives
for poor learners to drop out of school, or
some combination of these strategies. If em-
pirical evidence regarding the efficacy of any
or all of these approaches is lacking, as it is
typically for the reasons just stated, then the
legitimacy of the chosen reform strategy
must be found elsewhere.
Most often, that legitimacy is found in
what may be termed social validation, in the
mesh between prevalent social norms and
values and the norms and values perceived
to be reflected in the reform strategy (Par-
sons, 1956; Rowan, 1982). If an innovation
is perceived to be capable of moving educa-
tional or social goals in a direction consistent
with the goals, norms, or values of influential
social groups, the innovation will be well
received. So, the likelihood of adopting an
innovation is determined largely by how well
that innovation is perceived to incorporate
the goals, norms, or values of important
social constituencies (Cremin, 1961). The
legitimacy of most innovations then, derives
not from empirical evidence of their likely
effectiveness, but rather from the percep-
tions they evoke and the extent to which
those perceptions mesh with prevalent social
norms and values. In essence, innovations
are weighed in terms of whether they are
symbols of broader social values, and it is
this symbolic weighing of innovations that
determines their legitimacy in the public's
eyes. A brief examination of social symbols
and symbolism helps clarify this relation-
ship.
Symbols and Symbolism
A symbol is a concrete indicator of a more
general, abstract idea (Firth, 1973; White-
head, 1927). Symbols tend to be evocative
rather than analytical, emotional rather than
intellectual. That is to say, symbols evoke
not only concrete images, but also feelings,
values, emotions, and sentiments.

In addition to being representational and
infused with an affective component, sym-
bols flow from culture and depend on con-
vention, habit, or agreement within the
culture for their meanings and impor-
tance (Firth, l973;Peirce, 1940; Whitehead,
1927). Symbolism derives from the values
and ideals of the extant culture, not from
the concrete indicator itself For example,
the cultural dependence of language, the
most ubiquitous of symbolic forms, can be
seen in many examples of a given word
conveying different images in two cultures
(e.g., boot in the United States evokes an
image of footwear but in England boot
evokes an image of the empty space in the
back of a car). Because cultures change over
time, the meanings of symbols can change
and evolve. For example, less than 50 years
ago, test score disparities between black stu-
dents and white students were taken to sym-
bolize the "natural order of things" between
racial groups—a reaffirmation of the pre-
dominant hereditarian outlook (Cronbach,
1975). In more recent times, given changes
in cultural mores and expectations, differ-
ential school test performance among racial
or ethnic groups is taken to symbolize un-
equal educational opportunity, discrimina-
tion, and lack of progress in an important
area of social reform.
Central to the study of symbols is the
question of the relationship between a sym-
bol and the things or emotions it represents.
Why do some objects become powerful so-
cial symbols while others do not? The an-
swer to this question must reflect the fact
that a symbol derives from some common-
ality of experience or perception among
those who accept and use it. In this context,
symbols can be understood as shorthand,
mnemonic devices to remember and com-
municate experiences and emotions (Lo-
emker, 1962).
Symbols themselves can be categorized
into two domains: public and private (Firth,
1973). Both types are true symbols; they are
representational, often emotion laden, and
dependent on cultural experiences. Private
symbols reflect an individual's particular
idiosyncrasies, background, and experiences
State-Mandated Testing
and so have meaning only for that individ-
ual. For example, although each of us
dreams, the specific content and context of
our dreams vary from one person to another.
At the individual level, each person's dream
is a private symbol reflecting his or her idio-
syncrasies. Public symbols, on the other
hand, reflect common threads underlying
private symbols; they emerge from the reg-
ularities and commonalities that are con-
tained in and can be discerned from private
symbols. Thus, Freud's approach to the
analysis of dreams became viable when he
showed that the private symbols of one in-
dividual's dreams had underlying common-
alities with the private symbols of other in-
dividuals' dreams (Firth, 1973). The com-
monality sought in public symbols rests
largely on the cultural experiences and
shared understandings of the social group
being described (Lindblom & Cohen, 1979).
The reaction to the symbol is dependent on
this commonality, because the symbol ab-
stracts only the important aspects of the
referent, leaving the individual's memory to
fill in the details.
The common experiences or values un-
derlying public symbols shed light on the
issue of the political or social validity of
symbols. When an innovation contains ele-
ments that symbolize experiences or values
generally believed in the extant cultural
ethos to be necessary for solving a given
social or educational problem, the innova-
tion will achieve social legitimization and
acceptance. The degree to which the inno-
vation symbolizes something common or
readily understandable in an audience and
the degree to which that commonality is
believed to be necessary for resolving an
identified problem, dictate the extent to
which the innovation will be perceived to be
valid. Cremin's (1961) discussion of the rise
of the vocational education movement in
the 1880s and 1890s and Callahan's (1962)
description of the advent of the school effi-
ciency movement in the period 1910 to 1930
exemplify situations in which there was a
clear meshing of cultural values and educa-
tional responses. Symbolic validity then, is
based on existential or consensual valida-
303
Peter W. Airasian
tion, on public symbols and their value, and
emotional referents. This paradigm can be
useful in explaining the adoption or lack of
adoption of proposed, but untested, inno-
vations.
This paper explores the symbolism pos-
sessed by a new and rapidly spreading edu-
cational innovation, namely state-man-
dated, high-stakes testing programs (Aira-
sian, 1987c; Madaus, l985;Popham, 1987).
The results of such testing programs are
being used as the primary criteria in making
important educational decisions, such as
whether or not a pupil will receive a high
school diploma or be promoted to the next
grade, whether a teacher will be certified or
recertified to teach in a given state, and
whether remedial funds or performance bo-
nuses will be awarded to schools and teach-
ers. The tests appeared virtually out of no-
where about 10 years ago and have enjoyed
an acceptance and status that not only is
rarely seen in an educational innovation, but
also far outstrips available evidence of their
effectiveness. The following sections seek to
explain the unusual popularity and breadth
of implementation of such testing programs.
This paper does not deal with the techni-
cal, psychometric properties of state-man-
dated, high-stakes tests, but rather with pub-
lic perceptions of these tests. It does not ask
whether the tests are good and dependable
in a technical sense, but asks why people
have been so quick to adopt them in the
name of educational reform. In this regard,
it is necessary to point out that the percep-
tual issues addressed in this discussion have
created a too-willing audience for state-man-
dated, high-stakes testing programs, an au-
dience that needs to have its expectations
tempered by a more critical review and ex-
amination of the technical properties of such
tests.
The Case of State-Mandated, High-Stakes
Testing Programs
Documenting the Problem
In the late 1970s, after two decades of
focus on policies designed to enhance edu-
cational equity, concern began to be ex-
pressed about the quality of American
304
schools and pupils. There was a perception
that standards in schools had declined and
needed to be strengthened. The National
Commission on Educational Excellence
(1983) summarized the evidence advanced
to document this perception. Data showed
that SAT scores had declined annually be-
tween 1963 and 1980. The National Assess-
ment of Educational Progress results showed
performance declines over time in most
school subjects at most grade levels. On 19
different international achievement tests
American pupils never ranked first or sec-
ond, but ranked last seven times. Average
achievement of high school pupils on stand-
ardized tests was lower in the early 1980s
than it was 26 years ago when Sputnik was
launched. In 1980, one quarter of all the
math courses taught in public 4-year colleges
were remedial courses. The Department of
the Navy indicated that one quarter of its
recruits could not read at the ninth-grade
level. These and similar statistics conveyed
a picture of an educational system in which
standards and academic performance had
fallen to unacceptable levels. There was a
perceived need to focus on educational ex-
cellence and improve school standards and
attainments.
The American public clearly endorsed this
view. In a 1983 Gallup Poll on education
(Gallup, 1983), 74% of the American adults
polled agreed that the quality of education
in American public schools was only fair
and not improving. In 1984 (Gallup, 1984),
60% of adults polled believed that elemen-
tary school pupils were not made to work
hard enough in school. Sixty-seven percent
felt that high school pupils were not required
to work hard enough in school. In 1986
(Gallup, 1986), 70% of American adults sur-
veyed said that there should be stricter re-
quirements for both grade to grade promo-
tion and high school graduation. In the same
poll, 68% of respondents favored stricter
high school graduation requirements even if
it meant that fewer students would graduate.
Responding to the Perceived Need
Once the need for reform was docu-
mented by evidence such as this, many pol-

icy responses were advanced, and most of
these were tried in one form or another
(Doyle & Hartle, 1985). Among the wide
array of reform strategies advanced to im-
prove school quality, however, the predom-
inant strategy, in terms of the breadth of its
application and its public acceptance, has
been new forms of standardized testing and
accountability programs. Within the past 10
years all state legislatures have been con-
fronted by, and most have endorsed, new
state-mandated testing programs designed to
make students, teachers, and schools ac-
countable for their academic performance.
The new state-mandated tests are different
from more traditional standardized educa-
tional tests in three key ways (Airasian,
1987a). First, the new state tests are man-
dated for all schools and virtually all pupils
within a state. Second, the mandate elimi-
nates most of the local district discretion in
test selection, administration, content cov-
erage, scoring, and interpretation. A single,
state-approved test that is administered,
scored, and interpreted according to state
guidelines is used across school districts.
Third, the tests have built-in sanctions or a
so-called high stake associated with specific
levels of performance. Any individual's test
score can be compared to a statewide stand-
ard of satisfactory performance and a pass/
fail decision made about whether the indi-
vidual can graduate from high school, be
promoted to the next grade, receive a per-
formance bonus, and so on. Once the test is
scored, the school administrator has no dis-
cretion in whether to dole out the educa-
tional reward or sanction.
The logic behind these testing programs is
that when an important consequence, that
is, a high stake, such as receiving a high
school diploma or a teaching certificate is
tied to test performance, the content re-
flected in the test will become incorporated
into instruction. The high stakes associated
with test performance will force an instruc-
tional response to the test so that the test
content will, in current parlance, "drive"
instruction. This approach is practiced in
most countries of the world and exemplified
by the British O and A Level external ex-
State-Mandated Testing
amination system (Madaus & Airasian,
1977). Popham (1987) states
Measurement-driven instruction occurs
when a high stakes test of educational
achievement, because of the important
contingencies associated with the students'
performance, influences the instructional
program that prepares students for the test
. . . Teachers tend to focus a significant
portion of their instructional activities on
the knowledge and skills assessed by such
tests. A high-stakes test of educational
achievement, then, serves as a powerful
curricular magnet, (p. 680)
High-stakes standardized tests have be-
come cornerstones in state government ef-
forts to improve educational standards and
to gain control over education in local
school districts (Airasian, 1987a). Such tests
have assumed new and important gatekeep-
ing roles in the educational process. A survey
in Education Week ("Excellence," 1985) in-
dicated that 29 states require pupils to take
competency or proficiency tests at selected
points in the educational ladder. Over 20
states have passed legislation that requires
high school students to show mastery on a
state-mandated exit test as a prerequisite to
receipt of a regular high school diploma.
Eight states and several large cities tie grade
to grade promotion to pupils' performance
on standardized tests. As of April 1987,
44 states had implemented or decided to
implement a testing program to be used to
award prospective teachers state certifi-
cation. Twenty-seven states test or plan to
test applicants to teacher training programs
in state colleges and to determine admit-
tance on the basis of these test scores (Leh-
mann & Phillips, 1987; Rudner, 1987).
Other states use pupil performance on state-
mandated tests to allocate remedial funding
to local school districts and to award mon-
etary bonuses to schools and teachers based
on year-to-year pupil test score improve-
ment. California and Pennsylvania are
working on implementing "honors" tests
that will be used to certify advanced achieve-
ment in particular high school subjects
(Carlson, 1987; Hertzog & Lehr, 1987).
Tests have moved from recorders of the
305
Peter W. Airasian
effects of educational reform to initiators of
reform, from passive instruments to intru-
sive devices that threaten test takers and seek
to dictate the ends of the instructional sys-
tem. Tests have moved from descriptive in-
dices to certification devices. State-man-
dated, high-stakes tests used to determine
high school graduation, grade to grade pro-
motion, teacher certification, resource allo-
cation, and monetary bonuses abound in
American education. All this has occurred
in barely a 10-year period.
Clearly, the public supports the new forms
of educational testing, even though the
American educational system has had little
prior experience with them (Airasian, 1987c;
Madaus & Airasian, 1977). Gallup Poll re-
sults from 1983 (Gallup, 1983) indicated
that 75% of American adults believed that
children should be promoted from grade to
grade only if they can pass examinations. In
the same poll, three quarters of the respond-
ents endorsed testing pupils in local schools
with national tests so that educational
achievement could be compared across
communities. In 1984 (Gallup, 1984), about
two thirds of those polled agreed that all
high school students in the United States
should be required to pass a standard na-
tionwide examination to get a high school
diploma. The 1986 poll (Gallup, 1986) re-
vealed that 77% of respondents would like
to see national tests given in their local
schools so that achievement can be com-
pared across communities. In the 1987 Gal-
lup Poll of Education (Gallup & Clark,
1987), 74% of the public at large endorsed a
national testing program for public school
students, although only 31 % of professional
educators endorsed such a testing program
(Elam, 1987). These data reflect substantial
and ongoing public support for testing for-
mats and uses that were virtually unknown
and untried in the United States until
around 1980. Certainly little clear empirical
evidence of their effectiveness in increasing
standards and achievement in schools has
been produced, and their benefits are
debated (Airasian, 1987b; Madaus, 1988;
McDill, Natriello, & Pallas, 1985). Yet they
are endorsed heartily by the public at large.
The question is, Why?
306
Tests as Symbols
One answer to this question is that state-
mandated, high-stakes achievement tests
possess potent symbolic appeal. Certainly
the concept of a test is meaningful to most
Americans; it is in the realm of their expe-
rience and provides a contextual reference
capable of conjuring up many and rich men-
tal images. Although each individual may
attach idiosyncratic meaning and images to
the concept of a test, there are also more
general, public meanings and images that
reflect commonalities across individuals.
Rafferty (1985) examined the significance of
tests and examinations in society by survey-
ing the manner in which they are treated in
nontechnical writings such as biographies,
fiction, poetry, and drama. Rafferty has
shown both the permeation of the concept
of tests and examinations in biography and
fiction and the commonalities tests and ex-
aminations are perceived to symbolize (e.g.,
fear of failure, path to social mobility, main-
tenance of standards). Moreover, the word
test is not affectively neutral and most often
evokes images of an experience that was
unpleasant at best and anxiety-provoking
and obstructive at worst.
State-mandated, high-stakes tests possess
three generally distinct, though not mutually
exclusive, types of symbolic appeal. First,
they symbolize order and control in a system
where autonomous local control is perceived
to be weak. Second, they symbolize, in the
constructs they are perceived to measure,
important educational outcomes. Third,
they symbolize a distinct value or moral
outlook that the public wishes to see re-
flected in its schools.
Tests as symbols of order and control In
most communities and in the nation at large,
support for schools is conditioned by the
answer to the question: How are our schools
doing? The answer sought most often is
rooted in standardized test results because,
wisely or not, the public is no longer willing
to accept testimonials about the status of
education from teachers and school admin-
istrators. The credibility of school-based as-
sessment has diminished in the face of pes-
simistic news about educational standards

and pupil achievements. In addition, there
is a sense that we have lost control of edu-
cation in our local schools and that diversity,
competing interests, and politicization are
the order of the day in the public schools
(Airasian, 1987b; Kirst, 1984). Greene
(1987) has summed up this sense and its
resulting desire very well:
Even as we become increasingly aware of
the great diversity of nations and cultures
represented in our schools, of the unpre-
dictable and the uncontrollable factors to
be taken into account, we are afflicted more
and more by arguments against relativism
and multiplicity. At once, we are bom-
barded with demands for a "common core"
at least of information, for categorical
agreements that will overcome what strikes
people as cognitive as well as moral chaos.
Feeling uncertainties and fragmentations
all around, they ask implicitly for effective
systematizations, for what is "logically
basic." They want promises and prescrip-
tions for the children; many want single-
track highways with one-way signs leading
unswervingly to some kind of success, (pp.
6-7)
Coupled with increased social acceptance of
testing to insure industrial product safety
and standards, these perceptions provide
powerful motives for state-mandated, high-
stakes testing programs.
Most Americans trust and desire external
tests. The latest Gallup Poll of Education
(Gallup & Clark, 1987) indicates that 70%
of the adults surveyed support reporting
achievement test results publicly on a state
by state and community by community ba-
sis so that comparisons can be made among
schools. Only 14% of those polled opposed
public reporting for comparison purposes.
About 70% of the people polled by Gallup
also thought that the local schools should
not be given sole responsibility for policing
their compliance with state educational
standards. The public is seeking some exter-
nal assurance of the quality of the teaching
and learning process, and it turns to stand-
ardized tests as the one index it believes can
provide the evidence it seeks. State-man-
dated, high-stakes standardized tests are
being used to restore public trust in the
State-Mandated Testing
educational credentialing system because, at
least for the present, they have more legiti-
mizing power than locally derived tests and
testimonials.
In this regard, tests are perceived as sym-
bols of order and control in the midst of
diversity, chaos, competition for dwindling
school resources, and the need for external
verification of educational standards. State-
mandated, high-stakes tests are perceived to
be scientific because they produce a numer-
ical score; fair because they are standardized
across all pupils and school districts; and
objective because preestablished cut scores
and decision rules remove local teachers',
parents' or principals' influence in educa-
tional decisionmaking. There is a sense that
human judgment and biases are being re-
moved from the assessment process by ex-
ternally mandated high-stakes tests, with a
resulting increase in control, fairness, and
objectivity. External tests reinforce a sense
of the commonness of the common schools
and the perception that the distribution of
educational resources and credentials and
the social benefits that are believed to flow
from them (i.e., higher salaries, greater pres-
tige, and better jobs) is based on merit and
performance, not whim and political clout.
To provide a perspective on this symbol-
ism, let us recognize two important charac-
teristics of state-mandated, high-stakes tests.
First, as tests, they are fairly trivial instru-
ments in and of themselves. This is not to
say they are trivial either in their perceived
import or actual effect, but it is to point out
that most tests measure only a small portion
of student behavior vis a vis some content
domain. Important decisions often are based
on a one-shot, 30- or 45-item multiple
choice test. Certainly there are better and
often more valid ways to measure behavior
and to make important decisions about
pupils.
Second, the results of state-mandated,
high-stakes tests rarely tell us things we don't
already know about pupils, especially since
the decisions we require from such tests
pertain to minimum competence. Although
correlations between teacher judgments of
pupils' achievement and pupils' measured
achievement are typically in the .6 to .7
307
Peter W. Airasian
range (Brophy & Good, 1974), such studies
are based on asking teachers to discriminate
among pupils on a five- or six-point rating
scale. High-stakes testing programs do not
require such fine distinctions among pupils,
but instead seek to differentiate the most
extreme 10 to 15% of poor learners from
their peers. Teachers can make such differ-
entiations; they know which pupils are aca-
demically unprepared to be promoted, grad-
uated, certified, and so on. Although tests
do not, of course, produce information that
is perfectly isomorphic with teacher percep-
tion, there is substantial agreement between
the two sources of information (Kellaghan,
Madaus, & Airasian, 1982), especially for
the type of differentiation sought from state-
mandated, high-stakes tests. The tests supply
largely redundant information, but in a
manner that is perceived to be more objec-
tive and fair than school-based judgments.
If state-mandated, high-stakes tests are
one-shot, small sample measures of educa-
tional attainments and if they provide little
new information about test takers, some-
thing apart from the tests themselves and
the information they provide makes them
desired by the public. That something is
what the state-mandated, high-stakes tests
are perceived to represent: external control,
order, common standards, unbiased judg-
ments, and sanctions for poor performance.
The tests are symbols of these desired char-
acteristics.
Tests as symbols of important educational
constructs. The language associated with an
innovation is important in helping it achieve
social consensus and acceptance. Language
sets the perceptions of and expectations for
the innovation. State-mandated, high-stakes
testing programs are no exception, and typ-
ically are encased in language that is in-
tended to strike a responsive chord with the
public. The rhetoric of such testing programs
heavily relies on constructs such as compe-
tence, survival skills, functional literacy, ex-
cellence, academic standards, basic skills,
preparation for life, mastery, and more re-
cently, cultural literacy. Words like these
command attention and support. The rhet-
oric makes it important to insure that pupils
308
have attained these desired outcomes or
schooling.
It is hard to be against competence, stand-
ards, or functional literacy, and few politi-
cians or school board members would be
likely to state in public that they are not in
favor of such constructs. What, after all, is
someone who is against competence for—
incompetence? The language of state-man-
dated, high-stakes testing programs helps
crystalize public acceptance because words
like competence and standards conjure up
symbolically potent images.
Too often, however, the symbolic richness
of the language used to characterize an in-
novation confers a status on the innovation's
particular methods and procedures that goes
far beyond its inherent worth or meaning.
In state-mandated, high-stakes testing pro-
grams, there is often a confusion between
what the high-stakes tests are actually meas-
uring and what they are assumed to be meas-
uring by people who are reacting to the
symbolic richness of the language used to
characterize the testing program. For most
people a so-called "Functional Literacy
Test" or "Survival Skills Test" symbolizes
an array of competencies and proficiencies
that goes far beyond the 25 vocabulary and
25 basic math items that may actually com-
pose the test. When we name or describe the
essence of an innovation with constructs
that are socially desirable and complex, as it
is often politically beneficial to do, the ter-
minology we use makes the innovation itself
a symbol of the constructs.
Consider the language used by the Cali-
fornia State Legislature in 1980, when it
passed State Bill 3408 requiring junior and
senior high schools to adopt standards of
proficiency in basic skills, and tied high
school graduation to these standards: "Pu-
pils attending the public schools in Califor-
nia [should] acquire the knowledge, skills,
and confidence required to function effec-
tively in contemporary society." Such lan-
guage is not unique to California; similar
language can be found in bills of all states.
At first blush it is difficult to find fault with
such an intent. It holds out the promise of
knowledgeable, confident pupils successfully

carrying out their chosen social roles. Envi-
sion the characteristics of a pupil who fulfills
this expectation.
Consider more fully the language of the
law. What, for example, is the distinction
between knowledge and skills? Is knowledge
a familiarity with basic facts, such as the
multiplication tables or rules of punctua-
tion, while skills are applications of these
facts to real-world situations like filling out
a tax form or writing a coherent business
letter? And if this distinction is the intended
one, are we confident that schools can or
should teach such applications to all pupils?
More to the point, however, is the language
used in the latter portion of the law. Can
schools teach confidence? What exactly is
confidence? How much is too much and
how much is too little? What of the pupil
who demonstrates cognitive proficiency but
lacks "confidence"?
Finally, what skills and knowledge are
required to function effectively in contem-
porary society? Is there a universal norm of
effective functioning that can be applied to
all people in all contexts? Moreover, in a
society that is changing constantly, is it suf-
ficient and desirable to set one's goals on
functioning effectively in contemporary so-
ciety? Contemporary society will not be con-
temporary in 10 years, and the skills, knowl-
edge, and confidence required to function
today may be quite different from what is
required in 10 years.
In lieu of such an analysis, the terminol-
ogy used to describe this early version of a
state-mandated, high-stakes testing program
is seductive: "knowledge, skills, and confi-
dence to function effectively in contempo-
rary society." Regardless of the particular
domains that finally are measured by the
test used to determine whether or not a
student will be granted a diploma in Califor-
nia, performance on that test will symbolize
for most Californians the degree to which a
pupil possesses the "knowledge, skills, and
confidence to function effectively in con-
temporary society." It goes without saying
that the test will not assess such functioning
at a level that captures the richness and
complexity most people will read into the
State-Mandated Testing
results of the test. What actually is measured
is confused with what the language of the
testing program suggests is being measured.
Often the public doesn't even know the na-
ture of the content being tested. It knows
only that a functional literacy test or a min-
imum competency test will be administered,
and this seems to be enough to convince it
that the test is desirable.
In 1966, Coleman et. al. published Equal-
ity of Educational Opportunity, in which
they concluded that "school brings little in-
fluence to bear on a child's achievement that
is independent of his background and gen-
eral social context" (p. 325). The common
interpretation of this finding was that
schools don't make a difference and an enor-
mous amount of debate over the ineffective-
ness of schools ensued from this interpreta-
tion (Madaus, Airasian, & Kellaghan, 1980).
Although Coleman et. al. used a variety of
outcome measures to assess pupil perform-
ance, the measure used in the main analyses
was a single measure of general verbal ability
(Brimer et. al., 1978; Jencks, 1972). School
outcomes were assessed by a general verbal
ability test, which came to symbolize for the
American public the broad domain of learn-
ings it associated with school achievement.
Little wonder that the conclusion that
schools don't make a difference in pupils'
achievement created debate and soul search-
ing. In interpreting the results of the Cole-
man study, the construct verbal ability,
which described what actually was meas-
ured, became confused with the construct
school achievement and a status and inter-
pretation that was not warranted was con-
ferred on the verbal ability test.
Tests, and more particularly, the scores
pupils get on tests, make concrete abstract
terms like competence, functional literacy,
excellence, and school achievement. The test
score becomes a symbol of the broader con-
struct it was designed to measure. So, the
names we give tests and the constructs that
they are intended to assess serve as powerful
and persuasive social symbols for state-man-
dated, high-stakes testing programs.
Tests as moral symbols. In addition to
symbolizing order, control, and desired
309
Peter W. Airasian
school achievements, state-mandated, high-
stakes tests also symbolize a set of more basic
social values. Rafferty (1985) has shown that
in nontechnical writings there is a common­
ality of reaction to tests and examinations.
This reaction centers on testing as a symbol
of traditional social and educational values
such as hard work, reward for effort, and
punishment for the lazy (Airasian, 1987c;
Eraut, 1981). Most of the public prizes these
traditional values and would like to see more
of them reflected in its schools. The more
these values are reflected in school practices
and programs, the more the rewards of ed­
ucation (e.g., diplomas, honors, etc.) are
thought to be worth. Tests, particularly state-
mandated, high-stakes tests with their em­
phasis on common standards and sanctions
for poor performance, reinforce the impres­
sion that the emphasis in the schools is on
earning credentials by demonstrating good
performance. In a sense, the tests revalue a
debased educational currency, the high
school diploma, teaching certificate, etc. The
use of high-stakes tests to allocate educa­
tional benefits and credentials harkens back
to an earlier time (real or imagined) in which
people attained high school diplomas, grade
to grade promotions, and teacher certifi­
cations in the old-fashioned way: They
earned them (Armbruster, 1977).
But state-mandated, high-stakes tests are
perceived not only as a measure of hard
work and diligence in learning, they also are
perceived to be motivators of hard work.
The results of the most recent Gallup Poll
of Education (Gallup & Clark, 1987) shed
light on this characteristic of tests. A national
sample of adults was asked, "It is now being
proposed that educational achievement test
results be reported on a state-by-state basis,
so that comparisons can be made between
schools of similar size and racial and eco­
nomic make-up. Do you favor or oppose
this proposal?" Given the preceding discus­
sion, it should not be suφrising that 70% of
the sample favored the proposal, and only
14% opposed it.
Adults were then asked a second question:
"Let's assume that the students in the local
public schools received higher test results
310
than students in comparable schools else­
where. Do you think that this would serve
as an incentive for the local schools to do an
even better job or not?" Seventy-two percent
said yes, higher test performance than com­
parable schools would be an incentive to do
an even better job. Only 12% said no, higher
performance would not be an incentive to
do even better.
The final question posed to adults was the
reverse situation to the second question:
"Let's assume that the students in the public
schools in this community received lower
test scores than students in comparable
schools elsewhere. Do you think this would
encourage the local schools to try to do a
better job or not?" Again, 72% said yes,
lower test performance than comparable
schools would be an incentive to try to do a
better job. As with question 2, 12% said no,
lower test scores would not be an incentive
to try to do better.
Regardless of whether the results of a test­
ing program showed local school perform­
ance to be higher or lower than comparable
neighboring schools, over 70% of the adults
polled said test results would be an incentive
for their local school to improve. People
appear to perceive some value inherent in
the test or testing process that is independent
of the results attained on the test. Tests are
valued for what they stand for, not for them­
selves. Tests motivate people to strive to do
better, regardless of their current level of
performance, and this is desired. The mere
presence of tests in schools provides psychic
satisfaction to the patrons of high standards,
reward for hard work, and punishment for
laziness. Black (1982) echoes a common re­
frain that
if the student must face an accounting (in
the form of a test) of how well he or she
has mastered the subject, learning will take
place. Whether as a competitive exercise or
in isolation, standardized testing offers the
carrot (the satisfaction of doing well) and
the stick (the disappointment of doing
badly) both at once. (p. 245)
How much stronger is the belief in this
accounting when the standardized test is a

state-mandated, high-stakes test with real
consequences for test takers?
If this line of reasoning is correct, it is
likely that the particular content domain
assessed by the tests may be less important
than the fact that a test of some kind is being
administered and rewards or punishments
are doled out on the basis of performance.
The test becomes a surrogate or symbol for
the values people perceive in the act of test-
ing: hard work, nose to the grindstone, trial
by fire, and so on. The situation is somewhat
analogous to the issue of school prayer. Few
would argue strongly that daily recitation of
the Twenty-third Psalm or the observance
of a moment of silence at the start of the
school day, in and of itself, will have a major
impact on pupils' moral and ethical devel-
opment or demeanor. That is not the point.
For its advocates, the recitation of the
Twenty-third Psalm or a moment of silence
symbolizes a host of religious and moral
values that it is reassuring to believe are
present in the school environment to which
the young are sent to learn and be nurtured.
In the same manner, tests represent broad
social and educational values that the public
wants in the school environment.
Conclusion
As educational reform mechanisms, state-
mandated, high-stakes testing programs may
not be as powerful as many expect. The jury
is still out on the effects of such programs.
There is little question, however, that the
testing programs are powerful symbolically;
they strike a responsive chord in the public
at large and this response helps explain the
widespread and speedy adoption of an in-
novation that had virtually no track record
in American education before about 1979.
In state-mandated, high-stakes testing pro-
grams, tests symbolize order and control, a
focus on important educational outcomes,
and basic, traditional moral values. These
values are endorsed by the extant culture
and this endorsement provided the legiti-
mization necessary to justify their wide-
spread adoption in the name of educational
reform. Given a different extant culture,
state-mandated, high-stakes testing pro-
State-Mandated Testing
grams might have been considerably less
warmly endorsed.
Because of the public's current faith in
tests and testing, it would not be surprising
if the symbolic dimensions of tests produced
some effects of their own, quite apart from
the educational effects produced by the test-
ing programs. Given their potent symbol-
ism, an inherent danger in state-mandated,
high-stakes tests is that their mere existence
will convince the public that much is being
done to remedy recognized educational
problems, thereby directing attention away
from other, needed remedies. The symbolic
richness of high-stakes tests creates the dan-
ger that such tests can become a convenient,
though poor, substitute for other necessary
and meaningful actions to improve schools
and learning. Also, as noted earlier, focus on
the symbolism of the new testing programs
directs attention away from needed consid-
eration of the technical quality of the tests.
The symbolic role of testing often makes it
difficult to separate discussion of educa-
tional policy and standards from public re-
lations.
A second consequence of the symbolic
dimension of tests relates to the way that the
public defines and measures educational ex-
cellence. Because so much is read into test
scores by a public that confuses the construct
label of the test with the more narrow set of
items it contains, test scores and passing
rates on high-stakes tests are becoming the
educational equivalent to the Dow Jones
stock average. People follow both the Dow
Jones and test scores avidly in order to gauge
the financial and educational progress of
society.
Tests are socially valid, highly respected
symbols of a broad range of administrative,
academic, and moral virtues in our society.
When tests have the added features of cen-
tral control and sanctions for poor perform-
ance, we can expect that their symbolic im-
portance will be heightened. Thus, regardless
of the actual impact that state-mandated,
high-stakes tests have on pupils, teachers,
and the curriculum, they are likely to have
an important perceptual impact on the pub-
lic at large. This perceptual impact underlies
311
Peter W. Airasian
the social consensus or social validation that
provides the legitimization of testing as a
workable and desired reform strategy.
References
Airasian, P. W. (1983). Societal experimentation.
In G. Madaus, M. Scriven, & D. Stufïlebeam
(Eds.), Evaluation models (pp. 163-176). Bos-
ton: Kluwer-Nijhoff.
Airasian, P. W. (1987a). State mandated testing
and educational reform: Context and con-
sequences. American Journal of Education,
95(3), 393-412.
Airasian, P. W. (1987b, November). State-man-
dated, high-stakes testing programs: Challenges
to educators. Paper presented at the Association
for Moral Education annual conference, Har-
vard University, Cambridge, MA.
Airasian, P. W. (1987c). The consequences of
high school graduation testing programs.
NASSP Bulletin, 71(496), 54-67.
Armbruster, F. F. (1977). Our children's crippled
future. New York: Quadrangle Books.
Black, T. M. (1982). Straight talk about American
education. New York: Harcourt, Brace, Jova-
novich.
Brimer, A., Madaus, G. F., Chapman, B., Kel-
laghan, T., & Wood, R. (1978). Sources of
difference in school achievement. Windsor,
England: NFER.
Brophy, J. E., & Good, T. L. (1974). Teacher-
student relationships. New York: Holt, Rine-
hart and Winston.
Callahan, R. E. (1962). Education and the cult of
efficiency. Chicago: University of Chicago
Press.
Campbell, D. T. (1969). Reforms as experiments.
American Psychologist, 4, 409-429.
Carlson, D. C. (1987, April). Technical problems
associated with high-aspiration examinations.
Paper presented at the annual meeting of the
National Council on Measurement in Educa-
tion, Washington, DC.
Coleman, J. S., Campbell, E. Q., Hobson, C. J.,
McPartland, J., Mood, A. M., Weinfeld, F. D.,
& York, R. L. (1966). Equality of educational
opportunity. Washington, DC: Office of Edu-
cation, Department of Health, Education, and
Welfare.
Cremin, L. A. (1961). The transformation of the
school. New York: Alfred A. Knopf.
Cronbach, L. J. (1975). Five decades of public
controversy over mental testing. American Psy-
chologist, 30, 1-14.
Doyle, D. P. & Hartle, T. W. (1985). Excellence
312
in education: The states take charge. Washing-
ton, DC: American Enterprises Institute.
Excellence: A 50-State Survey. (1985, February
6). Education Week, 11-30.
Elam, S. (1987). Differences between educators
and the public on questions of education pol-
icy. Phi Delta Kappan, 69(4), 294-296.
Elmore, R. F, & McLaughlin, M. W. (1982).
Strategic choice in federal education policy:
The compliance-assistance trade off. In A. Lie-
berman & M. McLaughlin (Eds.), Policy mak-
ing in education. Eighty-first yearbook of the
National Society for the Study of Education,
Part I (pp. 159-194). Chicago: University of
Chicago Press.
Eraut, M. (1981). Accountability and evaluation.
In B. Simon & W. Taylor (Eds.), Education in
the eighties (pp. 145-162). London: Batsford
Academic and Educational Limited.
Firth, R. (1973). Symbols public and private.
Ithaca, NY: Cornell University Press.
Fullan, M. (1982). The meaning of educational
change. New York: Teachers College Press.
Gallup, G. H. (1983). The 15th annual Gallup
poll of the public's attitudes toward the public
schools. Phi Delta Kappan, 65(1), 33-47.
Gallup, G. H. (1984). The 16th annual Gallup
poll of the public's attitudes toward the public
schools. Phi Delta Kappan, 66(1), 23-38.
Gallup, G. H. (1986). The 18th annual Gallup
poll of the public's attitudes toward the public
schools. Phi Delta Kappan, 68{\\ 43-60.
Gallup, A. M., & Clark, D. L. (1987). The 19th
annual Gallup poll of the public's attitudes
toward the public schools. Phi Delta Kappan,
69(1), 17-30.
Greene, M. (1987, October). Quality in teacher
education. Paper delivered at the Northeast
Holmes Group Fall, 1987 Conference, Boston,
MA.
Hertzog, J. T, & Lehr, S. T. (1987, April). Con-
tent determination for honors tests. Paper pre-
sented at the annual meeting of the National
Council on Measurement in Education, Wash-
ington, DC.
Jencks, C. S. (1972). The quality of the data
collected by The equality of educational oppor-
tunity survey. In F. Mosteller & D. P. Moyni-
han (Eds.), On equality of educational oppor-
tunity (pp. 437-512). New York: Vintage
Books.
Kellaghan, T., Madaus, G. F., & Airasian, P. W.
(1982). The effects of standardized testing. Bos-
ton: Kluwer-Nijhoff.
Kirst, M. W. (1984). The changing balance of
state and local power to control education. Phi

Delta Kappan, 66(3), 189-191.
Lehmann, I. J. & Phillips, S. S. (1987). A survey
of state teacher-competency examination pro-
grams. Educational Measurement: Issues and
Practice, 6(1), 14-18.
Lindblom, C. E., & Cohen, D. K. (1979). Usable
knowledge. New Haven: Yale University Press.
Loemker, L. E. (1962). Symbol and myth in
philosophy. In T. Altizer, W. A. Beardslee, &
J. H. Young (Eds.), Truth, myth, and symbol
(pp. 109-127). Englewood Cliffs, NJ: Prentice-
Hall.
Madaus, G. F. (1985). Testing as an administra-
tive mechanism in educational policy: What
does the future hold? Educational Horizons,
63(5), 34-39.
Madaus, G. F. (1988). The influence of testing
on the curriculum. In L. Tanner (Ed.), Critical
issues in curriculum. The 87th yearbook of the
National Society for the Study of Education,
part I (pp. 83-121). Chicago: University of
Chicago Press.
Madaus, G. F., & Airasian, P. W. (1977). Issues
in evaluating student outcomes in competency-
based graduation programs. Journal of Re-
search and Development in Education, 10(3),
79-91.
Madaus, G. F., Airasian, P. W., & Kellaghan, T.
(1980). School effectiveness: A reassessment of
the evidence. New York: McGraw-Hill.
McDill, E. L., Natriello, G., & Pallas, A. M.
(1985). Raising standards and retaining stu-
dents: The impact of the reform recommen-
dations on potential dropouts. Review of Edu-
cational Research, 55, 415-433.
Moynihan, D. P. (1970). Maximum feasible mis-
understanding. New York: Free Press.
National Commission on Excellence in Educa-
tion. (1983). A nation at risk. Washington, DC:
U.S. Government Printing Office.
State-Mandated Testing
Parsons, T. (1956). Suggestions for a sociological
approach to the theory of organization-1. Ad-
ministrative Science Quarterly, 1, 63-85.
Peirce, C. S. (1940). The philosophy of Peirce;
selected writings. New York: Harcourt Brace.
Popham, W. J. (1987). The merits of measure-
ment-driven instruction. Phi Delta Kappan,
68(9), 679-682.
Rafferty, M. (1985, April). Examinations in lit-
erature: Perceptions from non-technical writers
of England and Ireland from 1850 to 1984.
Unpublished doctoral dissertation, Boston Col-
lege.
Rowan, B. (1982). Organizational structure and
the institutional environment: The case of the
public schools. Administrative Science Quar-
terly, 27, 259-279.
Rudner, L. M. (Ed.). (1987). What's happening
in teacher testing. Washington, DC: Office of
Educational Research and Improvement, U.S.
Department of Education.
Suchman, E. A. (1967). Evaluative research. New
York: Russell Sage Foundation.
Timpane, M. (1976). Into the maw: The uses of
policy in Washington. Phi Delta Kappan, 56(2),
177-178.
Whitehead, A. N. (1927). Symbolism, its mean-
ing and effect. New York: Macmillan.
Wise, A. E. (1979). Legislated learning. Berkeley
and Los Angeles: University of California
Press.
Author
PETER W. AIRASIAN, Professor and Chair of
Educational Foundations Division, School of
Education, McGuinn 600, Boston College,
Chestnut Hill, MA 02167. Specialization: test-
ing and public policy.
313