Вы находитесь на странице: 1из 22

15_WDR_Ch07.

qxd 8/14/03 9:19 AM Page 111

Basic education services

7
An institutional arrangement for basic edu- ter 1—unaffordable access, dysfunctional
cation should be judged by its production schools, low technical quality, low client
chapter of high-quality learning, equitably distrib-
uted. This requires that children be in
responsiveness, and stagnant productivity.
But not all countries face the same prob-
school and that they learn. This in turn rests lems. In many of the poorest countries
on education systems that create relation- there are enormous deficits in affordable
ships of accountability between citizens, access. Poor people have less access, lower
politicians, policymakers, and providers, attainment, and lower quality than those
with clear objectives, adequate resources, better off. In many countries public sector
capable and motivated providers, progress provision is close to dysfunctional and rife
assessments, and performance-oriented with corruption. The technical quality of
managements. instruction and learning outcomes are
Successful education systems vary widely. shockingly low, especially among poor peo-
Some systems are centralized, others decen- ple. And even the most advanced economies
tralized. Some have almost exclusively pub- struggle to make education systems more
lic schools, while others provide public sup- productive.
port to private providers. But not just Shortfalls in universal primary comple-
anything goes. tion—a combined result of children who
never enroll, children who do not progress,
• The politics of schooling—particularly
and children who drop out—reflect the fail-
the effectiveness of the voice of poor
ures in the system. In Madagascar only 52
people—determines both the school sys-
percent of 15- to 19-year-olds in the poorest
tem’s objectives and the public resources
20 percent of the population had ever
that go to education.
enrolled in school, and only 4 percent com-
• The compact between policymakers and pleted even grade 5 (figure 7.1). In Brazil 89
providers of schooling needs to balance percent of poor adolescents enrolled in
the autonomy of schools and teachers grade 1, but only 30 percent completed
with performance assessment. grade 5 because of high dropout and repeti-
• Schools (and school systems) must be tion rates. In Turkey high retention through
enabled to manage for performance— primary school, followed by a sharp drop in
and, particularly, to find effective ways to progress to the next level, suggests that sys-
train and motivate teachers. temic and institutional solutions are
• Direct parent and community participa- required to increase achievement. In Ban-
tion in schools, demand-side inducements gladesh only 60 percent of poor adolescents
to expand enrollments, and choice—if have completed grade 1, and only 36 per-
correctly designed—can be valuable parts cent have completed grade 5.
of an overall plan for school improvement.
Unaffordable access
Common problems Despite at least 55 years of acknowledgment
of service provision that universal literacy is the heart of develop-
Education systems face the common prob- ment, and despite repeated rhetorical com-
lems of service provision outlined in chap- mitments to universal enrollment, even the
111
15_WDR_Ch07.qxd 8/14/03 9:20 AM Page 112

112 WORLD DEVELOPMENT REPORT 2004

modest goal of universal primary school Bangladesh 30 percent of students who


completion has not been realized. Some completed grade 5 were not minimally
countries have made huge strides—average competent in reading; 70 percent were not
Figure 7.1 Poor children: less likely completion rates in Brazil expanded from less minimally competent in writing.316
to start school, more likely to drop out
15- to 19-year-olds who have than 50 percent in 1990 to more than 70 per- Evidence on learning outcomes is disap-
completed each grade cent in 2000. But if countries continue at only pointing even in middle-income countries.
Madagascar 1997 their recent rate of progress, universal pri- For instance, in the recent Programme for
Percent mary completion would come only after International Student Assessment of the
100 2020 in the Middle East and North Africa, achievement of 15-year-olds in school, only
80
after 2030 in South Asia, and not in the fore- 5 percent of Brazilian students reached the
seeable future in Sub-Saharan Africa. Organisation for Economic Co-operation
60 Richest
fifth In the very poorest countries the attain- and Development (OECD) median in
40 ment deficit is spread across the population, mathematical literacy (figure 7.2). Fifty-six
but in most it is concentrated among chil- percent of Brazilian students were at level 1
20 Poorest dren from poor households. In countries (of 5) in reading literacy, compared with 18
fifth with very low attainment, like Mali, most of percent for students in OECD countries.
0
0 1 2 3 4 5 6 7 8 9 the population is rural, and there are sub- Only 4 percent reached proficiency levels of
Grade
stantial deficits in primary completion even 4 or 5, compared with 31 percent for OECD
Turkey 1996–97
among relatively wealthier and urban fami- students.317 This is not to single out Brazil
Percent
100 lies. In India the rural poor (poorest 50 per- for poor performance: Brazil is widely rec-
cent) accounted for 72 percent of the deficit ognized for its advances, and its willingness
80 in completion of grade 5 among 15- to 19- to participate in the study and its courage in
60 year-olds, and completion is higher among releasing the results demonstrate a strong
boys than girls. In the Philippines the deficit commitment to education outcomes (other
40
is much lower, concentrated among the countries have participated in examinations
20 rural poor and higher among boys than and then refused to disclose the results). In
girls. addition, in an earlier comparison of 11
0
0 1 2 3 4 5 6 7 8 9 Latin American countries Brazil was tied
Grade Dysfunctional schools with Argentina for second place in the
Brazil 1996 Schooling completions and learning out- mathematics performance of 4th graders.
Percent
100
comes may fall short because providers are
dysfunctional. While most teachers try con- Low client responsiveness
80 scientiously to do their jobs, one recent sur- When communities are not involved in
60 vey found a third of all teachers in Uttar establishing, supporting, or overseeing a
Pradesh, India, absent. Cases of malfeasance school, the school is often seen as some-
40 by teachers are distressingly present in many thing alien. Villagers refer to “the govern-
20 settings: teachers show up drunk, are physi- ment’s” school, not “our” school. In Voices of
cally abusive, or simply do nothing. This is the Poor people often complain of absent or
0 not “low-quality” teaching—this is not teach- abusive teachers and demands for illegal
0 1 2 34 5 6 7 8 9
Grade ing at all.314 fees to get their children into school or to
Bangladesh 1996–97 influence examination results.318 A study of
Percent Low technical quality schooling in rural Nigeria found that vil-
100 lagers often stopped expecting anything
The quality of instruction can also be low
80 because of low capability, weak motivation, from government schools, shouldering the
and a lack of complementary inputs. In burden themselves.319
60
very-low-income settings learning out-
40 comes can be dismal. The 1994 Tanzania Stagnant productivity
Primary School Leavers Examination sug- Creating and maintaining an institutional
20
gested that the vast majority of students had environment that promotes higher produc-
0 learned almost nothing that was tested in tivity and more learning is not easy. A
0 1 2 3 4 5 6 7 8 9 their seven years of schooling—more than recent set of studies documented that
Grade
Source: Analysis of Demographic and Health
four-fifths scored less than 13 percent cor- spending per pupil in real terms has
Survey data. rect in language or mathematics.315 In increased by 50 percent or more, often two-
15_WDR_Ch07.qxd 8/14/03 9:20 AM Page 113

Basic education services 113

or threefold, in nearly all OECD and East Figure 7.2 Fifteen-year-olds in Brazil and Mexico perform substantially worse on
standardized tests than students in OECD countries
Asian countries. Yet in none of these coun-
tries have test scores improved commensu- Distribution of mathematics test scores Distribution of reading test scores
rately.320 The obvious implication of these OECD average
two facts is that measured learning achieve- 0.5 0.5
ment per dollar spent has fallen dramati- 50 percent
above 500 50 percent
cally in every country examined. above 500

0.0 0.0
367 500 625 366 500 623
For higher-quality systems, Mexico
strengthen the relationships 0.5 0.5

of accountability
8.6 percent 18 percent
Despite enormous differences in attain- above 500 above 500
ment, equity, and learning across countries, 0.0 0.0
the features of school systems are strikingly 281 387 496 311 422 535
Brazil
similar. Public production is almost always 0.5 0.5
the dominant—if not exclusive—means of
government support of education. Whether
4.4 percent 11 percent
in Argentina, Egypt, India, Indonesia, above 500 above 500
Paraguay, or Tanzania, public systems dis- 0.0 0.0
play age-grade organization of classrooms, 212 334 464 288 396 507
Normalized test score Normalized test score
replication of social structures and inequal-
Note: Distributions are approximated on the basis of the mean and standard deviation reported in the
ities, and similar ways of training, hiring, original source.
Source: OECD (2001).
compensating, and promoting teachers.
Despite these surface similarities, there are
widely different outcomes. Both Nigeria
and Singapore retain many of the organiza- • Voice, or how well citizens can hold the
tional elements of British education. Yet on state—politicians and policymakers—
one international achievement test in the accountable for performance in dis-
1980s Nigeria was among the worst per- charging its responsibility for education.
formers while Singapore is frequently • Compacts, or how well and how clearly
among the best. the responsibilities and objectives of
That public provision has often failed to public engagement are communicated to
create universally available and effective the public and to private organizations
schooling does not imply that the solution that provide services (Ministries of Edu-
is a radically different approach (complete cation, school districts).
decentralization, total control by parent • Management, or the actions that create
groups, generalized choice) or a narrow effective frontline providers (teachers,
focus on proximate determinants (more administrators) within organizations.
textbooks, more teacher training). Univer- • Client power, or how well citizens, as
sal and quality education can come from clients, can increase the accountability of
very centralized systems (France, Japan) or schools and school systems.
from very decentralized systems with con-
siderable local accountability and flexibility Effective solutions are likely to be mixtures
(United States). Many countries have little of voice, choice, direct participation, and orga-
private schooling, and some a great deal nizational command and control, with func-
(Holland). Classroom practice is what mat- tional responsibilities distributed among
ters. If the underlying causes of failure are central, regional, local, and school administra-
not addressed, all these approaches can fail. tions. The pieces have to fit together as a sys-
Chapters 3 through 6 developed a frame- tem. More scope for parental choice without
work for analyzing service provision, looking greater information about schooling outputs
at four relationships of accountability. In will not necessarily lead to better results. Infor-
education, these are: mation systems that produce data on inputs
15_WDR_Ch07.qxd 8/14/03 9:21 AM Page 114

114 WORLD DEVELOPMENT REPORT 2004

but do not change the capabilities or incentives mate determinants (box 7.1). When teachers
of frontline providers cannot improve quality. are not consulted in training design—often
Schools and teachers cannot be made more the case—poor implementation is the result.
accountable for results without also receiving Training may not be integrated into the sys-
sufficient autonomy and resources and the tem, as when teachers are trained in methods
opportunities to build capabilities. Conversely, inconsistent with public examinations and so
schools cannot be given autonomy unless they are reluctant to adopt them. Teachers often
are given clear objectives and regular assess- have little incentive besides professional pride
ments of progress. to adopt new methods.
What successful education systems share is If the underlying problems are not
a working structure of accountability: clear solved neither bureaucracy nor market will
objectives, adequate resources, and capable work well. Increasing client power, by creat-
and motivated providers. This Report focuses ing mechanisms for communities and par-
on institutional reforms to achieve that system ents to improve their local school, is impor-
of accountability—not on the proximate tant. But this short-route accountability is
determinants of success, such as curriculum not enough. Improving services also
design, pedagogical methods, textbooks, requires stronger mechanisms of long-route
teacher training, school construction, or new accountability—accountability of politi-
information technologies. Institutional re- cians and policymakers for education and
forms will achieve desired outcomes by affect- improved proficiency in public administra-
ing proximate determinants—and proximate tion with accountability of the education
determinants that produce good education are bureaucracy for outcomes. There is no
the outcome of well-structured and well-func- quick fix in an area as complex and exten-
tioning systems. But efforts to improve proxi- sive as schooling, only the hard slog of grad-
mate determinants through internal manage- ual improvement through strategic incre-
ment initiatives have usually failed. Why? Not mentalism, which links current operational
because of a lack of knowledge of what to do. actions with long-run institutional strate-
But because of lack of the sustained bureau- gies and goals.
cratic, market, parental, and political pressure
needed to make things work. Citizens and clients, politicians
The disappointing experience with teacher and policymakers: voice
training shows the limit of a focus on proxi-
In administration of all schools, it must be kept in
mind, what is to be done is not for the sake of the
BOX 7.1 The dismal state of teacher training in Pakistan pupils, but for the sake of the country.
circa 1990 —Mori Anori, Japanese Minister
of Education 1886–89
“Teacher training in this province is a mock- A national survey of Pakistan’s primary
ery. We should close down the teacher train- schools suggests that these anecdotal Politics plays a key role in establishing
ing institutes and stop this nonsense. I have accounts are only too true. Survey data on
been teaching in a B.A./B.Ed. program for teaching practices “provide no basis for
objectives for the education system—con-
many years and see no signs that I have any statements that . . . teacher training makes a cerning both distribution and quality—and
impact on the students I teach.” substantial difference to how teachers in mobilizing resources. The reason is that
—A university education instructor quoted teach.” A 1998 study of teacher training schooling, especially at the basic level, has
in Warwick and Reimers (1995). suggests that “staff and faculty are profes-
sionally untrained, political interference is become an important element in a child’s
“Most inmates of this system [two teacher common, resources and facilities are poor socialization.
training institutes] have no respect for and badly utilized, motivation and expecta- Those who control the state use schooling
themselves, hence they have no respect for tions are low and there is no system of
others.The teachers think the students are to promote beliefs they consider desirable.
accreditation to enforce standards.” Embed-
cheats, the students think the teachers have ded in an education system that was funda- Nearly everywhere this means that schools
shattered their ideals. Most of them are dis- mentally unaccountable and lacked any promote a sense of national identity, a
illusioned.They have no hopes, no aims, no outcome orientation, teacher training national language, and loyalty to the nation-
ambitions.They are living from day to day, reflected worst practice.
watching impersonally as the system crum- state—in competition with more local or
bles around them.” Sources: Warwick and Reimers (1995); Kizilbash ethnic affiliations—and, in more extreme
—Nauman (1990). (1998), p. 45. cases, a specific political indoctrination.
Modern states—from Third Republic France
15_WDR_Ch07.qxd 8/14/03 9:21 AM Page 115

Basic education services 115

to Ataturk’s Turkey—have also used public Because the educational system had no
schools to supplant or suppress religious coherent, consensual focus: “For reforms to
instruction.321 Authoritarian states have used stick, there first needs to be a vision for the
schooling to disseminate a single acceptable future with agreed long-term objectives
ideology—for example, Soeharto’s promo- derived from stakeholders: informed dia-
tion of the five principles of pancasila322 in logue with parents, employers, religious lead-
Indonesia. These examples are not the excep- ers, school leavers, and others. The absence of
tion but the rule: countries around the world such a shared long-term quality-of-services
explicitly use schooling to inculcate ideas strategy that focused scarce resources on
about the proper organization of society. quality rather than quantity has left the edu-
cation sector open to the imposition of ideas
Voice and the objectives of schooling from outside: from donors, with agendas of
Schooling has become a battleground for questionable value to the country’s situation,
political conflicts. Different groups want dif- or from graduates returning with overseas
ferent—often contradictory—things from degrees and ill-informed, though well-inten-
schooling. Poor parents see education as an tioned, agendas of their own.”323
opportunity for their children to lead better Democracy is not necessary for excellent
lives, but they may also want education to schools. The huge variation in commitment
reinforce traditional values. Elites may want to schooling across the states of India is
universal education but often promote pub- enough to suggest that electoral democracy is
lic spending on higher education for the also not sufficient for voice to lead to universal
benefit of their own children. Urban and education (see spotlight on Kerala and Uttar
business coalitions may favor more educa- Pradesh). But the absence of democracy or
tion because it increases the productivity of other means of effective citizen voice has a
their workers, or industrialists may quietly huge downside. While one-party states occa-
oppose “too much” education because it sionally produce good results (see spotlight
makes workers restive. One recent study of on Costa Rica and Cuba), many authoritarian
owners and managers of modern factories regimes have no interest in expanding educa-
in Northeast Brazil that were moving to cut- tion or improving its quality. There are two
ting-edge business practices revealed a dis- risks: the system is effective but its goals are
turbing lack of support for expanding edu- completely set by politicians and policymak-
cation. Many felt that a primary education ers, or the system is ineffective because politi-
(eight years) was helpful, but more than that cians and policymakers have goals other than
was “dangerous” because it created workers effective provision of services. The results: too
who were less docile. Many commented that few resources are allocated to education, too
“too much education is a bad thing.” few of those resources reach poor people, and
(Tendler 2003). Politicians may want to resources are allocated ineffectively (because
deliver on promises of universal schooling providers are more influential than citizens).
while also using the education system to As more countries move to more democ-
provide patronage jobs (the example of Pak- ratic modes of choosing leaders, citizen con-
istan, in box 5.3, is not unique). Teachers trol over the structure and content of curric-
and their unions want high-quality univer- ula gains prominence. Having a common
sal education but also higher wages. negotiated vision of the objectives of public
To get what you want, you need to know support for schooling makes it easier to move
what you want. But what a society wants to the other stages of improving the quality of
from its schools is not simple and cannot be schooling—mobilizing and allocating
decided by experts alone. A recent study of resources, communicating objectives to
attempts to improve the quantity and qual- providers, and delegating responsibility and
ity of basic education in an Asian country in autonomy to schools. Without a clear vision
the 1990s concluded that even many peda- of goals, reform is reduced to a focus on
gogically and internally sound reforms did inputs and process alone.
not have a sustained impact on teaching The greater the demand for education, the
practice or student learning. Why? sharper the vision. In Malawi, Uganda, and
15_WDR_Ch07.qxd 8/14/03 9:22 AM Page 116

116 WORLD DEVELOPMENT REPORT 2004

most recently Kenya, a commitment to uni- higher levels of education. Or systems are inef-
versal education was a popular stance— ficient in translating resources into outputs. A
although a difficult commitment to match common problem is that teacher salaries, even
with resources (see spotlight on Uganda). at very low wages, crowd out all other inputs. A
recent study found that 44 of 55 countries
Adequate resources, examined allocated more than 70 percent and
adequately distributed half (23) allocated more than 80 percent of
To achieve educational goals politicians and spending to salaries. Such levels of spending
policymakers—either autonomously or often imply either inadequate supplies to other
through the pressure of citizen voice—must inputs or formal or informal levies on parents.
provide adequate resources. To learn effec- Empirical studies also show that increases in
tively, children need affordable access to teacher salaries have little or no association
infrastructure, inputs, and instruction—far with learning outcomes (discussed further
from the case in many countries. A recent below). Many studies estimate the impact of
study of financing the global Education for selected classroom instructional materials or
All initiative compared successful and less school facilities to be some 10 times that of
successful countries along three dimensions: teacher salaries (this is not to say that simple
“equipment-based” approaches will succeed).
• Revenue mobilization for primary edu-
Another common problem is devoting
cation (overall taxation rates, the frac-
resources to reduce average class sizes, which
tion of spending on schooling, the frac-
often results in inefficiently small classes—
tion of that spent on primary schooling).
boosting unit costs and limiting access.
• Unit cost of a year of effective schooling Public resources are politically distrib-
(teacher salaries and class size). uted, so the effective distribution of resources
• Internal efficiency (years of schooling is an issue of voice. A review of the empirical
provided per primary school completer). evidence suggests that the common pattern
Even with adequate fiscal effort, reason- of too few resources to high-productivity
able costs, and internal efficiency, many inputs is so ubiquitous—figure 7.3 gives just
countries do not generate enough resources two of many possible examples—that it is
to achieve universal completion. For these likely generated by a political economy that
countries there is a compelling case for addi- fails to adequately incorporate the voice of
tional international assistance (see box 2.3). poor people. Changing this distribution of
But in many cases the resources are simply resources requires more than a technocratic
not used effectively. They are allocated to the adjustment—as Brazil has shown by its
wrong mix of inputs. Too great a share goes to reforms in the 1990s.
Because poor people are almost always the
Figure 7.3 Increases in test scores per dollar spent on different inputs last enrolled, additional spending that
Northeast Brazil (1980s) India (1990s) expands access is more favorable to poorer
households than existing spending. A study
Teacher salary 1 Teacher salary 1 in India found that even though educational
expenditures on average were not more pro-
Ensuring all
Facility poor than a uniform transfer would be, the
teachers have 3
years secondary
1.9 improvement 1.2 poor benefited more than proportionately at
intervention the margin when enrollments in primary
school
Teacher table, education expanded (since the better-off
One additional
pupil tables and
chairs, and other
7.7 square foot 1.7 were already in school).324 So education
per student
“hardware” expenditures that expand access are better
Packet of Full packet of targeted to poor people than resources that
instructional 19.4 instructional 14
materials materials exclusively raise quality.
But the quality-quantity tradeoff is not a
0 5 10 15 20 0 5 10 15 20
Increase in test score per dollar, Increase in test score per dollar, simple choice between creating additional
relative to teacher salary relative to teacher salary school places or improving instruction. A
Source: Pritchett and Filmer (1999). major problem for poor children in nearly
15_WDR_Ch07.qxd 8/14/03 9:22 AM Page 117

Basic education services 117

Table 7.1 In Madagascar, at higher levels of education unit costs are much higher and participation of the poor much lower
Ratio of cost of a year of Cumulative public spending Share of poorest 40 percent Share of poorest 40 percent in
higher education to the cost on graduates of each level in those who complete each those who reach each level
of a year of primary school (percent of GDP) level (percent) (percent)

No schooling 0 0 37.6 57.8

Primary (grades 1–5) 1 0.4 7.3 33.7

Lower secondary
(grades 6–9) 2.75 1.25 0.5 3.1

Upper secondary (3 years) 5.5 2.56 * *

Higher (4 years) 19.6 8.84 * *

* indistinguishable from zero.


Sources: World Bank (2001c) and analysis of Madagascar Demographic and Health Survey.

every environment is that they drop out of tute and disadvantaged has political dan-
school with greater frequency, in part because gers as well. Systems that focus mainly on
the quality of the schooling they receive is so poor citizens, leaving the middle classes no
low. So quality improvements need to accom- stake, tend to be financially less sustainable
pany quantity improvements. and to experience less pressure for account-
Spending on primary schooling is mildly ability—and so tend to be inefficient and
progressive, but that on higher levels of edu- unconcerned with quality.
cation is not. With children from the poorest
households unlikely to reach higher levels of Policymakers and organizational
schooling, and with greater per student providers: compacts
spending at higher levels than at lower, chil-
dren from richer households capture the bulk I do not care that teachers are offended by it. I am
less interested in the teacher’s method of teaching
of educational spending. In Madagascar a than in the result she achieves. . . . There should be
single year of higher education costs 20 times a test at the end to see whether the results are being
that of primary schooling—and only 3 per- achieved. . . . Let us who represent the community
cent of children completing lower secondary say here and now there should be a [test] no mat-
school are from the poorest 40 percent of ter who may oppose it. . . . If we want to see that a
certain standard is reached and we are paying the
households (table 7.1). Relative cost alone is money we have the right to see that something is
not the issue. It is whether funding across lev- secured for that money.
els is equitable and efficient—or driven Eamon de Valera, Irish Prime Minister, 1941326
exclusively by elite politics.
The political conditions required for ade- The line separating the state as education pol-
quate budget allocations for education are icymaker (setting the rules of the game) and
not obvious. Simple answers like “democ- as major organizational provider (running
racy” are attractive—but just not true. India, the school system) is typically blurred. The
democratic since independence, has wealth minister of education frequently wears both
gaps in education attainment larger than any hats. Often there is no interest in measuring
other country with comparable data. At least results, so there is no way of making the pub-
one empirical study suggests that nondemoc- lic provider accountable for results.
ratic countries spend more on education.325 Clarifying objectives and the roles of pol-
But there is a risk that these governments care icymakers and providers is a first step. With-
not about the quality of education but about out specifying desired outputs and outcomes
using schools for religious, secular, or there is no way to say whether resources are
national indoctrination. In countries with sufficient (sufficient to do what?) or used
democratic elections, schooling opportuni- effectively (relative to what goal?). Vague
ties can be limited and education resources oversight and vague goals reduce manage-
devoted to patronage and clientelism if ment to compliance with formal rules for
voice is weak and control rests with a nar- inputs and processes. The resulting lack of
row elite. Targeting resources to the desti- clarity often results in “mission drift” and
15_WDR_Ch07.qxd 8/14/03 9:22 AM Page 118

118 WORLD DEVELOPMENT REPORT 2004

distracting struggles within the ministry of tion-intensive. It has multiple outputs that
education. Lacking a clear mission, the edu- differ in measurability and in the difficulty
cation ministry is often accused of being cap- of attribution. And it involves a complex—
tured by a teachers’ union rather than repre- and not well understood—relationship
senting the collective interest in schooling. between inputs and outputs. High-perfor-
The Irish Prime Minister’s insistence on mance schooling conveys skills, attitudes,
testing is a common reaction to the perceived and values. Some steps in this process can
failure of schools: a temptation to define the be reduced to a detailed script. And some
output of the school system exclusively as test aspects of instruction can be replaced by
scores and then to hold schools accountable technology. But face-to-face interaction and
for those scores. But accountability too nar- flexibility are crucial to high-quality
rowly measured distorts the education sys- instruction. Instructors need to be capable
tem. Only what gets measured gets done. of exercising discretion—in assessing stu-
The strict primary completion examination dent mastery, providing feedback, and tai-
brought in so confidently by the Irish govern- loring the instructional mode to the student
ment in the 1940s was gone by the 1960s, in and subject matter. This classroom behavior
large part because of these concerns. is extremely difficult to monitor.
The compact between policymakers and Schooling has multiple outputs—some
organizational providers should create an easily assessed, others not. Assessing mastery
environment in which all schools have the of simple skills through standardized testing
means and motivation to provide high-qual- is fairly straightforward. But it is difficult to
ity learning. Whether there is public produc- assess how well schooling has conveyed a
tion or government funding of a range of conceptual mastery that allows application
providers, the compact should focus on out- to real-world problems. It is still more diffi-
puts and outcomes. This requires a means of cult to assess how well schooling has
assessing a school’s contribution to the col- encouraged creativity. And it is even more
lective objectives of education, and creating difficult to assess how well schooling has
an environment for organizations to inno- conveyed values. Assessing success is further
vate and bring those innovations to scale— complicated because different actors assign
school autonomy with accountability. different values to different objectives.
Designing an accountability system is
The use and abuse of accountability difficult because it is difficult to attribute
Creating accountability in schooling is diffi- specific outcomes—or even outputs—to
cult. Schooling is discretionary and transac- specific actors. If a 15-year-old has mastered

Table 7.2 Schools account for only a small part of variance in student learning outcomes (percent)
Share of total variance across students
I II III IV
Due to differences in student Due to differences across Fraction of total variation Share of total variation in
performance within schools schools attributable to student student test performance
background differences that is (a) school specific
across schools and (b) not attributable to
student background differences
across schools (II minus III)

Brazil 55 45 25 20

Russian Federation 63 37 17 20

Czech Republic 48 52 4 18

Korea, Rep. of 62 38 14 24

Mexico 46 54 32 22

Developed country average 66 34 20 14

Source: OECD (2001), Annex B1, table 2.4.


15_WDR_Ch07.qxd 8/14/03 9:23 AM Page 119

Basic education services 119

algebraic concepts, who deserves credit and many goals societies have for their schools.
in what proportion? The parents’ genes? Performance measurement is not an attempt
The child’s nutrition? The parents’ motiva- to reduce the output of schooling to the ability
tion and efforts? The child’s peers? The of students to answer questions on standard-
child’s primary school math teachers? ized examinations. The dangers of test-based
Another teacher who motivated the child to school accountability have been debated for at
do well in all subjects? The child’s current least 140 years (box 7.2). You get what you pay
algebra teacher? for. But there are also dangers in too little
Nearly all empirical studies of measured attention to performance. It is important to
learning achievement agree that home back- distinguish among the three types of assess-
ground accounts for most of the explainable ment: sample-based assessments to track per-
variation in learning outcomes, especially in formance over time, “gatekeeper” examina-
primary grades. The same studies disagree tions that are high stakes for students, and
widely about how much can be attributed to assessments of school performance.
a child’s school. The recent Programme for
International Student Assessment study Tracking progress. One way to strengthen
found wide variation in differences in student the compact between policymakers and edu-
performance within or between schools cation providers is to develop measurement
(table 7.2). Half or more of the variation in and reporting systems that allow investiga-
performance across schools was due to varia- tion of value for money. Standardized exami-
tion in students’ socioeconomic status, not to nations are a relatively inexpensive device for
factors under school control. In poorer coun- monitoring progress and effectiveness. But
tries the effect of schools is larger—and that few education systems in the developing
of parental background smaller. But, in gen- world have disaggregated the cost of running
eral, identifying the school’s value added is a school, and even fewer know how that cost
not simple. is associated with learning. So there is almost
Even for outputs easier to specify and no reporting based on such measurements.
measure, not much is known about how The lack of information leads to an inability
inputs affect them. Economists summarize to act accordingly.
this relationship under the metaphor of a When the data are revealed, they can be
“production function.” Little is known about surprising. One study that generated data
this function because instruction involves relating expenditures and learning at the
human beings—teachers and students—in
all their complexity. For instance, there is
ongoing, vigorous debate about the relevance BOX 7.2 Test-based accountability—nothing new under
of class size for student test scores. Some the sun
assert that class size is irrelevant, or nearly so.
Test-based school accountability might ments provided teachers (who at the time
Some assert that reductions in class size have
seem like the latest thing. It isn’t. British leg- had little training) with clear indications of
such a salutary impact on performance that islation for school funding in 1862 included what was valued and tangible awards for
they are a cost-effective means of improving a system of “payments for results.” In addi- achievement.
performance.327 After more than a century of tion to a base grant (based on number of Opponents raised the same arguments
children and attendance), schools received made today.Teachers will “teach to the test”
widespread use of classroom instruction, a grant for each student who passed a and ignore subjects not covered by the test
intelligent, well-meaning, and methodologi- series of tests given by school inspectors in (such as history and geography).Test-based
cally sophisticated researchers are still debat- reading, writing, and arithmetic. accountability will lead to teaching meth-
ing such a seemingly simple issue. That shows Proponents of the testing argued that ods that emphasize rote memorization and
performance-based transfers were only cramming. One educator argued that “pay-
how truly complex the research questions common sense since public money was ment for results” would “be remembered
are—the results will vary across time, con- involved. As one parliamentary proponent with shame.”
tent, and context. reasoned: paying for performance will This particular system of “payment for
either be cheap (because few schools meet results” was abolished in 1890. But the
the standard) or expensive (because many debate continues today.
Assessment systems students have high performance)—but it
National assessment systems are essential for will not be both expensive and ineffective. Sources: Based on Bowen (1981) and Good and
monitoring educational achievement. But per- Educational historians claim that the pay- Teller (1969).
formance measurement is as complex as the
15_WDR_Ch07.qxd 8/14/03 9:23 AM Page 120

120 WORLD DEVELOPMENT REPORT 2004

Figure 7.4 School success depends on more than chances, parents will exert pressure on the
spending per student
Primary school pass rate in Mauritania
school system for better examination
results. Where public examinations are lim-
Pass rate (percent) ited and educationally inadequate, perverse
100
pressures can worsen true educational qual-
80
ity in the interests of better examination
scores.
60
School-based accountability for examina-
40 tion results. School accountability is con-
troversial—with good reason. There is
20 empirical evidence that accountability mech-
anisms based on examination results lead to
0 “teaching to the test” and to attempts to
0 5,000 10,000 15,000 20,000 manipulate results. Evidence from locations
Unit cost (Mauritanian currency)
as diverse as rural Kenya (see chapter 11 and
box 7.5 later in this chapter) and urban
Source: Mingat (2003).
Chicago shows that accountability raised
examination scores—but also that teachers
school level in Africa found little connec- manipulated the students taking the exam,
tion—Mauritanian schools with similar and taught to the test.
spending had pass rates of less than 5 per- But teaching to the test is a criticism only
cent and more than 95 percent (figure 7.4). if the test is not a reliable assessment of the
Needed for active management are skills that are the objective of public sup-
data—on school costs, on the characteris- port for schooling—or if the tests divert
tics of students, and on school performance teachers from more productive activities,
on cognitive achievement tests. Once such as teaching higher-order thinking.
implemented, these sample-based systems There is a tradeoff between what the test
can be gradually scaled up to provide more costs (in design, testing, and scoring) and
census-like measurements. how well it captures desired schooling out-
put. Tests in some circumstances could
Gatekeeper examinations. In most coun- divert teachers from more productive to less
tries examinations are seen as a fair way of productive activities, such as “drills.” But in
allocating limited school places. One study many cases performance is so weak that
suggests that the impact on student perfor- even “less productive” but learning-ori-
mance of centralized curriculum-based ented activities would be an improvement.
examinations is as large as that associated
with differences in parental education or
with substantially more formal education Figure 7.5 Centralized exams have a strong impact
for teachers (figure 7.5). Since centralized on student performance
examinations make relevant information
Incremental test score
widely available, they can be useful for gen- 20
erating accountability.
Math
The impact of public examinations on 15
the incentives of various actors points to
Science
systemic considerations. For instance, 10
teacher training programs often attempt to
instill pedagogical techniques that promote 5
higher-order thinking skills. But when gate-
0
keeper public examinations assess only rote Centralized Parent completed Teacher with
memorization, teachers frequently revert to exams tertiary/secondary PhD/MA/
similar methods. And if public examina- education bachelor degree
tions have a major impact on students’ life Source: Wößmann (2003).
15_WDR_Ch07.qxd 8/14/03 9:24 AM Page 121

Basic education services 121

There are three important technical comes. The roles of the ministry of educa- Figure 7.6 In Chile, good schools
service students from every level of
design issues with school-based account- tion can be unbundled so as to separate edu- socioeconomic status
ability. First, the characteristics of students, cation policy from the operation of schools.
their peers, and their families are far and A more explicit compact relationship can be Fourth grade basic SIMCE score
100
away the largest determinants of variation in made with organizational providers, per-
performance. Any attempt to judge schools haps even with multiple providers within
80
on their level of performance will therefore the same jurisdiction. This structure can
be judging the socioeconomic composition give clearer guidance on desired outputs and
of the school—a “good” school might sim- outcomes, freeing school heads and teachers 60

ply have wealthier students. This is true on to pursue defined goals.


average. Some schools serving poorer popu- Nicaragua created autonomous public 40
lations perform well or even very well (fig- schools guided by a school directive council
ure 7.6). And some schools with wealthier comprising the school head, elected teach- 20
0 20 40 60 80 100
students are mediocre. ers, parents, and students. The school High Low
To focus on school value added rather retained revenue from students, and the Index of socioeconomic status
than differences based on school popula- council could make decisions about person- Note: The SIMCE is a standardized test in Chile.
tions, scores can be empirically adjusted for nel, finance, and pedagogy. The average Source: Mizala and Romaguera (2000).
the composition of the student body (box school autonomy reported by these schools
7.3). Or assessments can measure changes in was between that reported by traditional
student performance (which assumes that public schools (very little) and private
socioeconomic composition is roughly con- schools (almost complete). The degree of
stant). Or a threshold can be set that all self-reported school autonomy was posi-
schools—whatever their student composi- tively correlated with student performance
tion—are expected to achieve. on test scores at the primary level (though
A second design issue in school-based not at the secondary)—but autonomy on
accountability is statistical sampling. In paper was not. In a study in Chile very little
many schools the number of students is of the variation (less than 1 percent) in
small enough to result in considerable vari- three measures of self-reported autonomy
ability. That means that even schools with of teachers was between the four types of
strong improvements over time will have schools—public, private voucher, private
years when scores are lower than in previ- paid, and Catholic voucher. More of the
ous years—simply because of the mix and variation was between schools of the same
number of students. It also means that a
program of rewards or punishments for
performance would disproportionately BOX 7.3 School-based performance awards in Chile
reward and punish small schools relative to
large schools. The third design issue is Since 1996 Chile has had an award for “top- correlation between socioeconomic status
whether to reward good performance or performing” schools in each region. Ninety and awards.
percent of the award goes directly to teach- Next, an index of school performance is
intervene in bad performance—or both. ers (in proportion to their hours of employ- calculated based on standardized tests in
ment), and 10 percent is allocated to the Spanish and mathematics in grades 4, 8,
School autonomy schools.The awards are given every two and 10.The index is weighted for average
years. test level (37 percent) and improvement in
Accountability and autonomy are twins.
Schools are divided into comparison test scores (28 percent) and includes other
Traditional public sector bureaucracies have groups within each region of the country criteria such as “equality of opportunity” (22
little autonomy because accountability is based on location (rural, urban), education percent)—based on student retention and
linked to rules and procedures, which allows level (primary only, secondary with no “discriminatory practices”—and “initia-
primary), and socioeconomic status of par- tive” (6 percent)—based on regular devel-
for little discretion. The heads of individual
ents (according to information collected as opment of group pedagogical activities.
schools are often bound by process require- part of the examination and an official The program has been through three
ments and so have little autonomy to “index of vulnerability”). In 2000–01 this rounds of selection, with 2,520 schools hav-
actively manage their schools—to define a classification produced 104 comparison ing received awards once, 1,084 schools
groups. In this way the performance of poor twice, and 360 schools in all three rounds.
mission, choose instructional staff, inno- rural schools is not compared head-to-head
vate, or encourage performance. Granting with that of richer urban schools. Analysis
greater autonomy requires new forms of suggests that this procedure diminishes the Source: Mizala and Romaguera (2002).
accountability based on outputs and out-
15_WDR_Ch07.qxd 8/14/03 9:24 AM Page 122

122 WORLD DEVELOPMENT REPORT 2004

teaching methods, new instructional inputs,


BOX 7.4 Two large-scale cross-national assessments new use of the latest technology. The problem
of learning is that there is too little systematic learning
from innovation and too little replication of
The Third International Mathematics and formance data on 265,000 students in 32
proven innovations.
Science Study (TIMSS) created a data set on (mostly OECD) countries along with infor-
student performance and characteristics mation from students and principals about The contrasting use of rigorous evalua-
and on institutional characteristics of the themselves and schools.The conclusions for tions in health and education is striking. In
schooling system such as use of centralized what schools can do to make a difference most developed countries no drug can be
examinations and central, local, and school were:
decisionmaking responsibilities. Analysis of
used until it is proven safe and effective, and
the performance of more than 266,000 stu- • Students’ reported use of school the standard of proof is the randomized
resources was more closely associated
dents from some 6,000 schools in 39
with performance than principals’ reports
double-blind clinical trial. But in schooling,
(mostly OECD) countries yielded the follow- instructional practices for hundreds of mil-
of resource deficiencies.
ing conclusions:
• The ratio of students to teachers matters lions of children can be changed because a
• Money cannot buy quality in present where it is high, while in the typical range new technology appears promising. Or
schooling systems. there is a much weaker association with
because a group of experts thinks so. Or
• Incentives are the key to success. performance.
because the practice has been tried in a pilot
• Schools should be allowed to decide • Three factors of school policy are associ- program (and subject to “Hawthorne”
autonomously on operational tasks. ated with better student performance:
• Schools must be made accountable. school autonomy, teacher morale and effects, the nonreplicable impacts that
commitment, and other teacher factors occur simply as a result of the increased
• Teachers’ incentives have to focus on such as expectations.
improving student performance. attention from any innovation). Or because
• Competition between schools creates • Three classroom practices reported by it has been shown to be statistically corre-
students show a positive association with
incentives for improving performance.
performance: the extent to which teach- lated with success, subject to all the dangers
A second study, the OECD Programme ers emphasize performance, teacher-stu- of improperly inferring causation. There is
for International Student Assessment, dent relations, and the disciplinary
assessed “young people’s capacity to use
strikingly little use of randomized con-
climate of the classroom.
their knowledge and skills in order to meet trolled experiments as a routine manage-
real-life challenges, rather than merely look-
• Successful students are more likely to do ment practice—despite its eminent feasibil-
homework.
ing at how well they mastered a specific ity for many classroom practices (box 7.5).
school curriculum.”The study collected per- Sources: Wößmann (2003) and OECD (2001).
A recent example of evaluating a school-
ing innovation illustrates the power of flexi-
bility in design—and the power of evalua-
type (between 15 and 18 percent), and most tion. A remedial education program,
was between teachers in the same school.328 established as a collaboration between the
Teacher autonomy in classroom tasks government and a nongovernmental orga-
consistently emerges as a determinant of nization (NGO) in two cities in India
success (box 7.4). The principles developed (Mumbai and Vadodara), hired local
in chapter 6 are apt: discretion and deci- women to teach catch-up classes for stu-
sionmaking power need to be delegated to dents who were falling behind. The program
those with the relevant information and was inexpensive—$5 a child a year. A rigor-
professional skills. Centralized control of ous evaluation based on the randomized
teacher assignment and assessment can design of the program found it very effective
cause bureaucratic paralysis. But making at boosting learning, especially among
schools autonomous in curriculum design, poorer children. The evaluation showed
examinations, assessment, and finance can that, at the margin, extending the program
lead to excessive variability across schools. would be about five times more cost-effec-
tive than hiring new teachers.329 The pro-
Innovating, evaluating, gram is implemented now in 20 Indian
and scaling up cities, reaching tens of thousand of children.
The goal of school autonomy and account- But “there is a particular irony to educa-
ability is to create a system in which organiza- tion reform . . . [as] pockets of good education
tional providers have strong, sustained incen- practice . . . can be found almost anywhere,
tives to improve outputs. The problem is not signifying that good education is not the
a lack of innovation—there is a continual result of arcane knowledge. Yet the rate of
stream of new modes of teacher training, new uptake of effective practices is depressingly
15_WDR_Ch07.qxd 8/14/03 9:25 AM Page 123

Basic education services 123

BOX 7.5 Randomized experiments in Busia district, Kenya


Since 1996 a group of researchers has been pre-test.This suggests that because the treatment for hookworm, roundworm,
working with a Dutch nonprofit (International textbooks in this instance were too difficult for whipworm, and schistosomiasis found that it
Christelijk Steunfonds) supporting schools in the typical student, the books did not matter. reduced absenteeism by a quarter. Does this
rural Kenya to estimate the impact of various mean that health is all that matters? No. While
Teacher incentives. Everybody knows that
interventions.Through random selection some attendance improved, test scores did not.
teacher incentives are crucial since teachers are
schools were chosen to implement the Three observations. First, things that every-
undermotivated.Yet a study on incentives for
interventions first, with the other schools to fol- body “knows” to be important did not work as
teachers based on student test scores found
low.This allowed the researchers to test a num- planned, whereas the intervention with lower
that “teachers responded to the program
ber of ideas. expectations had large impacts. Second, these
primarily by seeking to manipulate short-run
results from a hundred schools in an isolated
test scores. . . . [T]eachers’ absence rates did not
Textbooks. Everybody knows that textbooks area of Kenya have been getting enormous aca-
decline, homework assignments did not
are important and that their lack is a major con- demic attention because there are so few rigor-
increase, teaching methods did not change.”
straint on effective instruction.Yet the first study ous, randomized evaluations of schooling inter-
Does this mean that teacher incentives don’t
found “no evidence that the provision of text- ventions.Third, the findings from each
matter? No.Teachers did change their
books in Kenyan primary schools led to a large intervention do not reveal universal,
behavior—they “conducted special coaching
positive impact on test scores, nor is there any immediately generalizable results, but they
sessions and encouraged students to take the
evidence that it affected daily attendance, grade reveal that specifics matter and that learning
test.”This suggests that you get what you pay
repetition, or dropout rates.” Does this mean about what works needs to be local to be useful.
for—whether you like it or not.
that textbooks don’t matter? No. Although text-
books did not increase the performance of the Deworming. Deworming does not feature Sources: Miguel and Kremer (2001); Glewwe, Ilias,
typical (median) student, they did improve per- widely in the education effectiveness literature. and Kremer (2000); Glewwe, Kremer, and Moulin
formance for students who did the best on the Yet a randomized trial of an inexpensive medical (1997).

low and effective schools are often found just Accountability is, of course, difficult to
a few blocks from dysfunctional ones.”330 define. Is it accountability within the
The U.S. Agency for International Develop- bureaucracy (so that policymakers choose
ment (USAID) attempted to refocus its and replace principals based on perfor-
efforts in education in Africa from “proxi- mance)? Is it the direct participation of par-
mate determinants” to a more systemic ents or school councils in choosing school
approach that focuses on internally driven management? Is it parental choice?
identification and scaling up of good prac- There are alternatives. One is to allow the
tices. A recent review of USAID projects most competent actors (principals, teach-
based on systemic reform found, not surpris- ers) to run more than one school. This
ingly, that implementation was difficult would allow the more competent to affect
because it went to the heart of the relation- greater numbers of children—and reduce
ships of accountability among actors in edu- the sphere of influence of the less compe-
cation—and that was intensely political. tent. A second way is to systematize a vari-
Even so, recent work at USAID explores ety of standard-provision models that are
solutions to the challenges of linking author- easy to replicate and franchise, whether the
ity, accountability, and transparency to franchise is a bureaucracy or a private
strengthen basic education through institu- provider. Franchise models should be based
tional reform. There are several ways of on local research on what capable principals
expanding and scaling up good practice.331 currently do in a variety of real settings as
The most obvious way is to use greater well as on citizen dialogue around the
school autonomy—leaving scope for school emerging models. Models could also be
management to define a school mission, based on statistical analysis of the maxi-
mandate, and tactics—and greater account- mum “output” produced by schools, using
ability to enable the monitoring of perfor- the average level of resources that schools
mance. The autonomy and accountability can typically mobilize.
create incentives to adopt proven successful None of these approaches to learning
practices, to evaluate the effectiveness of about learning is possible without assess-
homegrown initiatives, and to create a sense ments of outputs—not just standardized
of pride and commitment in the school. exams but assessment of all the relevant
15_WDR_Ch07.qxd 8/14/03 9:25 AM Page 124

124 WORLD DEVELOPMENT REPORT 2004

outputs of schooling. Nor is any possible of income) or too high (where pay is several
without enough organizational autonomy times higher than needed to attract a qual-
for individual schools or groups of schools ity pool of teachers). But appropriate com-
to decide how best to act. pensation involves more than the level of
pay. It is the overall attractiveness of the
Organizational and frontline profession and the structure of compensa-
providers: management tion that motivate performance.332 Teacher
Managing for effective services means get- pay is usually linked to factors that show lit-
ting people with the right skills and training tle association with student performance—
in place (capacity). It means giving them mainly seniority. Teacher earnings thus
the right infrastructure and inputs to work exhibit much less variance than earnings of
with (logistics). And it means ensuring the workers in other occupations. Compensa-
motivation (both extrinsic and intrinsic) of tion should reward good teaching, not just
frontline workers. The typical public school longevity.
is often handicapped in these endeavors in Motivation and capabilities
nearly every possible way. Individual school
The schooling process is so complex—the
managers often cannot choose their own
difficulties of attribution so severe—that
teachers and cannot dismiss them—even
simple proposals of “pay for performance”
for good cause. Teacher training and capac-
for individual teachers and principals have
ity building are often ill-designed and
rarely proved workable.333 But a total lack of
poorly integrated, and so become irrele-
connection between incentives and perfor-
vant. Logistical issues are beyond an indi-
mance allows excellent teachers working in
vidual school’s control—with decisions
centralized and bureaucratic. Compensa- adverse circumstances and those who never
tion structures tend to be tied to seniority show up to be paid the same amount. This
and level of education or training, not to undermines the morale of good teachers
demonstrated mastery of skills. And and drives them out of the profession.
although pay, or other extrinsic motivation, But motivation is affected by more than
is not the only motivator for education pro- money, as a study of teachers in three types of
fessionals, the typical structure of working schools in Merida, República Bolivariana de
conditions and pay undermines even the Venezuela (nonprofits, state, and national),
intrinsic motivation of providers. shows. Catholic Fe y Alegria schools—which
cater to low-income families—emphasize
Employment relationship school autonomy and teacher input in deci-
and structure of compensation sionmaking. Even though pay is roughly the
same as in state and national schools, teacher
There is no single best approach to com-
satisfaction—and student performance—are
pensation, capacity building, and classroom
much higher (table 7.3).
autonomy. Indeed, one of the major bene-
Enhancing teachers’ capabilities is clearly
fits of greater autonomy is that it allows
fundamental to good-quality schooling, but
more experimentation and more flexibility
experience with teacher training is frequently
in implementation and replication. With
disappointing, mainly because of too little
school autonomy, organizations can try dif-
transfer from training to classroom practice.
ferent compensation schemes, training
Teachers need training that lets them do their
methods, and modes of parent-teacher
job better. But autonomy, motivation, and
interaction and can evaluate them relative
assessments of providers (based on outputs
to output and outcome objectives. If the
and outcomes) are needed for training to
public sector can specify what it wants from
improve outcomes.
a school—a clear compact—it can leave
teacher compensation to school manage-
ment and let the best system win. Client power
Teacher pay can be too low (where infla- Client power is a weak force in public
tion has eroded real salaries to the point school systems. Channeled into narrow
where teachers resort to alternative sources interests, it has little impact. In nearly all
15_WDR_Ch07.qxd 8/14/03 9:26 AM Page 125

Basic education services 125

Table 7.3 Autonomy and outcome in Merida, República Bolivariana de Venezuela, in the mid-1990s
School type Cost per student and Salaries as share of Teacher satisfactiona Student performance Retention rate, grades
per student hour operating expenses (average of math, 1–6 (percent)
(bolivar) (percent) reading, writing,
percent)

State (escuelas
integrales) 190, 24 95 3.75 40 51
National 160, 32 99 3.57 39 42
Private (Fe y Alegria) 155, 31 88 4.02 53 100

a. Rating of 5 indicates complete agreement with the statement, “I’m satisfied with my work.”
Source: Navarro and de la Cruz (1998).

countries parents can choose schools for • The emergence of community-run


their children, within limits. But choice has schools in El Salvador showed that they
little or no overall impact on school quality performed as well in test scores and in stu-
because there is typically no effect either on dent dropout rates as schools operated by
schools that lose children or on those that the ministry of education, which catered
gain them. Even when parents abandon the to wealthier children (see the spotlight on
public system and pay for private schools— Educo).
as is happening in many countries—little • In Cambodia a donor-financed initiative
systemic pressure for change is created sought to improve schools by stimulat-
because government resources continue to ing greater community engagement in
flow into public schools. Direct parental schools and using direct budget transfers
participation in schools is also typically a to schools (box 7.6).
weak force since there is little about public
schools that parents can affect. Often the
• Evidence from Argentina supports the
idea that parental participation together
school head and teachers themselves have with school autonomy raises student per-
little or no autonomy to make changes. Par- formance.334
ent organizations are simply a means of
mobilizing additional resources for the • NGOs can help both through direct
engagement with communities and
school.
through creating and disseminating infor-
There are ways to change this, to use client
mation—as in the system of school infor-
power to improve outcomes. One is to
mation for communities in Nepal assisted
involve citizens directly in the assessment and
by Save the Children-UK.
operation of schools. Another is to use
demand-side subsidies to increase access for
poor people. A third is to make provider
resources depend on client choice—to have
money follow students. None is a panacea, BOX 7.6 School improvement in Cambodia
but each can be part of a strategy for school To improve school quality, the Education advise the government on how to improve
improvement. Quality Improvement Project in Cambodia its education policies.The animators are
uses a participatory approach and perfor- supported by a network of technical assis-
mance-based resource management. Oper- tants at the local level, who provide peda-
Direct participation: community ating in three provinces, the project covers gogical and organizational support.
involvement in schools 23 percent of the primary school The project has stimulated lively
population. Local school communities iden- dialogue at the school, cluster, and adminis-
Since students, and indirectly their parents,
tify their needs and make proposals for trative levels on how to improve schools. It
interact daily with the education system, they change and investment. Funds are delivered has also set in motion a process of change
have valuable information about provider directly to school clusters by the Ministry of in the administration of schooling and in
performance that tends to be ignored in Education. teaching and learning practices. As a result,
Change management is supported by unprecedented responsibility has been
purely bureaucratic systems. Several success- district-based “animators,” who draw gen- devolved to school and local administrators.
ful experiences with giving parents a formal eral lessons from the experience with the
role in school governance have heightened school’s quality improvement grants to Source: World Bank (2002c).

interest in this model:


15_WDR_Ch07.qxd 8/14/03 9:26 AM Page 126

126 WORLD DEVELOPMENT REPORT 2004

Informal and community schools exist in teaching methods and will encourage teach-
many settings, most often when parents take ing to the test when there are gatekeeper
matters into their own hands and arrange examinations. Ensuring that the poorest are
for teaching outside the formal system. This not excluded from this process is essen-
is often supported by NGOs and religious tial—and difficult. Experience with school-
organizations. A recent review of initiatives based control in South Africa suggests a key
in Ethiopia points to the potential of this role for training parent groups: without the
support for expanding access to schooling training the more advantaged populations
(box 7.7). The big question is how to link benefited while poorer and less powerful
these efforts to the formal system so that groups lost out.
informal schools are not a dead end. Direct participation in schools raises the
Greater parental involvement in school difficult issue of user fees and their relation-
management has its risks. Parents need ship to community engagement. Some
access to relevant information and the argue that as long as locally collected fees
power to effect change. Their focus should are retained by the school, fees are a good
be on performance, not on micromanaging thing, for two reasons. First, empirical
the classroom, where teachers should have studies suggest that centrally controlled
professional autonomy. It is fairly straight- resources are almost universally devoted
forward for parents to assess whether the largely to payroll, while resources collected
instructor is present and not abusive to stu- at the school level raise school quality by
dents. But high-quality teaching cannot be much more than equivalent resources from
reduced to scripted actions. Parents often higher levels.335 A study in Mali showed that
have a very conservative perspective on paying fees left parents better off (on aver-
age) because the value of increased school
quality was much larger than the fee
itself.336 Second, if communities are to feel
pride in their school and empowered by
BOX 7.7 Alternate routes to basic education in Ethiopia their participation, then parents should be
expected to make some contribution. Pay-
Ethiopia is a large country with a heteroge-
neous population. Education levels are low: • Involving community members in moni- ment may come in-kind, such as labor for
toring the attendance of teachers and construction of the school, rather than as
only 24 percent of children complete primary
students.
school.There are very few schools in poor direct fees for use.
and remote areas: only about 30 percent of • Targeting class sizes of about 35 But these potential benefits of greater
10-year-olds in rural areas have ever attended students.
school. But recent innovations sponsored by • Recruiting teachers and teaching assis- community engagement have to be weighed
NGOs show other ways of getting schools to tants from local areas and paying them against the apparently large negative effects
these children. less than professional teachers. on enrollments of even very low user fees in
Programs run by six NGOs reveal how
expanding school places is possible even in
• Spending more on textbooks, other poor countries and against the increases in
instructional materials, training, and
remote areas—at reasonable cost and with- supervision.
inequality from relying on fees (see box
out sacrificing quality.The NGO ActionAid 4.4). Some might argue the ideal is a com-
The results are promising. Children
proposed adapting school models used by
attending these schools continue on to promise of a fairly apportioned fee on com-
the Bangladesh Rural Advancement Commit-
higher grades. Moreover, learning does not munities to generate ownership but with
tee in Ethiopia, and since then several other
NGOs have sponsored similar programs.The
appear to have suffered.Test scores in the significant exemptions for poor households
second grade were about 20 percent higher
schools share several features: (subsidized from a central fund). Recent
than in government schools, and scores in
experience with such targeting (as in South
• Compressing four years of the official the fourth grade were only slightly lower,
even though the schools catered to Africa) suggests that it is difficult to make
curriculum into three years.
• Streamlining the curriculum to reduce children from poorer families. All this at a this work.337
repetitiveness and remove elements lower cost per student.
Issues remain, however—particularly
deemed irrelevant to local needs.
about scaling up these programs to reach
Demand-side transfers
• Using instructional routines that appeal more children, the more so since some ini- Many governments use scholarships or con-
to children, such as songs or teaching in
tial success was driven by a few energetic ditional transfers (households receive bene-
groups.
individuals.
• Scheduling classes on days and times fits if children are enrolled) to expand
approved by the community. Source: Ministry of Education Ethiopia (2000). enrollments. The Education, Health, and
Nutrition Program of Mexico (Progresa)
15_WDR_Ch07.qxd 8/14/03 9:27 AM Page 127

Basic education services 127

has drawn considerable attention because— ment support for these schools—as for
unusual for this type of effort—it was struc- Catholic schools in Argentina or Islamic
tured to allow rigorous impact evaluations. schools in Indonesia—or there is support to
The program has resulted in substantially parents who choose private schools—as in
higher transitions to secondary school.338 In Chile, the Czech Republic, the Netherlands,
Bangladesh a study found that conditional and New Zealand.
transfers of rice raised enrollments. The What kind of relationship should gov-
program was also cost-effective relative to ernments have with nonpublic providers?
other interventions, though the government One decision is whether to allow demand-
recently moved to monetize the benefits due side transfers or scholarships to be used in
to concerns about leakage.339 Indonesia nonpublic schools. Colombia used scholar-
introduced a large scholarship program in ship programs for private schools to expand
response to the economic crisis in 1997. The enrollments for poor students. The fact that
program helped maintain junior secondary participants were chosen randomly from a
school enrollments.340 pool of applicants allowed for rigorous
Conditional cash transfers have proved impact evaluation, which found significant
effective in expanding enrollments, but they positive impacts for scholarship recipi-
have shortcomings. They focus on enroll- ents.341 But even though the program was
ment without creating incentives for both targeted and apparently effective, it
improving quality. To the extent that was discontinued—for bureaucratic and
demand-side transfers use funds that would political reasons. A second decision involves
otherwise have been devoted to school more generalized support for nonpublic
improvement, there is the risk of expanding schools. In general, it is hard to say anything
quantity at the expense of quality. There about “choice” without provoking contro-
was widespread concern that school feeding versy, but here are four tries.
programs in India were “too successful” in
attracting students. Schools were flooded General subsidies to private schooling—
with underage children not ready for learn- neither disaster nor panacea. Although
ing, which put even more pressure on qual- there is a wide-ranging and still inconclusive
ity at the critical lower grades. empirical debate about the impact of gener-
alized choice, providing general subsidies to
Resources and client choice private schooling has never been a disas-
The Universal Declaration of Human Rights ter—or a panacea. The Netherlands has had
(Article 26) asserts that parents have a “right full school choice among public and
to choose the kind of education that shall be denominational providers since 1920, with-
given to their children.” Despite this apparent out terrible repercussions. Chile has had
endorsement of parental choice, there is little choice since 1980, and while there is some
consensus about its role. controversy about whether it has produced
In practice, there is a large amount of substantial gains in measured learning out-
choice. A substantial fraction of schooling is comes (Hsiao and Urquoila 2002), no one
carried out by a range of private providers: argues it has been a disaster. New Zealand has
for-profit schools, religious and denomina- had school choice since 1991, and in a recent
tional schools, NGO-operated schools, and assessment of 32 countries, came in third in
community-owned and -operated schools. math and sixth in reading and science liter-
In some countries the proportion of chil- acy. The Czech Republic and Sweden have
dren in private schooling is rising rapidly— had public financing of private schools since
even without public support. In Pakistan the 1990s.
the proportion of urban students in public So choice is neither an ivory tower
schools fell from 72 percent in 1991 to 60 notion that could never work in practice
percent in 1996 to 56 percent in 1998—with nor an ideological Trojan horse that would
most of the shift to private, nonreligious destroy public schooling. It is also not a
schools (religious schools accounted for universal remedy. The successful expansion
only 1 percent). Sometimes there is govern- of choice has nearly always been embedded
15_WDR_Ch07.qxd 8/14/03 9:27 AM Page 128

128 WORLD DEVELOPMENT REPORT 2004

in a more general program of school reform other elements as well. Choice as part of a
and improvement. package of reforms can have three benefits:
Parents who exercise choice perceive
themselves to be better off. But schooling • The introduction of choice forces an
unbundling of roles. To have effective
transmits beliefs and values, which implies
choice the government must be explicit
a distinction between meeting the collective
in its dual role of setting the rules for all
goals of citizens for publicly financed
providers and managing schools as the
schooling and satisfying the clients of
largest provider.
schooling. Parents acting as citizens may
want publicly supported schools to encour- • Deciding how to regulate private providers
age all children to be tolerant and respectful can force a discussion of the output and
of other people’s beliefs. Yet these same par- outcome goals of education that can
ents acting as clients may want their chil- improve accountability in all schools.
dren to receive instruction in the absolute • And choice often creates new acceptance of
correctness of a particular set of beliefs. A assessments for monitoring providers—
system that satisfies every individual par- which can be expanded to all schools.
ent’s demands as a client might fail to meet
the collective goals of citizens for publicly Designing choice around the politics. Pol-
supported schools. Doubts about choice icy decisions about choice are intrinsically
often arise from the impact of schooling on political. The United States prohibits public
socialization.342 But this argument cuts support to schools run by religious organi-
both ways: if socialization is chosen by an zations. Cordoba, Argentina, has actively
authoritarian government to repress indi- supported Catholic schools.343 Holland
vidual or group rights, choice is all the more explicitly supports both Catholic and
important. Protestant schools. Rather than being based
on the perceived relative effectiveness of the
Using taxes for private schools requires different schools, these policy choices seem
accountability. While parents should be to reflect differing public opinion at the
allowed to choose their child’s education time the decisions were taken—for exam-
and create their own accountability, using ple, historical concern about Catholic influ-
taxes for private schools requires public ence among the Protestant majority in the
accountability. For choice to be effective in United States, a predominantly Catholic
creating greater accountability, parents population in Cordoba, and a more even
need timely, relevant information. This will distribution of religions in Holland. Simi-
not necessarily emerge spontaneously larly, the suppression of Islamic schools in
because it depends on comparable assess- some countries and support for them in
ments across schools. Policymakers could others, or the decision to ban private
publicize that a specific school meets mini- schools in Pakistan and Nigeria in the
mum standards through easily visible infor- 1970s, has little to do with school effective-
mation tools, such as symbols prominently ness. The promotion of choice through
displayed in the school. A more sophisti- vouchers in the Czech Republic has been
cated approach could involve broadly dis- seen as a reaction to the use of schools for
seminated census-like information on out- political indoctrination.
puts and outcomes—perhaps normalized If school choice is a political given, an
by socioeconomic status. effective school system can be designed
around that constraint. If school choice is
politically precluded, an effective school sys-
Making choice part of a package of reforms.
tem can be designed around that as well.
The public sector always remains an impor-
tant provider, and choice complements
reforms to improve the public sector. Advo- Getting reform going
cates of school choice emphasize the poten- This chapter is about changing the relation-
tial beneficial effects of competition—for ships of accountability to produce better
which there is mixed evidence. But there are educational outcomes by creating the insti-
15_WDR_Ch07.qxd 8/14/03 9:28 AM Page 129

Basic education services 129

tutional conditions for the technically right strongly linked to economic performance.
things to happen. But how can institutions But it is much easier to mobilize technocratic
be changed? How do openings for reform policymaker support for certain types of
get created and exploited? Decentralization edcuation reform (narrow accountability)
can create opportunities. Reform champi- than others (pedagogical improvement).
ons can emerge from political, business,
professional, or parental interests. And Teachers and teachers’ unions. Effective
teacher groups can promote—or resist— teachers are the backbone of any educa-
change. tional system, but how can the power of
teachers be harnessed for educational
Decentralization improvement? Some believe that teachers
Decentralization can be driven by a desire have too little power, arguing that educa-
to move services closer to people. But suc- tional reforms ignore teachers. Followed
cess depends on how it affects relation- through, this view can lead to reforms that
ships of accountability. If decentralization ignore classroom and school-level realities,
just replaces the functions of the central further demoralizing teachers and under-
ministry with a slightly lower tier of gov- mining reforms. Others believe that teach-
ernment (a province or state), but every- ers, especially teachers’ unions, have too
thing else about the environment remains much power and focus exclusively on
the same—compact, management, and wages and working conditions (box 7.8).
client power—there is little reason to Both sides can marshal empirical evidence.
expect positive change. The assumption is Much of the debate stems from the joint
that decentralization works by enhancing function of teachers’ unions as profes-
citizens’ political voice in a way that results sional organizations, which exist to pro-
in improved services. But this could go mote efficacy, advance professional knowl-
either way on both theoretical and empiri- edge, and advocate views in public policy;
cal grounds. Decentralization is not magic. and as agents of collective bargaining,
It must reach the classroom. And it will which emphasize resources and working
work only to the extent that it creates conditions.
greater opportunities for school reform
(chapter 10).

Reform champions
BOX 7.8 Education reform and teachers’ unions in Latin
Getting education reform on the agenda is
no mean feat, and getting reform politically
America
supported and implemented is even more Reforms to promote greater parental Education of its intention to transfer teacher
difficult. While individual parents are pow- involvement, more school autonomy, more colleges to public universities set teachers
erful advocates for their children, that does emphasis on results, and changes in the and students at those colleges “rioting in the
training, selection, assignment, and streets, breaking windows, attacking police,
not necessarily translate into system compensation of teachers are politically throwing rocks, and setting cars on fire”
improvement. Educators and progressive explosive—particularly with teachers’ (images the government used to mobilize
forces among teachers often emerge as unions. A study of five attempts at educa- public opinion against the unions).
tion reforms that included many of these Teachers’unions wanted governments to
champions of education reform because
elements in Latin America in the 1990s address the issues of teachers’wages and
they are most acquainted with the prob- found that teachers’ unions opposed nearly working conditions and were concerned that
lems inside the classroom and school. But it all of them—emphatically and stridently. decentralization and school autonomy would
is much easier to mobilize educator sup- “Teacher’s unions in Mexico, Minas intrude on more familiar relationships and
Gerais, Brazil, Bolivia, Nicaragua, and Ecuador negotiations between a centralized school
port for certain types of education reforms
followed similar strategies in opposing edu- administration and a centralized union.
(system expansion, increased resources, cation reform. All used strikes to assert their Even when governments pushed
pedagogy improvement, technical curricu- power . . . against unwanted changes.The reforms through, conflicts with the unions
lar reform) than others (increased power to disrupt public life, to close down made implementation problematic, since
schools and ministries, to stop traffic in capi- successful reform requires teacher partici-
choice).344 tal cities, to appeal to public opinion—were pation.
Local or national politicians or tech- familiar actions to them.” In April 1999 the
nocrats can also be forces for education announcement by the Bolivian Ministry of Source: Grindle (forthcoming).
reform, particularly if they perceive it is
15_WDR_Ch07.qxd 8/14/03 9:28 AM Page 130

130 WORLD DEVELOPMENT REPORT 2004

In too many countries discussions can reinforce professional ethics and mutual
between the government and teachers’ accountability. They can be used to organize
unions are no different from discussions teacher input on technical issues of educa-
between a large company and its unions. The tional reform, such as assessment, classroom
relationship between policymakers and autonomy, student discipline, and teacher
teachers’ organizations needs to shift from a training. If unions refuse to take on that role,
pure bargaining game to a positive-sum preferring to concentrate on wages and work-
game. This is easier said than done. As profes- ing conditions, there are no firm guidelines
sional development bodies teachers’ unions for how reformers should cope with that.
16_pgs 131-132_Ch07Spot.qxd 8/14/03 3:28 PM Page 131

spotlight on Educo

Educación con Participación de la Comunidad en El Salvador


By contracting directly with communities, El Salvador dramatically increased the primary school enrollment of children
in poor and remote areas—without reducing the quality of learning.

E l Salvador was wracked by civil war


throughout the 1980s. Some 80,000
people died—in a total population
of roughly 5 million—and many more were
wounded and disabled. Income per capita
community schools, bearing the cost them-
selves and paying teachers when they could.
The government seized on this model of
community-based schooling as the basis for
a formal program that would be financially
percent of the new students enrolled in
Educo schools (figure 1). By 2001 there
were almost 260,000 students enrolled in
Educo primary schools, 41 percent of all
students enrolled in rural schools—and
fell almost 40 percent between 1978 and and administratively supported by the min- more than 100,000 children enrolled in
1983.345 In 1989 the conservative Republi- istry: Educación con Participación de la Educo preschools, 57 percent of all children
can Alliance Party won a majority in the Comunidad, or Educo, with the goal of in preschool.
national assembly, with Alfredo Cristiani as encouraging the establishment of preschools Even as enrollments increased rapidly,
president. Despite contentious negotia- and primary schools, or classrooms in exist- there is little evidence that learning quality
tions, a peace accord was signed in January ing schools. suffered. A survey of 30 Educo primary
1992.346 Begun in 1991, Educo targeted 78 of the schools and 101 traditional schools in 1996
The war had severely damaged the edu- country’s poorest rural municipalities (of found no significant differences in average
cation system. Communication between 221 urban and rural municipalities). By 1993 math and language test scores among third
the central ministry and schools broke the program was expanding to all rural graders in the two types of schools.352 A fol-
down, supervision collapsed, and many areas, including many areas formerly under low-up study in 1998 found that grade pro-
teachers, viewed by some as government opposition control. But not all of the “popu- motion and repetition were similar across
“agents” and by others as agents of social lar schools” established during the war were the two types of schools as well.353 As the
opposition, abandoned their posts. By 1988 incorporated into Educo. Some observers innovation matured, the institutional
more than a third of the country’s primary claimed there was selective inclusion based arrangements that it introduced took hold
schools had closed.347 And by the end of the on political favoritism; others saw not incor- and ensured rapid expansion of school
war some 1 million children were not in porating popular schools into a government places and enrollments of poor children,
school.348 program as a way of sustaining spontaneous seemingly without a substantial cost in
community-based education.350 quality.354
Establishing Educo— Each Educo school (or section within a
Education with the Participation traditional school) is operated by a Com- Parent visits to classrooms made much
of Communities munity Education Association (ACE)—an of the difference
The Ministry of Education quickly identi- elected committee made up primarily of That Educo schools served the poorest of El
fied expanding access to basic education students’ parents—that enters into a one- Salvador’s students, in the poorest areas,
and raising its quality as central goals— year renewable agreement with the min- makes these results all the more astonish-
both to rebuild national unity and to pro- istry. The agreement outlines rights, ing. How did they do it? Using retrospective
mote long-term economic development. responsibilities, and financial transfers. The data that allow controls for child, house-
Minister of Education Cecilia Gallardo de Ministry of Education oversees basic policy hold, teacher, and school characteristics—
Cano, a reform proponent from the “mod- and technical design. Using the money
ernizing” wing of the Republican Alliance directly transferred to them, ACEs select, Figure 1 Students enrolled in traditional rural
hire, monitor, and retain or dismiss teach- and in Educo primary classes
Party, was intent on lessening the distrust
between former combatants. ers. Teachers at Educo schools are hired on Thousands of students
one-year renewable contracts. Parents are 700
But skepticism was high. The Ministry
of Education was not trusted in many parts taught about school management and how 600

of the country and by organized groups to assist their children at home.351 500 Traditional
such as the National Association of Teach- 400
ers. Expansion of the traditional education Three-quarters of new enrollments 300
system was viewed suspiciously as a covert Educo succeeded in many respects. From a 200
means of reasserting national control and pilot phase of six ACEs in three depart- 100
building political support in opposition- ments, it scaled up nationally to all of the Educo
0
dominated areas.349 country’s departments by 1993. Rural pri- 1990 1992 1994 1996 1998 2000 2002
During the war many communities had mary enrollments increased from 476,000 Note: Figures for 2002 are estimates.
recruited local teachers and established in 1992 to 555,000 in 1995—with over 75 Source: El Salvador Ministry of Education.
16_pgs 131-132_Ch07Spot.qxd 8/14/03 3:28 PM Page 132

132 Spotlight on Educo

Figure 2 Educo promoted parent involvement, that teachers were less likely to be absent in Is the Educo model applicable
which boosted test scores Educo schools (averaging 1.2 days of elsewhere?
Parent visits to Increase in test scores absence a month rather than 1.4 days). Stu- Educo’s achievements might appear idio-
classrooms in previous associated with a visit dents in Educo schools were also absent less
month syncratic. The end of a bloody civil war that
(three fewer days a month) than students in had thrown the traditional education sys-
Number of visits Percent increase
6 5.7 6 5.7 traditional schools.356 In addition, Educo’s tem into chaos opened up a unique oppor-
more flexible compensation scheme tunity to change the way schools were man-
5 5
resulted in greater variability in teacher aged. Based in part on coping strategies
4 4 3.8 earnings, which suggests that parent associ- during the civil war, El Salvador had a his-
3 3 ations used compensation to motivate tory of community involvement in school
greater effort among teachers.357 Offering management. Indeed, the community asso-
2 1.6 2
or withholding future employment itself ciations appear to have worked better in
1 1 was an incentive, and one that ACEs used. places that had prior experience in commu-
0 0 Turnover among Educo teachers was high, nity organization.358 In addition, in the
Traditional Educo Math Language which suggests that job loss was not an idle aftermath of the war there was an unusually
schools schools threat. large pool of educated people without jobs
Source: Adapted from Jimenez and Sawada (1999).
(coinciding with the rapid expansion of
Converging with traditional university places fueled by opening higher
and statistically adjusting for the fact that schools education to the private sector).
unobserved abilities of children might sys- Educo’s administration has become These factors suggest that the Educo
tematically differ between the two types of embedded in the Ministry of Education, model might not be directly replicable in a
schools—researchers found that commu- and Educo has developed into a major different setting. But some lessons are gen-
nity involvement explains much of Educo’s schooling model in the country. Aspects of eral. First, with political will it is possible to
success. traditional and Educo schools have been change the relationships between the actors
Parents are more active in Educo converging. Traditional schools now have in basic education. Second, schools can be
schools. And their involvement affects more parent participation in school gover- transformed to work in ways that promote
learning (figure 2). Each classroom visit by nance and management, and are more enrollment, participation, and learning—
parents was associated with significantly autonomous with supporting block even for children from the poorest house-
higher math and language test scores financing. Similarly, the pay packages of holds. Third, getting parents to participate
regardless of the type of school. Parents teachers are more similar: Educo teachers effectively in managing schools can help
were more active informally as well: they receive the same salaries and benefits as overcome some of the potential pitfalls in
were more likely to meet with teachers or to teachers in traditional schools. Even so, a the provision of education services—espe-
assist teachers in monitoring attendance or key distinction remains: Educo teachers cially monitoring schooling in remote
maintaining school furniture.355 are hired (and potentially fired) by parent areas. Fourth, it is possible to scale up small
How did Educo and parent involvement committees while those in traditional innovations to have a significant impact on
affect test scores? At least part of the story is schools are not. national outcomes.

Вам также может понравиться