Вы находитесь на странице: 1из 29

642517

research-article2016
EPAXXX10.3102/0162373716642517Strunk et al.Innovation and a Return to the Status Quo

Educational Evaluation and Policy Analysis


September 2016, Vol. 38, No. 3, pp. 549577
DOI: 10.3102/0162373716642517
2016 AERA. http://eepa.aera.net

Innovation and a Return to the Status Quo: A Mixed-


Methods Study of School Reconstitution

Katharine O. Strunk
Julie A. Marsh
Ayesha K. Hashim
Susan Bush-Mecenas
University of Southern California

School reconstitution, a turnaround strategy that prescribes massive staffing turnover, is expected to
result in more committed and capable school staff and innovative practices. However, little evidence
supports this assumption. We use quasi-experimental designs to assess the impact of reconstitution
on student achievement and teacher mobility, finding that reconstitution affected teacher mobility
and improved student achievement in the first year of the reform, with continued but smaller impacts
in the out years. We draw on mutual learning theory to conduct an exploratory analysis of reform
implementation. We find that initial re-staffing and strategic planning may have promoted balance
between exploring new and exploiting existing knowledge. Over time, however, balanced, mutual
learning was not sustained.

Keywords: reconstitution, turnaround, accountability

Education leaders are increasingly relying on The federal government has included school
school reconstitution as a policy solution for turn- reconstitution in several of its recent reforms. For
ing around low-performing schools (Malen & instance, the No Child Left Behind Act (NCLB)
Rice, in press). This reform strategy targets the specifies school staff replacement as a potential
human capital stock of schools, replacing existing corrective action for schools that fail to meet aca-
teachers, administrators, and support staff with demic performance standards for 4 consecutive
more motivated, qualified, and collaborative years (Linn, 2005; Novak & Fuller, 2003).
employees. Typically, school reconstitution allows Similarly, the School Improvement Grants (SIG)
existing staff members to reapply for their jobs program features school reconstitution as the
and be rehired up until a certain threshold (usually turnaround model for school improvement
50% of the schools existing staff), or to otherwise (U.S. Department of Education, Office of
transfer to another position in the school district Elementary and Secondary Education, 2010).1
(Malen, Croninger, Muncey, & Redmond-Jones, Many states and school districts have also relied
2002; Malen & Rice, in press). In addition, recon- on school reconstitution to provide a fresh start
stituted schools may receive financial resources for low-performing schools, with reforms being
and technical assistance to facilitate shifts in orga- implemented in cities such as San Francisco,
nizational design and practice. Through these Chicago, Washington, D.C., Los Angeles,
efforts, reconstituted schools are expected to Colorado Springs, New York City, and
develop a more committed and capable school Philadelphia (Malen & Rice, in press).
staff, innovate school and classroom practices, Although informed by a deliberate theory of
and support rapid gains in student achievement. action, there is little empirical evidence to suggest
Strunk et al.

that school reconstitution actually transforms low- Next, we review the quantitative and qualitative
performing schools. Extant research has produced data and methods that we use in this study. We
mixed evidence on the impacts of school reconsti- follow this with a discussion of our results from
tution on student achievement (Dee, 2012; each of the three research questions and conclude
Hamilton, Heilig, & Pazy, 2014; Malen etal., with further discussion and implications for pol-
2002) and indicates that school reconstitution icy, practice, and future research.
might lead to negative and unintended conse-
quences for the human capital of schools (Malen Empirical and Conceptual Underpinnings:
etal., 2002; Malen & Rice, 2004; Rice & Grounding the Inquiry
Croninger, 2005; Rice & Malen, 2003). To pro-
vide further evidence to inform the use of reconsti- In this section, we outline the theory of action
tution in education reform, this article examines underpinning school reconstitution and summa-
the implementation and outcomes of three recon- rize the key findings of the school reconstitution
stituted schools in a large urban school district. literature regarding the effects of school reconsti-
Identified based on past performance, these low- tution on (a) staffing, focusing on teacher mobil-
performing schools engaged in a local school ity and quality; (b) school redesign and
redesign process, including reflection and detailed innovation; and (c) student achievement. We
instructional and operational planning, which cul- then review Marchs (1991, 1994) mutual learn-
minated with district-led reconstitution. In partic- ing model and describe how this theory of orga-
ular, we ask the following research questions: nizational learning can be applied to studies of
school reconstitution.
Research Question 1: How does school
reconstitution affect teacher mobility and The Theory of Action Behind School
the teacher workforce of low-performing Reconstitution
schools?
Research Question 2: How does school Although school districts have implemented
reconstitution affect student outcomes in school reconstitution under different labels (e.g.,
low-performing schools? turnaround, redesign, reengineering) and pro-
Research Question 3: What conditions gram designs, these reforms all share a common
and factors contributed to the observed theory of action. In particular, school reconstitu-
outcomes of reconstitution in our case tion efforts are grounded in research indicating
study schools? that teachers and administrators matter for stu-
dent achievement (e.g., Aaronson, Barrow, &
In answering these questions, our article con- Sander, 2007; Chetty, Friedman, & Rockoff,
tributes to the existing literature on school recon- 2014; Leithwood, Seashore, Anderson, &
stitution, as it is one of the first to use multiple Wahlstrom, 2004; Nye, Konstantopoulos, &
methods both to examine the implementation and Hedges, 2004; S. G. Rivkin, Hanushek, & Kain,
impacts of reconstitution on teacher and student 2005; Rockoff, 2004) and play a central role in
outcomes. In addition, we draw on mutual learn- reform implementation (e.g., Coburn & Stein,
ing theory (March, 1991, 1994) in our qualitative 2006; Spillane, Halverson, & Diamond, 2001).
analyses to posit a more nuanced understanding As such, efforts to improve school performance
of the reconstitution process, including giving must target the quality of teachers and school
attention to how and why changes in teacher and leadership. School reconstitution policies further
student outcomes may occur. assume that low-performing schools are dys-
The article proceeds as follows. In the next functional organizations, meaning that incre-
section, we outline the theory of action on which mental changes such as new teacher professional
school reconstitution is based, reviewing the development or evaluation procedures will not
empirical and conceptual literature that addresses improve organizational performance (Malen &
reconstitution and outlining Marchs (1991, Rice, in press, p. 5). Instead, low-performing
1994) mutual learning model, which we use as a schools must be reinvented anew by replacing
lens for the exploratory implementation analysis. existing school staff with highly qualified and

550
Figure 1. Theory of action of school reconstitution.

motivated educators, administrators, and support leaders provide school capacity reinforcements
personnel (Malen etal., 2002; Malen & Rice, in in the form of resources that are aligned to school
press). goals such as new personnel, technical assis-
Figure 1 outlines the sequence of events tance, and professional development (Malen &
expected to follow once low-performing schools Rice, 2004, in press; Rice & Malen, 2010). All
undergo re-staffing. First, new hires come with together, these incentives and capacity measures
more experience, higher qualifications, and greater are expected to promote organizational learning
levels of motivation than existing school staff, thus and changes to school practices (discussed later
improving the human capital stock of schools in this section), guaranteeing that school staff
(Malen etal., 2002; Malen & Rice, in press). Next, replacement eventually leads to improvements in
these reconfigured school staffs engage in school student achievement and school performance.
redesign (Malen etal., 2002), adopting innovative
and efficient school practices that improve student
A Review of the Empirical Literature on
achievement. Examples of such practices include
School Reconstitution
community-based governance, faculty collabora-
tion, tailor-made professional development, and There is little empirical evidence on the effi-
teacher-initiated, inquiry-based approaches to cacy of reconstitution reforms to improve student
instruction (Malen etal., 2002, p. 123). As part of and school outcomes. Although there is a larger
this school redesign process, new employees build and growing body of literature on the implemen-
an organizational culture of staff collaboration, tation and impacts of school turnarounda
learning, and adaptation (Malen etal., 2002; Rice broader reform movement that encompasses
& Croninger, 2005), which further enhances school multiple strategies for creating rapid, substantial,
capacity for innovation and improvement. and sustained improvements in school perfor-
Education leaders use two policy tools to mance (e.g., Herman etal., 2008; Knudson,
facilitate the above chain of events. First, educa- Shamburgh, & ODay, 2011)we focus on stud-
tion leaders rely on the threat of school reconsti- ies of school reconstitution efforts, which empha-
tution, often alongside other rewards and size the human capital component of school
sanctions in high-stakes school accountability change. In this section, we summarize the key
systems, to incentivize returning or newly hired findings of the school reconstitution literature in
staff to work more productively and make effi- terms of three key milestones as identified in our
cient use of school resources (Malen & Rice, in theory of action (Figure 1), namely, the effects of
press; Rice & Malen, 2010). Second, education reconstitution on (a) staffing (teacher mobility

551
Strunk et al.

and quality), (b) school redesign and innovation, inconclusive evidence about reconstituted
and (c) student achievement. We conclude with a schools capacity for innovation and improve-
review of Marchs (1991, 1994) mutual learning ment. Some studies suggest that school reconsti-
model, which we use as a lens in our qualitative tution enhances the cultural capital of schools
study, and a discussion of how this work contrib- when new staff come from similar ethnic back-
utes to the empirical and conceptual literature grounds as students (Malen etal., 2002; Rice &
base on school reconstitution. Croninger, 2005), as well as improves school
professional culture with new hires working lon-
Teacher Outcomes of School Reconstitu- ger hours and exerting more effort than previous
tion. Although the theory of action behind staff (Hamilton etal., 2014; Hansen, Kraetzer, &
school reconstitution hinges on school workforce Mukherjee, 1998; Rice & Malen, 2010). How-
improvements, research has shown that reconsti- ever, the extent to which these benefits translate
tuted schools experience negative human capital into innovative and effective school practices
consequences. A series of case studies of three that ultimately improve student achievement is
reconstituted schools in a single urban district unknown (Malen & Rice, in press; Rice & Malen,
find disruptive short-term outcomes including 2010). This knowledge gap is due, in part, to the
high rates of teacher exit, with first-year and non- fact that existing studies on school reconstitution
certified teachers often replacing more experi- have paid little attention to processes of organi-
enced and reputedly effective teachers (Malen zational learning that contribute to school
etal., 2002; Malen & Rice, 2004; Rice & Cron- improvement.
inger, 2005; Rice & Malen, 2003). These results The human capital advantages of school
suggest that newly hired teachers at reconstituted reconstitution can also be offset by the disrup-
schools might actually be less qualified and com- tive and punitive nature of this reform, which
mitted to school improvement than their prede- increases both the intensity of work and pres-
cessors (Malen & Rice, in press). Empirical sure for success (Malen etal., 2002; Malen &
studies of school reconstitution in Texas (Hamil- Rice, 2004, in press). These high-stress condi-
ton etal., 2014) and Chicago (de la Torre etal., tions can deplete the social capital of schools
2013; Hess, 2003), along with state and national (Malen & Rice, 2004, in press; Rice &
reports on SIG reforms (Center on Education Croninger, 2005), as well as focus staff efforts
Policy, 2012; U.S. Government Accountability on reestablishing organizational routines rather
Office, 2012), report similar findings. than implementing new and comprehensive
To explain these human capital challenges, school improvement strategies (Malen etal.,
education scholars suggest that challenging 2002; Malen & Rice, 2004, in press; Rice &
working conditions, the threat of future correc- Croninger, 2005). For example, due to frequent
tive action, and the stigma of working at a recon- staff and leader turnover, teachers at reconsti-
stituted school create strong incentives for tuted schools often find it difficult to develop
effective and veteran teachers to transfer to other collaborative relationships, coordinate efforts
district schools (Hamilton etal., 2014; Hess, for addressing student and school needs, or
2003; Malen etal., 2002; Rice & Malen, 2003). build institutional knowledge (Malen etal.,
In addition, school districts often face shortages 2002; Malen & Rice, 2004; Rice & Croninger,
of qualified and experienced teachers and school 2005; Rice & Malen, 2003). Observations and
administrators (Center on Education Policy, reports of classroom instruction further reflect
2012; de la Torre etal., 2013; Hamilton etal., superficial changes in teacher practices (Hess,
2014; LeFloch etal., 2014; Rice & Croninger, 2003; Malen etal., 2002), low-quality instruc-
2005), which reduces the quality of incoming tion (Rice & Malen, 2003), and the dilution of
staff assigned to reconstituted schools. academic standards (Hess, 2003). These chal-
lenges, in turn, can deplete teachers morale and
School Redesign and Innovation in Reconstitu- increase their sense of disillusionment, further
tion Reforms.Given the apparent negative diminishing schools human capital stock and
effects of school reconstitution on teacher qual- capacity for innovation and improvement
ity, it is not surprising that extant research finds (Malen etal., 2002; Malen & Rice, 2004).

552
Innovation and a Return to the Status Quo

Student Outcomes of School Reconstitution. Cur- innovate and grow as learning organizations as a
rently little is known about the impacts of school result of reconstitution efforts, and the extant
reconstitution on student and school perfor- work that exists suggests that school capacity,
mance. Most prior studies have focused on the culture, and climate suffer. Nonetheless, the
effects of school turnaround rather than on the long-term goal of reconstitutionimproved stu-
impacts of school reconstitution on student out- dent outcomesappears, from the literature, to
comes (de la Torre, etal., 2013; Strunk, Marsh, be achieved.
Hashim, Mecenas-Bush, & Weinstein, in press). As noted before, our article contributes to this
The few studies that have explicitly focused on literature by utilizing multiple methods to exam-
school reconstitution find either inconsistent ine the impacts of reconstitution on teacher and
trends in student achievement (Hamilton etal., student outcomes and explore the implementa-
2014; Malen etal., 2002) or short-term gains due tion of reconstitution in individual case sites. As
to reconstitution (Archibald & Odden, 2000; will be further described below, our qualitative
Dee, 2012; Hess, 2003; Mintrop & Trujillo, analyses draw on mutual learning theory (March,
2005; OBrien & Dervarics, 2013; Rojas, 1996). 1991, 1994) to posit a more nuanced understand-
Given the findings that paint a negative picture of ing of the reconstitution process, and how and
the effects of reconstitution on teacher quality why changes in teacher and student outcomes
and school capacity, the somewhat positive may occur.
results regarding performance outcomes are sur-
prising. However, as with the studies examining Using Mutual Organizational Learning to
the effects of reconstitution on teacher outcomes, Understand Reconstitution. Because the theory
most student achievement studies do not use of action on which reconstitution is based
methodologies that enable the assessment of assumes that treated schools will benefit from
causal impacts of school reconstitution and having a mix of new and existing staff, along
should be interpreted with caution. with the knowledge and skills that they possess,
One notable exception to this literature is a and enhanced school organizational learning, we
recent study by Dee (2012), which suggests that draw on Marchs (1991, 1994) mutual learning
school reconstitution can improve student out- theory to understand the nuances and intricacies
comes when coupled with complementary of how school reconstitution occurs and to
resources, such as additional funding, training, explore how and why reconstitution may influ-
and/or personnel to implement new programs. ence teacher and student outcomes.2 March pos-
Using a regression discontinuity design, Dee its that organizations allocate their limited
demonstrates that SIG-funded reforms in resources to exploring new possibilities and
California produce significant school perfor- exploiting old knowledge. Exploration describes
mance gains, with the school reconstitution the introduction of new knowledge into the orga-
model driving these achievement gains. Given nization, through both active (as in the case of
that the SIG program couples school reconstitu- purposeful search for new alternatives) and pas-
tion with sizable financial grants and local dis- sive processes (for example, the infusion of new
cretion for implementing school improvement ideas possessed by new employees). Thus, explo-
plans, Dees findings lend credence to the notion ration supports organizational adaptation through
that school reconstitution can improve school the discovery of more effective and efficient
performance when ample resources and local practices, but also involves risk as organizations
discretion are provided. must invest time and resources, in terms of finan-
As is clear from this review, not all of the cial, social, and human capital, to implement
tenets of the reconstitution theory of action hold unfamiliar practices that may or may not lead to
up to empirical examination. In particular, the improved performance. Exploitation, or drawing
descriptive work on the teacher outcomes of on existing knowledge within the organization, is
school reconstitution offers warnings of negative theorized to provide more certain and faster
consequences stemming from drastic reshaping returns. It is through exploitation that organiza-
of school-based labor forces. In addition, there tions may realize the returns on their investment
has been little work that examines how schools in new knowledge. Ultimately, organizations

553
Strunk et al.

establish balance between exploitation and and exploitation (March, 1991; Rivkin &
exploration to support ongoing learning and opti- Siggelkow, 2003) because both short-term pro-
mal performance. ductivity and long-term innovation are essential
Consider a simple case of mutual learning in a for organizational success and survival (Lavie &
school setting. A hypothetical high school oper- Rosenkopf, 2006, p. 800). These researchers
ates on a traditional schedule of 1-hour class acknowledge, however, the challenges (e.g., com-
periods. The school has long operated in this petency traps) that organizations face in pursuing
way, so teachers have developed 1-hour lessons two contradictory activities concurrently and
and students are comfortable with this structure. finding an appropriate balance (Levinthal &
The school, however, has seen declining student March, 1993; March, 1991). As a result, many
achievement in recent years, perhaps related to organizations seek to attain a balance between the
environmental changes such as decreased fund- two, such that they are exploring enough to be
ing and a shift to Common Corealigned curricu- able to adapt to changing conditions, but not so
lum. This school is used to exploiting its much that they either exhaust their resources dur-
knowledge about how to function with the cur- ing search or change course so frequently that the
rent schedule. Achievement may be declining organization never achieves competence or strong
due to environmental factors, but it does so returns on its investment.
steadily and predictably. We can interpret this Mutual learning theory asserts that individual,
school as having a high rate of environmental organizational, and environmental factors influ-
change and a low rate of exploration, which ence to what extent and how organizations
together reduce its efficiency and effectiveness. engage in exploitation (see Figure 2). The capac-
One day, a new teacher arrives at our hypo- ity and learning rate of individuals influence how
thetical high school. She finds it difficult to fit knowledge is found, managed, and used. Indeed,
her Common Corealigned lessons into the turnover of individuals itself promotes explora-
1-hour class periods, so she suggests to the tion as outside knowledge is introduced through
administration that they shift to a 2-hour block new hires (in contrast to the redundant knowl-
schedule. Agreeing with her idea, the administra- edge embodied by employees with longer tenure
tors invest time in convincing teachers that the in the organization). New employees, however,
2-hour block will improve student achievement are expected to learn the organizational code (or
and adjusting school operations. They also existing practices) over time, adapting to the
expend financial resources to provide profes- norms and practices of the organization and its
sional development to teachers about teaching in employees. The pace of learning among organi-
the extended-block format. Teachers, in turn, zational members, therefore, is another key influ-
invest time in redesigning their lesson plans. ence on mutual learning; if new employees
Students, too, will expend effort in modifying acclimate too quickly, the valuable integration of
their study routines. Considering these costs, it is outside knowledge is lessened.
possible that this new schedule will allow for Organizational characteristics may strongly
more in-depth instruction and enhanced student influence how individuals interact and the content
academic outcomes; however, it is also possible of their interactions (and thus, the extent to which
that the school may see null or negative growth exploitation and exploration occur). High aspira-
in student achievement. In this way, exploration tions, or shared performance goals of the organi-
involves a level of risk that exploitation does not, zation, are thought to promote exploration because
but also offers the potential of enhanced effec- the organization pursues multiple alternatives to
tiveness and efficiency. The return on investment reach high performance goals. Experimentation,
in the new schedule might only be realized once in turn, increases the organizations failure rate.
the new schedule has become routine (and is, at Over time, high failure rates may decrease the
that point, an exploited practice). organizations motivation to seek new alternatives
Clearly, solely focusing on either exploration and therefore lead organizations to rely on tried
or exploitation makes sense only in particular and true practices (i.e., exploitation).
environments. Researchers assert that organiza- Leadership style and structure is another orga-
tions need to strike a balance between exploration nizational dimension that influences mutual

554
Figure 2. Marchs (1991, 1994) mutual learning theory as a sensitizing concept for reconstitution.

learning (Pandey & Sharma, 2009; Vera & empirical research demonstrates the tendency of
Crossan, 2004). A leaders attitude toward cre- exploitation to crowd out exploration in organi-
ativity, emotional security, uncertainty tolerance, zations (Benner & Tushman, 2002; Lavie &
and sense of urgency influence the extent to Rosenkopf, 2006). Essentially, the theory of
which the organization and its members engage action of reconstitution implicitly communicates
in exploration or exploitation. In addition, the that low-performing schools are in need of an
frequency, content, and tenor of collaboration infusion of new knowledge and practices,
among an organizations members can support embodied in new staff and organizational pro-
experimentation and learning. Finally, environ- cesses, to supplement their existing knowledge
mental conditions may influence organizational and practices. The reconstitution process, as
learning. For example, organizations in turbulent shown in Figure 1, therefore depends on pro-
environments must establish more adaptive prac- cesses of mutual learning and improvement. As
tices and engage in more exploration than organi- such, we draw on Marchs framework for our
zations in relatively stable environments where exploratory analyses of the school reconstitution
existing knowledge and practices may prove to process in two schools.
be suitable for a longer period of time.
Marchs mutual learning model is well suited
Data and Methods
to the case of school reconstitution, as this strat-
egy assumes that dramatic staff turnover, matched Our study takes place in an urban school district3
with planning, professional development, and that reconstituted a small set of schools. In the
increased aspirations associated with account- 20102011 school year, the district identified three
ability, will promote exploration within organiza- low-performing schools in need of turnaround
tions that have over-exploited and are thought to based on student performance measures. During
be in need of new knowledge to attain balanced their reconstitution year, school leaders and outside
mutual learning. In fact, despite their best efforts partners engaged in designing and writing a school
at creating structures to balance these activities, improvement plan, subsequently approved by the

555
Strunk et al.

district. Reconstitution (re-staffing) occurred dur- K12 traditional schools and some charter
ing the spring and summer of 2011. These schools schools.5 Although our student panel extends till
received new administrators, interviewed existing 20132014, we only have achievement data up
teachers (retaining up to 50% of them), and until 20122013, as the district under study
selected new teachers to fill in remaining vacan- stopped administering assessments in 20132014
cies. During the first year of implementation as a when transitioning into the Common Core State
reconstituted school (in 20112012), the district Standards. As such, we can only estimate the
provided additional resources in the form of staff impacts of school reconstitution on student
professional development, after which this resource achievement during the first 2 full years of reform
was discontinued. We examine teacher mobility implementation (20112012 and 20122013). We
and school operations in the year of reconstitution limit our student sample to those students who
(20102011) and the following Implementation were enrolled in reconstituted schools or compa-
Years 1, 2, and 3 (20112012, 20122013, and rable sets of schools during these treatment years,
20132014), whereas our student outcomes analy- leaving us with approximately 500,000 student-
sis is limited to the reconstitution year and year observations or 270,215 individual student
Implementation Years 1 and 2 due to statewide observations.6 Finally, we combine our teacher-
changes to the standardized testing regime. and student-level panels with school demograph-
ics (e.g., percent of students who are
socioeconomically disadvantaged, underrepre-
Quantitative Data and Analytic Approach sented minorities, etc.) and enrollment data from
To estimate the impacts of school reconstitution 20032004 to 20132014.7
on teacher mobility and student achievement, we
rely on a rich panel of the districts teacher-, stu- Teacher Mobility: Focal Variables and Analytic
dent-, and school-level administrative data. Our Approach. To evaluate the impacts of school
teacher panel spans from 20072008 to 20142015 reconstitution on teacher mobility, we measure
and includes employee demographics (e.g., race/ three mutually exclusive mobility outcomes: (a) If
ethnicity, gender,), educational attainment, job teachers return to the same school, (b) if teachers
title, contract status (i.e., permanent, probationary, return to another school in the school district, and
etc.), school placement, and measures of teacher (c) if teachers leave the school district. In addition,
effectiveness (evaluation scores and value-added the school reconstitution literature suggests that,
measures [VAMs] of teachers contributions to stu- contrary to the outcomes hypothesized by the the-
dent achievement for teachers in tested grades and ory of action, treated schools are more likely to lose
subjects [see Appendix A in the online version of experienced and effective teachers and to hire inex-
the journal for detail about how we construct perienced and noncertified teachers. To observe
VAMs].) We restrict our sample to teachers who whether these trends manifest in our data, we exam-
have worked at a reconstituted or comparable ine the heterogeneous effects of school reconstitu-
school (defined later) at some point from 2007 tion based on the following teacher characteristics
2008 to 20142015. Our final analytic data set commonly used as indicators for teacher quality in
includes 11,715 teacher-year observations and research and practice: (a) experienced teachers
3,300 unique teacher observations of elementary, (with 10 or more years of experience),8 (b) teachers
secondary, or special education teachers at tradi- with low evaluation scores, which we define as
tional K-12 schools.4 those who receive a rating of needs improvement or
Our student panel spans from 20032004 to below standards on any area of a teachers annual
20132014 and includes information on student performance evaluation, and (c) teachers with high
demographics (e.g., race/ethnicity, gender, eligi- and low VAMs, which we define as teachers with
bility for free and reduced price lunch [FRL]), VAMs that are one standard deviation above or
school enrollment, and academic performance below the school districts mean.
(standardized test scores in ELA and math normed We utilize multinomial logistic regression
by grade and year). Within this panel, we limit models in a difference-in-differences framework
our sample to students who are enrolled in tested to run our mobility analysis. A difference-in-dif-
grade levels (i.e., second through 11th grades) at ferences approach allows us to separate the

556
Innovation and a Return to the Status Quo

impact of school reconstitution on teacher mobil- 3m, representing interactions between


ity from other hard-to-observe factors that might RECONSs and the year indicators, represents our
have simultaneously influenced teachers deci- main coefficients of interest. Interactions
sions to switch schools or leave the school dis- between RECONS s and the years leading up to
trict. This inference is made possible by reconstitution (t 2 and t 1, or 20082009 to
comparing the change in mean mobility rates of 20092010) show the log odds of teachers in
teachers at reconstituted schools with that of schools that would be reconstituted in the years
teachers in a comparable group of schools that before treatment switching schools or exiting the
were not affected by school reconstitution district, relative to teachers in comparison
(Bloom, 1999; Duflo, 2001; Shadish, Cook, & schools. Interactions between RECONS s and
Campbell, 2002). We define comparison schools the reconstitution year (t, 20102011) or the
as those that narrowly missed being selected for years post reconstitution (t + 1 through t + 3, or
treatment in the year that schools were reconsti- 20112012, 20122013, and 20132014) show
tuted.9 From this group of schools, we exclude the log odds of teachers in reconstituted schools
low-performing schools that went on to be iden- switching schools or exiting the district during
tified for other turnaround reforms by the school reconstitution and after schools have been recon-
district in the following year (i.e., the 20112012 stituted, again relative to teachers in comparison
school year; n = 23 comparison schools).10 schools. We then use these coefficients to esti-
We run the following multinomial regression mate the marginal effects of school reconstitu-
model: tion on teacher mobility in the reconstitution and
P ( m) posttreatment years relative to prior years
log ist = 0 m + 1m RECONS s (enabling our difference-in-differences estima-
Pist (0) tion). Finally, to estimate the heterogeneous
+ 2 mYEARt + 3m RECONS s effects described earlier, we re-run our models
YEARt + 4 m X ist with three-way interactions between our indica-
tor, year dummies, and teacher characteristics of
+ 5m S st + m ,
interest.12
where Pist(m) is the probability that a teacher
moves to a different school within the district or Student Achievement: Focal Variables and Analytic
leaves teaching in the district relative to staying in Approach. Our student outcomes of interest are
ELA and math scores, standardized by subject grade
the same school. We estimate the log odds of the
to have a mean of zero and a standard deviation of
P (m) one. To estimate the effects of school reconstitution
probability of these outcomes, log ist , as a
P ( 0 ) on student achievement, we use a Comparative
ist
function of our treatment indicator, RECONS s, Interrupted Time Series (CITS) estimation approach.
which equals one for reconstituted schools and The intuition behind this strategy is the same as our
zero for comparison schools. To estimate the year- difference-in-differences approach for teacher
by-year effects of school reconstitution, we interact mobility, except that we are now comparing changes
our treatment indicator with a vector of year dum- in both the mean and trend in student achievement
mies, YEARt . We also control for individual of reconstituted versus comparison schools.13 We
teacher characteristics, X ist , such as employment estimate the following model:
status (permanent vs. nonpermanent), job title (ele-
mentary, secondary, and special education teach- Yist = 0 + 1YEARt + 2 RECONSt
ers), highest degree of educational attainment + 3YEARS _ SINCE _ RECONSt
(bachelors, masters, or doctorate), years of expe- + 4 (Tis YEARt )
rience, underrepresented minority status, gender,
and school characteristics, S st , such as the natural + 5 (Tis RECONSt )
log of enrollment, and the percent of English lan- + 6 (Tis YEARS _ SINCE _ RECONSt )
guage learners, disabled, and socioeconomically
disadvantaged students. m is a random error with + 7 Z ist + 8 S st
mean zero and clustered at the teacher level.11 + i + ist ,

557
Strunk et al.

where Yist represents student is test scores in labeled as in Program Improvement Year 3 or
ELA and math, normed by gradesubjectyear in higher under NCLB classifications (i.e., those
School s in Year t. YEARt is a trend variable, consistently failing to make Adequate Yearly
which we center to begin at 1 in the 20032004 Progress (AYP) under NCLB; n = 247 PI3+
academic year, the first year for which we have schools). We find consistent results across these
student-level administrative data. RECONSt is a comparison groups, thus confirming the internal
dichotomous indicator variable equal to 1 in all validity of our main findings (see Appendix Table
years since schools were reconstituted and stu- B1 in the online version of the journal for results
dents would be exposed to the treatment condi- from these analyses).
tion (i.e., 20112012 and 20122013). Our final robustness check strictly concerns
YR _ SINCE _ RECONSt is defined as the num- our student-level models. We account for the fact
ber of years of reform implementation that have that the year in which schools were identified for
been completed or equivalently, YEARt 2012. reconstitution (i.e., 20102011) might have also
As such, this variable takes on a value of 1 in the affected student achievement (a sort of
second year of implementation (i.e., 20122013). Ashenfelters Dip). Because schools were
Z ist is a vector of time-varying student-level aware of the threat of reconstitution early in the
covariates including FRL eligibility, special edu- school year and engaged in planning activities in
cation status, an indicator of nonstructural stu- attempts to avoid reconstitution, these activities
dent mobility in each year,14 and lagged might have caused administrators, teachers, and
achievement scores. S st is a vector of the school- possibly students to behave differently and there-
level covariates described earlier.15 Ts is a time- fore may have affected student achievement. We
invariant dichotomous variable that measures the determine whether there is a reconstitution year
treatment imposed by school reconstitution. This effect by estimating the same CITS model as
variable equals 1 if a student is ever enrolled in a defined earlier, except that our indicators for
reconstituted school in the treatment years, and 0 RECONSt and YR _ SINCE _ RECONSt now
if the student is ever enrolled in a comparison switch from zero to one in 20102011 and 2011
school in these years. i is a student fixed-effect, 2012 respectively. We do not observe significant
and st is a random error with mean zero that is effects of reconstitution identification on student
clustered at the school level.16 Overall, this speci- achievement (see Appendix Table B2 in the online
fication allows for a school reconstitution effect version of the journal for results).
that can be reflected in both a level shift in the
outcome variable ( 5 ) as a well as a shift in the
Qualitative Data and Method
achievement trend ( 6 ), or alternatively an over-
all effect of 5 + 6 . To explore the implementation of school
reconstitution, we collected data on two of the
Robustness Checks for Teacher and Student three reconstituted schools included in the quan-
Results. Given the comparative nature of our titative analyses. This article draws on data from
difference-in-differences and CITS models, our a larger study that examined a broad set of
ability to make causal inferences relies on the schools engaged in reform. As such, we purpose-
identification of plausible comparison groups. To fully sampled two reconstituted schools, along
account for potential biases arising from our with a subset of other school reform efforts.
selection of comparison schools, we re-run our These two schools were located within the same
models with variations of our comparison group. urban region with similar staff and student demo-
First, we expand our control group to include graphics. The schools differed substantially,
comparison schools that were selected for turn- however, in their level of outside funding and
around reforms in the following school year, as support (e.g., grants and programs). Given the
these soon-to-be treated schools are arguably the limits of our sample, we consider this analysis
most similar to the reconstituted schools in our exploratory.17
analysis (n = 29 comparison schools). Next, we We visited the schools at least once annually
compare reconstituted schools to all low-per- for their first 3 years of reform implementation as
forming schools in the school districtthose reconstituted schools. During these visits, we

558
Innovation and a Return to the Status Quo

conducted semistructured interviews with princi- we utilized it as a sensitizing concept. Sensitizing


pals (n = 6), other school administrators (n = 5), concepts are commonly understood as useful
and teachers (n = 22). During their first year of tools for framing qualitative research (Bowen,
reconstitution, we also conducted focus groups 2006; Patton, 2002). Rather than provide pre-
with parents (n = 2 focus groups, 18 parents). scriptions of what to see, they merely suggest
The content of these interviews and focus groups directions along which to look (Blumer, 1954,
included aspects of school operations and con- p. 7). Applied to qualitative data, sensitizing con-
text, such as staffing procedures, motivation and cepts offer ways of seeing, organizing, and
morale, school culture and teacher collaboration, understanding experience (Charmaz, 2003, p.
instructional approach, professional develop- 259). In this case, we applied the mutual learning
ment, and management (see Appendix C in the model as a sensitizing framework, post hoc, as
online version of the journal). We also inter- part of our iterative analytic process.
viewed district leaders (n = 35) to inform our Specifically, we developed codes according
understanding of the policy and its implementa- to key ideas, including individual capacity, staff
tion generally.18 All interviews were transcribed turnover and staffing procedures, organizational
and coded, using axial and selective methods capacity, leadership, collaboration, and envi-
(Strauss & Corbin, 1990) in NVivo qualitative ronmental turbulence, to examine how Marchs
research software. concepts of knowledge exploration and exploi-
We utilized the constant comparative method tation played out in school reconstitution (see
(Glaser & Strauss, 1967) to guide our data col- Appendix C in the online version of the journal
lection and analysis. That is, after identifying for a full list of codes, the interview protocol
school turnaround as a phenomenon of interest, questions from which coded material derived,
we guided our interviews to examine key con- and example coded text). Coded data were ana-
cepts from the literature. Following each phase lyzed using matrix displays (Miles, Huberman,
of data collection, we coded and analyzed data & Saldaa, 2013) to allow us to uncover pat-
within and between cases, and within and across terns, trends, and conflicts in the data. Using
years. Specifically, we completed detailed, coded Marchs lens in this way, we were better able to
case reports, which served as our source for com- explore how and why reconstitution led to stu-
parison. These reports were organized around the dent and teacher outcomes identified by our
key areas of reconstitution implementation, such quantitative analyses.
as staffing procedures, leadership, levels of col- For example, we observed and measured the
laboration, professional culture, parent engage- construct of balance in our data by comparing
ment, professional development, curriculum and the amount and quality of evidence coded as
instruction, use of data, and student school cli- exploration (reported use of new practice) to the
mate. We utilized these formative analyses to amount and quality of evidence coded as exploi-
guide our future data collection. For example, we tation (reported use of existing practice or rever-
modified interview protocols from year to year sion to previous practice). Of course, balanced
and adjusted the time spent on particular con- mutual learning is not necessarily achieved when
cepts of school turnaround and reconstitution exploitation and exploration are utilized in equal
during interviews. proportion; it is context-dependent and, in the
We began our analysis by conducting cross- extant literature, is the point at which an organi-
case analysis using our case reports. That is, we zation demonstrates short-term productivity and
compared each thematic section across schools long-term competitiveness (Lavie & Rosenkopf,
and analyzed emergent patterns. Interestingly, 2006; March, 1991; J. W. Rivkin & Siggelkow,
our cases demonstrated parallel trends. To help 2003). To examine these patterns in our qualita-
understand these trends, we then experimented tive data, we estimated relative balance based on
with several relevant theoretical lenses to exam- three factors: (a) whether new practice was
ine our results. Marchs (1991, 1994) mutual reported, (b) whether new practice was attributed
learning model aligned with the policys underly- to the reconstitution (new staff, disruption,
ing theory of action and best captured the pat- change in motivation) and/or improvement plan,
terns in our cases in early secondary analyses, so and (c) how the balance ratio changed over time.

559
Strunk et al.

We do not present numerical ratios or frequen- for two of the three reconstituted schools in the
cies, as these formats do not adequately reflect treatment set, and because we match these
the nuances and intricacies of interview data. schools to similar comparison schools for our
Rather, we rely on thorough triangulation of data quantitative analyses, we believe that we have
to confirm and validate our findings. Therefore, addressed potential limitations of our research
we are limited to presenting a selection of our design to the best of our ability. Future research
data; the quotations presented in our discussion on reconstituted schools will help us to under-
below represent broader findings validated stand whether our results can be generalized to
through triangulation. Excerpts from interviews reconstitution in other contexts and with more
included throughout this article illustrate the typ- schools.
ical language used in and across sites. Although
the qualitative analysis is exploratory in nature, Findings
we believe that it is an important first step in
How Does School Reconstitution Affect Teacher
examining the inner workings of reconstitution
Mobility and the Teacher Workforce of Low-
and that it lays the groundwork for future research
Performing Schools?
in this area.
Teacher Mobility. Our teacher mobility analyses
reveal that, on average, teachers at reconstituted
Limitations
schools are significantly more likely than teach-
Both our quantitative and qualitative analyses ers in similar schools that are not reconstituted to
rely on a small sample of reconstituted schools, switch schools and leave the district. Given the
which limit both the internal and external valid- massive re-staffing that initiates the reconstitu-
ity of our findings. Although we account for this tion process, these mobility trends are, unsurpris-
small sample size in our student-level analyses ingly, particularly high in the year in which
by clustering our standard errors to the school schools are reconstituted and undergo re-staffing,
level, we realize that this adjustment still may be and either diminish (in the case of switching
insufficient for estimating correct standard errors schools) or escalate (in the case of leaving the
and making valid statistical inferences (Donald school district) over time. Tables 1 and 2 present
& Lang, 2007). As noted earlier, we do not clus- the results from these mobility analyses. The left
ter our standard errors in our teacher-level analy- column of Table 1 shows the results from our
ses because of how we define our treatment main difference-in-differences estimates com-
variables of interest. Although we cannot rule out paring the mobility rates of teachers in reconsti-
school-level factors that might affect the preci- tuted schools with those in comparison schools.
sion of our teacher-level results, school-clustered The middle and right columns of Table 1 and all
effects are less important in an exploratory study of Table 2 provide these results for specific sub-
of this size. Moreover, our teacher-level models groups of teachers including those with 10 or
still allow us to describe how trends in teacher more years of experience, low evaluation ratings,
mobility at reconstituted schools vary across and low and high VAMs. Each row in Tables 1
teachers with different attributes. and 2 is calculated as the marginal probability of
Although our small sample restricts the gener- a teacher in reconstituted schools switching
alizability of our findings from an empirical schools or leaving the school district in the recon-
standpoint, our results are still highly relevant stitution year or a posttreatment year, relative to
from a policy standpoint. This is because school teachers in comparison schools and relative to
reconstitution is a high-intensity intervention, the reconstitution or pretreatment year.
and as such, districts generally do not reconsti- The top left panel in Table 1 shows that teach-
tute large quantities of schools in any given year, ers at reconstituted schools, relative to teachers
or even over multiple years.19 In fact, undertak- in comparison schools, are 41.8% and 37.4%
ing three reconstitutions in a single year taxes the more likely to switch schools in the year when
resources of even large school districts. Given schools are identified for reconstitution (2010
that we utilize administrative data for all three 2011) than in the pretreatment years (20082009
reconstituted schools and qualitative case data and 20092010, respectively). This is to be

560
Table 1
Difference-in-Differences Estimates of the Marginal Probabilities of Teachers With Varying Characteristics at
Reconstituted Schools Switching Schools or Leaving the School District (Relative to Staying in Their Current
School) in Reconstitution and Posttreatment Years Relative to Prior Years

Overall 10+ years exp Low evaluation

Switch Leave Switch Leave Switch Leave


vs. stay vs. stay vs. stay vs. stay vs. stay vs. stay

Reconstitution year (t, 20102011)


20102011 to 20082009 (t 2) .418*** .070** .501*** .118** .681*** .146
(.035) (.026) (.045) (.035) (.126) (.097)
20102011 to 20092010 (t 1) .374*** .071* .487*** .105** .615*** .150
(.038) (.027) (.046) (.036) (.135) (.101)
Posttreatment year (t + 1, 20112012)
20112012 to 20082009 (t 2) .190*** .009 .346*** .026 .406*** .035
(.034) (.024) (.059) (.035) (.095) (.034)
20112012 to 20092010 (t 1) .146*** .010 .332*** .013 .340** .032
(.037) (.025) (.060) (.036) (.107) (.045)
20112012 to 20102011 (t) .228*** .061* .155* .092* .275 .181
(.047) (.030) (.069) (.044) (.148) (.098)
Posttreatment year (t + 2, 20122013)
20122013 to 20082009 (t 2) .067 .122** .080 .125 .259** .085
(.039) (.041) (.064) (.064) (.092) (.067)
20122013 to 20092010 (t 1) .023 .123** .066 .113 .193 .089
(.041) (.041) (.065) (.065) (.105) (.073)
20122013 to 20102011 (t) .351*** .052 .421*** .008 .422** .061
(.052) (.044) (.076) (.069) (.153) (.114)
20122013 to 20112012 (t + 1) .123* .113** .266** .099 .147 .121
(.051) (.044) (.084) (.069) (.125) (.068)
Posttreatment year (t + 3, 20132014)
20132014 to 20082009 (t 2) .068* .151** .017 .167* .057** .414*
(.031) (.046) (.030) (.067) (.021) (.172)
20132014 to 20092010 (t 1) .025 .152** .003 .154* .009 .418*
(.034) (.046) (.031) (.067) (.053) (.174)
20132014 to 20102011 (t) .350*** .081 .484*** .049 .624*** .268
(.044) (.049) (.050) (.072) (.124) (.195)
20132014 to 20112012 (t + 1) .121** .142** .329*** .141 .349*** .450**
(.044) (.048) (.063) (.072) (.093) (.172)
20132014 to 20122013 (t + 2) .001 .029 .062 .041 .202* .329
(.047) (.058) (.067) (.089) (.090) (.181)

No. of teachers 2,680 2,680 2,680 2,680 1,943 1,943


No. of schools 25 25 25 25 25 25

Note. Difference-in-differences estimates are calculated by subtracting the marginal probability to switch schools or leave the
school district for all teachers or teachers with certain attributes (e.g., 10 years or more of experience) at comparison schools
from that of reconstituted schools in each year. We then subtract these net differences in each year from one another to observe
how the mobility rates of teachers at reconstituted schools change over time. These marginal effects are calculated from models
that control for individual teacher characteristics (employment status, job title, educational attainment, years of experience, race/
ethnicity, gender) and school characteristics (natural log of enrollment, and percent of students who are English language learn-
ers, disabled, and socioeconomically disadvantaged students). VAM = value-added measures.

p < .10. *p < .05. **p < .01. ***p < .001.

561
Table 2
Difference-in-Differences Estimates of the Marginal Probabilities of Teachers With High or Low VAMs at
Reconstituted Schools Switching Schools or Leaving the School District (Relative to Staying in Their Current
School) in Reconstitution and Posttreatment Years Relative to Prior Years

Low VAMs High VAMs

Switch Leave Switch Leave


vs. stay vs. stay vs. stay vs. stay

Reconstitution year (t, 20102011)


20102011 to 20082009 (t 2) .484*** .024 .317* .070
(.090) (.054) (.136) (.092)
20102011 to 20092010 (t 1) .411*** .009 .324* .084
(.109) (.065) (.138) (.099)
Posttreatment year (t + 1, 20112012)
20112012 to 20082009 (t 2) .220* .026 .033 .125
(.109) (.029) (.087) (.122)
20112012 to 20092010 (t 1) .148 .058 .040 .139
(.125) (.049) (.091) (.128)
20112012 to 20102011 (t) .264 .050 .284 .055
(.142) (.050) (.154) (.134)
Posttreatment year (t + 2, 20122013)
20122013 to 20082009 (t 2) .066 .172 .275 .124
(.104) (.131) (.201) (.134)
20122013 to 20092010 (t 1) .007 .140 .283 .138
(.119) (.120) (.203) (.140)
20122013 to 20102011 (t) .418** .148 .042 .055
(.131) (.119) (.238) (.153)
20122013 to 20112012 (t + 1) .203 .198 .242 .001
(.314) (.112) (.219) (.180)

No. of teachers 1,077 1,077 1,077 1,077


No. of schools 25 25 25 25

Note. Low VAMs are defined as one standard deviation below the school districts mean, whereas high VAMs are defined as
one standard deviation above the school districts mean. Difference-in-differences estimates are calculated by subtracting the
marginal probability to switch schools or leave the school district for teachers with low or high VAMs scores at comparison
schools from that of similar teachers at reconstituted schools in each year. We then subtract these net differences in each year
from one another to observe how the mobility rates of teachers at reconstituted schools change over time. These marginal
effects are calculated from models that control for individual teacher characteristics (employment status, job title, educational
attainment, years of experience, race/ethnicity, gender) and school characteristics (natural log of enrollment, and percent of
students who are English language learners, disabled, and socioeconomically disadvantaged students). VAM = value-added
measures.

p < .10. *p < .05. **p < .01. ***p < .001.

expected, given that reconstituted schools do not district and be picked up by other schools with
rehire the majority of existing staff. In addition, vacancies. That teachers from reconstituted
teachers at reconstituted schools are approxi- schools are significantly more likely to leave
mately 7% more likely to leave the school district suggests that the reform may be driving them
in the year of reconstitution than in the pretreat- from the district.
ment years. This is less expected, given that the The remaining panels in the left column of
non-re-hired teachers from reconstituted schools Table 1 show that the relatively heavy rates of
are supposed to return to the teacher pool in the school switching from reconstituted schools fade

562
Innovation and a Return to the Status Quo

over time and eventually tail off completely. By However, over time, low-quality teachers in
the end of 20132014, teachers in reconstituted reconstituted schools become less likely to
schools are 35.0% less likely to switch schools switch to other district schools, but are more
than in the reconstitution year itself, 12.1% less likely to exit the district entirely (in the case of
likely to switch schools than in the year follow- observational evaluations only, where low rat-
ing reconstitution, and no more likely to switch ings indicate subpar quality).21 Veteran teachers
schools than in the year before reconstitution. are also less likely to switch to other schools,
However, these patterns are different for rates of whereas teachers with high VAMs are no more or
district exit. Specifically, teaches are less likely less likely to leave reconstituted schools. We
to exit the school district in the year after recon- interpret these results as suggesting that the ini-
stitution than they were in the year of reconstitu- tial gains in teacher quality at reconstituted
tion itself, but then again grow more likely to exit schools likely diminish over time.22
the district in the out years.
How Does School Reconstitution Affect Student
Teacher Quality. Although increases in teacher
Outcomes in Low-Performing Schools?
mobility may create harmful instability in schools
(Guin, 2004; Ronfeldt, Loeb, & Wyckoff, 2013), Given the patterns in teacher mobility
it might be beneficial depending on the quality of described above, we next focus on how school
teachers who are leaving and entering schools. reconstitution affected student achievement.
Our results suggest that reconstituted schools Table 3 presents results from our CITS models
may experience net-gains in teacher quality in comparing the ELA and math achievement of
the year of reconstitution, but less so in subse- students enrolled in reconstituted schools versus
quent years. The middle and right panels of Table comparison schools. Overall, we find that stu-
1 and all of Table 2 show that, first, reconstituted dents in reconstituted schools experience signifi-
schools experience initial gains from low-quality cant and sizable gains in ELA achievement in the
instructors transferring to other district schools first and second years of reconstitution, and posi-
during the re-staffing process. In particular, tive but insignificant effects for math.
teachers with low evaluation ratings at reconsti- Specifically, in the first year, students in reconsti-
tuted schools, relative to teachers with low evalu- tuted schools demonstrate 8.2% of a standard
ation ratings at comparison schools, are 62% and deviation unit increase in ELA achievement rela-
68% more likely to switch schools in the recon- tive to students in comparison schools. In the
stitution year than in the first and second years second year, students in reconstituted schools
pretreatment, respectively. Similarly, teachers continue to demonstrate significant gains in ELA
with low VAMs at reconstituted schools, relative achievement, albeit at a smaller magnitude (5.7%
to low VAM teachers in comparison schools, are of a standard deviation unit increase). Both of
49% and 41% more likely to switch schools in these effects are relatively large by education
the reconstitution year than in the first and sec- reform standards. The overall change in ELA
ond years pretreatment, respectively. At the same performance of students after 2 years of reconsti-
time, reconstituted schools also lose experienced tution is the sum of these individual year effects,
and high-quality instructors: Teachers with 10 or which amounts to 13.9% of a standard deviation
more years of experience at reconstituted schools, increase, as shown in the bottom row of Table 3.
relative to senior teachers at comparison schools, We find positive but insignificant effects of
are approximately 50% more likely to move to reconstitution on math achievement. In the first
other schools in the district,20 and teachers with year of reconstitution, students demonstrate
high VAMs, relative to high-VAM teachers in 1.6% of a standard deviation unit increase in
comparison schools, are 32% more likely to math achievement relative to students in com-
switch schools in the reconstitution year relative parison schools. In the second year, students in
to either pretreatment period. Notably, however, reconstituted schools demonstrate an additional
the propensity for high-VAM teachers to switch 2.8% of a standard deviation unit increase in
from reconstituted schools is still much lower math achievement. The overall change in math
than that observed for low-VAM teachers. achievement after 2 years of reconstitution is

563
Table 3
ELA and Math Achievement for Students Enrolled in Reconstituted Relative to Comparison Schools

ELA Math

YEARt .024*** .017


(.006) (.013)
RECONSt .057*** .044**
(.012) (.017)
YEARS_SINCE_RECONSt .086*** .017
(.018) (.031)
Tis YEARt .018*** .009
(.003) (.008)
Tis RECONSt .082** .016
(.025) (.052)
Tis YEARS_SINCE_RECONSt .057** .028
(.021) (.040)
FRL ELIGIBLE .009 .015*
(.006) (.007)
SPED ELIGIBLE .063*** .034**
(.009) (.011)
MOBILITY .009 .058***
(.009) (.016)
LAG ACHIEVEMENT SCORE .095*** .102***
(.007) (.009)
% SOCIOECONOMICALLY DISADV .084* .003
(.040) (.068)
% ENGLISH LANGUAGE LEARNER .221*** .114*
(.033) (.051)
LN(ENROLLMENT) .009 .001
(.009) (.015)
MIDDLE SCHOOL .012 .098*
(.022) (.040)
HIGH SCHOOL .109** .112
(.037) (.074)
CONSTANT .082 .003
(.061) (.127)
R2 adj. .762 .649

No. of students 28,620 27,984


No. of schools 26 26
F stat. 40.402 47.769
Total effect (as of 20112012) 0.139*** 0.044
(0.032) (0.055)

Note. Results are from our main CITS specification comparing the ELA achievement of students enrolled in reconstituted
schools versus comparison schools (not including comparison schools that went on to be restructured by the school district in
the following year). Our main variables of interest are Tis RECONSt, which represents the effect of school reconstitution on
achievement in the first year of the reform; Tis YEARS_SINCE_RECONSt, which represents the growth effect of school
reconstitution in the second year of the reform; and the total effect, which is the sum of the above first year and growth effects.
Standard errors are clustered to the school level. ELA = English Language Arts; FRL = free and reduced price lunch; CITS =
Comparative Interrupted Time Series.

p < .10. *p < .05. **p < .01. ***p < .001.

564
Innovation and a Return to the Status Quo

4.4% of a standard deviation unit increase, evidence below, organized by the level of influ-
although not significant at conventional levels.23 ence on organizational learning: individual, orga-
How and why did reconstitution result in nizational, and environmental (see Figure 2).
some achievement gains and increased teacher
mobility in the early years, but greater challenges Improvements in human capital (individ-
in strategic staffing during subsequent years? ual). Although our quantitative analyses showed
What trends in student achievement might we the immediate removal of lower quality teachers,
expect to see in future years? We turn to our case staff at both case schools questioned whether or
studies to explore these key issues. not the reform actually drove improvements in
the human capital at their schools.

What Conditions and Factors Contributed to the Teacher turnover


Observed Outcomes of Reconstitution in Our On one hand, respondents did not report see-
Case Study Schools? ing an overall shift in the quality of the staff that
was hired into the schools. One administrator
Our qualitative analyses provide important
noted, The ones [new teachers] weve gotten in
insights into the mechanisms of reconstitution
their [removed teachers] place are not any better
that is, what may have occurred within the
than they were . . . Respondents attributed this
schools that likely contributed to the observed
perceived failure to improve the quality of the
outcomes over time. Guided by mutual learning
workforce to the fact that the re-staffing process
theory, we uncover two distinct phases of change
itself was not especially rigorous at either case
within the schools: a shift toward balance
site. Due to time constraints, limited resources,
between exploration and exploitation in the first
and the restriction of staffing to district applicant
year (Phase I), followed by a gradual regres-
pools, principals reported experiencing chal-
sion back to exploitation and reliance on old
lenges finding high-quality applicants for open
practices in subsequent years (Phase II).
positions.
Individual capacity, school culture and leader-
On the other hand, case respondents noted
ship, and environmental conditions greatly
that the reconstitution process was valuable as a
affected these organizational shifts. Together,
way to remove teachers who were burnt out or
these findings illustrate the challenges of sustain-
not a good fit. In particular, reconstitution
ing organizational learning and change after the
along with planning allowed administrators to
initial re-staffing of a reconstituted school.
develop a staff willing to try new ideas. As one
principal shared,
Phase I: Gradual Shift Toward Balance. Consis-
tent with observed positive gains in student per- So, it really got us a chance [to get rid] of people who
formance and improvement in teacher quality in werent really on board with certain things. We got an
the early years, our case study data indicate that opportunity to give those people another place to be,
and gave them an opportunity to go somewhere else,
after the mandatory re-staffing and adoption of if they were interested in being somewhere else.
an improvement plan, reconstituted schools
appeared to approach balanced, mutual learning, Furthermore, one teacher reported that his prac-
which is predicted to achieve improved organiza- tice had dramatically shifted and he had engaged
tional performance. This balance resulted in part in experimentation in practice due to exposure
from an immediate removal of lower quality to new teachers:
teachers, but also from a more gradual process of
improving morale and building a culture of learn- The principal] did [his] best to hire very new, a very
ing and collaboration. The exodus of uncommit- young staff with the intention of having a student
ted (and perhaps lower quality) staff, a school first, lets change things mentality. There is very
improvement plan, committed and distributed little of the old guard left and very few teachers at my
level of seniority even hired in, which I think is the
leadership, and calm environmental conditions best thing in my educational practice, because right
played critical roles in this recalibration and now in the other meeting thats my team and two of
change within the schools. We examine the them are brand new teachers. I threw away all my old

565
Strunk et al.

lesson plans and we were really reinventing and part of the reconstitution effort), improvements
starting from scratch. I had the advantage of in professional development and opportunities
experience, but I am longer relying on, oh, I think
that lesson is going to go well, I have done that
for collaboration, and new behavioral support
before. I am not relying on that crutch. I am really systems for students.
experimenting with new practice. And they have the
enthusiasm that first year teachers bring to the table, I Strengthened instructional leadership
miss it, so it is fantastic. In both cases, new principal leaders were well
received by teachers in the first year of the reform
Improved teacher morale and culture and there was evidence of distributed leadership
Rehired teachers believed that the fact that among other administrators, coaches, and teach-
they were kept at the school spoke to the quality ers. One teacher noted the importance of active
of their existing practices, whereas new teachers leadership and, in particular, principal involve-
believed that the schools turnaround process ment in instruction:
was an opportunity to innovate. As one teacher
said, The one thing that I have really appreciated [is that
he] expects instruction to be delivered. And when
The idea that my team is meeting right now to plan thats not happening [he] takes the measures that a lot
next weeks lesson together so that all fits together, of administrator dont. I mean [he] has been very
its a big part of [the schools improvement] plan . . . proactive about that, which has taken up a lot of [his]
When the principal was hiring me, [he] says, Well time for other stuff, but I think that it needs to be done
this is the plan that weve got, and I understand its especially when you are starting fresh.
very similar to the sort of thing you are interested in.
I was like . . . [this is my] dream school. At both case sites, the principals significantly
increased the focus on data, and thus may have
In addition, and as expected, the substantial increased the aspirations (and likelihood of explo-
turnover in staff also positively disrupted exist- ration) of the organizations. Teachers reported that
ing relationships and dynamics within the school, the principal conveys targets and dates and data
enabling exploration. A teacher shared, goals consistently, and that data has become an
important piece of our instructional reflection.
We have a whole bunch of new people . . . [before Similarly, teachers noted that they have increased
reconstitution] our turnover rate was lownobody
left. So it was like now you got a whole new group of
the use of data, not just the frequency, but the dis-
people, whole new personality set, whole new value cussion . . . its not perfect yet, but we are starting
system. The whole dynamic was changed. to use it to inform our practices.

Previous networks were replaced with planned Improvement planning


collaboration, as stipulated in each schools In particular, the school improvement plans
improvement plan. In this first year of implemen- essentially functioned as both a search mechanism
tation, teachers who had been retained noted that (enabling exploration) and a source of organiza-
collaboration was substantially improved and tional memory (enabling exploitation). Embedded
more focused on instructional matters (rather in the plans were previously successful school
than previous collaboration targeting student practices along with new ideas and protocols for
behavioral problems). finding new practices (such as the provision of
outside consulting or professional development).
Mechanisms for innovation and heightened
aspirations (organizational).In addition to Professional development and
evidence of improvement in human capital at collaboration
the reconstituted sites, we found evidence that In addition to, and as a part of, the school
several aspects of the reform activities, occur- plan, professional development programs may
ring at the organizational level, may have led to have enabled improvements in instruction and
a balance between exploration and exploitation. morale. One teacher highlighted the planned pro-
These include strengthened instructional leader- fessional development, along with staffing turn-
ship, the schools improvement plan (written as over, as a source of exploration. He explained,

566
Innovation and a Return to the Status Quo

We have been seeing an improvement though with the might have previously been absent due to
PDs [professional developments] only because there suspensions.24
has been a bigger focus on teachers leading the PDs.
And when you have the best teachers in the school
Together, these purposeful changes to school
leading the PDs, thats when the teachersthe other leadership, school culture, and student behavior
teacherslike benefit a lot. So there is a push for best management communicated an overall increase
practices, yeah. It did exist before [reconstitution] in the expectations for the students, in terms of
because it [the school] did have phenomenal teachers their behavior and performance potential; the
but what also existed before were really bad teachers
that just like didnt really care . . . So I am really
staff, in terms of their capacity to better serve stu-
thankful actually for this bigthe big push this year dents and continually improve their practice; and
to make PDs be way better. And what helps is that it is the school, in its ability as an organization to
the new people that are hired in because they have a function more efficiently and effectively. As
fresh perspective and really we have a good variety of March (1994) suggests, increased aspirations of
teachers this year.
this kind may increase the likelihood of explora-
tion, as individuals and organizations seek out
At another case, a new teacher noted that the
new practices to reach their heightened goals.
principal had encouraged him to try out new
ideas regarding interdisciplinary planning and Insulation from policy change (environmen-
team teaching and allowed one grade level to tal). Environmental turbulence is commonly felt
experiment with an alternative collaboration in school settings, as a result of changing federal
arrangement. or state laws, state or district policies, or man-
dated district practices. As part of their reconsti-
New behavioral support systems tution process, however, our case schools were
Notably, both schools engaged in substan- effectively sheltered from changes in district
tially reworking their student behavior manage- policy during their first year. That is, because
ment plans, using positive (rather than punitive) the schools had undergone such a dramatic turn-
methods and incentives to motivate students. around effort, the district intentionally protected
Teachers at both schools reported a substantial them from other interventions and policies (e.g.,
shift in student behavior during the first year of release from participation in standardized pro-
implementation. In particular, one administrator fessional development). Furthermore, they were
observed, encouraged, in this first year, to implement the
[The school is] certainly moving in a good direction. instructional and operational strategies outlined
Like the actual behavior of the kids is 1,000 times during their school planning process. As a result,
better than it used to be. They used to . . . do whatever this first phase represented a period of relative
they wantedcrazy. Its not like that anymore. calm in the environment surrounding the school,
Theyre very compliant in class. I think the problem
which likely facilitated more optimal organiza-
that were having now is like you used to spend all
period telling them sit down, be quiet, dont get out tional learning.
of your seat, sit down, be quiet. Now theyre sitting In summary, multiple factors appeared to
down being quiet and you actually have to teach them enable the reconstituted schools in our case sam-
something and . . . some people are not used to that. ple to move toward balance between exploration
They might have taught for 30 minutes before [and]
and exploitation in the first year. As anticipated
they now have 50 minutes to teach because the kids
are sitting there. So thats where I think were by the theory of action behind school reconstitu-
struggling to really increase the level of instruction. tion (Figure 1), turnover in personnel alongside
various supports likely moved reconstituted
Although this administrator acknowledged the schools in a positive direction. In particular, addi-
need for further instructional improvement, tional professional development time, adherence
teachers across both schools agreed that student to school improvement plans, and increased col-
behavior improvements allowed them to engage laboration time facilitated investments in devel-
in more rigorous and engaging instructional oping teacher capacity, improvements in school
strategies. These changes were also accompanied operations, and school culture. These substantive
by a sharp decline in student suspensions, result- changes to practice are likely to have improved
ing in more instructional time for students who instruction and contributed to improvements in

567
Strunk et al.

student performance, in part by raising the aspira- In the second and third years post reconstitu-
tions of the individuals and the organization. tion, staff turnover, declining morale and collab-
oration, and increased environmental turbulence
Phase II: Balance in Peril.Although student began to chip away at the organizational learning
achievement continued to increase in the second and improvement experienced in the first year. In
year of reform (at least in ELA), our case data fact, much of the data collected from our case
suggest that changes in individual, organiza- sites suggest a shift away from exploration and a
tional, and environmental conditions began to return to the pre-reform status quo. In what fol-
threaten balanced mutual learning and the pros- lows, we examine how the second and third years
pects of sustaining this improvement in later of operations post reconstitution unfolded in
years of the reform. In their second year after ways that moved our case schools toward a reli-
reconstitution, staff at both schools expressed ance on exploitation.
that initial improvements in the school were not
fully sustained. As one administrator noted, Challenges sustaining improvements in
human capital (individual). Consistent with the
The big bang was this whole restructuring of the mobility analyses presented earlier, staff reported
school and the community also seeing that this is a high levels of teacher turnover in the second year
new [school] and were making changes, were as well as declining staff morale. For example,
making strides, were improving. I almost feel that
that sort of energy and that vision is just going away,
one teacher noted that
like its just kind of slowly melting away.
a lot of people are leaving this school year. I know a
lot of people are choosing to transfer to other school
Similarly, a teacher expressed concerns that the sites because it feels unbearable at times, and I dont
schools improvement plan, in particular the know what to tell you [is causing this].
more innovative aspects, was not being imple-
mented. In his words, Reflecting on how the reconstitution process
influenced staff capacity, one teacher reported
Honestly, I dont know that anybodys even mentioned little change in quality:
the plan in the last two yearsfunctionally, were
running the same way that we did prior to the plan.
I believe that mix of teachers didnt change quality
Theres been more of an emphasis on [collaboration]
much. I believe that even though they shook the bag
. . . but that was something that waswe were moving
up, you still pulled out some of the same quality of
towards anyway, prior to the [reconstitution] . . .
teachers; maybe not the same ones. You might end up
Really, nothing has changed.
with fewer of them, but ultimately you kind of get the
teachers that represent the district.
By Year 3, these concerns escalated to a marked
return to pre-reconstitution practices. Two teach- Administrators noted that efforts to improve the
ers described this Phase II shift in this way: quality of the teaching staff were constrained by
the cessation of staffing flexibilities. In particu-
I would say that . . . year one and two, we were
making some positive gains in a lot of the components: lar, administrators reported that they would have
instructionally, and the school culture was getting benefited from the ability to make changes to
better. Now we are six, seven years back again. We staffing and avoid district layoffs in the years fol-
are back in the position that got us into [reconstitution] lowing reconstitution. In essence, reconstituted
in the first place.
schools received staffing autonomy during their
reconstitution year that was not sustained during
We were making gains. We were improving, and we
decided for adult agendas to change some things . . . subsequent years. According to Marchs mutual
Now were back in a mess again . . . We were on an learning framework, we speculate that this struc-
upward trajectory for the last two years and had some tural rigidity (in contrast to flexible staffing in
things in place. I think that last year [year 2], while, the year of reconstitution itself) may have con-
not everything was working perfectly, it was working strained the ability of schools to maintain the
a whole lot better. Now people are, like I said, were
going to see a huge turnover in staff. Were going to optimal mix (and turnover rate) of teachers.
see people that are just going to wash their hands of In Year 3, the schools continued to experience
this and walk away. challenges in recruiting, hiring, and retaining

568
Innovation and a Return to the Status Quo

highly qualified teachers, which may have under- Its a very territorial fiefdom-oriented, step-on-my-
cut the exploratory intent of staffing turnover. curb mentality that still exists here. Once I eradicate
that and we play as a team, this whole school [will]
Much like the teacher cited above who compared blossom . . . In the vacuum of leadership, they have
staffing with randomly picking teachers from a relied on each other, which is what humans do, but
bag, an administrator cited similar staffing what Im saying now is there is a chain of command.
challenges:
This shift toward top-down, hierarchical leader-
We are really, really doing all that we can to hire ship was evident in both schools, albeit in a
really qualified staff, and it is the most difficult thing
milder form at the other case site.
to do . . . because sometimes you have to go through
those pools, which is a very, very difficult thing to do. In addition, administrators felt pressure to
To get from out from underneath the poor hiring make adjustments and implement competing
practices or poor teachers that have been retained and practices, whereas teachers reported feeling that
put into the pool has been extremely difficult. there was inadequate flexibility to deviate from
their school improvement plan. One teacher said,
A decline in staff morale, as well as in the you want to have the flexibility to throw some
schools professional culture and school climate, of those things out if theyre not working instead
was also evident in both case sites. In Year 3, of being rigid. I feel like some of that rigidness
teacher morale continued to decline, particularly isit impedes our growth. Together, the shifts
in one school where teachers reported feeling toward relying on past practice (e.g., going back
unheard, fed up with student behavior, and to collaboration focused on student behavior)
burned out, and described the school culture as and responding to turbulence with rigidity indi-
toxic and dysfunctional. cate that both case schools engaged in more
Finally, teachers reported strained relation- exploitation than exploration during their second
ships with administrators. In Year 2, one teacher year. A teacher at one case school said,
attributed the slow pace of change during the
schools second year to these relationship issues: The big bang was this whole restructuring of the
Its been a frustrating year. The first semester, school, and the community also seeing that this is a
the staff morale was bottomed out. I think the new [school], and were making changes, were
making strides, were improving. I almost feel that
second semester the admin figured out that the that sort of energy and that vision is just going away,
staff was not happy with them, so they just quit like its just kind of slowly melting away.
pushing anything. Concurrently, administrators
reported experiencing mounting stress due to Although flexibility from school improvement
increased environmental turbulence, saying plans was necessary to meet changing school
every year I feel like Im starting over. I can conditions, it appears that such changes may have
never be on solid ground and move up. diluted the balance of new and existing knowl-
Specifically, the schools were subject to several edge within the school, and lowered the height-
new state and district mandated policy changes ened aspirations codified in the plan. By Year 3,
and practices. Faced with competing demands, teachers noted that the school was running the
one principal noted, I think that there needs to same way we did prior to the reconstitution and
be some clarity around: What are we following? that the improvement plan was probably on top
Whats important? If youre writing one [school of a filing cabinet somewhere in somebodys
improvement] plan, why do you have five or office. Moreover, the planned collaborative
three or two of them? What are we doing? structures intended to promote teacher capacity
building and organizational learning became less
Aspirations lowered due to organizational frequent and more focused on student behavior in
challenges (organizational). Despite early posi- both schools. Teachers shared that its been chal-
tive perceptions of leadership, staff attributed lenging to build that sense of collaboration that
declining staff morale to leadership changes at Ive known and theres no team building activi-
both schools. One new principal reported plans ties. By Year 3, decreased motivation and chal-
to shake up the power dynamics he encountered lenges to professional culture also appeared to
on arrival: influence the schools aspirations. As these two

569
Strunk et al.

teachers state, expectations for school improve- In the words of one teacher, I think whats
ment, staff efficacy, and student potential declined happened is because theres been so much change
substantially. . . . I think people have just basically hunkered
Other data from our cases echoed these find- down to do what they do best. That is, do what
ings, with staff noting that efforts to implement they know best. At the other case, the principal
positive behavior management with students had reported that the staff tune out when faced with
been diluted over time and student behavior changing policies, plans, and practices.
improvements had been lost. As a result, teachers Interestingly, one teacher demonstrated an aware-
reported that student behavior had returned to ness of the reconstitution context and highlighted
pre-reconstitution norms. As one teacher shared, the crucial importance of continued exploration
Its out of control. For students, there is a big to respond to environmental turbulence:
push for positive behavior support, restorative
justice, and that has translated to zero discipline. Yes, were teachers and education, but its still a
business, so this is no different than a company
The kids pretty much know that there are no con-
reorganizing itself, in going through a reorganization
sequences. Of course, increasing exploration period, because these is new leadership or the climate
activities in an organization necessarily involves in the particular market has changed. Therefore, you
the risk of failure. Unfortunately, in these cases, cant use those same value systems anymore, so I
challenges during implementation may have con- think thats what a lot of it was going on. The value
systems in the community, and society as a whole, has
tributed to lowered staff morale and reliance on
changed, whereas you cant do the same. The way you
knowledge exploitation. taught in 73, you cant do that in 2013 now, so you
have to adapt and change. Yeah, but you have to
Turbulence in the external policy context (envi- remain flexible. I think the thing about [school
ronmental).By Year 3, the case schools were turnaround] is that, with anything, if youonce you
stop becoming flexible, your lifes going to be
feeling increasing pressures stemming from the dis- stressed, because theres always going to be
tricts larger policy and budgetary environment. For something.
example, during their third year, the new Common
Core curriculum was rolled out. Staff struggled to Although student achievement data are not avail-
reconcile these new demands with their planned able to verify the effects of this shift back to
practices. In addition, constrained budget and staff- exploitation, these qualitative findings raise
ing stemming from the recession and associated questions about the prospects of sustaining the
decreases in funding affected schools abilities to learning gains schools experienced in the early
implement elements of their school improvement years of reconstitution.
plans. For instance, externally developed profes-
sional development and interdisciplinary teacher
Discussion and Implications
collaboration ceased as funding dried up.
In summary, the combined changes in exter- Overall, our analysis of student outcomes sug-
nal policy environment, diminishing aspirations, gests that school reconstitution in the early years
decline in morale, shift in power dynamics, and led to positive gains in student achievement in
challenges with staff turnover appeared to ELA that then diminished over time, and to no sig-
dampen teachers willingness and ability to nificant effects on student achievement in math.
experiment and innovate. In fact, staff in both These results go hand-in-hand with some of the
case sites reported returning to exploiting past insights from our analysis of teacher outcomes,
knowledge and using past practice. In particular, where we find that reconstituted schools experi-
both schools reverted to previous student behav- enced significant turnover in their teaching staff
ior management practices, which, according to and a seeming tendency to retain lower quality
many teachers, resulted in major student behav- teachers after the first year of reform implementa-
ior challenges in their third year of implementa- tion. Taken together, our student and teacher find-
tion. Similarly, staff collaboration time and ings suggest that the challenges of retaining a
professional development once again focused on stable and effective teaching workforce might
student behavior and professional culture, rather have slowed down or prevented school progress in
than instructional improvement. improving student achievement.

570
Innovation and a Return to the Status Quo

Our qualitative analyses also align with our planning process, it may be helpful to provide
findings related to diminishing gains in student flexibility for schools from new, potentially con-
outcomes. In particular, our case data suggest that flicting policies and programs. Furthermore,
new practices and conditions that might have ini- holding schools accountable for achievable,
tially driven gains in student achievementsuch short-term goals may help schools to maintain
as positive student behavior management, high aspirations without experiencing reform
improved professional development, shared, open fatigue. For example, benchmarking schools on
leadership, and increased, meaningful collabora- their implementation of experimental programs
tiondiminished in later years of implementa- and expected performance indicators may help to
tion at reconstituted schools. The cases also show sustain high organizational aspirations while
how reconstituted schools struggled to maintain avoiding reform fatigue resulting from a sense of
mutual learning balance in turbulent environ- failure or lack of monitoring or follow-through.
ments. Although these schools demonstrated ded- Another policy change to consider is allowing
ication to experimentation and use of prior best reconstituted schools ongoing staffing flexibility
practices during their first year, the willingness of as well as time to engage in rigorous staffing pro-
staff to try new practices decreased substantially cedures. In particular, principals reported want-
in their second and third years of implementation. ing to have the time and resources to select
School staff attributed the decline in intermediate employees based on observation of classroom
school outcomes to funding constraints, staffing practices in addition to interviews. After pur-
challenges, and leadership problems. These on- posefully staffing for heterogeneity, our case
the-ground developments help us understand our principals experienced ongoing district layoffs,
mixed student outcomes, as we see that schools which shifted their staff make-up. When faced
initially increased their exploration practices to with district reductions in force, these principals
reach better balance and subsequently reverted to requested the ability to select which staff to
exploiting prior knowledge more and more. remove or receive an exemption from layoffs.
Such flexibility may help schools sustain balance
between exploration and exploitation over time.
Policy and Practice
One-time infusions of new ideas or people may
Given the increasing popularity of reconstitu- not be sufficient to lead to longer term, balanced
tion methods of school turnaround and the lim- organizational learning.
ited empirical literature available on this topic, In addition, it may be beneficial to identify
this article contributes new evidence that can and invest in new ways to retain high-quality
inform broader policy efforts focused on system- teachers to ensure that initial gains in organiza-
atic school reform. Our findings indicate that, tional learning (and student performance) are
although reconstitution has the potential to facili- sustained. For example, one case principal advo-
tate organizational learning and improve student cated the use of performance-based compensa-
outcomes, schools may experience challenges tion to entice high-quality teachers to stay at the
sustaining these improvements within turbulent school (particularly given the challenging con-
environments. text of working in an urban school). Furthermore,
Case reports indicate that issues related to our case reports indicate that shifts to a more top-
resources, aspirations (and reform fatigue), down, hierarchical approach to leadership may
autonomy, leadership, and stability may have have alienated teachers who value experimenta-
hindered school progress. For this reason, it may tion and autonomy. One policy solution may be
be beneficial for district administrators to explore to create shared leadership structures and path-
ways to insulate reconstituted schools from ways for teachers to take on additional responsi-
excessive turbulence, at least in early years. In bility and leadership. Finally, purposeful search
particular, substantial shifts in funding, which or experimentation structures may facilitate
resulted in loss of staffing, were particularly dis- ongoing exploration while also enhancing
ruptive to teacher collaboration and trust in lead- teacher engagement. For example, new practices
ership. Following the purposeful search and developed by teachers may be purposefully
codification processes embedded in the school implemented in pilot lab classrooms to

571
Strunk et al.

encourage ongoing search as well as codification use techniques to recruit, develop, and retain tal-
of practice into the organizational memory. ent in hard-to-staff environments.
We also suggest that mutual learning, as a sen-
Research and Theory sitizing framework, may be useful in the study of
other school improvement efforts. In particular,
Our methods provide a promising model for studies of charter schools may shed additional
future study of related phenomenon. The use of light on how to promote and sustain optimal orga-
purposive, rigorous qualitative case studies com- nizational learning. Studies with larger samples
bined with our causal analyses uniquely high- of reconstituted or turnaround schools would also
lights the successes and challenges of allow researchers to systematically test the appli-
reconstitution, as well as the nuances involved in cability of the mutual learning model and the pre-
ongoing implementation. Expanding on our liminary case study findings presented herein.
work, we suggest that future research examine Additional work examining the relationship
how reconstitution plays out over the longer between indicators of teacher quality, heterogene-
term. Do learning gains persist? Do schools sus- ity of teachers within schools, and school organi-
tain the learning and changes in practice over zational factors may help to inform researchers
time? Given the exploratory nature of our quali- and practitioners about the efficacy and scalabil-
tative research, future studies could investigate ity of reconstitution as a reform strategy.
the individual, organizational, and environmen-
tal factors we identified as important in larger Declaration of Conflicting Interests
samples and with other methods to further our The author(s) declared no potential conflicts of inter-
understanding of the mechanisms operating est with respect to the research, authorship, and/or
within reconstitution, and how improvements in publication of this article.
performance might be sustained. This knowledge
could further help policymakers decide whether Funding
and when reconstitution is an appropriate fit for
The authors received external funding for this research
their context. from the U.S. Department of Education Investing in
With the promise of enhanced measures of Innovation Grant #U396C100336.
teacher qualityfrom rigorous observations of
teacher practice, stakeholder surveys, and VAMs Notes
of effectivenessthe study of teacher mobility
patterns is an important area for future research. 1. The turnaround model in School Improvement
Grants (SIG) requires school districts to replace school
When examining interventions that rely on staff-
leadership, rehire no more than 50% of existing teach-
ing changes, it is crucial to understand the char- ing staff, and grant new school principals sufficient
acteristics of teachers working in reconstituted or flexibility to implement a comprehensive plan for
otherwise re-staffed schools. Our results from school improvement (U.S. Department of Education,
this study suggest that reconstitution is, at least Office of Elementary and Secondary Education, 2010).
initially, successful in removing low-quality 2. Although we acknowledge that there are several
teachers from schools. However, it appears that other theories that may contribute to and/or explain
the success with which low-quality teachers are how reconstitution played out (e.g., goal theory, social
exited from reconstituted schools diminishes capital, threat rigidity), the mutual learning model fit
over time. This may well be tied to the aforemen- best with the data we collected and generated the great-
tioned removal of staffing flexibilities, as well as est insight in this case (we discuss this further below).
3. To ensure anonymity, we have not named the
the ways in which professional environments
district, schools, or individuals participating in this
conducive to autonomy and collaboration dimin- research, and we use only male gender pronouns in
ished over the post-reconstitution period. reference to respondents throughout this article.
However, we also find that very experienced 4. We exclude magnet schools and other alterna-
teachers are more likely initially to exit and to tive schools of choice, and do not have data for teach-
continue exiting reconstituted schools. These ers at independent charter schools. The time frame of
findings point to important areas for future our analysis coincides with a time period when many
research on the ways that re-staffed schools can school districts were downsizing their workforce.

572
Innovation and a Return to the Status Quo

As such, we exclude from our sample any teacher teachers with 3 to 5 years of experience only repre-
who received a Reduction in Force (RIF) notice and sented 7.73% of all non-RIFed teachers and teach-
might therefore have switched schools or left the ers with 3 to 7 years of experience only represented
school district for reasons unrelated to school recon- 16.16% of all non-RIFed teachers. We understand that
stitution. We also account for another school district it might be more conceptually interesting to examine
policy of breaking up large school campuses (mainly the effects of school reconstitution on teachers with
high schools) into smaller academies or building new fewer years of experience and also run our models
campuses to relieve overcrowding in neighborhood identifying teachers with 5 or more years of experi-
schools. Because teachers movement between these ence and with 7 or more years of experience as teach-
schools is a function of larger structural changes as ers. Our results remain largely consistent across these
opposed to individual decision making in response to different specifications, demonstrating that experi-
school reconstitution, we exclude these teachers from enced teachers are more likely to switch schools in
our analysis. the year of reconstitution, relative to similarly experi-
5. Charter schools are not required to provide enced teachers in comparison schools, and less likely
student-level data to this school district. However, we to switch schools in the years following reconstitution
gained permission to use data from all reconstituted (results available on request). Although the most inter-
schools (charter and traditional) and for 50% of the esting comparison might be that between teachers with
charter schools in our control groups. Our sample 10 or more years of experience (as a possible indicator
excludes the 25% of enrolled students who are in non- of high quality) and those with 3 or fewer, we cannot
tested grades (K, 1, and 12) and students in charter run the model looking at inexperienced teachers (those
schools for which we could not obtain data. Our sam- with 3 or fewer years of experience) because there
ple excludes 1.2% of students who do not demonstrate were very few inexperienced teachers in the district.
normal grade progression, as we are concerned that 9. Specifically, schools were selected for treatment
these students grades are miscoded. Normal grade based on meeting a set of indicators for low-school
progression is defined as students who progress one performance, including (a) school program improve-
grade in each year, who are ever retained in the same ment status (needs improvement under No Child Left
grade, or who skip ahead a grade once in 2 consecu- Behind [NCLB]), (b) an aggregate score consisting
tive years. of a weighted combination of various achievement
6. This sample includes students at reconstituted measures, generated by the state department of educa-
schools and multiple comparison groups that we use tion, (c) percentage of students scoring proficient or
for our main analyses and specification checks. advanced on the state achievement tests, (d) meeting
7. As public school data are currently not avail- Adequate Yearly Progress (AYP) targets, and (e) drop-
able for 20132014, we imputed our school-level data out rates. Comparison schools missed any one of these
from the districts student-level data. To ensure that indicators of low performance regardless of whether
our imputed data are accurate, we correlate our aggre- or not that data point was missing.
gated student-level measures with available school- 10. Our comparison group is slightly smaller (n
level data in prior years, finding that most variables are = 22) for our teacher-level models as we do not have
highly correlated at .80 or higher. The one exception teacher data from independent charter schools.
is our aggregated measure for the percent of students 11. Because we measure the effects of school
who are socioeconomically disadvantaged, which is reconstitution on teachers decisions to leave recon-
only correlated at .62. To ensure that this measure does stituted schools, our treatment indicator, RECONSt ,
not bias our analyses that draw on 20132014 data, restricts our sample to only teachers who work at treat-
we re-run our main models by substituting the percent ment and comparison schools. Given the small number
of students who are socioeconomically disadvantaged of treatment and comparison schools in our study (n =
with the percent of students who are underrepresented 25), we do not cluster our teacher mobility results to
minorities, confirming that our main results do not the school level as our models, which include in excess
change. of 30 covariates, would not have sufficient degrees of
8. We focus on teachers with 10 or more years of freedom for estimation.
experience because, given budgetary reductions and 12. We only report marginal effects in this article.
associated RIFs, they represent the largest group of Results as log odds ratios are available on request.
teachers in the district with the greatest potential influ- Most of the teacher covariates used to estimate the
ence on teacher turnover in schools. In 20102011, heterogeneous effects of school reconstitution are
the year of school reconstitution in our study, teachers specified as teacher-level controls in our main model,
with 10 or more years of experience represented 72% except for our indicator for teachers who are rated
of all non-RIFed teachers in the district. In contrast, as needing improvement or below standards on their

573
Strunk et al.

performance evaluation and our value-added measure component of our study in terms of school staff, stu-
(VAM) estimates. We exclude these covariates because dent demographics, and academic performance.
only a subset of teachers participate in a performance 18. Again, these data were drawn from a larger
evaluation in each year and because we only have data study, which accounts for the high number of dis-
on teachers VAMs up until the 20122013 school trict administrator interviews relative to case-level
year, reducing our sample size substantially. We run interviews.
our main models with these covariates included to 19. Extant studies of reconstitution efforts sug-
make sure that our main results do not change (results gest that most districts cannot or do not reconstitute
available on request). large quantities of schools in a given year. Chicago
13. A Comparative Interrupted Time Series (CITS) reconstituted seven schools in 1997, after which they
approach is appropriate for our student-level analy- reconstituted 16 schools between 2006 and 2010,
ses given our ability to observe the trend in student averaging three to four schools per year (de la Torre
achievement for several years (from 20032004 etal., 2013). In the studies of reconstituted schools in a
to 20102011) prior to school reconstitution. This large Maryland school district, Rice and Malen (2010)
approach is not appropriate for our teacher-level anal- note that the district reconstituted six schools (also see
yses for which we have less sufficient pretreatment Malen, Croninger, Muncey, & Redmond-Jones, 2002).
data. Nevertheless, we observe consistent results when 20. We observe similar findings when we define
we re-run our teacher-level analyses in a CITS frame- experienced teachers as those with 5 or more or 7 or
work and account for the fact that we are comparing more years of experience (results available on request).
teacher mobility in the postintervention period with a 21. Although we see that teachers with low evalu-
pretreatment trend as opposed to teacher mobility in ation ratings are significantly more likely to leave the
individual years of the pretreatment period (results school district in 20132014, these results might be
available from authors on request). driven by a concurrent change in the school districts
14. We control for student mobility because endog- teacher performance evaluation system. As such, we
enous sorting of students between reconstituted and view these results as suggestive.
comparison schools could threaten the internal valid- 22. We note that we cannot make definitive claims
ity of our design. We also confirm that school recon- about the net teacher quality gain in reconstituted
stitution did not have a significant impact on student schools relative to comparison schools given that we
mobility during the time frame of our analysis (results do not examine the quality of entrants into reconsti-
available on request). Although we might account for tuted versus comparison schools. As such, again, we
endogenous student mobility by comparing students view these results as suggestive.
who are consistently enrolled in reconstituted schools 23. It is unclear why we observe gains in ELA but
in the pre- and postintervention period with those con- not math. However, anecdotal evidence from discus-
sistently enrolled in comparison schools, this would sions with district and school personnel indicate that
substantially narrow our sample to those students in treated schools focused primarily on ELA instruction.
elementary or secondary grade levels who continue at 24. Simultaneous with this reform, the district
their original school for the entire time frame of analy- enacted an effort to decrease suspension rates across
sis, reducing the generalizability of our results. all schools.
15. We do not control for the percent of disabled
students in our CITS model, as we do not have these
public data in earlier years of our analysis. In our References
student-level models, we also control for school-level Aaronson, D., Barrow, L., & Sander, W. (2007).
(e.g., elementary, middle, high), which we specify as a Teachers and student achievement in the Chicago
teacher covariate in our teacher-level models. public high schools. Journal of Labor Economics,
16. The treatment indicator in our student-level 25, 95135.
models, Ts, is more conservative in that we are mea- Archibald, S., & Odden, A. (2000). A case study
suring the effects of school reconstitution on student of resource reallocation to implement a whole
achievement in all postintervention years (i.e., 2011 school reform model and boost student achieve-
2012 and 20122013) even if students were only ment: Parnell Elementary School. Retrieved
enrolled in a treated school for just one of these years. from http://cpre.wceruw.org/papers/parnell%20
Because students can be enrolled in other schools in sf%203-00.pdf
the pre- and postintervention period, we cluster our Benner, M. J., & Tushman, M. (2002). Process man-
error term to the school level. agement and technological innovation: A longitu-
17. Case schools are also comparable with the dinal study of the photography and paint industries.
reconstituted school excluded from the qualitative Administrative Science Quarterly, 47, 676707.

574
Innovation and a Return to the Status Quo

Bloom, H. S. (1999). Estimating program impacts on Hansen, J. B., Kraetzer, A. V., & Mukherjee, P. (1998,
student achievement using short interrupted time August). Adams-Hunt achievement initiative final
series. New York, NY: Manpower Demonstration evaluation report (Rep. prepared for the Colorado
Research Corporation. Springs School District 11, Colorado Springs, CO).
Blumer, H. (1954). What is wrong with social theory? Paper presented at American Educational Research
American Sociological Review, 18, 310. Association Annual Conference, Montreal,
Bowen, G. (2008). Grounded theory and sensitizing Quebec, Canada.
concepts. International Journal of Qualitative Herman, R., Dawson, P., Dee, T., Greene, J., Maynard,
Methods, 5(3), 1223. R., Redding, S., & Darwin, M. (2008). Turning
Center on Education Policy. (2012). Opportunities and around chronically low-performing schools: IES
obstacles: Implementing stimulus-funded School practice guide. Washington, DC: National Center
Improvement Grants in Maryland, Michigan, and for Education Evaluation and Regional Assistance.
Idaho. Washington, DC: Author. Hess, G. A. (2003). ReconstitutionThree years later
Charmaz, K. (2003). Grounded theory: Objectivist monitoring the effect of sanctions on Chicago
and constructivist methods. In N. K. Denzin & Y. high schools. Education and Urban Society, 35,
S. Lincoln (Eds.), Strategies for qualitative inquiry 300327.
(2nd ed., pp. 249291). Thousand Oaks, CA: SAGE. Knudson, J., Shamburgh, L., & ODay, J. A. (2011).
Chetty, R., Friedman, J. N., & Rockoff, J. E. (2014). Beyond the school: Exploring a systematic
Measuring the impacts of teachers II: Teacher approach to school turnaround. San Mateo:
value-added and student outcomes in adulthood. California Collaborative on District Reform.
The American Economic Review, 104, 26332679. Lavie, D., & Rosenkopf, L. (2006). Balancing explo-
Coburn, C. E., & Stein, M. K. (2006). Communities of ration and exploitation in alliance formation.
practice theory and the role of teacher professional Academy of Management Journal, 49, 797818.
community in policy implementation. In M. Honig LeFloch, K. C., Birman, B., ODay, J., Hurlburt,
(Ed.), New directions in education policy imple- S., Mercado-Garcia, D., Goff, R., . . . Wei, T. E.
mentation: Confronting complexity (pp. 2546). (2014). Case studies of schools receiving School
Albany, NY: State University of New York Press. Improvement Grants: Findings after the first year
Dee, T. (2012, April). School turnarounds: Evidence of implementation. Washington, DC: National
from the 2009 stimulus (Working Paper No. Center for Education Evaluation and Regional
17990). Retrieved from http://www.nber.org/ Assistance, Institute of Education Sciences.
papers/w17990 Leithwood, K., Seashore, L. K., Anderson, S., &
de la Torre, M., Allensworth, E., Jagesic, S., Sebastian, Wahlstrom, K. (2004). How leadership influ-
J., Salmonowicz, M., Meyers, C., & Gerdeman, ences student learning. New York, NY: Wallace
R. D. (2013). Turning around low-performing Foundation.
schools in Chicago. Chicago, IL: The University Levinthal, D. A., & March, J. G. (1993). The myo-
of Chicagos Consortium on Chicago School pia of learning. Strategic Management Journal,
Research. 14(Suppl. 2), 95112.
Donald, S. G., & Lang, K. (2007). Inference with Linn, R. L. (2005). Conflicting demands of No Child
difference-in-differences and other panel data. The Left Behind and state systems: Mixed messages
Review of Economics and Statistics, 89, 221233. about school performance. Education Policy
Duflo, E. (2001). Schooling and labor market con- Analysis Archives, 13(33). Retrieved from http://
sequences of school construction in Indonesia: files.eric.ed.gov.libproxy1.usc.edu/fulltext/
Evidence from an unusual policy experiment. The EJ846738.pdf
American Economic Review, 91, 795813. Malen, B., Croninger, R., Muncey, D., & Redmond-
Glaser, B. G., & Strauss, A. L. (1967). The discov- Jones, D. (2002). Reconstituting schools: Testing
ery of grounded theory: Strategies for qualitative the theory of action. Educational Evaluation and
research. Chicago, IL: Aldine Transaction. Policy Analysis, 24, 113132.
Guin, K. (2004). Chronic teacher turnover in urban Malen, B., & Rice, J. K. (2004). A framework for
elementary schools. Education Policy Analysis assessing the impact of education reforms on
Archives, 12(42). Retrieved from http://epaa.asu school capacity: Insights from studies of high-
.edu/ojs/article/view/197 stakes accountability initiatives. Educational
Hamilton, M. P., Heilig, J. V., & Pazy, B. L. (2014). Policy, 18, 631660.
A nostrum of school reform? Turning around Malen, B., & Rice, J. K. (in press). School reconstitu-
reconstituted urban Texas high schools. Urban tion as a turnaround strategy: An analysis of the
Education, 49, 182215. evidence. In J. Mathis & T. Trujillo (Eds.), Test-

575
Strunk et al.

based education reforms: Lessons from a failed Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005).
agenda. Greenwich, CT: Information Age. Teachers, schools, and academic achievement.
March, J. G. (1991). Exploration and exploitation in Econometrica, 73, 417458.
organizational learning. Organization Science, 2, Rockoff, J. E. (2004). The Impact of Individual
7187. Teachers on Student Achievement: Evidence from
March, J. G. (1994). Primer on decision making: panel data. The American Economic Review, 94,
How decisions happen. New York, NY: Simon & 247252.
Schuster. Rojas, W. (1996). Reconstitution, reculturing, and
Miles, M. B., Huberman, A. M., & Saldaa, J. (2013). reform: Adding options for urban education
Qualitative data analysis: A methods sourcebook. (Unpublished doctoral dissertation). Teachers
Thousand Oaks, CA: SAGE. College, Columbia University, New York, NY.
Mintrop, H., & Trujillo, T. (2005). Corrective action in Ronfeldt, M., Loeb, S., & Wyckoff, J. H. (2013).
low performing schools: Lessons for NCLB imple- How teacher turnover harms student achieve-
mentation from first-generation accountability sys- ment. American Educational Research Journal,
tems. Education Policy Analysis Archives, 13(48). 50, 436.
Retrieved from http://epaa.asu.edu.libproxy1.usc Shadish, W. R., Cook, T. D., & Campbell, D. T.
.edu/ojs/index.php/epaa/article/view/153 (2002). Experimental and quasi-experimental
Novak, J. R., & Fuller, B. (2003). Penalizing diverse designs for generalized causal inference. Boston,
schools? Similar test scores, but different stu- MA: Houghton-Mifflin.
dents, bring federal sanctions (Policy Brief 03-4). Spillane, J. P., Halverson, R., & Diamond, J. B.
Berkley, CA: Policy Analysis of California (2001). Investigating school leadership practice: A
Education. distributed perspective. Educational Researcher,
Nye, B., Konstantopoulos, S., & Hedges, L. V. 30(3), 2328.
(2004). How large are teacher effects? Educational Strauss, A., & Corbin, J. M. (1990). Basics of quali-
Evaluation and Policy Analysis, 26, 237257. tative research: Grounded theory procedures and
OBrien, E. M., & Dervarics, C. J. (2013). Which way techniques. Thousand Oaks, CA: SAGE.
up? What research says about school turnaround Strunk, K. O., Marsh, J. A., Hashim, A. K., Mecenas-
strategies. Center for Public Education, National Bush, S. C., & Weinstein, T. L. (in press). The
School Boards Association. Retrieved from http:// impact of turnaround reform on student outcomes:
www.centerforpubliceducation.org/Main-Menu/ Evidence and insights from the Los Angeles
Policies/Which-Way-Up-At-a-glance/Which- Unified School District. Education Finance Policy.
Way-Up-Full-Report.pdf U.S. Department of Education, Office of Elementary
Pandey, S., & Sharma, R. R. K. (2009). Organizational and Secondary Education. (2010). Guidance on fis-
factors for exploration and exploitation. Journal cal year 2010 School Improvement Grants under
of Technology Management & Innovation, 4, section 1003(G) of the Elementary and Secondary
4858. Education Act of 1965. Washington, DC: U.S.
Patton, M. Q. (2002). Designing qualitative studies. In Department of Education.
Qualitative research and evaluation methods (Vol. U.S. Government Accountability Office. (2012).
3, pp. 230246). Thousand Oaks, CA: SAGE. School Improvement Grants: Education should
Rice, J. K., & Croninger, R. G. (2005). Resource gen- take additional steps to enhance accountability for
eration, reallocation, or depletion: An analysis of schools and contractors. Washington, DC: Author.
the impact of reconstitution on school capacity. Vera, D., & Crossan, M. (2004). Strategic learn-
Leadership and Policy in Schools, 4, 73103. ing and organizational learning. The Academy of
Rice, J. K., & Malen, B. (2003). The human costs of Management Review, 29, 222240.
education reform: The case of school reconstitu-
tion. Educational Administration Quarterly, 39,
635666. Authors
Rice, J. K., & Malen, B. (2010). School reconstitu- Katharine O. Strunk, PhD, is an associate
tion as an education reform strategy: A synopsis professor of education and policy at the University
of the evidence. Atlanta, GA: National Education of Southern California. Her research centers on
Association. K12 education policy with a focus on education
Rivkin, J. W., & Siggelkow, N. (2003). Balancing labor markets, accountability policies, and gover-
search and stability: Interdependencies among nance. In particular, she studies teachers unions and
elements of organizational design. Management the collective bargaining agreements they negotiate
Science, 49, 290311. with district administrators and the implementation

576
Innovation and a Return to the Status Quo

and impacts of various teacher retention, evaluation, Ayesha K. Hashim is a Deans PhD fellow at the
and compensation policies, as well as broader University of Southern Californias Rossier School of
accountability policies. Education. Her research focuses on the implementa-
tion and effects of instructional reform, accountability,
Julie A. Marsh, PhD, is an associate professor of
and school choice policies.
education policy at the University of Southern
Californias Rossier School of Education, who spe- Susan Bush-Mecenas is a Provosts PhD fel-
cializes in research on K12 policy. Her research low at the University of Southern Californias Rossier
blends perspectives in education, sociology, and School of Education. Her research interests include
political science. Her research focuses on the imple- organizational learning, district reform, district and
mentation and effects of accountability and instruc- school capacity building, and accountability.
tional reform policies, including the roles of central
office administrators, intermediary organizations, Manuscript received May 11, 2015
and community members in educational reform and Revision received February 5, 2016
the use of data to guide decision making. Accepted March 10, 2016

577

Вам также может понравиться