Extension

R ESEARCH
Analysis of Secondary School

Efficiency: Final Report
Professor Peter C Smith

Dr Andrew Street
Research Report RR788

Research Report
No 788
Analysis of Secondary School

Efficiency: Final Report
Professor Peter C Smith

Dr Andrew Street
The views expressed in this report are the authors’ and do not necessarily reflect those of the Department
for Education and Skills.
© Professor Peter C Smith 2006

ISBN 1 84478 788 5
Contents
Introduction ............................................................................................................................. 3
1 The concept of efficiency ................................................................................................ 4
Organisational efficiency ...........................................................................................4
Weight restrictions and cross-efficiency models .......................................................9
Decomposing efficiency ..........................................................................................13
Returns to scale ........................................................................................................15
The cost function......................................................................................................15
Analytical techniques: SFA and DEA .....................................................................16
2 Issues in model construction ........................................................................................ 18

Unit of analysis ........................................................................................................19
Specifying inputs and outputs..................................................................................20
Labour inputs .......................................................................................................21
Capital inputs.......................................................................................................22
Environmental constraints .......................................................................................23
Practical challenges..................................................................................................25
Conclusions..............................................................................................................26
3 Data and descriptive analysis ...................................................................................... 28

4 Regression analysis ....................................................................................................... 33
Modelling strategy ...................................................................................................33
Model specification..................................................................................................33
Estimation results.....................................................................................................36
5 Data envelopment analysis........................................................................................... 40

Model specification..................................................................................................41
Estimation results.....................................................................................................42
6 Commentary and next steps ........................................................................................ 44

The dataset ...............................................................................................................44
Stochastic frontier analysis and model specification ...............................................45
Endogeneity .............................................................................................................45
Multilevel modelling................................................................................................48
Concluding comments .............................................................................................49
Acknowledgement ................................................................................................................. 49
References .............................................................................................................................. 50
Appendix 1: Regression results ............................................................................................ 52
Full sample...............................................................................................................52
Schools with 6th forms .............................................................................................55
Schools without 6th forms ........................................................................................58
Appendix 2: The Malmquist index of productivity change ............................................... 60
1
2
Introduction
The Department for Education and Skills (DfES) commissioned us to deliver a report
detailing how current techniques used to model multi-input, multi-output production
process, such as Data Envelopment Analysis (DEA) and Stochastic Frontier Analysis
(SFA), can be applied to measure allocative and technical efficiency in the education
sector. We were asked to:
• provide a critical appraisal of the different methods that can be used to
calculate technical and allocative efficiency, by reviewing the current
literature and assessing its value to the school context.
• apply the techniques of DEA and SFA to analyse the relative performance of
secondary schools in England.
• make recommendations about the priorities for future research by the DfES.
This report examines the efficiency of English secondary schools in academic year
2003-04. It uses data from 2,928 schools to develop and test models of efficiency
using standard parametric and non-parametric statistical methods.
Section 1 examines the issues involved in measuring the efficiency of public service
organisations such as schools. Section 2 then discusses the elements of the efficiency
model, such as the units of analysis, the choice of inputs and outputs, and the impact
of external constraints on levels of attainment. For those familiar with the economic
ideas of efficiency and productivity, sections 1 and 2 can be safely skipped.
Section 3 describes the data used in this study. Section 4 describes the development of
parametric models of school efficiency, using the methods of ordinary least squares
regression analysis and stochastic frontier analysis. Section 5 describes the
development of analogous non-parametric models, based on data envelopment
analysis. Section 6 comments on the findings and discusses the priorities for future
work.
3
1 The concept of efficiency
The pursuit of efficiency has become a central objective of policy makers within most
public services. The reasons are manifest. In developed countries, expenditure on such
services amounts to a sizeable proportion of gross domestic product. Policy makers
need to be assured that such expenditure is in line with citizens’ preferences,
particularly when many sources of finance, such as tax revenues, are under acute
pressure.
The analysis and measurement of efficiency is a complex undertaking, especially

when – as in most public services – there exist conceptual challenges, multiple
objectives and great scope for measurement error. To address this complexity there
has developed a flourishing research discipline of organisational efficiency analysis.
Following pioneering studies by Farrell (Farrell, 1957), statisticians, econometricians
and management scientists have developed tools to a high level of analytic
sophistication that seek to measure the productive efficiency of organisations and
systems. This report examines their specific relevance to secondary schooling in
England.
Notwithstanding the apparent simplicity of the concept, there is a great deal of

confusion in both popular and professional discussion about what is meant by
efficiency. In this section we first discuss the reasons for wishing to measure
efficiency, and then define the concepts of organisational efficiency deployed in this
report. Throughout the report we take the view that educational objectives are known
and agreed. In practice, we acknowledge that objectives and priorities may be highly
contested. However, we set aside such debates for the purposes of this analysis.
Organisational efficiency
The focus of efficiency analysis is as an organisational locus of production, often

referred to as a decision-making unit (DMU). In this study the DMU is assumed
usually to be the school, although there may be circumstances when the Local
Authority (LA) is the organisation under scrutiny. The DMUs consume various costly
inputs (labour, capital, etc.) and produce valued outputs. Efficiency analysis is
centrally concerned with measuring the competence with which inputs are converted
into valued outputs. In general, it treats the organisation as a black box, and does not
seek to explain why it exhibits a particular level of efficiency (Fried et al., 1993).
The terms ‘productivity’ and ‘efficiency’ are often used interchangeably, which is
unfortunate since they are not precisely the same thing. Productivity is the ratio of
some (or all) valued outputs that an organisation produces to some (or all) inputs used
in the production process. Efficiency incorporates the concept of the production
possibility frontier, which indicates feasible output levels given the scale of operation.
Thus the concept of productivity may embrace but is not confined to the notion of
efficiency.
A starting point for examining the basic notion of efficiency is shown in Figure 1.1,
which illustrates the case of just one input and one output. The line OC indicates the
simplest of all technologies: no fixed costs and constant returns to scale. A technically
efficient organisation would then produce somewhere on this line, which can be
4
thought of as the production possibility frontier. Any element of inefficiency would
result in an observation lying strictly below the line OC. For an inefficient
organisation located at P0, the ratio X0P0/ X0P0* offers an indication of how far short
of the production frontier it is falling, and therefore a measure of its efficiency level.
Figure 1.1: Efficiency measurement under constant returns to scale
C
Output
*
P0
P0
X0 Input
0
Many other technologies are possible. For example, the curve OV indicates a frontier
with variable returns to scale. Up to the point P0*, the ratio of output to input
decreases (increasing returns to scale), thereafter it increases (decreasing returns to
scale).
The notion of a production frontier can be extended to multiple outputs and a single
input (say, costs). Figure 1.2 illustrates the case with two outputs. For the given
technology, the isocost curve CC gives the feasible combination of outputs that can be
secured for a given input. At a higher level of costs the isocost curve moves out to
C'C'. These curves indicate the shape of the production possibility frontiers at given
levels of input. An inefficient DMU lies inside this frontier. We define the marginal
rate of transformation to be the sacrifice of output 2 required to produce a unit of
output 1, indicated at any particular point on CC by the slope of the curve − ( P2 P1 ) .
It is usually assumed that – as in this Figure – for a given level of input the slope
becomes steeper as the volume of output 1 produced increases.
5
Figure 1.2: The case of two outputs
Output 2
C'
C
P2
P1 C C' Output 1
Likewise in input space we examine the case of two inputs and one output, as in
Figure 1.3. The isoquant QQ indicates the feasible mix of inputs that can secure a
given level of output, with inefficient DMUs lying beyond this curve.
Figure 1.3: The case of two inputs
Input 2
Q
Input 1
Extending the analysis to the general case of multiple inputs and multiple outputs, we
define the overall efficiency eff0 of organisation zero to be the ratio of a weighted sum
of outputs to a weighted sum of inputs. Mathematically, if organisation 0 consumes a
6
vector of m inputs X0 and produces a vector of s outputs Y0, its overall efficiency is
measured by applying weight vectors U and V to yield:
S
∑U s Y s0
eff 0 = s=1
M
(1.1)
∑V
m=1
m X m0
where:
Ys0 is the amount of the sth output produced by organisation 0;
Us is the weight given to sth output;
Xm0 is the amount of the mth input consumed by organisation 0;
Vm is the weight given to the mth input.
The weights U and V indicate the relative importance of an additional unit of output
or input. On the input side, the weights V might reflect the relative market prices of
different inputs. It is often the case that – with the notable exception of capital inputs
– these can be measured with some accuracy. Then, if the actual input costs incurred
by organisation 0 are C0, the ratio:
M
∑V m X mi 0
Ceff 0 = m =1
(1.2)
C0
indicates the extent to which the organisation is purchasing its chosen mix of inputs
efficiently (that is, the extent to which it is purchasing its chosen inputs at lowest
possible prices).
However, the organisation may not be using the correct mix of inputs. This can be
illustrated using a simple two input model. For some known production process, the
isoquant QQ in Figure 1.4 shows the use of minimum inputs required to produce a
unit of a single output. The points P1 and P2 lie on the isoquant and therefore – given
the chosen mix of inputs – cannot produce more outputs.
When the unit costs of inputs are known, it is possible to examine the input price (or
allocative) efficiency of the two units. Suppose the market prices are V1* and V2*.
Then the cost minimizing point on the isoquant occurs where the slope is -V1*/V2*
(shown by the straight line BB). In Figure 1.4 this is the point P1, which is input price
efficient. However, although technically efficient, the point P2 is not efficient with
respect to prices, as a reduction in costs of P2P2* is possible. The price efficiency of P2
is therefore given by the ratio OP2*/OP2.
7
Figure 1.4: Allocative efficiency with two inputs
Q
Input 2 B
P1
P2
*
P2
Q
B
Input 1
0
Analogous arguments can be deployed to examine the allocative efficiency of

organisations in output space. Figure 1.5 illustrates the case where a single input is
used to produce two outputs. If the relative values U1 and U2 of the outputs are known,
and the production possibilities are given by the curve CC, then organisation P1 is
producing at its allocatively efficient point while organisation P2 exhibits some
allocative inefficiency.
Figure 1.5: Allocative efficiency with two outputs
Output 2
P1
P2
C Output 1
0
Although organisations may exhibit allocative inefficiency in purchasing the wrong

mix of inputs or producing the wrong mix of outputs, we have so far explored only
those organisations that lie on the frontier of technical production possibilities.
However, it is likely that, particularly in a non-market environment, many
organisations are not operating on the frontier. That is, they also exhibit an element of
technical inefficiency (also referred to as managerial inefficiency or X-inefficiency).
8
This is illustrated in Figure 1.6 by the points P3 and P4. Organisation P3 purchases the
correct mix of inputs, but lies inside the isoquant QQ. It therefore exhibits a degree of
technical inefficiency, as indicated by the ratio OP1/OP3. Organisation P4 both
purchases an incorrect mix of inputs and lies inside the isoquant QQ. Its technical
inefficiency is indicated by the ratio OP2/OP4. Thus its overall level of inefficiency
OP2*/OP4 can be thought of as the product of two components: technical inefficiency
OP2/OP4 and allocative inefficiency OP2*/OP2.
Figure 1.6: Technical and allocative inefficiency
Q
Input 2 B
P3
P1
P4
P2
*
P2
Q
B
Input 1
0
Weight restrictions and cross-efficiency models
There are important questions relating to the objectives encompassed by any index of
efficiency. Is it legitimate for the central policy maker to attach a uniform set of
objectives to all organisations? If so, is it further legitimate to apply a uniform set of
weights to these objectives? If so, how should they be chosen? If not, what is the
extent of legitimate variation, and who should choose? These are fundamental issues,
the answers to which determine whether or not creating a single measure of
organisational performance is warranted. In our view, organisations can be ranked in
this way only if the policy maker may legitimately (a) set objectives and (b) attach
weights to those objectives.
When considering organizations working in the public sector, the set of output
weights ought to reflect societal values. However it is not a simple matter to derive
such weights, particularly when organisations face multiple objectives and there is
disagreement as to organisational priorities. Ultimately the selection of objectives in
the public services is a job for the politicians charged with reconciling conflicting
claims on public resources. The main role for analysts is to clarify the choices
required of policy makers, to provide evidence on popular preferences, and to develop
measurement instruments that most faithfully reflect the chosen objectives. Note that
9
policy makers are effectively attaching a zero weight to any output that is excluded
from the efficiency index.
In principle, the set of weights to be used in an efficiency index could be derived from
a range of sources, such as economic studies of willingness to pay or conjoint analysis.
However, rather than being externally agreed upon, in most efficiency analysis studies
the weights are generated as a by-product of the statistical estimation process. Indeed,
some see this as an attractive feature of the methods (Cooper et al., 2000).
When undertaking econometric analysis, it is not a trivial matter to take account of

multiple outputs. Approaches include the creation of a single index of outputs,
estimation of a cost rather than production function, or the use of distance functions
(Coelli and Perelman, 2000, Lothgren, 2000, Shephard, 1970). Irrespective of the
approach, the estimated magnitude of the weight for each output usually corresponds
to the value implicit in the sample mean cost of producing an additional unit of output.
Using a linear model, the weight attached to an output indicates the value of an
additional unit of that output, which remains constant for all levels of attainment of
the output. If a logarithmic model is used, the weight indicates the percentage increase
in attainment implied by a one percent increase in the output. Hence the econometric
approach is conservative in the sense that it implies that the existing output choices of
organisations (on average) reflect the values placed by society on the outputs. If this is
not true, the estimated weights will not be appropriate.
In conventional DEA both input and output weights are allowed to vary freely, so that
each organisation is evaluated in the best possible light. Indeed one quite frequently
finds in unconstrained DEA that the highest efficiency score for an organisation can
be secured simply by assigning a zero weight to one or more outputs on which it
performs poorly. In order to understand the role played by weights in DEA it is
useful to examine the simple two input model (Figure 1.7). For some known
production process, the isoquant QQ shows the use of minimum inputs required to
produce a unit of output, assuming constant returns to scale. The points P* and PT lie
on the isoquant and are therefore efficient in Farrell's technical sense.
10
Figure 1.7: Fixed and variable weights
Input 2 Q
B
P*
P
PT
PA Q
B
O
Input 1
However, when the unit costs of inputs are known, it is possible in addition to
examine the input price (or allocative) efficiency of the two units. Suppose the costs
are C1* and C2*. Then the cost minimizing point on the isoquant occurs where the
slope is -C1*/C2* (shown by the straight line BB). In Figure 1.7 this is the point P*, so
that point is also allocatively efficient. However, the point PT is not efficient with
respect to prices, as a reduction in costs is feasible. The price efficiency of PT is
therefore given by the ratio OPA/OPT. The DMU represented by point P also exhibits
technical efficiency, indicated by the ratio OPT/OP.
The classical application of DEA places no constraints (other than positivity) on

weights, and so allows a DMU's efficiency to be assessed using the weights most
favourable to its own circumstances. It therefore yields a measure of technical
efficiency. In this example, therefore, DMU P will be evaluated using weights that
place the allocatively efficient point of the frontier at PT rather than PA. These optimal
weights C1P and C2P for DMU P are represented by the slope of the tangent at the
point PT, indicated by the dotted line with slope -C1P/C2P in the diagram. Clearly very
‘extreme’ DMUs would favour extreme weights, attaching a zero weight to one or
other of the inputs. (Note that the weights can be multiplied by an arbitrary scalar, so
it is only the ratio of the weights – the slope of the dotted line – in which we are
interested.)
This ‘weight flexibility’ gives rise to a conundrum. If the inputs or outputs given zero
weight in the analysis for a particular DMU are important, then the measure of
efficiency must surely be deficient in placing no consequence on them. If on the other
hand they are not important, then why were they included in the analysis in the first
place? It is concerns of this sort that have led to the development of various schemes
for placing limits on variations in weights.
11
By placing constraints on weights, the region of search for those weights is reduced,
and so a DMU's efficiency assessment cannot increase, and may decrease. At the
extreme, if no flexibility in weights is allowed, DEA becomes classical ratio analysis,
in which a unit's efficiency is measured as its ratio of weighted outputs to weighted
inputs, with weights equivalent to the known input and output prices. In Figure 1.7
this measure is equivalent to ‘overall’ efficiency, measured relative to other DMUs.
Thus, as increasingly severe constraints are placed on weights, so the measure of
efficiency derived moves from one of relative technical efficiency to one of relative
overall efficiency.
An intermediate approach to imposing weight restrictions is illustrated in Figure 1.8.

Here the dotted lines represented the maximum and minimum ratio of weights the
analysts feels are acceptable. For DMUs with optimal weights between these two
limits, there is no change to the DEA assessment – the units continue to be assessed
with respect to the isoquant QQ between points M1 and M2. However, for DMUs with
optimal weights outside this range, efficiency is assessed with respect to the dotted
tangent lines representing the weight limits. For example, the efficiency of point R is
assessed relative to point RR on the tangent rather than point RQ on the frontier.
Imposing weight restrictions has the effect of limiting the ability of DMUs to ‘choose’
extreme weights, and therefore reduces the assessed efficiency of DMUs with extreme
optimal weights, whilst leaving unchanged the efficiency of DMUs with less extreme
choices.
Figure 1.8: Weight restrictions
Input 2 Q
RQ
RR
M1
P
PT
M2
Q
O
Input 1
There has been some attention to rules for restricting the flexibility of weight
variations (Allen et al., 1997), but the efforts to date have been poorly informed by
economic theory, and mainly confined to technical considerations (Pedraja-Chaparro
et al., 1999). The lack of a single set of weights implies that it is never appropriate to
12
rank DEA efficiency scores in a conventional ‘league table’ format. The fundamental
difficulty with applying weight restrictions is the same as encountered whenever
seeking to choose any set of weights – who should choose the restrictions, and how
should they choose?
A number of approaches have been suggested as mechanisms for addressing the lack
of knowledge about the magnitude of weights, and the arbitrary efficiency scores that
DEA can yield for ‘extreme’ DMUs. Amongst the most informative is the calculation
of ‘cross-efficiencies’ (Doyle and Green, 1994). For each DMU 0 a set of efficiencies
{e0j} is calculated, where j ranges from 1 to N, the number of DMUs in the sample.
Efficiency e0j indicates the efficiency score for DMU 0 if the optimal weights selected
by DMU j are applied. Thus, e00 is the conventional DEA score for DMU 0, and
because this seeks to cast DMU 0 in the best possible light, none of the remaining e0j
scores can be higher than e00. Cross-efficiencies offer valuable insights into the
‘robustness’ of a DMUs unconstrained efficiency score.
However, it is worth noting that – once DEA moves beyond simple models with just a
handful of inputs and outputs – there are often multiple solutions for optimal weights
that offer identical efficiency measures. That is, there is often not a unique set of
weights that yields the DMU’s maximum efficiency level. This offers challenges for
the systematic application of the cross-efficiency principle.
Decomposing efficiency
Figure 1.9 illustrates the various notions of efficiency that can be examined in the
case of just two inputs and a single output, with constant returns to scale. The figure
shows the inputs required to produce a given level of output. The isoquant QQ shows
the production frontier, in the sense that for any level of input 1 it shows the minimum
level of input 2 required to produce the output. All DMUs must therefore lie on or to
the north east of the isoquant.
13
Figure 1.9: Production possibilities with two inputs, one output
Input 2 Q
B
P*
x2 P
PT
PA Q
B
O
x1 Input 1
Consider a DMU P that consumes physical inputs x1 and x2 to produce the output. It is
technically inefficient because it does not lie on the isoquant. The traditional radial
measure of inefficiency is OPT/OP, the proportion by which it is feasible to reduce all
inputs and still produce the required output.
A technically efficient production choice such as PT may not be allocatively efficient

because – given the relative prices of the two inputs w1 and w2 – the output can be
produced at lower total cost. The line BB has slope -w1/w2 and passes through the
allocatively efficient point P*. It indicates the minimum cost point of production.
The overall inefficiency of DMU P is therefore indicated by the ratio OPA/OP which
can be written as the product of OPT/OP and OPA/OPT. The first ratio is the technical
inefficiency, the second is allocative inefficiency.
This analysis assumes that, although the DMU makes an incorrect choice of inputs,
and uses them inefficiently, it nevertheless purchases them at the correct market
prices w1 and w2, with total cost W* = x1w1 + x2w2. It is however possible that it
purchases inefficiently, at a higher total cost WP<W*. This suggests a third component
of inefficiency, indicated by the ratio W*/WP. Total inefficiency can now be
decomposed into:
• Technical inefficiency OPT/OP;
• Allocative inefficiency OPA/OPT;
• Cost inefficiency W*/WP;
This analysis suggests a need to calculate inefficiency using three different types of
input measure:
• Disaggregated physical inputs
• Physical inputs aggregated at market prices
• Actual costs.
14
If DEA is used, this implies a need to run the model first with disaggregated physical
inputs and then with the inputs aggregated at market prices in order to estimate the
ratios OPT/OP and OPA/OP. The cost inefficiency can be calculated trivially as the
ratio W*/WP without resort to DEA.
Returns to scale
We have so far assumed constant returns to scale. That is, the production process is
such that the optimal mix of inputs and outputs is independent of scale of operation.
In practice there exist important economies and diseconomies of scale in most
production processes, so an important influence on eff0 (from equation 1.1) may be
the chosen scale of operation. This is illustrated in Figure 1.10 for the case of one
input and one output. The production frontier is illustrated by the curve OV, which
suggests regions of increasing and decreasing returns to scale. The optimal scale of
production is at the point P* where the ratio of output to input is maximised. Although
lying on the frontier, the points P1 and P2 secure lower ratios because they are
operating below and above (respectively) the scale efficient point of production. They
therefore exhibit levels of scale inefficiency given by:
OY1 OY2
OX 1 OX 2
Seff1 = *
and Seff 2 = *
(1.3)
OY *
OY *
OX OX
Figure 1.10: Economies of scale
Output
P2 V
Y2
Y* *
P
Y1 P1
X1 X* X2 Input
0
The cost function
The fundamental building block of the economic analysis of organisational efficiency

is the production function or its counterpart the cost function. The production function
15
models the maximum (single) output an organisation could secure, given its mix of
inputs.
The cost function simplifies the input side of the production process by deploying a
single measure of the inputs used, rather than a vector. It indicates the minimum cost
that an organisation can incur in seeking to produce a set of valued outputs. Using the
notation introduced above, a cost function can be written in general terms as C0* =
f(Y0).
The cost function combines all inputs into a single metric (costs), and does not model
the mix of inputs employed, or their prices. In practice, as noted above, the costs
incurred by an organisation might be higher than those implied by the cost function
for three reasons. First, it may purchase inputs at higher than market prices (cost
inefficiency). Second, given prevailing prices, it may employ an inefficient mix of
inputs (allocative inefficiency). And third, it may not secure the maximum output
attainable given its inputs (technical inefficiency). However, if no measures of
physical inputs are available, and only aggregate measures of costs are available, it is
impossible to distinguish between these causes of deviation from the cost function.
Therefore, notwithstanding its practical usefulness, a cost function offers little help
with detailed understanding of the input side of efficiency.
Inefficiency can be defined as the extent to which an organisation’s costs exceed

those predicted by the cost function (or the extent to which its output falls short of that
predicted by the production function).
Analytical techniques: SFA and DEA
Inefficiency is inherently unobservable. This means that estimates of efficiency have

to be derived indirectly after taking account of observable phenomena. This very
broadly involves the following process:
1. Observable phenomena are measured such as outputs and inputs (costs).
2. Some form of relationship between these phenomena is specified. If a
parametric method is used and differences in cost are the focus of the exercise, a
production or cost function is estimated. If a non-parametric method is used, an
efficiency frontier is derived.
3. Efficient behaviour is then predicted on the basis of the definition of technical
efficiency.
4. The difference between each DMU’s observed data and the maximum
achievable is then calculated, as predicted by the frontier.
5. The difference (or some portion of it) is defined as inefficiency.
A number of analytic techniques have been developed to estimate the form of cost and
production frontiers and the associated inefficiency of individual organisations (Coelli
et al., 1998). These are covered in the subsequent sections of this report, and can be
divided into two broad categories: parametric methods, which use econometric
methods to estimate the parameters of a specific functional form of cost or production
function, and non-parametric methods, which place no conditions on the functional
form, and use observed data to infer the shape of the frontier.
16
The pre-eminent form of parametric method now in use is stochastic frontier analysis
(SFA). This is similar to conventional regression analysis, but decomposes the
unexplained error in the estimated function into two components: inefficiency (which,
in the case of a cost function will always be positive), and the more conventional two-
sided random error. Cost functions are used extensively in parametric analysis of
efficiency because the alternative strategy - estimating models with both multiple
inputs and multiple outputs - is methodologically challenging and demanding in terms
of data requirements. In contrast, single input cost functions and single output
production functions can be readily estimated using standard econometric methods, or
straightforward variants.
Most non-parametric methods take the form of data envelopment analysis (DEA) and
its many variants. These were stimulated by the pioneering work of Farrell (Farrell,
1957), later generalised by Charnes, Cooper and Rhodes (Charnes et al., 1978). DEA
uses linear programming methods to infer a piecewise linear production possibility
frontier, in effect seeking out those efficient observations that dominate (or ‘envelop’)
the others. In contrast to parametric methods, DEA can handle multiple inputs and
outputs without difficulty.
The distinctive focus of modern efficiency analysis is to seek – in addition to

estimating the frontier – to provide an estimate of how far each observation falls short
of the estimated frontier. This emphasis on the residual for each observation marks an
unusual departure from conventional statistical and econometric analyses, which are
in the main preoccupied with estimated coefficients (that is, relationships displayed by
the population of observations as a whole) rather than individual observations. This
novel focus gives rise to important methodological complications.
17
2 Issues in model construction
An organisation’s efficiency is considered to be the ratio of the value of outputs it
produces to the value of inputs it consumes. Figure 2.1 summarises the principles
underlying this viewpoint. The organisation consumes a series of M physical
resources, referred to as inputs, and valued in total as X by society. Some
transformation process takes place, the details of which do not immediately concern
us. This leads to the production of S outputs, which society values in aggregate as Y.
Our summary measure of ‘efficiency’ is the ratio of Y to X – what might be more
accurately referred to as cost-effectiveness. This is analogous to the DfES’s schema
which depicts the process as comprising three phases, termed economy, efficiency
and effectiveness.
Figure 2.1: The naïve model of organisational performance
Input 1 Output 1
Decision
Costs Input 2 Output 2 Benefits
Making
X : : Y
Unit
Input m Output s
Economy Efficiency Effectiveness

Procurement of inputs at Use of inputs to produce Choice of output mix to
minimum costs maximum output produce maximum impact
on outcomes
Models of public service efficiency almost always entail consideration of multiple

outputs. Central to the calculation of Y is therefore the relative weight Us attached to
each output s. These weights reflect the relative importance attached to an additional
unit of production of each output, and allow us to calculate for organisation 0 the
S
valuation of outputs Y = ∑U s Y s 0 as discussed in section one. In the same way, when
s=1
there are multiple inputs, the relative weight Vm attached to input m allow us to
M
calculate the valuation of inputs X =∑V m X m 0 . If we have secure information on the
m=1
magnitudes of U and V we can readily compute the efficiency as the ratio Y/X. In
particular, in competitive markets, both U and V might be readily observed as prices.
In such circumstances there may be no need to use the analytic techniques of SFA and
DEA. Instead, comparative efficiency can be readily computed using the exogenously
observed weights. However, a special feature of public services such as schools is that
such prices are rarely observed, particularly on the output side. It is in such
circumstances that the analytic techniques can be deployed in order to furnish
evidence on the weights U and V.
18
Although beguilingly simple, numerous complex issues are raised when seeking to
use this framework to develop operational models of organisational efficiency. The
complexity involved in developing an operational framework reflects the complexity
of the production process. The production of the majority of public service output
rarely conforms to a production-line type technology, where a set of clearly
identifiable inputs are used to produce a standard type of output. This means that the
production process is much less clearly defined and there is the potential for
considerable heterogeneity in how and what outputs are produced. Contributions to
public service outcomes are often made by multiple agents or organisations, a service
package may be delivered over multiple time periods and in different settings, and the
responsibilities for delivery may vary from place to place and over time.
We address model-building principles by considering five issues:

• What is the appropriate unit of analysis?
• What are the system outputs?
• What value should be attached to these outputs?
• What inputs are used in the production of these outputs and how should these be
valued?
• What environmental constraints are faced?
We then discuss practical facets of undertaking efficiency analysis, which is often

constrained by the scope and nature of data availability.
Unit of analysis
It is important that the boundaries of any efficiency analysis should be clearly defined.
A fundamental question to ask is: what is the unit of organisation in which we are
interested? Any efficiency analysis should have a clear idea of the entity it is
examining, but should also recognise that its achievements are likely to be influenced
by the actions of other organisations or factors beyond its immediate control. This is
especially likely when multiple agencies or organisations are involved in joint
production. Three criteria should guide the choice of analytical unit:
• First, the unit of analysis should capture the entire production process of interest.
This may entail defining artificial units of analysis if there is variation among
organisations in how the production process is divided between them.
• Second, they should be ‘decision-making units’ (DMUs). In a strict sense this
requires that their function is to convert inputs into outputs, and that the DMUs
have discretion about the technological process by which this conversion takes
place. But a weaker definition of DMUs requires only that they play an
organising function, establishing the rules and conditions to which producers
have to adhere. This definition would allow government bodies to be considered
as DMUs.
• Third, the units comprising the analytical sample should be comparable,
particularly in the sense that they are seeking to produce the same set of outputs.
In the current context the choice of analytical unit is fairly straightforward, with the
focus on school performance. Nevertheless, there may be elements of the process of
the production of education that are provided outside the school (eg by parents) or that
19
are organised differently from one school to the next (for instance, some schools may
be in a better position to take advantage of educational opportunities provided by
other agencies in their area).
Specifying inputs and outputs
Central to the development of any efficiency model is the specification of the inputs
consumed by the DMU and the valued outputs it produces. Indeed, although
efficiency analysts usually refer to the ‘outputs’ of the production process, regulators
and other decision-makers are more usually interested in the ‘outcomes’ of education
services, in terms of the impact they have on individual and social welfare.
Throughout this report we adhere to convention by referring to the ‘outputs’ of the
schools. However, it should be understood that – wherever possible – the analyst
should seek to measure outputs in terms of their ultimate outcomes on pupils and
society.
In competitive industries the physical output of the organisation is usually a traded

product. Of course, even in a reasonably homogeneous market, such products (say, a
refrigerator) can vary enormously in various dimensions of ‘quality’, such as
reliability, looks and temperature range. The quality of the product is intrinsic to its
social value, but that value can be readily inferred by observing the price people are
prepared to pay. Usually, therefore, there is no need explicitly to consider the ultimate
‘outcome’ of the product, in terms of the value it bestows on the consumer.
In many parts of the economy, however, not only do prices not exist, but outputs are
difficult to define. In particular this is true for many of the goods and services to
which government spending is devoted (Atkinson, 2005). In the context of efficiency
analysis, two fundamental issues need to be considered. How should the outputs of
the public services sector be defined? And what value should be attached to these
outputs?
Public service organisations rarely collect routine information about the outcomes
they produce. More commonly the analyst is forced to rely on comparing
organisations in terms of the quantity and type of activities they undertake. For
example, in the educational sector, numbers of pupils taught are readily measured.
However, in schooling we also have available an important intermediate measure of
outcome, expressed in the form of examination results. Although these do not
measure the eventual outcomes achieved, in terms of intellectual development,
citizenry and ‘life chances’, they are acknowledged to offer a useful indicator of
success in some important dimensions of school attainment.
The input side of efficiency analysis is usually considered less problematic than the
output side. Physical inputs can often be measured more accurately than outputs, or
can be summarised in the form of a measure of costs. However, even the specification
of inputs can give rise to serious conceptual and practical difficulties in efficiency
analysis. This section briefly summarises some of the major issues.
A fundamental decision that must be taken is the level of disaggregation of inputs to

be specified. At one extreme, a single measure of aggregate inputs (in the form of
20
total costs) might be used. The efficiency model then effectively becomes a cost
function. This approach assumes that the organisations under scrutiny are free to
deploy inputs efficiently, taking account of relative prices. Any failure to do so (price
inefficiency) will be reflected in a lower estimate of measured efficiency. Use of a
single measure of costs therefore adopts a long-run perspective as it assumes – for
example – that organisations can freely adopt an optimal mix of capital and labour.
It may also be important to consider a short-term perspective, in which certain aspects

of the input mix are considered beyond the control of the organisation. In these
circumstances, it becomes necessary to disaggregate the inputs in order to capture the
different input mixes that organisations have inherited. In particular, disaggregation of
labour and capital may be required. We consider these two classes of input in turn.
Labour inputs
Labour inputs can usually be measured with some degree of accuracy, often
disaggregated by skill level. An important issue is then how much aggregation of
labour inputs to use before pursuing an efficiency analysis. Unless there is a specific
interest in the deployment of different labour types, it may be appropriate to aggregate
into a single measure of labour input, weighting the various labour inputs by their
relative wages. A central contribution of efficiency techniques is to offer evidence on
efficiency when there is no direct information on the relative value placed on inputs or
outputs. Where such evidence does exist, as for example on the market price of inputs,
there may be little merit in disaggregation unless there is a specific interest in input
allocative efficiency. Aggregation leads to a more parsimonious model, thereby
allowing the analyst to focus attention on aspects of the production process where
there is less secure evidence on weightings.
However, there may be an interest in the relationship between efficiency and the mix
of labour inputs employed. Under such circumstances, a model using measures of
labour input disaggregated by skill type may be valuable. This might yield useful
policy recommendations about (say) substituting some types of labour for another.
It may be that labour inputs are measured in either physical units (hours of labour) or
costs of labour. Which should be used in an efficiency analysis depends on context.
Use of physical inputs will fail to capture any variations in organisations’ wage rates.
This may be desirable (for example if there are variations in pay levels beyond the
control of organisations) or undesirable (if there is believed to be input price
inefficiency, in the form of different pay levels for identical workers).
Organisations such as schools may vary in the extent to which they ‘buy in’ certain
services, rather than directly employ labour. For example, some DMUs may purchase
cleaning services from independent contractors and have little idea of the associated
labour inputs. Cleaning services might therefore appear as a ‘goods and services
purchased’ input rather than labour and capital inputs. If other DMUs directly employ
cleaning personnel, it may be the case that such inputs are not being treated on a
strictly comparable basis. In such circumstances, the analyst may need to resort to a
single measure of inputs, in the form of total costs.
21
Capital inputs
Incorporating measures of capital into the efficiency analysis is more challenging.

This is partly because of the difficulty of measuring capital stock and partly due to
problems in attributing its use to any particular period.
Measures of capital are often very rudimentary, and even misleading. For example,
accounting measures of the depreciation of physical stock usually offer little
meaningful indication of capital consumed. Indeed, in practice, analysts may have to
resort to very crude measures. Furthermore, public services often invest in important
non-physical capital inputs, such as staff development.
In principle, an efficiency model should use the capital consumed in the current
period as an input to the production process. But capital is by its nature deployed
across time. On the one hand, contemporary output may rely on capital investment
that took place in previous periods, whilst on the other hand, some current activities
are investments that are intended to contribute to future rather than contemporary
outputs.
Figure 2.2: The dynamic nature of organisational performance
Outputs Outputs Outputs

Year t-1 Year t Year t+1
Organization Organization Organization

Inputs Inputs Inputs

The accounting difficulties capital inputs give rise to can be summarised in the form
of a diagram, as shown in Figure 2.2. In each period, inputs are consumed and direct
outputs emerge from the organisation. A rudimentary efficiency analysis will examine
the ratio of outputs to inputs for only a single time period t. Yet crucially the
organisation in year t has enjoyed the benefits of past investments. And it also leaves
an ‘endowment’ for future periods in the form of investments undertaken in this and
preceding periods. The endowment might be in the form of real capital (buildings or
personnel) or investment in social ‘capital’ of some sort. In principle the endowment
may be an important aspect of both the inputs to and outputs of the system. In practice
it can be very difficult to measure.
22
A central issue in the treatment of capital is the extent to which short-run or long-run
efficiency is under scrutiny. In the short run, it makes sense for organisations to take
advantage of whatever investment infrastructure they have available, and optimise
subject to any constraints imposed by that infrastructure. So, for example, short run
efficiency should be judged in the light of the capital configuration that a DMU has
available. Yet in the longer run one might expect the DMU to reconfigure its capital
resources when this can bring about efficiency improvements.
Environmental constraints
In many contexts, a separate class of factor affects organisational capacity, which we

classify as the ‘environmental’ determinants of performance. These are exogenous
influences on the organisation’s production function, beyond its control, that reflect
the external environment within which it must operate. In particular, many of the
outcomes secured by public service organisations are highly dependent on the
characteristics of the population group they serve and the public bodies to which they
are responsible. For example, in education:
• levels of achievement may be related to the inherent ability of pupils. Ability at
intake may vary from school to school because of local socio-economic
conditions or because of the operation of selection processes;
• attainment is likely to be partly due to the efforts of the pupil’s parents or
guardians, and these efforts may vary systematically across the country;
• schools have differing freedoms and constraints, as they are answerable to both
central government and their Local Authority.
There is often considerable debate as to what environmental factors are considered

‘controllable’ by the DMUs under scrutiny. For example, to what extent should
engagement with parents be considered a controllable influence on school
performance? The analyst’s choice is likely to be heavily dependent on whether the
purpose of the analysis is short run and tactical, or longer run and strategic. In many
circumstances it will be appropriate to undertake both short run and long run analysis.
The performance of a school is in part dependent on inputs from outside agencies,

such as other public services, the voluntary sector or private families. In principle we
should recognise this when modelling efficiency, and there may be a difficulty in
identifying the element of outcome that is attributable specifically to the school. The
danger is either (i) a school’s contribution towards some outcomes is ignored in the
analysis (under-attribution) or (ii) the contribution of other external agencies towards
outcome is ignored (over-attribution). Whether these external efforts should be treated
as exogenous may depend on the extent to which the behaviour of external agencies is
amenable to influence by the DMU under scrutiny.
Broadly speaking, there are three ways in which environmental factors can be taken
into account in efficiency analyses:
• restrict comparison only to organisations within a similarly constrained
environment;
• model the constraints explicitly, as being analogous to ‘factors’ in the production
process;
• undertake risk adjustment.
23
The first approach to accommodating environmental influences is to cluster
organisations into similar ‘families’, using techniques such as cluster analysis (Everitt
et al., 2001). The intention is then to compare only like with like. Of course this begs
the question as to what criteria are to be used to create the families. Statistical
examination of the link between putative exogenous influences and performance is
often ruled out because efficiency too is correlated with performance. Unless
exogenous influences are known to be entirely independent of efficiency levels, it
becomes impossible to determine whether a correlation with performance is reflecting
exogenous influences or efficiency variations. The analyst will therefore often have to
resort to informed judgement in choosing the basis for creating families.
A further problem with the use of families is that it will reduce sample size, as it rules
out extrapolation of performance of one type of organisation as a basis for comparison
with another type. This may be appropriate to ensure robust comparisons, but with
parametric methods can seriously affect the confidence with which efficiency
judgements can be made, for example by leading to smaller samples and larger
standard errors. Furthermore, of course, useful lessons that organisations might learn
from being compared to a more heterogeneous sample will be lost. In this report we
employ the clustering approach when analysing separately schools with and without
sixth forms.
The second approach is to incorporate environmental factors directly into the

production model, often treating them as an exogenous ‘input’ analogous to labour or
capital. This approach effectively generalises the clustering approach by allowing
extrapolation from one class of organisation to another. For example, an
environmental factor might be included as an explanatory variable in the parametric
modelling approach or as an input or output in the non-parametric approach. Whilst
leading to a more general specification of the efficiency model than the clustering
approach, the direct incorporation of environmental factors into the efficiency model
leads to new modelling challenges that are considered further later in this report. We
also employ this approach when we introduce variables such as the proportion of
pupils entitled to free school meals into the models.
The final method to control for variation in environmental circumstances is the family
of techniques known as ‘risk adjustment’. These methods adjust organisational
outputs for differences in circumstances before they are deployed in an efficiency
model, and are – where feasible – often the most sensible approach to treating
environmental factors. In particular, they permit the analyst to adjust each output for
only those factors that apply specifically to that output, rather than use environmental
factors as a general adjustment for all outputs. For example, as in this report, the
outcomes secured by pupils (examination results) are adjusted for some measure of
prior attainment.
The methods of risk adjustment are often a highly efficient means of controlling an
output measure for a multiplicity of environmental factors. The risk-adjusted output
can then be entered into the efficiency model without any further need to enter
environmental factors. The methods of risk adjustment have been developed to a high
level of refinement (Iezzoni, 2003). However, it must be noted that risk adjustment
usually has demanding data requirements, in the form of information on the
24
circumstances of individual pupils. And even when adequate data do exist it is often
difficult to secure scientific consensus on the most appropriate way of undertaking the
risk adjustment.
Including a factor in a risk-adjusted output measure should preclude the need to

consider the factor further in any efficiency analysis. It therefore considerably
simplifies the efficiency modelling process. Yet it is important to bear in mind that
conventional risk adjustment usually requires the assumption that the sample average
outcome for a particular population group is a suitable benchmark against which
individual or organisational performance can be measured. If for some reason certain
population groups receive systematically poor services, it may be inappropriate to
include such groups in the risk adjustment process, particularly if the intention is to
highlight such underperformance and promote better performance amongst providers
of services for such groups.
One particularly important situation arises when organisations have been allocated
funds to deliver some standard level of care, for example when a school funding
formulae adjusts the ‘per pupil’ allocation for certain pupil characteristics. The
purpose of such formulae is to adjust for legitimate environmental circumstances, so
that in principle all organisations are operating on a ‘level playing field’. In these
special circumstances there may be no need explicitly to consider any exogenous
variables at all on the input side. Efficiency analysis should focus solely on variations
in organisational outputs, as all inputs have been equalised. This argument, of course,
presumes that the funding formulae have been correctly designed, which is rarely the
case. It is nevertheless important to note the link between funding mechanisms and
models of efficiency.
Practical challenges
Notwithstanding the important model-building considerations discussed in this

section, it is often the case that the most serious difficulties for the analyst arise from
the scope and nature of the available data sources. Numerous difficulties can arise:
• There is often a serious lack of information on some important dimensions of

performance, most notably the service outocmes. This is especially problematic in
efficiency analysis, when the ambition is to present a rounded view of
performance.
• Where measurement instruments are available, they may be incomplete or highly
imperfect metrics with large random elements.
• Time series are often short, or interrupted by structural changes.
• As always, there may be missing data for some observations, leading to difficult
technical choices as to whether to broaden the scope of the model (and so
effectively reduce sample size) or use a more circumscribed model with a larger
sample.
• More generally, given the complexity of the production process in many of the
public services, sample sizes may often be too small to draw secure inferences on
the nature of the preferred model.
Such difficulties will be familiar to those working in most domains of quantitative

analysis. However, they can become particularly problematic in efficiency analysis
25
because of the great attention placed on unexplained variations from the estimated
frontier. Results may be highly sensitivity to model specification and data errors.
This suggests a need for great attention to sensitivity analysis, experimentation with
different model specifications, and caution about drawing definitive judgements on
the efficiency of individual organisations. It is important to bear in mind that a great
deal of conventional uncertainty analysis merely models uncertainty in the data, and
not the implications of wrongly specifying the underlying model (Smith, 1997).
Conventional measures of uncertainty may therefore seriously overestimate the
degree of confidence in efficiency scores, and should be viewed with caution.
In short, data constraints will often circumscribe the ability to answer the questions of
public service regulators in their entirety. Instead, the analyst must adopt pragmatic
solutions to otherwise infeasible modelling demands. Under these circumstances, the
imperative is to state what technical assumptions have been made, and to
communicate clearly the limitations of the analysis.
Conclusions
The beguilingly simple notion of efficiency disguises a series of thorny conceptual

and methodological problems. Setting aside the obvious measurement difficulties, the
structural problem can be illustrated as in Figure 2.3. Naïve efficiency analysis
involves examining the ratio of outputs to inputs (the shaded boxes). Yet system
inputs should also incorporate previous investments (endowments) and exogenous
inputs (such as other organisations and pupil characteristics). And system outputs
should also include endowments for the future, joint outputs and outputs not directly
related to the service under scrutiny, such as enhanced economic productivity arising
from public service performance.
Figure 2.3: The broader context of efficiency analysis
Joint outputs: Outputs External outputs:

Research, training Year t Productivity
Etc. Etc.
Endowments Organization Endowments

Exogenous inputs: System constraints:

Other organizations Policy constraints
Population characteristics Inputs Physical constraints
Etc. Year t Etc.
26
It will never be feasible to accommodate all the issues summarised in Figure 2.3 into
an efficiency analysis. Rather, the analyst should be aware of which factors are likely
to be material for the application under consideration, and seek to offer guidance on
the implications of serious omissions from the efficiency model.
27
3 Data and descriptive analysis
The variables used to analyse the performance of English secondary schools are listed
in Table 3.1, with summary statistics provided in Table 2. The total sample available
for analysis comprises 2,928 schools, out of a possible 2,997. Sixty-nine schools are
omitted because of either missing data (n=55) or extreme – probably erroneous -
values (n=14) for one or more variables.
Table 3.1: Variable descriptions

Variable | Description
y gcsevauncap | GCSE uncapped value added score

teachpp | Number of teachers per 1000 pupils
x1m learnsuppp | Number of learning support staff per 1000 pupils
adminclpp | Number of administrative and clerical staff per 1000 pupils
eLearnITpp | Expenditure on Learning and ICT resources per 1000 pupils
e_exp | Total expected expenditure
x2
nfsm | % pupils not eligible for free school meals
zk nsen | % pupils without special education needs
nsenwout | % pupils without special education needs statements
eal | % pupils with English as additional language
eal2 | eal squared
eal3 | eal cubed
cat_sixth | If 1, school has a sixth form
f
cat_religi~s | If 1, religious domination school
dd cat_rural | If 1, school is rural
cat_eicclu~s | If 1, school in excellence in city programme
cat_specia~s | If 1, specialist school
cat_bursar | If 1, school has a bursar
avftepupils | Average number of full time equivalent pupils
s
The key performance measure (y) is the GCSE uncapped value added score
(gcsevauncap), measured for each school in academic year 2003-04. This captures
the "value added" attainment of pupils as it compares their key stage 4, GCSE results,
against the average level that they would be expected to attain given their key stage 3
attainment scores (sat 2 years earlier). The measure is termed “uncapped”, because it
captures the total sum of all GCSE’s that a pupil sits as opposed to the “capped”
measure which only includes a pupil’s top 8 GCSE results. This measure is
standardised around 100, as it is measured as deviations from the average expected
attainment plus 100 to avoid negative numbers. A unit increase in the measure is
equal to one GCSE grade on this scale. As table 3.2 shows, the measure is subject to
wide variation, ranging from 88.5 to 128.2.
Table 3.2. Descriptive statistics

Variable | Obs Mean Std. Dev. Min Max
y gcsevauncap | 2928 99.83064 4.562992 88.5 128.2

teachpp | 2928 62.22643 5.568889 46.59661 94.8949
x1m learnsuppp | 2928 15.98078 6.73287 .5590622 57.53579
adminclpp | 2928 7.850698 2.568798 1.227646 30.54874
eLearnITpp | 2928 237.4525 93.42784 38.37225 1040.047
e_exp | 2943 2476.203 340.7276 1129.734 11487.96
x2
nfsm | 2928 84.75535 13.28694 18.93 99.89
zk nsen | 2928 83.6953 9.297719 38.77 100
nsenwout | 2928 97.51585 1.644732 87.76 100
eal | 2928 9.023672 16.82746 0 99.65301
cat_sixth | 2928 .5717213 .4949138 0 1
f
28
cat_religi~s | 2928 .1653005 .371515 0 1
dd cat_rural | 2928 .1410519 .3481345 0 1
cat_eicclu~s | 2928 .0502049 .2184048 0 1
cat_specia~s | 2928 .3934426 .488597 0 1
cat_bursar | 2928 .8295765 .3760687 0 1
avftepupils | 2928 1033.487 335.7743 174 2622.5
s
A number of factors might have influence on the attainment of pupils in each school
and explain this variation in GCSE results, one of which is the relative efficiency of
secondary schools. The purpose of the analysis is to identify these influences. We
separate observable factors into two broad categories: inputs of production and
exogenous constraints upon production.
Inputs (x’s) include staffing and resources such as teaching materials and equipment.
Note that no measures of capital inputs are included. This means that the analytical
focus is on the efficiency of how schools utilise their recurrent funding streams, not
their capital budgets.
Inputs can be considered in physical terms, from which it is possible to assess

technical efficiency, or by taking account of their relative value, in order to estimate
allocative efficiency.
In our technical efficiency model specifications, inputs appear as a vector physical

units (x1), measuring the number of teachers (teachpp), the number of learning
support staff (learnsuppp), the number of administrative and clerical staff
(adminclpp), and expenditure on learning resources (including ICT learning
resources) (ave19e20pp, renamed eLearnITpp). All of these variables are measured
per 1,000 pupils in each school. The source of the data on staff input is the PLASC
return, and information is averaged over academic years 2002-2003 and 2003-2004.
Expenditure on learning and IT learning resources is derived from the Consistent
Financial Reporting (CFR) returns. On average, a school employs 62 teachers, 16
learning support staff and 8 administrative and clerical staff per 1,000 pupils.
In our allocative efficiency models, total expected expenditure on inputs is included

( x2 ). Total expected expenditure is calculated by weighting staff numbers by their
respective wage rate, and summing across input categories (including eLearnITpp).
National wage rates are listed in table 3.3. By taking account of the relative price of
inputs insight can be gained into the extent to which schools might be mis-allocating
resources across input types.
Table 3.3: Salary assumptions

Average salary Notes
Secondary teachers £30,000 FTE average salary of regular qualified
classroom teachers in maintained secondary
schools, 13 March 2004. Source: DTR
Admin & Clerical £13,932 FTE average annual salary for school
secretaries. Source: ASHE 2004, code 4213
Education Support £11,602 FTE average annual salary for educational
assistants. Source: ASHE 2004, code 6124
Figure 3.1 plots the relationship between GCSE results and each physical input and
expenditure on learning resources. There is no visually obvious relationship between
these measures, although the correlation coefficients reported in Table 3.4 suggest a
29
positive, but weak, relationship between GCSE results and the number of teachers
(r=0.1) and expenditure on learning resources (r=0.16).
Figure 3.1: Relationship between GCSE scores and inputs

(a) GCSE score and teaching staff
130 (b) GCSE scores and learning support staff
130
120
120
gcsevauncap
gcsevauncap
110
110
100
100
90
90
40 60 80 100 0 20 40 60
teachpp learnsuppp
(c) GCSE scores and admin/clerical staff (d) GCSE scores and expenditure on learning &
ICT learning resources
130
130
120
120
gcsevauncap
gcsevauncap
110
110
100
100
90
90
0 10 20 30 0 200 400 600 800 1000

adminclpp ave19e20pp
Table 3.4 Correlation coefficients

| gcseva~p teachpp learns~p adminc~p eLearn~p nfsm eal nsen
-------------+---------------------------------------------------------------------
gcsevauncap |
teachpp | 0.1004*
learnsuppp | 0.0036 0.3188*
adminclpp | 0.0125 0.2167* 0.4156*
eLearnITpp | 0.1698* 0.2627* 0.2497* 0.2246*
nfsm | -0.1053* -0.4319* -0.4594* -0.1930* -0.2645*
eal | 0.2833* 0.3410* 0.1360* 0.0798* 0.1644* -0.5638*
nsen | 0.0248 -0.3191* -0.5374* -0.2481* -0.1577* 0.5916* -0.2271*
nsenwout | 0.0769* -0.2282* -0.5149* -0.1861* -0.0656* 0.3195* -0.0361 0.5049*
* statistically significant at p<0.05
Schools face a number of constraints in helping pupils to attain high GCSE results.
We take account of some of these constraints, by considering measured characteristics
of pupils in each school. These variables are expressed in a form such that an increase
in their value is expected to lead to an increase in GCSE results (a characteristic
termed “positive isotonicity”, which is a required condition for Data Envelopment
Analysis). The variables are:
• The percentage of pupils in school who are not eligible for free school meals
(nfsm). It is hypothesised that the higher this percentage, the better the GCSE
30
scores are likely to be. Figure 3.2a and a weakly negative correlation coefficient
(r=-0.1) suggest this hypothesis might be misplaced however.
• The percentage of pupils that speak English as an additional language (eal). The
relationship with GCSE results is difficult to anticipate. Initially, such children
might be constrained by the language barrier, and schools might need to devote
additional resources to these pupils, implying that lower levels of attainment are
achievable the higher the proportion of such pupils. Once the barrier has been
overcome, the attainment of such children should be similar to that of other
children. Within schools, attainment may even be above average, the reason being
that pupils with English as an additional language are often concentrated in
particular schools that offer a different – and perhaps more conducive - learning
environment. If so, the direction of influence of this variable on attainment might
be positive. Figure 3.2b and the significantly positive correlation coefficient
(r=0.28) suggest that schools with a higher percentage of pupils with English as
an additional language are likely to have above average value added GCSE results.
In our regression analyses, we explore a flexible functional form to capture the
relationship between attainment and this variable, by including higher order
powers of eal. A F test is conducted to test the significance of the relationship.
This flexibility does not prove important for schools without 6th forms, but
improves the model of technical efficiency for schools with 6th forms.
• The percentage of pupils that do not have Special Education Needs (nsen) nor
Special Education Needs with statements (nsenwout). The higher the proportion
in each category, the better the value added GCSE scores are likely to be. To see
what these two variables capture and how they are related, consider figure 3.3
below. nsen captures an average of 85% of pupils, represented as area A, who are
not considered a constraining factor on achievement. nsenwout captures an
average of 96% of pupils, represented as area A+B, and this variable represents a
“medium” constraining factor. The omitted (reference) group, area C, are pupils
with SEN statements. nsen and nsenwout are correlated, though not strongly so
(r=0.5). Correlation is to be expected, given that nsen is a subset of nsenwout.
Finally, we have a set of school level characteristics, indicating such things as

whether or not they have a 6th form, are in rural or urban locations, or are specialist
schools. These take the form of dummy variables, with a value 1 if the school has the
characteristic in question, 0 otherwise.
31
Figure 3.2: Relationship between GCSE scores and pupil characteristics
(a) GCSE scores and % no free school meals (b) GCSE scores and % EAL
130
130
120
120
gcsevauncap
gcsevauncap
110
110
100
100
90
90
20 40 60 80 100 0 20 40 60 80 100
nfsm eal
(c) GCSE scores and % no SEN (b) GCSE scores and % no SEN without statement
130
130
120
120
gcsevauncap
gcsevauncap
110
110
100
100
90
90
40 60 80 100 85 90 95 100
nsen nsenwout
Figure 3.3 Percentage of pupils with SEN and SEN statements
A B C
with
statements
without
Pupils:
SEN
SEN
SEN
% 85 96 100
32
4 Regression analysis
Modelling strategy
Any statistical analysis requires a modelling strategy in order to guide the choice of
recommended model. In this study, numerous choices had to made, including:
• Whether to examine the entire sample, or to model separately schools with and
without sixth forms;
• What type of functional form to employ;
• Which explanatory variables to include;
• Whether to use ordinary least squares or stochastic frontier methods;
• Whether to model the hierarchical organisation of schools within local authorities.
A summary of our modelling strategy is presented in figure 4.1. This summarises the
various steps taken in analysing the data, and the basis for selecting among modelling
choices.
Model specification
Our baseline model (Model 1) is formulated to explain the variation in GCSE results
among the i schools as a function of the type and amount of staffing and learning
resources, controlling for pupil characteristics. This model takes the form:
M K
yi = α + ∑ β m x1mi + ∑ β k z ki + γ f i + ε i (4.1a)
m =1 k =1
where yi is the GCSE uncapped value added score; x1mi is a vector of m physical
input variables, z ki is a vector of k exogenous variables, and fi is a dummy variable
indicating whether or not the school has a 6th form. All variables appear in linear form
and the model is estimated by ordinary least squares in Stata/SE 8.2.
We performed the Ramsey RESET test, to test for functional form and omitted
variables. A significant test statistic of p<0.05 suggests that the model is misspecified
in some way (Ramsey, 1969). This test indicated that different models should be used
for schools with sixth forms and those without, the main reason being the relationship
between attainment and the percentage of pupils with English as an additional
language. For schools with 6th forms, the relationship between these variables is not
straightforward, and a flexible functional form is required, so higher powers of eal are
included (eal2, eal3). For schools without 6th forms, modelling the relationship in
linear form was sufficient to satisfy the RESET test.
Subject to ε i being non-normally distributed, the above model is also estimated as a

stochastic frontier model by maximum likelihood in Limdep (Greene, 1995). This
procedure partitions the error term into two components: vi which captures random
(stochastic) error and ui which captures inefficiency in production.
M K
yi = α + ∑ β m x1mi + ∑ β k z ki + γ f i + vi − ui (4.1b)
m =1 k =1
33
Figure 4.1: Schema of modelling strategy
Modelling decision Basis for decision
Pooled or split sample? t test of 6th form dummy

variable
Choice of functional RESET test;

form: Skew test of OLS residual
linear, logarithmic,
translog
Choice of explanatory RESET test

variables
Inclusion of higher order F test

powers for EAL?
OLS estimation of
models
M1, M2, M3, M4
Stochastic frontier Skew test of OLS residual

estimation of models
M1, M2, M3, M4
Consideration of Local Breusch-Pagan Lagrangian

Authority effects multiplier test
If ε i is normally distributed, all residual variance is interpreted as arising from

random noise and measurement error: in other words, there is no evidence of
inefficiency in the sample (Schmidt and Lin, 1984, Wagstaff, 1989).
34
Model 2 replaces the vector x1mi with total expected expenditure x2i , in which the
number of staff in each category is weighted by the average rate of pay.
K
yi = α + x2i + ∑ β k z ki + γ f i + ε i (4.2a)
k =1
K
yi = α + x2i + ∑ β k z ki + γ f i + vi − ui (4.2b)
k =1
Model 3 drops all input variables, and includes the vector z ki , with the specification
taking the form:
K
yi = α + ∑ β k z ki + γ fi + ε i (4.3a)
k =1
K
yi = α + ∑ β k z ki + γ fi + vi − ui (4.3b)
k =1
This parsimonious specification includes only those variables that are considered
exogenous constraints on production. It has been argued that, in efficiency analysis,
all variables that are endogenous to the production decision should be excluded
(Giuffrida et al., 2000). So, the choice about the technical production process, such as
what level and mix of staffing to employ, should be excluded from the model on the
grounds that these are decided by the organisation. The analytical task is to construct
a behavioural function that reflects the feasible production possibilities within a
constrained environment (Smith, 1981). Of course, the choice of what constitute
exogenous variables and over what timeframe these constraints are binding is likely to
be contentious. Recognising this, we specify a fourth model, in which school size s is
introduced, along with a vector of dummy variables d d capturing various school
characteristics:
K D
yi = α + ∑ β k z ki + γ f i + δ1 si + δ 2 si2 + δ 3 si3i + ∑ β d d di + ε i (4.4a)
k =1 d =1
K D
yi = α + ∑ β k z ki + γ f + δ1 si + δ 2 si2 + δ 3 si3i + ∑ β d d di + vi − ui (4.4b)
k =1 d =1
After omitting outliers, these models were estimated in linear form for the full sample
of schools, and separately for schools with 6th forms and for schools without 6th forms.
There are sound theoretical arguments for estimating production functions specified in
Cobb-Douglas or translog form (Christensen et al., 1973, Intriligator, 1978). These
functional forms entail transforming variables into logarithmic form and including
35
higher order powers of the explanatory variables. These specifications may provide
additional insights into the relationship between GCSE results and the explanatory
variables.
However, transformation in this way is less likely to yield models that are estimable
as Stochastic Frontier models, because the composite error ε i is less likely to be
skewed, for two reasons. First, logarithmic transformation tends to normalise the
variables that are transformed and, by implication, the unexplained residual will also
approach normalisation (Cook and Weisberg, 1982). Second, the more explanatory
variables included in the model and the more flexible their functional form, the less
unexplained variance there is to be captured by the error.
Finally, we consider the hierarchical nature of the data, whereby schools are clustered
within Local Authorities. School performance may be partially influenced by the LA
in which it is based, because of the influence that the LA has over school funding and
policy. Schools, therefore, are likely to share closer similarities to other schools
within the same LA, than to schools elsewhere. Importantly for estimation, this
clustering implies that schools cannot be considered independent observations. To
account for this inter-dependence among schools, we estimate a two-level fixed
effects model, where yij represents the GCSE uncapped value added score of the ith
within the jth LA and κ j captures the LA fixed effect:
M K
yij = α + ∑ β m x1mij + ∑ β k z kij + γ f ij + κ j + ε ij (4.5)
m =1 k =1
The Breusch-Pagan Lagrangian multiplier test is conducted to determine whether

clustering is important. A significant χ 2 statistic implies that the fixed effects
estimator is preferable to OLS.
Estimation results
Model 1:Technical efficiency
OLS results for the full sample are shown in the first set of columns in Table 4.1
below. This model passes the Ramsey RESET test, suggesting that the model is not
misspecified – ie there are no omitted variables. There are a number of notable
features:
• The number of staff, of whatever category, per pupil does not explain variation
in GCSE results among schools. This may be due partly to the limited
variation among schools in staffing levels (the standard deviations reported in
Table 3.2 are relatively small). It may also be indicative of endogeneity
between performance and staffing levels. On the one hand, pupils in schools
that are better resourced may have higher levels of achievement – so resource
levels explain achievement. On the other hand, schools with low achievement
may be provided with extra resources to help them improve their performance,
in which case achievement explains the level of resource.
36
• Higher expenditure on learning resources (including ICT learning resources,
eLearnITpp) is significantly related to higher GCSE results.
• Higher GCSE results are associated with a higher proportion of pupils who do
not receive free school meals (nfsm), who speak English as an additional
language (eal), and who do not have special education needs (nsen,
nsenwout).
• Schools with 6th forms achieve lower GCSE results than those without.
• The residual was not skewed. This has two implications. First, it rules out
Stochastic Frontier analysis because, in the absence of a skewed error term,
decomposition is unfeasible, and the entire error must be interpreted capturing
random noise. In other words, under this specification, there is no evidence of
differential efficiency among schools. Second, given that the primary interest
is in extracting estimates of efficiency rather than the parameter estimates,
exploration of logarithmic or translog functional forms need not be pursued.
Table 4.1: Model 1

All schools With 6th forms Without 6th forms
Coeff SE t Coeff SE t Coeff SE t
teachpp .027 .018 1.49 .038 .023 1.64 .006 .028 0.20
learnsuppp .009 .017 0.51 .005 .024 0.22 .017 .024 0.71
adminclpp -.035 .037 -0.96 -.074 .050 -1.49 -.013 .054 -0.25
eLearnITpp .007 .001 6.76 .007 .001 5.59 .006 .002 3.87
nfsm .027 .011 2.36 .017 .018 0.94 .035 .014 2.45
eal .082 .030 2.75 .097 .042 2.31 .100 .009 10.59
eal2 -.001 .001 -0.79 -.002 .002 -1.29
eal3 .000 .000 1.31 .000 .000 1.54
nsen .030 .014 2.10 .043 .018 2.42 .016 .022 0.72
nsenwout .170 .066 2.59 .218 .096 2.26 .109 .090 1.22
cat_sixth -.390 .181 -2.15
_cons 74.8 6.371 11.74 69.0 9.306 7.42 82.2 8.92 9.21
obs 2928 1674 1254

R2 0.117 0.098 0.146
p RESET 0.661 0.658 0.582
p F test eal, eal2, eal3 0.000 0.000
OLS results for schools with sixth forms are presented in the second set of columns in
Table 4.1. Compared to estimation results for the full sample, the following is worth
noting:
• For schools with 6th forms, the proportion of pupils that do not have free
school meals does not explain variation in GCSE results
OLS results for schools without 6th forms are shown in the final set of columns of
Table 4.a. The main difference between these results and those for the full sample is
that:
• eal appears in linear form only.
37
• There no longer appears to be a relationship between GCSE results and the
proportion of pupils who do not have special education needs (nsen,
nsenwout)
Rather than reproducing all of the regression results from the other models in the main
text, a summary of the significant findings is contained in Table 4.2 below, in which
the symbols can be interpreted as:
• 0 : non-significant relationship (p>0.05)
• + : significant positive relationship (p<0.05)
• - : significant negative relationship (p<0.05)
• F tests are performed to ascertain if the size variables are jointly significant
(p<0.05)
• RESET p / f : pass / fail Ramsey RESET test
• SFA no : ε i normally distributed, so stochastic frontier analysis not possible
Full results for these models are provided in Appendix 1.
Table 4.2: Summary of regression results

Full sample 6th form No 6th form
Eq 1 2 3 4 1 2 3 4 1 2 3 4
teachpp 0 0 0
learnsu 0 0 0
admin 0 0 0
eLearnIT + + +
e_exp + + +
nfsm + + 0 0 0 0 0 0 + + 0 0
eal,2,3 + + + + + + + +
eal + + + +
nsen + + + 0 + + + 0 0 0 0 0
nsenw/o + + 0 + + + + + 0 0 0 0
6thform - - 0 0
size,2,3 0 0 0
religio + + +
rural + + 0
excel 0 0 0
special + + +
bursar 0 0 0
RESET p f f p p f f f p p p p
SFA No No No No No No No No No No No No
The main findings of interest are:

• For schools with 6th forms, all but Model 1 appear misspecified.
• The choice between x1 ji (Model 1) and x2i (Model 2) has no qualitative
impact on the results.
• Omission of the input vector x1 ji (Model 3) results in a misspecified model for
the full sample, and for both the full sample and for schools without 6th forms,
the significance of nfsm and nsenwout is now eliminated. Qualitative results
for schools with 6th forms are unaffected.
38
• Inclusion of additional school characteristics d d (Model 4) eliminates the
significance of nsen and, for schools without 6th forms, nsenwout.
• Size is not a significant explanator of GCSE results.
• Under Model 4, religious and specialist schools are associated with higher
GCSE results. Rural schools also appear to perform better than urban schools,
but this holds only for the full sample and schools with 6th forms, where this
model appears misspecified.
• None of the specifications produces a skewed residual, ruling out the
possibility of stochastic frontier analysis.
Finally, we report results of estimating equation 4.5 for schools with and without 6th
forms in table 4.3 below. This clustering is clearly important for schools with sixth
forms, as indicated by the significance of the Breusch-Pagan test, p < 0.000 . The LA
effect appears less important for schools without sixth forms, where p = 0.046 . The
apparent differential importance of the LA effect may partly explain why the OLS
models fail the RESET test when applied to schools with 6th forms, but not appear
misspecified for schools without 6th forms.
Other notable points are:

• around 25% of the variation in the data is due to variation among Local
Authorities (rho=0.266 and 0.246)
• nsen and nsenwout no longer appear significant explanators of performance
for schools with 6th forms. This could be taken as an indication that schools
are compensated adequately to overcome these additional education needs, as
discussed earlier (page 25).
• estimation results for schools without 6th forms are not qualitatively different
from those reported in Table 4.1.
Table 4.3: Model 1 – with LA fixed effects

With 6th forms Without 6th forms
Coeff SE t Coeff SE t
teachpp .044 .025 1.76 .006 .029 0.22

learnsuppp .004 .027 0.13 .043 .028 1.57
adminclpp -.043 .052 -0.84 -.023 .056 -0.41
eLearnITpp .008 .001 5.51 .004 .002 2.33
nfsm .041 .019 2.12 .086 .019 4.41
eal .027 .044 0.61 .092 .012 7.75
eal2 -.001 .001 -0.99
eal3 .000 .000 1.98
nsen .026 .020 1.32 -.018 .021 -0.86
nsenwout .137 .110 1.24 .111 .108 1.02
_cons 76.0 10.60 7.17 80.95 11.04 7.33
obs 1674 1254

R2 within 0.064 0.061
R2 between 0.073 0.209
R2 overall 0.080 0.117
sigma u 2.460 2.429
sigma e 4.087 4.248
rho .266 .246
p Bruesch-Pagan test 0.000 0.046
39
5 Data envelopment analysis
Data envelopment analysis is based on similar economic principles to the parametric
methods described above, but employs a dramatically different estimation method,
using linear programming methods. In effect, it draws an ‘envelope’ around the most
extreme (best performing) schools, and estimates the performance of all other schools
relative to that envelope.
DEA assesses efficiency in two stages. First, a frontier is identified based on either
those organisations using the lowest input mix to produce their outputs or those
achieving the highest output mix given their inputs (the input or output orientation).
Second, each organisation is assigned an efficiency score by comparing its
output/input ratio to that of efficient organisations that form a piece-wise linear
‘envelope’ of surfaces in multi-dimensional space. If there are M inputs and S outputs,
then the production frontier becomes a surface in M+S dimensional space. The
efficiency of a DMU is the distance it lies from this surface - the maximum extent by
which it could improve its outputs given its current level of inputs (or reduce its
inputs given its current level of outputs).
Efficiency in DEA is therefore defined as the ratio of the weighted sum of outputs of a
DMU divided by a weighted sum of its inputs. A separate linear programme is
estimated for each DMU. It seeks out for DMU0 the set of output weights us and input
weights vm that maximizes the efficiency of DMU0, subject to the important constraint
that – when they are applied to all other DMUs – none can have an efficiency greater
than 1. The weights can take any non-negative value, and in general a different set of
weights is computed for each DMU. Thus, the weights us and vm are a central feature
of DEA. They are chosen to cast the DMU in the ‘best possible light’, in the sense
that no other set of weights will yield a higher level of efficiency.
In creating the efficient frontier, DEA selects a subset of ‘extreme’ DMUs that are
deemed efficient. The frontier is merely a linear combination of these efficient DMUs,
so that every inefficient DMU is assessed relative to some linear combination of
efficient units. The chosen efficient units are referred to as its ‘peers’.
DEA also yields specific input or output targets for each DMU, depending on whether
the input or output orientation has been used. For example, under input orientation,
these indicate the specific amounts by which a particular DMU should be able to
reduce its consumption of particular inputs without reducing output. DEA preserves
the input-output mix of DMU, which is therefore compared to a linear combination of
efficient peers that uses similar or identical input-output ratios, but at more efficient
levels.
One final detailed point is that in its original form DEA assumes all inputs and
outputs are measured in ‘absolute’ units, reflecting the size of the DMU. However, in
this application, inputs and outputs are measured as ratios (e.g. staff per pupil), which
are independent of school size. As explained by Hollingsworth and Smith
(Hollingsworth and Smith, 2003), it is important under such circumstances always to
use the modification of DEA advocated by Banker, Charnes and Cooper (Banker et al.,
1984).
40
Model specification
Three DEA models were run using software developed and used by Peter Smith over
a twenty year period. This software is written in FORTRAN, and employs a
Numerical Algorithms Group (NAG) optimization routine. It has been successfully
used in numerous academic and commercial DEA applications since 1984, although it
has not been marketed commercially. Where direct comparison has been possible, the
results secured in this study agree completely with the PIMSOFT results.
All three DEA models included the single output, gcsevauncap, but differed
according to how inputs were specified. These models are directly analogous to the
regression models described in section 4 (equations 4.1, 4.2 and 4.3).
Models 1 and 2 were also estimated as both output-oriented and input-oriented models.
An output-orientation implies that inputs are fixed, and the objective is to maximise
output with these fixed inputs. The input-orientation is analogous to a situation in
which a school has to achieve a target level of attainment, and the task is to minimise
the inputs utilised in meeting this target. The estimates of inefficiency for schools
below the frontier will be sensitive to choice of orientation.
The first model includes the vector of x1 ji is a vector of j physical input variables, and
the hypothesised constraints on production, represented as vector z k , as follows:
gcsevauncap
(5.1)
teachpp,adminpp,learnpp,eLearnITpp, zk
Referring back to section 1, the intention is to identify technical efficiency, without

reference to whether the optimal mix of inputs is being deployed by the schools.
The second DEA model uses total expected expenditure (e_exp) as an input. This is
calculated by aggregating information about physical resources, using relative factor
prices reported in Table 3 as weights.
gcsevauncap
(5.2)
Expected expenditure, zk
The purpose of this model is to identify the ‘overall’ efficiency of the school, given
the inputs it consumes. It will be the product of technical and allocative efficiency.
The third DEA model estimates attainment, controlling for production constraints.
This can be estimated assuming an output-orientation only, given that these
constraints are assumed fixed, in the sense of being beyond the control of the school:
gcsevauncap
(5.3)
zk
This model therefore indicates the extent to which schools secure the best possible
results given their external environments.
41
Estimation results
These models were estimated for schools with and without 6th forms. This generated
the following results:
Table 5.1: DEA efficiency estimates
Model 1 1 2 2 3
Orientation input output input output output
With 6th form

mean 0.884 0.907 0.832 0.876 0.855
min 0.582 0.764 0.535 0.738 0.738
100% 196 189 60 60 34
Without 6th form

mean 0.899 0.911 0.855 0.879 0.861
min 0.646 0.761 0.525 0.761 0.761
100% 198 192 62 62 32
Model 1 indicates the extent to which schools are technical inefficient. Referring back
to Figure 1.9, the ratio OPT/OP for the average school with a 6th form amounts to
0.884 – in other words the average school is operating 88.4% of the level of efficiency
achieved by the most efficient schools.
Model 2 indicates the total extent of (technical and allocative) inefficiency, OPA/OP
in Figure 1.9. School specific estimates of allocative inefficiency can therefore be
derived as the ratio of efficiency estimates from model 2 and model 1, which captures
the distance OPA/OPT. Taking the input orientation for schools with 6th forms, average
allocative efficiency is calculated as 0.832/0.884=0.941 - thus, the average level of
allocative efficiency among these schools is estimated as being 94.1%. 60 schools
with 6th forms are allocatively efficient and one school has an estimated level of
allocative efficiency of 75.4%, illustrating the most inefficient mix of inputs in this
sub-sample. Summarised details for the two samples of schools under input and
output oriented models are provided in table 5.2.
Table 5.2: Allocative inefficiency
Allocative efficiency
Orientation input output
With 6th form

mean 0.941 0.966
min 0.754 0.794
100% 60 81
Without 6th form

mean 0.952 0.965
min 0.674 0.802
100% 62 115
42
The allocative efficiency results may depend on the weights attached to each of the
physical inputs. We have adopted what we consider to be reasonable weights that
reflect estimates of the national average unit cost of each input (predominantly the
different categories of staff) – as shown in Table 3.3. The results are not sensitive to
minor perturbations of these weights. However, if radically different weights were
adopted, these could alter estimates of allocative efficiency.
For example, purely for illustrative purposes, we adopted a weighting system that
assumed equal weights for all categories of staff, and zero weight for non-staff inputs.
For the input-oriented model, this resulted in some quite large changes in estimates of
allocative efficiency. For schools with a sixth form, the average level of allocative
efficiency dropped a modest 0.9%. However, changes for individual schools ranged
from an increase of 17.4% to a decrease of 14.5%. The equivalent average change in
allocative efficiency for schools without sixth forms was from 95.2% to 92.2%, with
individual changes varying from an increase of 14.8% to a decrease of 16.0%. We
would not suggest that such a radical change in weights could ever be justified. But
these results do indicate sensitivity amongst some schools to the choice of weights.
Finally, we can consider the impact that choices about input use or expenditure have
on efficiency, over and above the set of exogenous constraints. This involves
comparison of the estimates from models 3 and 1 and models 3 and 2.
Table 5.3: The impact of environmental constraints
Constraining factors Inputs Expenditure
With 6th form

mean 0.943 0.976
min 0.755 0.814
100% 55 303
Without 6th form

mean 0.946 0.980
min 0.769 0.791
100% 85 526
Taking schools with 6th forms as an example and focusing on the impact of
expenditure, the results confirm the crucial importance of the external constraining
factors in explaining efficiency. Adding expenditure to the model improves average
efficiency by only 2.4% (1-0.976).
43
6 Commentary and next steps
This report has laid out the foundations for efficiency analysis and described the
cross-sectional data for 2003-04 available for all secondary schools in England. We
have analysed these data using regression methods and data envelopment analysis. An
important focus of the work was a desire to explore levels of technical and allocative
efficiency. However, in addition to addressing these issues, the analysis has brought
to light a number of fundamental modelling issues that have an important impact on
results.
The regression analysis provides insight into the influence of explanatory variables on
GCSE results. We describe the numerous modelling choices that must be made in
developing such models, and present only a subset of possible specifications. These
are deliberately chosen to be parsimonious, and are estimated in linear form only. It
was not possible to undertake Stochastic Frontier Analysis of these models, because
the composite residual was not skewed.
Data envelopment analysis has been conducted using our own in-house programme.
The results of our analyses will be compared to those produced by the PIMSoft
package, the development of which has been supported by the Department for
Education and Skills. Reassuringly, results for the models compared to date are
identical.
The analysis offers estimates of technical and allocative inefficiency in the secondary
school sector. It suggests that levels of allocative efficiency are relatively high, and
that – on average – the more important element of school inefficiency is technical
inefficiency.
In undertaking the analysis, we have identified a number of modelling issues that in

our view deserve further analysis: enhancing the data set; the use of SFA; endogeneity;
and multilevel modelling. We conclude by discussing these in turn.
The dataset
As is always the case, the variables available for this analysis are limited in scope.
Clearly more robust and persuasive results could be secured by expanding the dataset.
Most notably, we had available no measure of capital input, and this may explain
some of the variations in efficiency we identify.
Furthermore, we had available only a cross-section of data from one academic year.
For the parametric modelling, extension of the dataset to a longer series of years
would yield more robust estimates, although it would also introduce new modelling
challenges, for example requiring decisions about how to model the changes in
efficiency over time.
If suitable time series data are available, they can also be used to explore productivity
gains of schools over time. In particular, Appendix 2 illustrates how DEA can be used
to decompose changes of efficiency over time into technological progress and
efficiency gains, employing a Malmquist index approach. This report has
concentrated on the role of efficiency models in offering insights into the efficiency of
44
individual schools. However, the Malmquist approach may be a useful focus for
future work if there is also a policy interest in efficiency gains for the whole
secondary school sector.
Stochastic frontier analysis and model specification
We have been unsuccessful in being able to develop SFA models, because none of the
models examined exhibited the requisite asymmetric error term required to model
‘inefficiency’ in SFA.
When undertaking SFA from a regulatory perspective, a necessary condition is that all
variables included as regressors are indicators of environmental factors beyond
organisational control (Giuffrida and Gravelle and Sutton, 2000). This contrasts with
the traditional approach to statistical model building, where the aim is to select a set
of explanatory variables that best explain variation in the dependent variable. It is
quite likely that many potential explanatory variables are indicators of both
environmental effects (which we would wish to include in the model) and policy or
efficiency effects (which we would wish to exclude from the model). Most statistical
modelling of efficiency relies on traditional variable selection devices to test whether
an environmental variable should be included in the model.
This approach cannot be used in SFA, because the analyst must make a joint decision
regarding the variables to include and the model’s error structure. Unexplained
variation from predicted output is decomposed into two parts: symmetric random
error and one-sided inefficiency. Suppose therefore that we wish to test an
environmental variable for inclusion in a SFA model. It will be a candidate for
inclusion if it ‘explains’ a material proportion of the overall residual and therefore
exhibits what is conventionally termed a statistically significant model coefficient.
However, the attribution of a statistically significant effect to the additional
explanatory variable may be for one or both of the following reasons:
1. It explains some of the random error. This implies that the original model suffered
from omitted variable or functional form misspecification.
2. It explains some of the inefficiency error. This implies that the variable is
correlated with the original estimates of inefficiency.
Whether the new variable should be included depends on whether it is judged to
measure an unavoidable hindrance to reaching the estimated frontier (in which case it
would be included) or some potentially controllable characteristic of inefficient
organisations (in which case it would be omitted). There is no scientific guide to
making this judgement. It can be made only with reference to the context of the
specific production process under scrutiny.
Endogeneity
Underlying most models of productivity is an implicit assumption of a straightforward

causality from inputs to outputs, and few published efficiency studies have examined
the implication of assumptions other than this simple relationship between inputs and
outputs. Yet, although common sense suggests an impact of inputs on outputs, it is
equally plausible, at least in the public sector, to suggest that there is a simultaneous
feedback from outputs to inputs. When output is measured by well-publicized
performance indicators such as examination pass rates, such feedback is likely to be
45
an intrinsic part of any model of performance. The feedback may arise, for example,
because of the importance attached amongst public sector DMUs to securing national
targets.
In particular, schools for which published examination results are poor are likely to
come under pressure - from residents, central government and other political
sources - to promote improved performance. One manifestation of a high political
priority being given to schooling is likely to be increased resource provision. Thus, as
well as the common sense positive causality from inputs to outputs, there is also
likely - using the terminology of systems dynamics - to be a negative feedback from
outputs to resources. This leads to one or more input variables being dependent on
output measures, and therefore being endogenous. Other things being equal, this
process will lead to a tendency for educational output levels to converge, as the desire
for equity leads to poorly performing schools securing more resources, even if such
activity is not allocatively efficient. Smith calls this process negative political
feedback (Smith, 1995).
When using econometric methods, the failure to model endogeneity may have
profound policy implications in two important areas. First, it may result in the
misspecification of any model which is employed to estimate the production function.
As a consequence of this, faulty inferences may be made about the importance of
input variables as determinants of those output possibilities which are of particular
interest. And second, the incorrect specification of the production function may lead
to errors in assessments of the managerial efficiency of individual organizations.
Econometricians have developed various approaches, such as instrumental variable
methods, to address such issues, although there have been few applications in the
productivity literature.
In contrast, it has usually been assumed that endogeneity causes no material problems
in DEA, since it is a boundary technique that merely models feasible relationships
between inputs and outputs. Orme and Smith challenge the assumption that
endogeneity is always irrelevant to DEA (Orme and Smith, 1996). They introduce the
notion of sparsity bias in DEA efficiency assessments, and suggest that this may
become problematic when the number of observations is small.
The implications of endogeneity for regression analysis are illustrated in Figure 6.1,
which shows the use of x (resources) and s (environmental input) for a given level of
output. The truly efficient frontier is indicated by the isoquant FF. All observations
lie within the frontier, but, because of the link between inefficiency and resource
inputs, there is a tendency for DMUs with low levels of input x to be closer to FF than
those with higher resource input levels. Instead of being parallel to FF, therefore,
OLS statistical methods yield a misleading estimate RR of "average" performance.
46
Figure 6.1: True frontier FF and OLS frontier RR with endogeneity
The analogous implication for DEA is that, while the method will certainly indicate
feasible improvements in output, the targets set may be more demanding for DMUs
adopting low levels of resource inputs than for those using high levels of resource
inputs. This phenomenon arises because, as the preceding argument suggests, at high
levels of x there is likely to be a smaller proportion of DMUs close to the true frontier
FF than at lower levels of x. Thus simultaneity introduces into DEA the possibility
that efficiency estimates (and therefore output targets) may be biased, in the sense that
different parts of the estimated DEA frontier are comprised of DMUs with different
levels of true efficiency. Efficiency improvement targets yielded by DEA in the
presence of endogeneity may therefore be unfair, in the sense that inefficient DMUs
using low levels of resources x may be asked to move closer to the truly efficient
frontier than equally inefficient units (as measured by ε) using higher levels of
resources.
Figure 6. 2 illustrates the effect of such endogeneity on DEA. Because of the sparsity
of efficient DMUs at high levels of x, the estimated isoquant EE is further from the
true isoquant FF there than at low levels of x. Thus two DMUs Ph and Pl with similar
levels of efficiency with respect to FF are indicated as having very different levels of
efficiency according to DEA, with Ph being deemed efficient and Pl inefficient. We
term the distorted view of relative efficiency this gives rise to sparsity bias.
47
Figure 6.2: True frontier FF and DEA frontier EE with endogeneity
Smith and Orme examine the relatively favourable treatment of DMUs with high
levels of x using Monte Carlo simulation. They find that endogeneity in DEA can be
problematic when using a small number of observations, but that its importance
declines as the sample size increases. In the context of this schools example,
endogeneity is unlikely to be a major problem when using DEA.
Multilevel modelling
English schools are currently organised hierarchically within local authorities. It

therefore seems to be natural incorporate the hierarchical structure into the parametric
modelling, using multilevel modelling techniques. We have demonstrated the impact of
this modelling choice in section 4, and find important local authority effects on
attainment levels.
Whether the multilevel formulation should be used may depend on the purpose of the
analysis. For example, the use of a simple ‘fixed effects’ multilevel specification
effectively removes the local authority effect from estimates of a school’s efficiency.
This may be useful if the intention is to examine variations within a local authority.
However, it may be less helpful if the intention is to compare schools across jurisdictions.
The fixed effects model also yields estimates of the ‘fixed’ local authority effect on
efficiency across all schools within its responsibility. This may be useful information if
the interest is in examining local authority (rather than school) efficiency. In practice, we
suspect that there will be policy interest in models both with and without the multilevel
structure.
48
Concluding comments
In the light of this discussion, our strongly held view is that – in the absence of
unequivocal evidence on the nature of environmental constraints on schooling – the
use of SFA is premature. The development of parametric models with asymmetric
effort terms is not in our judgement a priority at present, and instead we would
recommend attention is focused on:
• Expanding the data set to embrace more inputs and outputs, covering a greater
number of years;
• Developing statistically robust regression models based on symmetric errors;
• Examining in more detail the issue of endogeneity, and testing whether it is a
material influence on results;
• Developing multilevel models of performance that reflect the hierarchical
nature of school organisation in England, as a complement to the models
described here.
DEA does not suffer from the same potential for endogeneity problems as regression
methods. It also produces useful, school-specific information on levels of technical
and allocative efficiency, and is relatively easy to justify and explain. In the current
context, its main weaknesses are sensitivity to the inclusion of high attainment
outliers, and the lack of tests to identify whether a model is correctly specified. We
should therefore recommend that DEA can form a useful basis for inter-school
benchmarking, but that in doing so the limitations of data and modelling assumptions
need to be very carefully explained. If additional years’ data become available, DEA
could also serve as a basis for examining productivity gains, using the Malmquist
index approach.
Acknowledgement
We should like to thank Darren Pigg for his constructive help and thoughtful
comments throughout the project.
49
References
Allen, R., Athanassopoulos, A., Dyson, R.G., Thanassoulis, E., 1997, Weight
restrictions and value judgements in data envelopment analysis: evolution,
development and future directions, Annals of Operations Research 73, 13-34.
Atkinson, T., 2005, Atkinson Review: Final Report. Measurement of Government
Output and Productivity for the National Accounts (Palgrave Macmillan,
Basingstoke).
Banker, R.D., Charnes, A., Cooper, W.W., 1984, Some models for estimating
technical and scale inefficiencies in data envelopment analysis, Management
Science 30, 1078-1092.
Charnes, A., Cooper, W.W., Rhodes, E., 1978, Measuring the efficiency of decision
making units, European Journal of Operational Research 2, 429-444.
Christensen, L.R., Jorgenson, D.W., Lau, L.J., 1973, Transcendental logarithmic
production functions, Review of Economics and Statistics 55, 28-45.
Coelli, T., Perelman, S., 2000, Technical efficiency of European railways: a distance
function approach, Applied Economics 32, 1967-1976.
Coelli, T., Rao, D., Battese, G., 1998, An Introduction to Efficiency and Productivity
Analysis (Kluwer Academic Publishers, Boston).
Cook, R.D., Weisberg, S., 1982, Residuals and Influence in Regression (Chapman
and Hall, London).
Cooper, W.W., Seiford, L.M., Tone, K., 2000, Data Envelopment Analysis: a
comprehensive text with models, applications and references (Kluwer, Boston).
Doyle, J., Green, R., 1994, Efficiency and cross-efficiency in DEA: derivations,
meanings and uses, Journal of the Operational Research Society 45, 567-578.
Everitt, B., Landau, S., Leese, M., 2001, Cluster analysis (Arnold, London).
Farrell, M.J., 1957, The measurement of productive efficiency, Journal of the Royal
Statistical Society Series A 120, 253-290.
Fried, H.O., Lovell, C.A.K., Schmidt, S.S.e., 1993, The Measurement of Productive
Efficiency (Oxford University Press, Oxford).
Giuffrida, A., Gravelle, H., Sutton, M., 2000, Efficiency and administrative costs in
primary care, Journal of Health Economics 19, 983-1006.
Greene, W.H., 1995, Limdep Version 7.0 User's Manual. Castle Hill: Econometric
Software, Inc.).
Hollingsworth, B., Smith, P.C., 2003, The use of ratios in data envelopment analysis,
Applied Economics Letters 10, 733-735.
Iezzoni, L.I., 2003, Risk adjustment for measuring health outcomes (Health
Administration Press, Baltimore).
Intriligator, M.D., 1978, Econometric Models, Techniques and Applications
(Prentice-Hall Inc, Englewood Cliffs, New Jersey).
Jacobs, R., 2001, Alternative methods to examine hospital efficiency: data
envelopment analysis and stochastic frontier analysis, Health Care Management
Science 4, 103-116.
Lothgren, M., 2000, Specification and estimation of stochastic multiple-output
production and technical inefficiency, Applied Economics 32, 1533-1540.
Orme, C., Smith, P.C., 1996, The potential for endogeneity bias in data envelopment
analysis, Journal of the Operational Research Society 47, 73-83.
50
Pedraja-Chaparro, F., Salinas-Jimenez, J., Smith, P., 1999, On the quality of the Data
Envelopment Analysis model, Journal of the Operational Research Society 50,
636-644.
Ramsey, J.B., 1969, Tests for specification errors in classical linear least squares
regression analysis, Journal of the Royal Statistical Society 31, 350-371.
Salinas-Jiménez, J., Pedraja-Chaparro, F., Smith, P.C., 2003, Evaluating the
introduction of a quasi-market in community care: assessment of a Malmquist
index approach, Socio-Economic Planning Sciences 37, 1-13.
Schmidt, P., Lin, T., 1984, Simple tests of alternative specifications in stochastic
frontier models., Journal of Econometrics 24, 349-361.
Shephard, R.W., 1970, Theory of cost and production functions (Princeton University
Press, Princeton).
Smith, P.C., 1995. Performance indicators and control in the public sector, in: Berry,
A.J., Broadbent, J. Otley, D., eds., Management Control: Theories, Issues and
Practices, (Macmillan, Basingstoke) 163-178.
Smith, P.C., 1997, Model misspecification in data envelopment analysis, Annals of
Operations Research 73, 233-252.
Smith, V.K., 1981, Elasticities of substitution for a regulated cost function, Economic
Letters 7, 215-219.
Wagstaff, A., 1989, Estimating efficiency in the hospital sector: a comparison of three
statistical cost frontier models, Applied Economics 21, 659-672.
51
Appendix 1: Regression results
Full sample
. * Equation 4.2 full sample
Regression with robust standard errors Number of obs = 2928

F( 8, 2919) = 29.91
Prob > F = 0.0000
R-squared = 0.1062
Root MSE = 4.3197
------------------------------------------------------------------------------
| Robust
gcsevauncap | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
e_exp | .0021147 .0003995 5.29 0.000 .0013314 .0028979
nfsm | .0253548 .0112963 2.24 0.025 .0032053 .0475043
eal | .0708859 .0298966 2.37 0.018 .0122654 .1295065
eal2 | -.0006092 .0010647 -0.57 0.567 -.0026969 .0014784
eal3 | .000011 9.24e-06 1.19 0.232 -7.08e-06 .0000292
nsen | .0368197 .0141206 2.61 0.009 .0091323 .064507
nsenwout | .2251183 .0638767 3.52 0.000 .0998703 .3503662
cat_sixth | -.3832179 .178845 -2.14 0.032 -.733893 -.0325427
_cons | 66.99142 6.118748 10.95 0.000 54.99392 78.98892
------------------------------------------------------------------------------
. test eal eal2 eal3
( 1) eal = 0
( 2) eal2 = 0
( 3) eal3 = 0
F( 3, 2919) = 44.85
Prob > F = 0.0000
. ovtest
Ramsey RESET test using powers of the fitted values of gcsevauncap

Ho: model has no omitted variables
F(3, 2916) = 1.25
Prob > F = 0.2895
52
. * Equation 4.3: exogenous variables full sample
. reg gcsevauncap nfsm eal eal2 eal3 nsen nsenwout if outlier==0, robust

F( 6, 2921) = 33.47
Prob > F = 0.0000
R-squared = 0.0954
Root MSE = 4.3442
------------------------------------------------------------------------------
| Robust
-------------+----------------------------------------------------------------
nfsm | .0061649 .0104098 0.59 0.554 -.0142463 .0265761
eal | .0754249 .0300698 2.51 0.012 .0164648 .1343851
eal2 | -.0007394 .001074 -0.69 0.491 -.0028453 .0013665
eal3 | .0000122 9.34e-06 1.30 0.193 -6.15e-06 .0000305
nsen | .0246316 .0139762 1.76 0.078 -.0027726 .0520357
nsenwout | .1535022 .0636668 2.41 0.016 .0286659 .2783385
_cons | 81.60338 5.73392 14.23 0.000 70.36044 92.84631
------------------------------------------------------------------------------
( 1) eal = 0
( 2) eal2 = 0
( 3) eal3 = 0
F( 3, 2921) = 47.00
Prob > F = 0.0000
. ovtest

F(3, 2918) = 0.77
Prob > F = 0.5124
53
. * Equation 4.4: exogenous and categorical variables
. * full sample
. reg gcsevauncap nfsm eal eal2 eal3 nsen nsenwout avfte* cat* if outlier==0,
> robust

F( 14, 2912) = .
Prob > F = .
R-squared = 0.1499
Root MSE = 4.218
------------------------------------------------------------------------------
| Robust
-------------+----------------------------------------------------------------
nfsm | .0019275 .0107489 0.18 0.858 -.0191487 .0230037
eal | .0909546 .0294639 3.09 0.002 .0331824 .1487268
eal2 | -.0012839 .0010441 -1.23 0.219 -.0033312 .0007634
eal3 | .0000164 9.05e-06 1.81 0.070 -1.35e-06 .0000341
nsen | .0164977 .0137138 1.20 0.229 -.010392 .0433874
nsenwout | .1469254 .0621325 2.36 0.018 .0250974 .2687535
avftepupils | -.0057575 .0038721 -1.49 0.137 -.01335 .0018349
avftepsq | 4.00e-06 3.24e-06 1.23 0.218 -2.36e-06 .0000104
avftepcu | -8.24e-10 8.57e-10 -0.96 0.336 -2.50e-09 8.56e-10
cat_religi~s | 1.258029 .2245764 5.60 0.000 .8176845 1.698374
cat_rural | .4247131 .2324907 1.83 0.068 -.0311499 .880576
cat_eicclu~s | .054268 .3817073 0.14 0.887 -.6941756 .8027116
cat_specia~s | 1.986463 .1711574 11.61 0.000 1.650861 2.322065
cat_bursar | -.3067835 .2077901 -1.48 0.140 -.7142139 .1006469
cat_sixth | -.2719149 .1861522 -1.46 0.144 -.6369182 .0930884
_cons | 85.05352 5.664218 15.02 0.000 73.94725 96.1598
------------------------------------------------------------------------------
( 1) eal = 0
( 2) eal2 = 0
( 3) eal3 = 0
F( 3, 2912) = 49.24
Prob > F = 0.0000
. test avftepupils avftepsq avftepcu
( 1) avftepupils = 0
( 2) avftepsq = 0
( 3) avftepcu = 0
Constraint 3 dropped
F( 2, 2912) = 2.06
Prob > F = 0.1273
. ovtest

F(3, 2909) = 1.76
Prob > F = 0.1527
54
Schools with 6th forms
. * Equation 4.2 schools with sixth form
. reg gcsevauncap e_exp nfsm eal eal2 eal3 nsen nsenwout if outlier==0 & cat_s
> ixth==1, robust

F( 7, 1666) = 14.14
Prob > F = 0.0000
R-squared = 0.0854
Root MSE = 4.2728
------------------------------------------------------------------------------
| Robust
-------------+----------------------------------------------------------------
e_exp | .0023284 .0005357 4.35 0.000 .0012776 .0033792
nfsm | .0171531 .0184079 0.93 0.352 -.0189519 .0532582
eal | .0859107 .0425679 2.02 0.044 .0024186 .1694029
eal2 | -.0018642 .0017006 -1.10 0.273 -.0051997 .0014712
eal3 | .0000231 .0000165 1.40 0.162 -9.30e-06 .0000555
nsen | .0528169 .018281 2.89 0.004 .0169607 .0886731
nsenwout | .2941143 .0865426 3.40 0.001 .1243706 .4638581
_cons | 58.7303 8.283696 7.09 0.000 42.48275 74.97785
------------------------------------------------------------------------------
( 1) eal = 0
( 2) eal2 = 0
( 3) eal3 = 0
F( 3, 1666) = 13.53
Prob > F = 0.0000
. ovtest

F(3, 1663) = 0.67
Prob > F = 0.5684
55
. reg gcsevauncap nfsm eal eal2 eal3 nsen nsenwout if outlier==0 & cat_sixth==
> 1, robust

F( 6, 1667) = 13.59
Prob > F = 0.0000
R-squared = 0.0738
Root MSE = 4.2987
------------------------------------------------------------------------------
| Robust
-------------+----------------------------------------------------------------
nfsm | .0014568 .0182116 0.08 0.936 -.0342631 .0371768
eal | .0960775 .0428984 2.24 0.025 .0119372 .1802178
eal2 | -.0020952 .0017034 -1.23 0.219 -.0054363 .0012458
eal3 | .0000249 .0000165 1.51 0.132 -7.50e-06 .0000572
nsen | .0442916 .0182441 2.43 0.015 .0085078 .0800754
nsenwout | .2316594 .0860164 2.69 0.007 .0629479 .4003708
_cons | 72.63389 7.727842 9.40 0.000 57.47659 87.79119
------------------------------------------------------------------------------
( 1) eal = 0
( 2) eal2 = 0
( 3) eal3 = 0
F( 3, 1667) = 14.87
Prob > F = 0.0000
. ovtest

F(3, 1664) = 1.77
Prob > F = 0.1511
56
. reg gcsevauncap nfsm eal eal2 eal3 nsen nsenwout avfte* cat* if outlier==0 &
> cat_sixth==1, robust

F( 13, 1659) = .
Prob > F = .
R-squared = 0.1316
Root MSE = 4.1724
------------------------------------------------------------------------------
| Robust
-------------+----------------------------------------------------------------
nfsm | .0009442 .0178281 0.05 0.958 -.0340237 .035912
eal | .1022575 .0416528 2.45 0.014 .0205599 .1839551
eal2 | -.0022498 .0016387 -1.37 0.170 -.005464 .0009643
eal3 | .0000256 .0000157 1.63 0.104 -5.26e-06 .0000565
nsen | .0319112 .0177897 1.79 0.073 -.0029815 .0668039
nsenwout | .2048671 .0846562 2.42 0.016 .0388228 .3709113
avftepupils | -.0083306 .0060696 -1.37 0.170 -.0202356 .0035743
avftepsq | 5.66e-06 4.72e-06 1.20 0.231 -3.60e-06 .0000149
avftepcu | -1.17e-09 1.17e-09 -1.00 0.317 -3.46e-09 1.12e-09
cat_religi~s | 1.4176 .2943041 4.82 0.000 .840353 1.994846
cat_rural | .6028934 .2904641 2.08 0.038 .0331787 1.172608
cat_eicclu~s | -.6115292 .6147881 -0.99 0.320 -1.817371 .5943129
cat_specia~s | 1.848931 .2147227 8.61 0.000 1.427774 2.270087
cat_bursar | -.1675917 .2800421 -0.60 0.550 -.7168648 .3816814
cat_sixth | (dropped)
_cons | 79.06749 7.787543 10.15 0.000 63.79304 94.34194
------------------------------------------------------------------------------
( 1) eal = 0
( 2) eal2 = 0
( 3) eal3 = 0
F( 3, 1659) = 16.16
Prob > F = 0.0000
( 2) avftepsq = 0
( 3) avftepcu = 0
F( 2, 1659) = 1.53
Prob > F = 0.2176
. ovtest

F(3, 1656) = 3.08
Prob > F = 0.0266
57
Schools without 6th forms
. * Equation 4.2 schools without sixth form
. reg gcsevauncap e_exp nfsm eal nsen nsenwout if outlier==0 & cat_sixth==0, r
> obust

F( 5, 1248) = 33.00
Prob > F = 0.0000
R-squared = 0.1372
Root MSE = 4.3748
------------------------------------------------------------------------------
| Robust
-------------+----------------------------------------------------------------
e_exp | .0016778 .0006008 2.79 0.005 .0004991 .0028564
nfsm | .0324484 .014124 2.30 0.022 .004739 .0601577
eal | .0981797 .0093498 10.50 0.000 .0798366 .1165228
nsen | .018164 .0216753 0.84 0.402 -.0243601 .0606881
nsenwout | .1392957 .0905176 1.54 0.124 -.0382878 .3168792
_cons | 77.11407 8.942679 8.62 0.000 59.56972 94.65841
------------------------------------------------------------------------------
. ovtest

F(3, 1245) = 1.12
Prob > F = 0.3382
58
. reg gcsevauncap nfsm eal nsen nsenwout if outlier==0 & cat_sixth==0, robust

F( 4, 1249) = 37.47
Prob > F = 0.0000
R-squared = 0.1302
Root MSE = 4.3906
------------------------------------------------------------------------------
| Robust
-------------+----------------------------------------------------------------
nfsm | .0174858 .0131488 1.33 0.184 -.0083103 .0432819
eal | .0992881 .0092913 10.69 0.000 .08106 .1175163
nsen | .0052605 .0209965 0.25 0.802 -.0359318 .0464527
nsenwout | .0757416 .0901768 0.84 0.401 -.101173 .2526563
_cons | 89.67447 8.307904 10.79 0.000 73.37548 105.9735
------------------------------------------------------------------------------
. ovtest

F(3, 1246) = 0.66
Prob > F = 0.5753

. reg gcsevauncap nfsm eal nsen nsenwout avfte* cat* if outlier==0 & cat_sixth
> ==0, robust

F( 11, 1241) = .
Prob > F = .
R-squared = 0.1845
Root MSE = 4.2651
------------------------------------------------------------------------------
| Robust
-------------+----------------------------------------------------------------
nfsm | .0053815 .0132991 0.40 0.686 -.0207097 .0314728
eal | .0966751 .0089221 10.84 0.000 .079171 .1141792
nsen | -.0026198 .0212028 -0.12 0.902 -.0442171 .0389775
nsenwout | .0569815 .087895 0.65 0.517 -.1154577 .2294207
avftepupils | -.0046562 .0097066 -0.48 0.632 -.0236994 .0143871
avftepsq | 2.91e-06 .0000107 0.27 0.786 -.0000181 .0000239
avftepcu | -2.87e-10 3.75e-09 -0.08 0.939 -7.65e-09 7.08e-09
cat_religi~s | 1.002352 .3473405 2.89 0.004 .3209124 1.683791
cat_rural | .3718704 .3846203 0.97 0.334 -.3827073 1.126448
cat_eicclu~s | .2663331 .4918984 0.54 0.588 -.6987112 1.231377
cat_specia~s | 2.197616 .2819458 7.79 0.000 1.644473 2.750759
cat_bursar | -.5423945 .3058863 -1.77 0.076 -1.142506 .0577169
cat_sixth | (dropped)
_cons | 94.50605 8.49998 11.12 0.000 77.83013 111.182
------------------------------------------------------------------------------
( 2) avftepsq = 0
( 3) avftepcu = 0
F( 2, 1241) = 0.86
Prob > F = 0.4222
F(3, 1238) = 1.72
Prob > F = 0.1613
59
Appendix 2: The Malmquist index of productivity change
The three indices used most frequently to measure changes of productivity are the
Törnqvist Index, Fisher's Ideal Index (the geometric mean of the Laspeyres and
Paasche indices) and the Malmquist Productivity Index. The first two require the
calculation of both the amounts and the prices of all inputs and outputs. In contrast,
the Malmquist Index has the advantage that no information is needed on the prices of
inputs and outputs. Furthermore, calculation of the Malmquist Index requires no
restrictive assumptions regarding whether the units under analysis are benefit
maximizers or cost minimizers. These two characteristics make the Malmquist Index
a particularly suitable tool for the analysis of productivity change in the public sector,
where output prices are not in general available. A further advantage of the
Malmquist approach is that it decomposes productivity into three parts that capture
changes in the level of technical efficiency, changes in the level of scale efficiency,
and changes due to technical progress.
The usual approach uses the non-parametric methods of data envelopment analysis
(DEA) to develop a series of production frontiers under different assumptions. The
analytic framework is described in detail in the Appendix. It can be illustrated
graphically by means of Figure 1, which seeks to explain the Malmquist indices in
intuitive form for a technology exhibiting variable returns to scale with just one input
x and one output y. There are two time periods, t and t+1. The variable returns to
scale (VRS) technology estimated by DEA in period t is represented by the frontier
StVRS, while the notional constant returns to scale (CRS) technology is indicated by
the line StCRS. The unit of interest consumes input xt and produces output yt in year t.
Then we can examine the Malmquist Index as comprising three elements, M = (P x S)
x T, as follows.
The pure efficiency change P between years is given by the ratio

(Oe/Oq)
P= .
(Od/Op)
This simply indicates the change in the unit’s distance from the current technically
efficient frontier from one year to the next. The change in scale efficiency S is given
by
(Oc/Oq)
S = (Oe/Oq) ’
(Ob/Op)
(Of/Op)
whilst the change in the scale efficient technology indicated by the CRS frontiers is
estimated by
 (Og/Oq) (Ob/Op) 
T=  . .
 (Oc/Oq) (Oa/Op) 
In this simple graphical example, the two components of T are identical, but this will
not in general be the case.
60
Output
St+1CRS StCRS St+1VRS
y
StVRS
(xt+1,yt+1)
! ! ! ! !
(xt,yt)
! ! ! ! !
O a bcde f ghp q Input x

Figure A1: Illustration of productivity change with one input, one output
The Malmquist Index is then given by:

 (Oc/Oq) 
(Oe/Oq)  (Oe/Oq)   (Og/Oq) (Ob/Op) 
M= .
(Ob/Op)   (Oc/Oq) (Oa/Op) 
. . .
(Od/Op)  
 (Of/Op) 
That is, the unit’s productivity change is expressed as the product of pure efficiency
change, scale efficiency change, and an estimate of technological progress.
In order to make the index operational, it is necessary to consider its four constituent
distance functions. This can be achieved using parametric or non-parametric
approaches. In our view, the non-parametric approach (DEA) offers distinct
advantages for analyzing public sector units. It requires fewer assumptions than
parametric methods regarding the functional form of the production technology; it can
readily handle multiple inputs and outputs; it can yield useful results with small
numbers of observations; and it is less vulnerable than parametric methods to
statistical biases, such as endogeneity and omitted variables. We therefore recommend
implementation of these principles using successive formulations of DEA models.
Numerous applications of Malmquist index methods can be found in the literature. An

example is the examination of productivity change in hospitals (Jacobs, 2001) and
social care (Salinas-Jiménez et al., 2003). It should be noted that – whilst such
methods offer interesting insights into progress over time – the estimates they yield
can be quite volatile and sensitive to model specification. Modelling therefore
requires careful implementation and interpretation.
61
Copies of this publication can be obtained from:
DfES Publications
P.O. Box 5050
Sherwood Park
Annesley
Nottingham
NG15 0DJ
Tel: 0845 60 222 60

Fax: 0845 60 333 60
Minicom: 0845 60 555 60
Online: www.dfespublications.gov.uk
© Professor Peter C Smith 2006
Produced by the Department for Education and Skills
ISBN 1 84478 788 5

Ref No: RR788
www.dfes.go.uk/research

Extension

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Extension

Загружено:

Авторское право:

Доступные форматы

R ESEARCH

Analysis of Secondary School

Professor Peter C Smith

Research Report RR788

Analysis of Secondary School

Professor Peter C Smith

© Professor Peter C Smith 2006

2 Issues in model construction ........................................................................................ 18

3 Data and descriptive analysis ...................................................................................... 28

5 Data envelopment analysis........................................................................................... 40

6 Commentary and next steps ........................................................................................ 44

Appendix 2: The Malmquist index of productivity change ............................................... 60

The analysis and measurement of efficiency is a complex undertaking, especially

Notwithstanding the apparent simplicity of the concept, there is a great deal of

The focus of efficiency analysis is as an organisational locus of production, often

Figure 1.1: Efficiency measurement under constant returns to scale

Figure 1.3: The case of two inputs

Analogous arguments can be deployed to examine the allocative efficiency of

Figure 1.5: Allocative efficiency with two outputs

Although organisations may exhibit allocative inefficiency in purchasing the wrong

Figure 1.6: Technical and allocative inefficiency

Weight restrictions and cross-efficiency models

When undertaking econometric analysis, it is not a trivial matter to take account of

The classical application of DEA places no constraints (other than positivity) on

An intermediate approach to imposing weight restrictions is illustrated in Figure 1.8.

Figure 1.8: Weight restrictions

A technically efficient production choice such as PT may not be allocatively efficient

Figure 1.10: Economies of scale

The cost function

The fundamental building block of the economic analysis of organisational efficiency

Inefficiency can be defined as the extent to which an organisation’s costs exceed

Analytical techniques: SFA and DEA

Inefficiency is inherently unobservable. This means that estimates of efficiency have

The distinctive focus of modern efficiency analysis is to seek – in addition to

Figure 2.1: The naïve model of organisational performance

Economy Efficiency Effectiveness

Models of public service efficiency almost always entail consideration of multiple

We address model-building principles by considering five issues:

We then discuss practical facets of undertaking efficiency analysis, which is often

Specifying inputs and outputs

In competitive industries the physical output of the organisation is usually a traded

A fundamental decision that must be taken is the level of disaggregation of inputs to

It may also be important to consider a short-term perspective, in which certain aspects

Incorporating measures of capital into the efficiency analysis is more challenging.

Figure 2.2: The dynamic nature of organisational performance

Outputs Outputs Outputs

Organization Organization Organization

Inputs Inputs Inputs

In many contexts, a separate class of factor affects organisational capacity, which we

There is often considerable debate as to what environmental factors are considered

The performance of a school is in part dependent on inputs from outside agencies,

The second approach is to incorporate environmental factors directly into the

Including a factor in a risk-adjusted output measure should preclude the need to

Notwithstanding the important model-building considerations discussed in this

• There is often a serious lack of information on some important dimensions of

Such difficulties will be familiar to those working in most domains of quantitative

The beguilingly simple notion of efficiency disguises a series of thorny conceptual

Figure 2.3: The broader context of efficiency analysis

Joint outputs: Outputs External outputs:

Endowments Organization Endowments

Exogenous inputs: System constraints:

Table 3.1: Variable descriptions

y gcsevauncap | GCSE uncapped value added score

Table 3.2. Descriptive statistics