BENCHMARKING UK GOVERNMENT

PROCUREMENT PERFORMANCE IN
CONSTRUCTION PROJECTS
Robin Holt and Andrew Graves
Robin Holt, Management Research Officer, took his PhD in Government at LSE, and is now investigating links
between social theory, management practice and construction. Andrew Graves, MIT Professor of Automotive
Management and Director of Cross-Industry Research at the University of Bath, is responsible for research on
developing ``lean production'' in the car, aerospace and construction industries.

Abstract Benchmarking is introduced as a practice of description of a pilot study, designed to begin a

non-financial assessment that promotes continual performance
improvement. Its relevance to and possible consequences for the
public sector are discussed in relation to a case study in construction
procurement. A pilot study investigating the achievements of
government clients in construction procurement has identified a need
for better client ``ownership'' of project risk and opportunity. The
article argues that benchmarking can provide the vehicle for this
``ownership''. In conjunction with the clients and HM Treasury, a
second stage of project assessment has just been completed and the
methods and results described. The aim is to realize consistent,
relevant and feasible metrics, co-operatively authored by client
practitioners and academics with reference to private construction
organizations, that will then be used for purposes of on-going
self-assessment at both project and strategic levels.
Keywords Benchmarking, Public sector, Performance,
Risk management, United Kingdom
Introduction
The UK Government is increasingly interested in policy
reform that seeks to tie the realization of the public
interest to specific performance improvements in service
provision (Grout, 1997). The definition and attainment
of these service levels is, we argue, determinable through
a benchmarked procedure of iterative improvement
where links between policy and results can be made
explicit (Coe, 1999). This article describes how the
measurement and consequent development of such
service provision has been undertaken with specific
regard to the procurement of construction projects. We
address criteria informing public sector perspectives that
bear on the development and use of metrics and that
potentially threaten the widespread and prolonged
adoption and adaptation of a benchmarking process.
Informed by these criteria, we discuss the development
of our benchmarking method and its intent. From the
benchmarking process amongst key central government
clients, we present a refined set of metrics that support a
second phase of research and then discuss the findings.
In conclusion we argue that unless good performance
indicators are used in conjunction with one another, and
are ``owned'' by practitioners, any performance
improvement is at best sporadic and often temporary.
Benchmarking as a process
Benchmarking operates at two levels: a method for
measurement and a focus of activity. As a method for
measurement, benchmarking assesses relative
performance (in terms of productivity, or profit, or
speed) to establish metrics that reveal good and bad
performance. The aim is to convey the existing delivery
of value as against potential delivery. Metrics only make
sense through their capture of narrowly defined aspects
of value. For example, a metric of safety is the number of
reportable accidents; but the validity of this metric stems
not from itself, but the quality of life it helps promote for
operatives; the benefits realized from an absence of
disruption and so on. Metrics must also be consistent
with one another ± their aggregated portrayal of cost,
function and quality must be comprehensive so that, for
example, cost metrics are understood in terms of their
impact upon function and quality. Metrics can frustrate
the process of benchmarking if their definition fails to
promote genuine comparison. In the context of
construction, for example, different procurement
options incur the transfer of different risks. More
traditional contracts require the client to retain design
responsibility, whereas recent ``one stop shop'' relations
The current issue and full text archive of this journal is available at
http://www.emerald-library.com/ft

Measuring Business Excellence 5,4 2001, pp. 13-21, # MCB University Press, 1368-3047
1 3
transfer this risk to the constructor. As a consequence, transport network) (Ammons, 1999). As such, through
variations in cost must be identified at agreed points in benchmarking, the UK Government can assess and
the procurement process and be related to a particular refine what it means by the delivery of ``value for
procurement option. Under ``divided'' procurement money''.
variations are more prevalent than under ``one stop Attending to the exact scope of any metrics requires,
shop'' or ``turnkey'' arrangements and do not necessarily first, that the ``worldview'' of those undertaking
impinge on performance. Metrics are also required that benchmarking is acknowledged, in order that what is
allow for the inclusion of stakeholders. In public measured is appropriate and meaningful to practitioners.
construction projects these include: users; taxpayers; In this case, the scope of the metrics needs to be alive to
local residents; environmental and economic pressure the prevailing and emerging influences upon public
groups; planning authorities; future generations; managers within construction, not least of which is the
national and international political interests; financial growing use of performance measurement itself. These
interests; and the construction suppliers (Kouzmin et al., influences can be summarized as follows.
1998).
With regard to benchmarking being a focus for Departing from the culture of regulation
business activity, it looks at measuring the impact of The growth of procedural constraints experienced by
initiatives that seek to attain, sustain and exceed both the public managers, designed as controls that make
performance and increased managerial discretion
representational capability of more auditable, can stymie
these metrics. Benchmarking is initiative and incur considerable
not a list or even a procedure, it ``Benchmarking the direct and indirect cost in terms
is a responsive activity that of initiation, monitoring and
requires practitioners to delivery of construction compliance (Hood et al., 1998).
understand the circumstances of Benchmarking risks being seen
achievement as much as the level services can extend as yet one more mode of
of achievements. Any from the analysis of a regulation and surveillance. To
organisation is required by the overcome this problem, the
benchmarking process to ask specific process to . . . research adopted the use of
what was learnt through the ``good'' practice and not `best'
metrics, and so address their a general social vision.'' practice. Best practice is static
adaptation, rather than just their and suggests performance is
adoption (Ammons, 1999). measured as the ability to
Bestowing such a strategic role imitate. Good practice is more dynamic, inviting
upon metrics must be accompanied by a dynamic consideration by offering itself as something to be taken
interaction with practitioners, so as to invest in them a up and adapted. In this way the idea that benchmarking
progressive ``ownership'' of emerging metrics (Hinks and is primarily a sub-optimal strategy of ``catch-up'' (Cox
McNay, 1999). and Thompson, 1998; Davis, 1998) is avoided.
The public sector and construction context Benchmarking is seen as setting horizons through an
The concern of current UK Government policy is to active and on-going awareness of where others have set,
achieve value for money in the delivery of services such reached and re-configured theirs.
as construction, of which in the UK domestic market of Having identified good practice, the recipient must
£55bn (FT, 1999) the Government procures 40 percent not only recognise the benefits of using metrics but also
(Graves et al., 1998a). This recent emphasis upon be in a financial and cultural position to use them.
delivering ``value'' enhances the scope for using Public clients of different asset strength, maturity or
``evidence'' as a decision tool in public policy because ``worldview'' are often able to appreciate the good
metrics can be used to discover ``what works'' (Davies et practices of others without having the time, resources or
al., 1999), and what could work better, not just in terms context to apply them (O'Dell and Grayson, 1998). This
of price, but in terms of out-turn cost linked to function source of inertia may prove particularly powerful when
and quality. Benchmarking the delivery of construction considered in conjunction with the current status of
services can extend from the analysis of a specific process performance reporting in the public sector. Bespoke
(for example, the use and relative success of using management systems of reactive design (Kouzmin et al.,
different types of contract forms in procurement of 1998) and partial coverage of collected information
construction services) to cover a general social vision (Hyndman and Anderson, 1998) serve to dilute the
that looks to use metrics as milestones for general quality argument for the radical level of change necessary to
of life issues (for example, the creation of an effective realise the potential of benchmarking.

Measuring Business Excellence 5,4 2001

1 4
A dynamic culture
The reliance of public sector bureaucracies on explicit,
codified information is also a source of frustration for
would-be benchmarking practitioners. Any metrics used
will tend to orient themselves to what is most readily
captured in terms of existing data, rather than to the
tacit knowledge of relationships and personal experience
(Nonaka et al., 1998). As a process to catalyse change,
benchmarking has to be carefully managed to avoid any
resulting undue restriction of scope. The desire to avoid
uncomfortable investigations and outcomes by limiting
the scope of investigation and reported evidence would
promote benchmarking on the basis of what was feasible
rather than on rational decision (Dorsch and Yasin,
1998; Leicester, 1999). One example might be the
tendency to look at capabilities, as opposed to satisfying
utility preferences (Sen, 1999). The measurement of
capabilities (such as road space provision) is typically far
easier than utility preference (in this case, the means for
satisfying travel need, that
may include roads amongst a
host of alternatives), because
benchmarking is reliant upon ``. . . benchmarking has
measurement, and metrics
themselves tend to to be carefully managed to
concentrate upon explicit, avoid any resulting undue
quantifiable data, the policy of
using metrics itself might restriction of scope.''
privilege a capability above a
utility view.
Political influence
Political realities face those implementing
benchmarking. The multiple reality facing the new
public manager (Dunleavy and Hood, 1994), where
efficiency and the public interest need to be reconciled,
needs to be understood to pre-empt conflict in the
design and implementation of metrics and promote their
appropriate interpretation (Holt and Rowe, 2000).
Political cycles form a persistent reality and any
benchmarking strategy must inform itself of prevailing
political norms and emerging shifts in power. This
responsive nature of a civil service requires any
medium- to long-term strategy be malleable and
under-determined. Metrics and their application should
not, therefore, form a dominating strategy.
The dimension of complexity
Construction is a complex activity. The structural
variations between projects in terms of potential
influence (political, social, economic, environmental,
skill availability) and scope (planning, material use and
re-use, budget, urgency, type (road, rail, water,
building), life-span, flexibility) impact heavily upon the
value of information comparison both between
construction projects and between construction projects
Measuring Business Excellence 5,4 2001
1 5
and manufacturing processes (Mohamed, 1996). We
have already argued that, to achieve a workable balance
of support metrics must be relevant, applicable,
approachable and practical. Metrics must also embrace
the structural variations of their context. This means the
performance measurement capacity of benchmarking is
tied intimately with the mapping and management of
risk.
Research methodology
Informed by the foregoing discussion, our research
embraces an iterative strategy to implement performance
improvement using benchmarking within central
government. The process has undergone two cycles of
development: the pilot and Stage two studies. These
have progressed using the following generally
characterised methodology:
^ Client anonymity has been preserved to promote
confidence through a non-threatening environment.
^ The project selection criteria addressed: stage of
completeness;
contemporaneity; and
value thresholds.
^ Data anomalies evident
in the samples for each of
the two studies were
``cleaned'' using
follow-up interviews.
Clients were asked to make
project selections without
regard to the perceived
outcome or history of projects. The widespread variation
in performance (discussed later) was in line with UK
Audit Commission's findings of local authority
construction procurement and the experience of
benchmarking in the private sector. This provided a
metric of confidence that the aim for representative
selection was at least moderately successful. Key
characteristics of the two mutually exclusive data
samples are shown in Table I.
Pilot study
The pilot study began with a focus group of the
Government Construction Client Panel (GCCP),
comprising representatives from 13 government clients.
The group identified hypotheses that represented
procurement risks and proposed metrics to inform those
hypotheses. The approach was pragmatic: the number
and scope of metrics were geared towards assured data
capture; and the themes addressed by metrics were
limited principally to information of established interest
to the client. The pilot study focused on project output
metrics to test the hypotheses. It was assumed that
capture, analysis and presentation of the information
would stimulate dialogue and an exchange of
perspectives as to why particular projects might have
 Table I Ð Comparison of pilot and stage two data samples
Data sample characteristic
Number of projects
Total value of projects
Number of client departments/agencies
Project type
Building
Infrastructure
Project status
New build
Refurbishment
Average service commissioning date for projects
Notes: a Clients submitting four or more projects; b
All clients submitting projects
excelled or gone awry. Informed by revealed trends and
feedback from the process of data capture itself, a
process of client-informed metric refinement could be
initiated.
A key role of the pilot study was to demonstrate the
post-project availability of data and the feasibility of
capture through a generic format. We felt that the
delicately balanced task of building a data model capable
of population without excessive investigation yet
sufficient for robust and challenging interpretations
should not be under-stated. The need to balance
achievability and usefulness is an important lesson
drawn from benchmarking experiences in other
industrial sectors.
Stage two study
The second stage of work continued to promote close
contact with the GCCP in developing a second
questionnaire. A joint responsibility was assumed for its
promotion within each client between the research team
and client representatives. Stage two constituted both
reinforcement of the successes of the pilot and an
opportunity to establish a maturing model. Feedback
from the pilot study was taken from the sampled clients as
to the validity and reliability of the metrics and the
appropriateness of the benchmarking study, given the
criteria detailed in the earlier discussion. This feedback
was used, in conjunction with further focus group
sessions of the GCCP and expert consultations with
construction research groups, to re-configure metrics so
as to better reveal existing and potential project risks and
so increase the relevance and ``ownership'' of the
benchmarking practices. The risk identification and
control model that emerged is shown in Figure 1. It is
divided into four areas, each with associated metrics,
reflecting the extended concern of the Stage two study
over the pilot study. The metric count illustrates how the
focus on outputs as seen by project funders and end users
in the pilot study was supplanted by a balanced
measurement of risk, practice usage, and the success of all
project participants/stakeholders in the Stage two study.
Measuring Business Excellence 5,4 2001
1 6
Pilot Stage two
60 65
£500 million £500 million
6a (8b) 7a (10b)
100% 52%
± 48%
62% 48%
38% 52%
June 1995 January 1998
Pilot study findings and discussion
The evaluated metrics demonstrated considerable
variation in performance and suggested a lack of
consistency in approach, both within and between
clients. The two main findings were as follows.
A culture of limited expectation
Notions of possible ``role fuzziness'' and confusion
surrounding the performance of product and service
were endorsed by the juxtaposition of generally high
levels of client and end-user satisfaction with poor cost
and time predictability. Given the UK Government's
insistence upon delivering performance improvements,
the pilot study provided a steady reminder that value
assessment was so judgmental that even poor cost and
programme performance did not meet with a rating of
unsatisfactory performance. The pilot study revealed
an extensive reality gap between performance and
performance assessment by practitioners. This gap
could be the result of the public managers looking
foremost to rules of accountability, judging their
``success'' in terms of their having fulfilled their
required functional role. The results describing this
``gap'' are shown in Figure 2.
Underestimating risks
The pilot study found that clients accepted two thirds
of realised programme risk. When compared to the
cost and programme overrun figures in Figure 2, this
suggests an underestimation of risk. Figure 3 shows
how in the majority of projects investigated, the risk
was not spread through suppliers using partnering or
other teamworking agreements, but retained by the
client.
The case for reviewing exposure to risk was enhanced
by the fact that one third of the projects were procured
on the basis of ``one stop shop'' style contracts such as
Design and Build, a mechanism that is intended to
determine much of the client's risk in terms of a
premium built into the tender price paid to suppliers
(rather than explicit determination at the end of
Figure 1 Ð Linking metrics with themes under the risk-based evaluation framework

g
y
y V

y g

Figure 2 Ð Client perception of success measured against programme and cost in construction phase

Measuring Business Excellence 5,4 2001

1 7
 Figure 3 Ð Use of partnering or teamworking set against cost growth in the construction phase
construction based on the actual risks encountered).
This analysis lends further support to an approach to
benchmarking that is underpinned through more
effective risk mapping to promote shared ownership of
risk. Where risk management did occur, and it was
limited as is as shown in Figure 4, it was not consistent
in delivering good project performance, further
suggesting that the type of risk management systems in
place required review.
Towards stage two
Analyses were undertaken to investigate relationships
between metrics, in particular between each metric and
cost and time. This failed to establish strong trends and
supported the notion that construction projects bear a
complex structural identity in which the specific
demographic, financial and political context has a large
influence. This general point of learning, combined with
the issues of unpredictable cost and time, was seminal in
moving us towards articulating the project to include
many more non-financial metrics, hence shifting the
emphasis in Stage two towards risk identification and its
control.
Figure 4 Ð The use of risk management set against cost growth in the construction phase
Measuring Business Excellence 5,4 2001
1 8
Stage two study findings and discussion
The research highlighted variations in the application of
good practices. There remained a ``reality gap'' between
performance and performance assessment. Figure 5
shows that construction cost overrun amongst the clients
remained high.
Despite this, and in line with the findings of the pilot
study, supplier satisfaction ratings made by the clients
remained consistently high, as shown in Figure 6.
Perhaps the most significant finding was that ``bolting
on'' specific good practices did not result in performance
improvement (see Figure 7).
Where improvement was experienced, however, was
in those projects where clients used a ``basket'' of good
practices; combining strategic relationship assessment
with teambuilding with safety audits and so on (see
Figure 8).
Conclusions
Overall, the public sector client was found to have both a
strategic and operational role in achieving performance
improvement in value for money service delivery, but
that this had been underutilised. It is suggested that
there needs to be greater ``ownership'' of projects from
 Figure 5 Ð Construction cost increase from budget
Figure 6 Ð Supplier assessment of client
public sector clients and risk sharing with suppliers.
Through benchmarking, public sector clients can better
assess the relevance of their existing aims and purposes;
and can appreciate how best to pursue those aims. It is
also suggested that public sector management needs to
expand its scope to deliver whole life value for money.
The development of benchmarking to include
whole life issues, value and the embodiment of both
service-oriented and product-oriented approaches to
procurement have a long way to go. We are optimistic
Measuring Business Excellence 5,4 2001
1 9
that the gaining momentum of benchmarking will
continue, gathering the support of new clients in the
UK and internationally. There remains, however, a
suspicion that metrics amount to a regulatory audit (if
not now, then at sometime) and that those insisting
upon value for money criteria are doing so for reasons
of political expediency as much as a genuine desire to
create an open, innovative public management
culture. MBE
 Figure 7 Ð Plotting outcome success against the use of mature realtionships
Figure 8 Ð Plotting outcome success against a ``basket'' of good practices
References
Ammons, D. (1999), ``A proper mentality for benchmarking'', Public
Administration Review, Vol. 59 No. 2, pp. 105-14.
Coe, C. (1999), ``Local government benchmarking: lessons from two
major multigovernment efforts'', Public Administration Review,
Vol. 59 No. 2, pp. 110-21.
Cox, A. and Thompson, I. (1998), ``On the appropriateness of
benchmarking'', Journal of General Management, Vol. 23 No. 3,
pp. 1-20.
Davies, H., Nutley, S. and Smith, P. (1999), ``Editorial'', Public Money
and Management, January-March.
Davis, P. (1998), ``The burgeoning of benchmarking in British local
government'', Benchmarking for Quality Management and
Technology, Vol. 5 No. 4, pp. 261-70.
Measuring Business Excellence 5,4 2001
2 0
Dorsch, D. and Yasin, M. (1998), ``A framework for benchmarking in
the public sector'', International Journal of Public Sector
Management, Vol. 11 No. 2/3, pp. 90-115.
Dunleavy, P. and Hood, C. (1994), ``From old public administration
to new public management'', Public Money and Management,
Vol. 14 No. 3, July-September, pp. 9-16.
Graves, A., Rowe, D., Sheath, D. and Sykes, M. (1998a), Constructing
the Government Client, HMSO, London.
Graves A., Rowe, D., Sheath, D. (1998b) Pilot Benchmarking Study,
HMSO, London.
Grout, P. (1997), ``The economics of the private finance initiative'',
Oxford Review of Economic Policy, Vol. 13 No. 4, pp. 53-66.
Hinks, J. and McNay, P. (1999), ``The creation of a management by
variance tool'', Facilities, Vol. 17 No. 1/2, pp. 31-53.
Hood, C., James, O., Jones, G., Scott, C. and Travers, T. (1998),
``Regulation inside government: where new public management
meets the audit explosion'', Public Money and Management,
April-June, pp. 61-8.
Holt, R. and Rowe, D. (2000), ``Total quality, public management and
critical leadership in civil construction projects'', International
Journal of Quality & Reliability Management, Vol. 17 No. 4/5,
pp. 541-53.
Hyndman, N. and Anderson, R. (1998), ``Performance information,
accountability and executive agencies'', Public Money and
Management, April-June, pp. 23-30.
Kouzmin, A., LoÏffler, E., Klages, H. and Korac-Kakbadse, N. (1998),
``Benchmarking and performance measurement in public
sectors'', International Journal of Public Sector Management,
Vol. 12 No. 2, pp. 121-44.
Measuring Business Excellence 5,4 2001
2 1
Leicester, G. (1999), ``The seven enemies of evidence based policy'',
Public Money and Management, January-March, pp. 5-7.
Mohamed, S. (1996), ``Benchmarking and improving construction
productivity'', Benchmarking for Quality Management &
Technology, Vol. 3 No. 3, pp. 50-8.
Nonaka, I., Reinmoeller, P. and Senoo, D. (1998), ``The art of
knowledge: systems to capitalise on market knowledge'',
European Management Journal, Vol. 16 No. 4, pp. 673-84.
O'Dell, C. and Grayson, C. (1998), ``If only we knew what we know:
identification and transfer of internal best practices'', California
Management Review, Vol. 40 No. 3, pp. 154-74.
Sen, A. (1999), Commodities and Capabilities, Oxford University Press,
Oxford.