Академический Документы
Профессиональный Документы
Культура Документы
51
52
PERFORMANCE-BASED MANAGEMENT
Performance-based management for highway construction was
initially referred to as a transportation asset management system by
practicing foreign countries. Canada, Britain, Japan, Australia,
and New Zealand are currently recognized as innovative leaders in
performance-based management and have been applying the methodology for nearly two decades with significant results (6, 7). Several
of these countries infrastructure agencies established themselves on
principles of performance measurement, such as Japans Ministry of
Land, Infrastructure, and Transportation and Queensland, Australias
Department of Transport and Main Roads; others attained this
program by way of government policies and acts requiring the use
of measurable standards and targets for government ministries
and agencies, similar to Canadas British Columbia Ministry of
Transportation (6). No matter the journey, various developed countries around the world are currently profiting from its use, whereas
others, including the United States, are only beginning to realize the
systems effectiveness and potential for inducing success.
AASHTOs task force on performance management has developed
a basic framework for transportation performance management. The
association states that a proper framework must first be practical,
focused, cost-effective, and consistent with what is measured, how
it is measured, and how the measured data are presented while also
quantifying agency performance, driving interest and participation
in achieving improvement, and aligning the measurement activities
with outcomes and objectives (8). As shown in Figure 1, AASHTOs
fundamental process for performance management consists of five
phases. The first phase is the selection of appropriate performance
measures to evaluate the agency in critical program and service areas.
That is followed by monitoring and reporting the performance results.
It is this phase in which results must be monitored on the agency-
DEFINITIONS
State DOTs use a divergent array of language to describe their
own unique performance management techniques. For simplicity
and consistency, this research defines the terms state policy, performance objective, performance measure, and key performance
indicator as overarching terms for the array of individual DOT
terms.
Statewide policy goals are established by the state government
for each transportation agency. Statewide policy goals typically
include areas such as safety, congestion and mobility, environmental
compliance, stewardship, and preservation.
Performance objectives are DOT mission statements or objectives
for each state policy area guiding the decisions made by the state
DOT and contractors during the course of the project. The performance objectives symbolically represent a promise to the traveling
public for achieving positive results in each performance area.
Performance measures are derived from the programmatic levels
of service sought by the transportation agency and imposed contractually as broad classifications of desired outcomes required of the
contractor.
KPIs are more project-specific milestones or components in
performance measures serving as precursors to indicate progress
toward the eventual achievement of the desired performance measures.
As broad classifications of desired outcomes, performance measures are most often reflected by contractual goals and statements of
increasing or decreasing specifics, ensuring a specific, establishing
a specific, or implementing a specific in one of the general performance areas of safety, mobility (highway congestion), environmental
stewardship, preservation, or public information management. Key
performance indicators typically include, but are not limited to,
elements such as project benchmarks, targets, milestone dates,
numbers, percentages, variances, distributions, rates, time, cost,
indexes, ratios, survey data, and report data. These targets and data
provide a state DOT with tangible evidence as to how a contractor is performing. It allows the agency to determine whether or
not it will achieve its set performance measures on the basis of the
positive and negative contributions of individual projects toward
those measures. These definitions are useful in understanding the
results and conclusions examining the information gathered from
each case study.
RESEARCH APPROACH
Continue to monitor and report
progress
Literature Review
The research team conducted a thorough review of existing documentation for performance-based management in highway agencies
and projects. This task focused on data collection concerning current
national and international industry approaches to KPIs and more
general performance measures. The objective of the data collection
effort was to provide information to define the state of the practice.
An extensive literature review was conducted by using general
Internet search engines (Google, Google Scholar, Yahoo, and so
forth); TRBs Transportation Research Information Systems Online
(TRIS Online); academic engineering databases (e.g., LexisNexis
and Engineering Village 2); academic business databases (EBSCO
Business Source Complete and Management and Organizational
Studies); ASCE civil engineering database; and selected transportation
agency websites and reports.
The team was able to gather more than 40 different scholarly
articles and research reports from these sources. The team extracted
more than 1,700 performance measures and indicators in use or previously used by different highway agencies relating to all aspects of a
project, including design, construction, operation, and maintenance.
This extraction was useful in defining performance measures and
indicators for the purpose of this study, as well as familiarizing the
team with possible measures and indicators for use in examining the
different case study projects.
Data Collection
The scope of the data collection involved gathering information from
four DB project case studies. The project case studies also required
the collection of agency data on statewide policy goals, performance
objectives, and programmatic performance measures. Before beginning the case studies examination, the research team conducted a
content analysis of the existing literature and contract documents the
team had in its current database. Subsequently, the team developed
a data collection instrument in the form of a case study protocol to
collect comprehensive and consistent information from the domestic
practitioners.
The first step in obtaining detailed information was conducting a
content analysis of the case study requests for proposal (RFPs) before
creating the data collection instrument. The purpose of the content
analysis was to (a) identify measures and procedures used in policy
and implementation documents and (b) minimize the burden on
the national respondents. A formal content analysis involves a set of
procedures for collecting and organizing information in a standardized
format that allows for the transformation of nonstructured information
into a format that permits analysis. The basic steps of the content
analysis method are to (a) decide to use content analysis, (b) determine
what material should be included in content analysis, (c) select units
of analysis, (d) develop coding categories, (e) code the material, and
( f ) analyze and interpret the results (9). The results of the content
analysis were a listing of performance measures and KPIs that
the team used as a basis for designing the case study protocol and
supplementing the data collection from the national practitioners.
To preserve the quality of the research, the case studies were
required to maintain trustworthiness, credibility, conformability,
53
and data dependability. Four validity tests were used to determine the
quality of the case study research: (a) construct validity, (b) internal
validity, (c) external validity, and (d) reliability (10). Construct
validity deals with peoples opinions and biasessubjectivity should
be avoided in the research. This study dealt with the construct
validity by using multiple sources to collect the data for the research
(e.g., both agency and contractor representatives). Internal validity
refers to the interpretation of the data, especially when some causal
factors are not recognized or are omitted during the analysis of the
data. To ensure the internal validity of this study, the team used
the technique of pattern matching across the case studies. External
validity in this study involved the transfer of the results to practice.
The external validity test was met by replicating the findings developed in one case study in another. Reliability is the capability of
replicating the findings if the same steps are repeated. The condition
of reliability was met by drafting a detailed protocol that guided the
formulation of propositions, research questions, case study design,
and data collection.
Synthesis
The research team synthesized the data by exploring patterns across
multiple case studies and the literature. The team used the performance
categories discovered in the content analysis to organize the results.
The categories, which were used for the presentation of results, were
also chosen to align with AASHTOs performance management
model as closely as possible. They included the performance areas
of (a) cost and schedule monitoring, (b) quality monitoring, (c) safety
management, (d) public information management, (e) environmental
stewardship, and ( f ) traffic reliability during construction. Design
and construction performance measures and KPIs were then identified
for each.
54
TABLE 1
Project Name
State DOT
DesignBuilder
Location
Washington State
Atkinson Construction,
LLC of Renton
Downtown Bellevue,
Washington
255
Missouri
KTU Constructors
700
Georgia
Matthews Contracting,
Inc.ARCADIS
LundaAmes
1.5
81
34
225265
Minnesota
Cost ($ million)
55
the frontage road. The department selected C. W. Matthews Contracting, Inc./ARCADIS as the DB team to complete the project. On
May 25, 2007, the notice to proceed was given to the contractor with
a contract completion date set for December 31, 2008. The total cost
of the project was $81 million.
Georgia DOT constructed the contracts RFP special provisions
to require the designbuilder to propose state-of-the-art methods
and innovative strategies to monitor and report on the achievements
of KPI targets in the performance areas of safety, congestion minimization, quality, and user satisfaction. Measuring performance in
these areas as project deliverables and tying these deliverables to an
incentivedisincentive program was a first for Georgia DOT. In doing
so, Georgia DOT made the project eligible to receive a $1 million
Highways for LIFE (HfL) grant, a program dedicated to improving
highway safety and the quality of infrastructure as well as reducing
the delays experienced by commuters caused by construction by
offering project grants as incentives for including strategies for
monitoring performance in these areas in project contracts.
TABLE 2
Performance Objective
Performance Measure
Progress reports
Executive summary of project achievements and difficulties
Analysis of project condition with respect to on-time and on-budget performance
Discussion of contract milestones and the project critical path
Detailed review of each disciplines past performance and future goals
Status of required submittals, RFIs, and design approvals that impact timely
performance
Contractor self-assessment of performance against incentive performance issues
Monthly schedule updates
Calculations, reports, and narratives regarding the project schedule and budget
Statistical comparison of the current schedule to the previous and expected final
schedule
Changes to activity logic and durations and expected impact of the changes
Discussion of milestones that are being calculated to finish later than the contract
milestone completion dates and how they might be improved
NOTE: Performance area: cost and schedule monitoring; policy: stewardship (State Policy 5).
56
KPIs
20092015 Strategic Plan (11, 12). The last column of Table 2 presents language used in the project RFP indicating the use or further
development of KPIs (or performance management tools with characteristics similar to formal KPIs). For example, under progress
reports the requirement of the designbuilder to provide an analysis
of the project condition with respect to on-time and on-budget performance gives Washington State DOT tangible results on the projects delivery status and whether or not this individual project is
contributing to the departments performance measure and objective
of using public resources effectively and efficiently. In this example,
Washington State DOT allows the designbuilder the freedom to
propose an analysis framework for on-time and on-budget performance
for acceptance, whereas in using a formal KPI, the department would
have set the standard and framework for the designbuilder (1315).
The findings of the West Point I-85 Interchange case study for the
performance area of quality monitoring are presented in Table 3.
TABLE 4
Performance Objective
Performance Measures
1. Customer involvement in
transportation decision making,
2. Personal, fast, courteous and
understandable response to
customer requests (inbound),
and
3. Accurate, timely,
understandable and proactive
transportation information
(outbound)
TABLE 5
57
Performance Objective
Performance Measures
NOTE: Performance Area 6: traffic management; policy: travel within the Twin Cities metro area.
CONCLUSIONS
The use of performance management in highway design and construction is rapidly evolving. This research found that agencies with
a strong focus in performance management are applying agency-level
performance measures, objectives, and policies, as well as projectlevel KPIs. These elements were found in all four case studies.
However, the term key performance indicator is not common.
These elements provide each department with tangible evidence
of how designbuilders are performing. The six major design and
construction performance areas were found to be cost and schedule
monitoring, quality monitoring, safety management, environmental
stewardship, public information management, and traffic management.
Each case study exhibits examples of RFP language or requirements
in the majority of these performance areas, indicating the use or
further development of KPIs.
The opportunities for using KPIs in DB contracts allow for a more
formalized process. This process follows that seen in international
publicprivate partnership project contracts using stated performance
measures and KPIs. That is to say that these opportunities reflect the
58
general structure of aligning themselves with agency-level performance measures and objectives. As the federal government and state
agencies move toward agency-level performance management
systems, positive contributions must be made by individual projects.
An agency cannot reach a performance measure related to reducing
work-zone-related injuries without reducing work-zone-related injuries
in each of its individual projects. Each of the four case studies shows
progression toward this type of performance-based management
structure. It is hoped that the definitions and examples provided in
this paper will help the highway sector move toward an integrated
performance management system.
ACKNOWLEDGMENTS
This research was sponsored by FHWAs Office of International
Programs. The authors thank the case study sponsors and project
personnel who contributed their time to this study. The authors also
thank FHWAs Construction Management Expert Task Group for
its guidance.
REFERENCES
1. State DOT Performance Management Programs: Select Examples.
AASHTO, Washington, D.C., 2007.
2. AASHTO Authorization Policy Topic I Performance Management.
AASHTO, Washington, D.C., 2008.
3. Task Force on Performance Management. A Primer on PerformanceBased Highway Program Management Examples for Select States.
AASHTO, Washington, D.C., 2008.
4. Molenaar, K. R., G. Proctor, and D. Navarro. Key Performance Indicators
in PublicPrivate Contracts Partnerships: A State of Practice Report.
Federal Highway Administration Report No. FHWA-PL-10-02 U.S.
Department of Transportation, 2011.
5. ASCE. Report Card for Americas Infrastructure. ASCE Report Card
for Americas Infrastructure. www.asce.org/reportcard/2009/grades.cfm.
Accessed July 19, 2009.
6. MacDonald, D., C. Yew, R. Arnold, J. Baxter, R. Halvorson, H. Kassoff,
M. Meyer, K. Philmus, J. Price, D. Rose, M. Walton, and W. White.
Transportation Performance Measures in Australia, Canada, Japan, and
New Zealand. American Trade Initiatives, Inc., LGB & Associates, Inc.,
2004, pp. 184.
7. Booz, Allen, Hamilton. Relationships Between Asset Management and
Travel Demand. FHWA, U.S. Department of Transportation, 2008,
pp. 199.
8. Measurement Working Group. Area Performance Indicator (API)
Handbook. Highways Agency, United Kingdom, 2007, pp. 160.
9. Content Analysis: A Methodology for Structuring and Analyzing Written
Material. GAO/PEMD-10.1.3. General Accounting Office, Washington,
D.C., 1996, pp. 177.
10. Yin, R. K. Case Study Research: Design and Methods, 2nd ed. Applied
Social Research Methods Series, Vol. 5. Sage Publications, Thousand
Oaks, Calif., 1994.
11. Washington State Department of Transportation. Business Directions:
WSDOTs 20092015 Strategic Plan. Washington State Department of
Transportation, Olympia. www.wsdot.wa.gov/Accountability/Performance
Reporting/StrategicPlan.htm. Accessed March 1, 2010.
12. Washington State Department of Transportation. The Gray Notebook.
Edition 35. Jan. 2009. www.wsdot.wa.gov/Accountability/GrayNotebook/
navigateGNB.htm. Accessed March 1, 2010.
13. Washington State Department of Transportation. Instructions to Proposers: I-405/NE 8th St to SR 520 Braided RampsInterchange Improvements. 2009. www.wsdot.wa.gov/biz/contaa/designbuildcontracts/bellevue
%20braids/default.htm. Accessed March 1, 2010.
14. Washington State Department of Transportation. Design-Build Contract,
Chapter 1 General Provisions: I-405/NE 8th St to SR 520 Braided
RampsInterchange Improvements. 2009. www.wsdot.wa.gov/biz/
contaa/designbuildcontracts/bellevue%20braids/default.htm. Accessed
March 1, 2010.
15. Washington State Department of Transportation. Design-Build Contract,
Chapter 2 Technical Requirements: I-405/NE 8th St to SR 520 Braided
RampsInterchange Improvements. 2009. www.wsdot.wa.gov/biz/
contaa/designbuildcontracts/bellevue%20braids/default.htm. Accessed
March 1, 2010.
16. Cambridge Systematics, Inc. 20052035 Georgia Statewide Transportation Plan Update. 2009. www.dot.state.ga.us/informationcenter/
programs/transportation/Pages/swtp.aspx. Accessed March 1, 2010.
17. FHWA. Highways for Life: Final Report. 2009. www.fhwa.dot.gov/
hfl/summary/ga/index.cfm. Accessed March 1, 2010.
18. Proposal: Interchange Construction on I-85/SR 403 at CR 98. Georgia
Department of Transportation, Atlanta, 2007.
19. Tracker: Measures of Departmental Performance. Missouri Department
of Transportation, Jefferson City, 2009, pp. 1233.
20. Missouri Department of Transportation. MoDOT Tracker. 2010. www.
modot.mo.gov/about/general_info/Tracker.htm. Accessed March 1,
2010.
21. Missouri Department of Transportation. Request for Proposals for the
Bridge Improvements Project: Book 1Design Build Contract. 2008.
http://search.mo.gov/search?q=safe+and+sound+RFP+book+2&btn
G=MoDOT+Search&Submit.x=0&Submit.y=0&output=xml_no_dtd&
client=modot&num=10&proxystylesheet=modot&sort=date:D:L:d1&
entqr=3&oe=UTF-8&ie=UTF-8&ud=1&site=modot. Accessed March 1,
2010.
22. Missouri Department of Transportation. Request for Proposals for the
Bridge Improvements Project: Book 2Performance Requirements.
2008. http://search.mo.gov/search?q=safe+and+sound+RFP+book+2&
btnG=MoDOT+Search&Submit.x=0&Submit.y=0&output=xml_no_
dtd&client=modot&num=10&proxystylesheet=modot&sort=date:D:
L:d1&entqr=3&oe=UTF-8&ie=UTF-8&ud=1&site=modot. Accessed
March 1, 2010.
23. Missouri Department of Transportation. Request for Proposals for the
Bridge Improvements Project: Instructions to Proposers. 2008. http://
search.mo.gov/search?q=safe+and+sound+RFP+book+2&btnG=MoD
OT+Search&Submit.x=0&Submit.y=0&output=xml_no_dtd&client=
modot&num=10&proxystylesheet=modot&sort=date:D:L:d1&entqr=
3&oe=UTF-8&ie=UTF-8&ud=1&site=modot. Accessed March 1, 2010.
24. Minnesota Department of Transportation. Minnesota Statewide Transportation Policy Plan 20092028. Jan. 2009. www.dot.state.mn.us/
planning/stateplan/download.html. Accessed March 1, 2010.
25. Minnesota Department of Transportation. Design Build Contract Book 1:
TH 61 Hastings Bridge Design-Build Project. Jan. 2009. www.dot.state.
mn.us/designbuild/hastingsbridge/. Accessed March 1, 2010.
26. Minnesota Department of Transportation. Design Build Contract Book 2:
TH 61 Hastings Bridge Design-Build Project. Jan. 2009. www.dot.state.
mn.us/designbuild/hastingsbridge/. Accessed March 1, 2010.
27. Minnesota Department of Transportation. Design Build Contract Book 3:
TH 61 Hastings Bridge Design-Build Project. Jan. 2009. www.dot.state.
mn.us/designbuild/hastingsbridge/. Accessed March 1, 2010.
28. Minnesota Department of Transportation. Design Build Contract
Instructions to Proposers: TH 61 Hastings Bridge Design-Build Project.
Jan. 2009. www.dot.state.mn.us/designbuild/hastingsbridge/. Accessed
March 1, 2010.
The Project Delivery Methods Committee peer-reviewed this paper.