HALCROW FOX

Contents
Page SUMMARY FICHE 0 EXECUTIVE SUMMARY 0.1 0.2 0.3 0.4 Introduction Methodology Applications of the PAM Conclusions 0-1 0-2 0-7 0-12

INTRODUCTION 1.1 1.2 1.3 Background Terms of Reference Report Content 1-1 1-1 1-3

POLICY ASSESSMENT METHODOLOGY 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 The Decision Model Alternative Appraisal Methods Cost-Benefit Analysis Planning Balance Sheet The Goals Achievement Matrix Comparison of Alternative Methods The Policy Assessment Models Role in Transport Planning Study Methodology Benefits of Goals Achievement Matrices in Transport Planning 2-1 2-3 2-4 2-5 2-6 2-6 2-8 2-10 2-14

TOP-DOWN ANALYSIS - POLICY DATA AND MEASURES OF PERFORMANCE 3.1 3.2 3.3 Introduction The Policy Structuring Process Identification of Measures of Performance 3-1 3-3 3-7

HALCROW FOX

Contents cont./
Page 4 BOTTOM-UP ANALYSIS - EX-POST EVALUATIONS, REVIEW OF PREDICTIVE MODELS AND THE GROUP IMPACT FRAMEWORK 4.1 4.2 4.3 Introduction Ex-Post Evaluations The Group Impact Framework 4-1 4-1 4-5

OPERATIONALISING THE POLICY ASSESSMENT MODEL 5.1 5.2 5.3 5.4 5.5 5.6 Introduction Model Structure Weighting Systems Background Data Measures of Performance - An Example Application of Weightings - An Example 5-1 5-1 5-3 5-6 5-7 5-9

TESTING THE POLICY ASSESSMENT MODEL 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 Introduction The IC5/IC25 Toll Motorway - Guimares/Chaves Border, Portugal The Igoumenitsa - Volos/Lamia Motorway, Greece TGV Nord, France Decin-Praha-Breclav Rail Upgrade, Czech Republic Twente-Mittelland Kanal, Germany - The Netherlands The Amsterdam Ring Road, The Netherlands Refinements to the Policy Assessment Model 6-1 6-2 6-8 6-12 6-21 6-26 6-29 6-33

7.

DISCUSSION AND SUMMARY FINDINGS 7.1 7.2 7.3 7.4 7.5 Introduction Principles of a Goals Achievement Matrix Policy Related Issues Performance of the Policy Assessment Model Policy Assessment Model Data Requirements 7-1 7-1 7-2 7-5 7-8

HALCROW FOX

Contents cont./
Page

8.

FURTHER RESEARCH AREAS 8.1 8.2 Potential Research Areas Links to Other Parts of Tenassess 8-1 8-2

REFERENCES

APPENDIX VOLUME 1 A B C D List of Tenassess Task 32 Deliverables Deliverable D(E) - Group Impact Framework Deliverable D(G) - Measures of Performance Sample Model Information

APPENDIX VOLUME 2 E Deliverable D(J) - Results of Demonstration Studies

Figures

HALCROW FOX
Page 0.1 2.1 2.2 2.3 2.4 2.5 2.6 2.7 6.1 6.1a 6.2 6.2a 6.3 6.3a 6.4 6.5 6.5a Structure of Task 32 The Classic Decision Model The Role of the Policy Assessment Model The Policy Assessment Model in Context with Transport Planning Structure of the Policy Assessment Model Structure of Task 32 Value Systems, Goals and Appraisal Methods Applying the Policy Assessment Model Planned Portuguese Toll Motorway Network Scoring of IC5/IC25 against national Portuguese policy objectives Igoumenitsa - Volos / Lamia Motorway - Route Options Comparison of three different policy scenarios of Greek motorway project TGV Nord Case Study Alternative Route Alignment Comparison of unweighted ex-post and ex-ante scores for the TGV Nord project Czech Rail Upgrade Case Study Rail Upgrade Route Schematic Diagram of Amsterdam Ring Road Unweighted Dutch national scores for the Amsterdam Ring Road 0-3 2-2 2-7 2-9 2-11 2-14 2-15 2-19 6-3 6-7 6-9 6-12 6-17 6-21 6-22 6-30 6-31

Tables
0.1 3.1 3.2 4.1 4.2 4.3 5.1 6.1 6.2 6.3 PAM Example Application - TGV Nord Ex-Post Assessment with Respect to French National Policy Initial Grouping of Austrian Policy Data into Aggregate Areas Policy Areas and Key Benchmarks - A Summary The Group Impact Framework (schematic) - Functional Relations between Groups and Impacts Financial Equity Summary Table Master Table PAM Example of Scores and Weightings - Czech Rail Upgrade Travel Time Savings, National Context Travel Time Savings, European Context Summary Case Study Results 0-7 3-4 3-19 4-7 4-23 4-27 5-10 6-14 6-15 6-32

Summary Fiche
The Tenassess project covers Task 32 and Task 36 of the Fourth Framework Strategic Transport Research Programme. The project team comprises 15 main partners from 9 different European countries, under the overall co-ordination of the ICCR, Austria. The Task 32 project team is led by Halcrow Fox in conjunction with ICCR, NEI, University of Cologne, PLANCO, DROMOS, EXACTO, INRETS and FACTUM. The objective of the Task 32 component of the Tenassess project is to produce a preliminary assessment methodology related to decisions on transport infrastructure investments and service (pricing and regulation) evaluations; more specifically, to develop a methodology that could be utilised in the assessment of the Trans-European Networks and other major infrastructure projects related to the Common Transport Policy (CTP). The general approach to this study has been to develop a version of a Goals Achievement Matrix, termed the Policy Assessment Model (PAM). This is a multi-criteria appraisal tool, which seeks to measure the extent to which a given transportation project achieves, or constrains, explicit policy objectives. This tool scores the impacts of large scale transportation projects against the policy objectives of decision makers. The research has identified important project impacts and whether / how they can be measured to produce the required information in the requisite format. The heart of the PAM is a series of relationships that relate the project impacts to specific, measurable policy objectives via, what has been termed, measures of performance. These mathematical relationships transform project impacts into a score that assesses whether the project contributes or detracts from policy goals at different levels in the decision-making hierarchy. The PAM is not intended to act as a replacement for traditional project appraisal techniques such as CBA and Environmental Impact Assessment (EIA). One potential role identified for such a methodology is as a form of screening tool with which a large number of projects or project variants can be sifted with relative ease by decision-makers, to ascertain an early view on whether projects broadly conform to policy objectives or not. Results are displayed in both disaggregate (at the level of individual policy objectives) and weighted (the model user can use weights to place greater emphasis on certain policy areas). Graphical display options are also available.

HALCROW FOX

0.
0.1 0.1.1

Executive Summary
INTRODUCTION The Tenassess project comprises Tasks 32 and Tasks 36 of the Strategic Transport section of the Fourth Framework Research Programme. The Tenassess project team comprises 15 main partners from 9 different European Countries under the overall co-ordination of the ICCR, Austria. The Task 32 project team is led by Halcrow Fox in conjunction with ICCR, NEI, University of Cologne, PLANCO, DROMOS, EXACTO, INRETS and FACTUM. The objective of the Task 32 component of the Tenassess project is to produce a preliminary assessment methodology related to decisions on transport infrastructure investments and service (pricing and regulation) evaluations; more specifically, to develop a methodology that could be utilised in the assessment of the Trans-European Networks and other major infrastructure projects related to the Common Transport Policy (CTP). The focus of the work has thus been to develop a method for assessing the impacts of major transport projects on European policies - the ordering of these terms is crucial to the understanding of the method and should be borne in mind throughout this document. To reiterate, the focus is on how projects contribute to policy objectives. The work has been methodological in nature; that is, identifying a suitable, preliminary approach to policy assessment. Nevertheless, in order to fully test and demonstrate the method, considerable progress has been made in specifying the detailed elements and programming a policy assessment model. It must be emphasised at the outset that the method presented is not intended to be the final version of any approach to policy assessment. Rather, in line with the study Terms of Reference, it represents an initial, developed, approach. The general approach to this study has been to develop a version of a Goals Achievement Matrix, termed the Policy Assessment Model (PAM). This is a multi-criteria appraisal tool, which seeks to measure the extent to which a given transportation project achieves, or constrains, explicit policy objectives. The benefits of such an approach are considerable: a PAM provides a dual focus upon policies and the impacts of the projects in question; the theory under-pinning a PAM reflects the value system of the policies as opposed to externally imposed value systems such as those of economic and financial appraisal methods; the PAM is a highly versatile tool, able to evaluate policies from a wide range of sectors; the PAM is flexible, able to reflect changes in policy or the relative importance of policies; 0-1

0.1.2

0.1.3

0.1.4

0.1.5

0.1.6

HALCROW FOX
the PAM is a transparent and, therefore, democratic technique, in which the entire basis for the assessment is explicit and open to inspection and audit. 0.1.7 It should also be noted that the PAM developed here is compatible and able to interact with other elements and tools of the Tenassess project. Indeed, several of the assessment tools being developed as part of the Task 36 side of the Tenassess project are of direct relevance and use to the PAM described in this report. The PAM is not intended to act as a replacement for traditional project appraisal techniques such as CBA and Environmental Impact Assessment (EIA). The main role identified for such a methodology is as a form of screening tool with which a large number of projects or project variants can be sifted with relative ease by decision-makers, to ascertain an early view on whether projects broadly conform to policy objectives or not. Once a reduced number of projects have been identified, more detailed project evaluation with traditional techniques can be undertaken to provide a robust estimate of project viability. Given the long lead time from project inception to project implementation, the PAM can also be used as a monitoring tool to assess at regular intervals, whether projects selected through the initial screening process continue to be in line with policy objectives that are continually evolving in terms of priority and relevance.

0.1.8

0.1.9

0.2 0.2.1

METHODOLOGY Figure 0.1 provides an overview of the PAM. The figure also summarises the process by which the PAM has been developed.

Figure 0.1: Structure of Task 32

Topdown

Stage 1

Policies, impacts, objectives

Stage 3

Goals achievement matrix, measures of success & failure, weighting system Bottom -up
Stage 2

Stage 4

Stage 5

Case Studies
refinement & adjustment

Conclusions

feedback

Impact measurement

One case study

0-2

HALCROW FOX

0.2.2

The PAM can be seen to comprise a top-down and a bottom-up module. The topdown module is policy-focused, generating information on relevant policies, their objectives and allocating these to a number of policy areas. These policy areas include: fair and efficient pricing; traffic restraint; modal split targets; accessibility; environmental quality; regional development; European harmonisation; deregulation and liberalisation; safety; public acceptability; and financial viability.

0.2.3

Allocation of specific policies to these policy areas has allowed explicit, quantifiable objectives to be specified for each area. There are circumstances in which policies that are specific to a particular country or region do not fit into this framework. Where this occurs, the method allows additional objectives to be defined. Policies are drawn from local, regional, national and super-national levels and relate to all sectors of relevance. A key component of this module is the identification of explicit points of success and failure for each policy area. Once the end points have been identified, an explicit mathematical relationship is defined that traces the path between failure and success. These relationships have been termed measures of performance. In most instances, a linear relationship has been defined but there is no reason, in principle, for other mathematical forms - such as an all or nothing approach where anything but total success is viewed as total failure. Project impacts are then assessed and scaled as to whether they further progress towards the successful achievement of an area of policy, or contribute towards failure. This is achieved by specifying the end points of the measure of performance as -5 and +5. The impacts of a particular project score somewhere between these two delimiting values, dependant upon whether the impacts are in line with policy objectives (a positive score) or in conflict with them (a negative score). For example, in assessing projects with respect to policies of economic development, World Bank information dated from 1994 has been used. The data are average economic rates of return of implemented infrastructure projects, disaggregated by mode. The figures range from between 14% for railway projects

0.2.4

0.2.5

0.2.6

0.2.7

0-3

HALCROW FOX
and 25% for highways. These benchmarks have been translated into scores, by assigning a project with : the benchmark rate of return a score of zero; more than double the benchmark a maximum score (5); and less than zero rate of return a minimum score (-5). Intermediate scores are assigned by linear interpolation. 0.2.8 The bottom-up module is project-focused, identifying the main impacts of projects and means of measuring and forecasting these. Impacts include social, economic, environmental and other fields of relevance to major transport projects. A notable element of this module is a Group-Impact Framework (GIF) listing and cataloguing the main impacts of transport projects and the groups on which they fall. The GIF also contains details of existing measurement and forecasting models, relevant to each type of impact. This is a pragmatic part of the methodology, in that any impact for which there is not an available measurement technique has been excluded from the analysis. It should be stressed at this point that the Tenassess project does not specifically provide any new tools for the measurement of the impacts of large scale European transport projects. Whilst the methodology has designed a framework for assessing the impact of projects on policy objectives, it is left to other models to provide the specific transportation and non-transportation impacts of specific projects. In the case studies analysed as part of the Tenassess projects, information has been taken from existing models and reports. It is fully intended that the transportation modelling projects being conducted elsewhere within the Fourth Framework Research Programme should be utilised in order to better understand project impacts at the European level.

0.2.9

0.2.10 The central part of the PAM integrates the top-down and bottom-up elements, producing a set of quantifiable measures of performance for each policy area, as outlined above. These form the heart of the methodology, identifying the extent to which the impacts of the project in question support the policies or constrain them. A weighting system is also built into the PAM, allowing users to reflect the comparative importance of policies, as well as the intrinsic value of impacts. 0.2.11 This central, operational, part of the PAM scores impacts on each policy area using a cardinal scale, ranging from 5 (highly negative) to +5 (highly supportive). These scores identify the value of the impacts upon each policy area. The weighting system applies a weight (between 0 and 3) to each value score to reflect the comparative importance of each policy area. The final score is the product of the two. These are summed across all policy areas to produce a final score for each run of the model. 0.2.12 The choice of scoring and weighting system has been driven by a common sense approach and a desire to make the operation of the model as understandable to the non-technician as possible. The -5 to +5 scoring broadly corresponds to a commonly used scale outside and within the transport sector, where respondents

0-4

HALCROW FOX
are asked to rank alternative between very poor, poor, neutral, good and very good. It is the intention that such a system will be easily recognisable to the user of the PAM. The weighting system, on the other hand, allows certain policy areas to be neutralised (a weighted score of zero) or others to be given greater levels of importance (a two for moderate importance and a three for very important). This then leaves the number one vacant to represent the unweighted, but standardised (or normalised) score. 0.2.13 The adoption of alternative weighting systems is not prohibited in order to make the PAM function, but a robust justification for the adoption of an alternative would need to be provided.

0-5

HALCROW FOX

0.2.14 It is important to note that individual scores are not, in themselves, an important output of the PAM. Indeed, the PAM is not designed to generate a single figure summary of the worthiness of a project. Rather, the PAM is an analytical tool, designed to test a variety of options, refinements and weights in order to generate a measure of the overall worthiness and robustness of a project. The PAM allows the user to explore, not only which project, or option, is optimal, but also why a project performs better, or worse, than others. 0.2.15 Table 0.1 demonstrates the relationships between impacts and both raw and weighted scores, using a case study as an example. The TGV nord case study is used, and the assessment is made from the perspective of French national policy in 1996, after the project's completion (ex-post). 0.2.16 For each policy area the necessary impact required for a maximum possible score is exemplified (table, column 2), as is the actual project impact (column 3). In column 4 each policy is assigned a weighting which seeks to reflect French national policy with respect to transport. Column 5 shows the raw score, i.e. one that is based on impacts and not on weightings. The final column presents scores which take account of the policy weightings: the weightings are factored down so that they have a mean value of one, and are multiplied by the raw score. 0.2.17 It can be seen from Table 0.1 that the project scores well with respect to French specific policy areas: 'regional acceptability', 'promotion of technological ingenuity' and 'French accessibility criteria'. The project is in conflict with the policy area of 'deregulation, privatisation and liberalisation', which is given a negative score and a high priority weight of 3. Other policy areas of maximum weighting have a positive score, indicating that the project is complying with them. 0.2.18 The overall raw score totals 11.4, whereas the total weighted score is 14.6. This can be interpreted as meaning that the project is more attractive with respect to French national policy than a 'nave' (unweighted) assessment would imply.

0.3 0.3.1

APPLICATIONS OF THE PAM The PAM has been tested on six case studies in order to refine and demonstrate the method. All are ex-post assessments (although the TGV Nord study does provide the opportunity for an ex-post / ex-ante comparison) and together, they involve a mix or road, rail and waterway projects. The case studies are:

0-6

HALCROW FOX

Table 0.1: PAM Example Application - TGV Nord Ex-Post Assessment with Respect to French National Policy Policy Area
Environment Fair & Efficient Pricing Improve Transport Accessibility & Regional Development Harmonisation & Common Market Traffic Avoidance Reduce price of transport Restrict road transport Improve Safety Deregulation, privatisation & liberalisation Financial acceptability Public acceptability Regional acceptability Promotion of Technological Ingenuity French Accessibility Criteria

Requirement for Full Score

Emission reductions per annum: CO2 -0.3%, CxHy -8%, NOz -13%, particulates -3% Ratio of user cost to social cost is the same for each modes No increase in car or road freight mode share 28% economic rate of return (RoR) Ratio of cross-border growth in traffic relative to internal growth is 140% The project halts any increase in traffic intensity User benefits per annum > 6.15% of total investment costs Not applicable - no policy on this 7.4%reduction per annum in injuries and fatalities Internal rate of return (IRR) of 24% Qualitative assessment Qualitative assessment Qualitative assessment Qualitative assessment Qualitative assessment

Project Impact
Emission reductions per annum: CO2 0.6%, CxHy -2%, NOz -0.6%, particulates -1.5% Approximately no change in ratio, though changes in costs occur Road transport mode share decreases 10% economic RoR Ratio is 128% The project has negligible impact on traffic intensity User benefits per annum are 2.3% of investment cost N/A 0.4% reduction per annum in injuries and fatalities IRR of 6% N/A N/A N/A N/A N/A

Policy Weight (0 to 3)
2 2 3 2 3 1 3 1 2 3 1 3 3 2 3

Raw Score (-5 to 5)

3.7 0 1.0 -1.4 3.4 0.01 0.8 0 -4.5 -2.5 0 4 4 3 0

Weighted Score
3.2 0 1.3 -1.2 4.5 0.0 1.1 0 -4.0 -3.3 0 5.2 5.2 2.6 0

0-7

HALCROW FOX
0.3.2 IC5/IC25 Motorway, Portugal; Igoumenitsa-Volos/Lamia Motorway, Greece; TGV Nord High Speed Railway, France; Decin-Prague-Breclav Rail Corridor, Czech Republic; Twente-Mittelland Kanal, Germany; and The Amsterdam Ring Road, The Netherlands.

In general, the case studies demonstrate the success and potential value of the PAM, in a variety of contexts. They have also permitted a number of initial amendments to be made to the model. These have included the addition of extra policy areas (where project geography or local political constraints deem it necessary) and amendments to the PAM to increase its sensitivity to regional and economic development objectives in peripheral regions of the EU. For example, a specific French regional accessibility criteria was deemed necessary in order to reflect a very explicit French policy goal. Similarly, additional policy areas were required for the Portuguese case study to reflect some specific Portuguese developmental objectives. In other instances, some of the original policy areas proved irrelevant to the policy debate - for example, traffic avoidance and traffic restraint policies do not feature at all in the policy debate that currently exists in Greece or Portugal. As such, this reflects a difference in policy objectives between northern and southern/eastern European countries, but it also reflects a perceived difference between objectives in the large northern European countries when compared to the smaller ones. In the more densely populated regions of northern Europe, issues surrounding the negative side effects of road transport tend to dominate the policy debate. In southern, eastern and other peripheral regions, economic growth, with the associated increases in (largely road) traffic, is the most important issue on the political agenda. The observed difference between large and small northern European countries is that infrastructure investment project are still seen as being an important solution to transport problems in those countries where geographic space does not constrain its construction. This conclusion supports the conclusions of the work earlier parts of the Tenassess project which examined national transport policies throughout Europe in far more detail than has been done here. The Czech case study provides another interesting angle on this policy issues debate, insofar as the project is being promoted on grounds of economic growth, but also to improve the acceptability of the Czech Republic as a new member of the European Community. The core / periphery policy divide identified in the preceding sections has several implications for individual areas of policy. The most transparent of these is the treatment of the environmental externalities associated with all forms of transport. The relevance of these issues to the policy debate is a direct function of the level of development of the basic transportation network. Where this minimum level of 0-8

0.3.3

0.3.4

0.3.5

0.3.6

0.3.7

HALCROW FOX
infrastructure is lacking, environmental concerns (at the strategic level) tend to be well down the policy hierarchy. In the more highly developed region and countries of the European Union, where the base network is well developed, the negative aspects of transport are far higher up the political agenda. 0.3.8 If the policies of the European Commission are taken in their entirety, a number of potential policy conflicts present themselves. The recognition of several broad thrusts to Community level policy does, however, explain these perceived conflicts. For example, policies aimed at economic growth and social cohesion are, in the main, focused in peripheral regions, whereas the more restrictive polices of say, fair and efficient pricing and sustainable mobility are aimed largely at the core of the Union. In this, Commission policy is very much in step with the picture that emerges from an assessment of national or regional policies. The bulk of the transport related policies identified during the production of the Tenassess country reports [Country Reports, Internal Report D(K), Tenassess Project Team, January 1997] tended to focus on the interaction of road and public transport, particularly the desire in many countries to achieve a more sustainable balance between road and public transport modes.

0.3.9

0.3.10 In the sphere of Trans European Networks, high speed rails main competitor is, in many cases, air transport rather than the private road transport sector. The Policy Assessment Model has been modified to examine these rail / air issues in the case of the TGV Nord case study. In terms of policy conclusions, these issues do not appear to be well articulated in the policy documents of many countries.

Infrastructure, Pricing and Regulation 0.3.11 Whilst the six case studies undertaken are all transport infrastructure investment projects, pricing and regulatory initiatives can be assessed in an identical manner, insofar as the Policy Assessment Model operates by scoring the impacts of any project against policy objectives. Additionally, many pricing and regulatory issues are already operationalised within the Policy Assessment Model in terms of the measures of performance that operationalise policy objectives. The case studies in themselves bring out some pricing issues such as prices in the Czech Republic generally being too low to justify the capital costs of high speed rail and Portuguese values of time being too low allow high toll charges that are required to make the projects financially viable. 0.3.12 As part of the initial policy structuring process, a clear distinction between laws and policies has been observed. It is an important task to make this distinction, insofar as the Tenassess project is concerned with the latter rather than the former. The dividing line that has been adopted for the purposes of the development of the Tenassess Policy Assessment Model is the recognition that laws are a mandatory requirement that all proposed transport projects must satisfy. In other words, there is no room for trading project impacts against the legal framework - the laws of the European Community and the Member States essentially form the boundaries within which the Policy Assessment Model must operate. This is not to suggest

0-9

HALCROW FOX
that conflicts between different laws do not exist but rather stipulates that it is not the purpose of a policy research project to address these issues. 0.3.13 Policies, unlike legal requirements, represent a desired end state in a given area of society. In an ideal world, all policies would be achieved in their entirety but there is no legal compulsion to achieve them. It is this distinction that defines the subject matter of the Tenassess project. 0.3.14 The Policy Assessment Model clearly highlights conflict within the policy hierarchy. The standard deviation of the individual policy area scores, together with the graphical presentation of the results, gives a clear indication of the degree to which projects are in line with policy objectives or in conflict with them. 0.3.15 In terms of policy scores, road projects tend to have a high standard deviation and rail projects a low one. In general, rail projects tend to score positively across the entire range of policy areas whereas road projects tend to score very highly in some areas and poorly in others. This finding would tend to support the hypothesis that the current policy thrust in many European countries is towards rail and away from road projects. 0.3.16 Whilst most of the case study results have been described in terms of overall scores, these numbers should not be interpreted as giving a definitive answer on project performance. It is only through extensive use of the method on a large number of different projects (similar projects in the same country or across many projects in many countries) that the total scores start to become meaningful in their own right. With a limited number of studies available for comparison, the total score should be interpreted as an indication of the most likely outcome, rather than the definitive answer. 0.3.17 The ex-post / ex-ante project analysis for TGV Nord highlights how the model can show how changing policy objectives or optimistic project performance figures can produce a very different result. 0.3.18 Following from the previous point, the ex-ante / ex-post analysis of TGV Nord has highlighted that the model results essentially confirm the reasons for the projects progression through the planning process to construction. The model also performs well in predicting the preferred route option. 0.3.19 The Czech case study highlights that the incorporation of the policy objectives of other countries whom are affected by the project can have a deciding impact upon the project score. One of the most important issues here would appear to be: if the project benefits are felt abroad, can any of the project costs be recouped from the beneficiary state?

0-10

HALCROW FOX
0.3.20 As far as the practical application of the PAM, the assessment of a mere six case studies has necessitated the collation of a large amount of project specific and background data. The latter is required in order to operationalise many of the policy measures of performance which are, by their very nature, of a temporal nature. 0.3.21 The quality of background data1 varies enormously across the European Union and very little forecast data exists. Given that many policy objectives specify the attainment of a certain level or value by a given date in the future, this forecast data is of great importance to the assessment process. It is hoped that other projects within the Fourth Framework Research programme will be able to advise on a wider range of robust forecasting data for the areas required by the model. 0.3.22 The Policy Assessment Model also specifies that it should receive data in a certain format. The conclusion from the testing work done to date is that an assessment of this type should start with a detailed modelling exercise in order to produce the requisite material in the correct format. Post-processing of existing project impact data and estimation of missing components is not a trivial task. This conclusion has implications for the scope and depth of project appraisal undertaken in different countries, as reported in other areas of European research2. In order, however, to fulfil the screening role of the PAM, utilisation of existing sparse project impact data is justifiable if its robustness can be verified. In many countries, this level of data would equate to that normally found in a project prefeasibility study, albeit with additions to address the specific policy areas that may be of relevance. 0.3.23 The definition of the project corridor and competing projects is crucial to the adequate analysis of the project. The Twente-Mittelland project highlights these issues through its competitor project the Betuwe rail line. Analysis of these conflicts, interactions and policy preferences for each scheme is crucial to an adequate understanding of the decision making process. 0.3.24 Project evaluations normally report project impacts as total figures for the project as a whole. As the Twente case study has show (this conclusion would seem particularly applicable to cross-border projects), the distribution of impacts is of central importance to the understanding of the policy objectives that lie behind a project and its chances of success.

0.4 0.4.1

CONCLUSIONS The principal conclusion from this part of the Tenassess project is that the PAM is an appropriate assessment tool for policy appraisal, worthy of further development. The model focuses upon the key area of project impacts on policies

EUROSTAT sources have provided the bulk of the background data material used within the Policy Assessment Model. See for example, the APAS report commissioned by DGVII on Methodologies for Impact Assessment.

0-11

HALCROW FOX
and appears to be a powerful analytical tool for exploring factors underlying the project-policy relationship. Other attributes include: flexibility the PAM addresses a wide variety of policy areas and is amenable to changing policies and new policy emphases; versatility the PAM can assess policies from a variety of sectors; transparency all inputs, transformations, weights and outputs are open to inspection and audit; Analytical power the case studies demonstrate the wide variety of issues that the PAM can address, along with its potential for use as an analytical tool. 0.4.2 The PAM is also compatible with other forms of assessment (economic, financial, environmental, etc.). It is not intended to replace other assessment tools, rather, to support them and allow a more comprehensive evaluation of major projects with significant policy implications. A number of other attributes of the PAM deserve note: The weighting system allows users to reflect differing levels of importance afforded to various policy areas (environmental, social, economic, etc.); this enables various perspectives to be explored and their implications assessed in advance of the actual decision-making process. The model is adept at illustrating and exploring policy conflicts, between sectors, different levels in the policy hierarchy, etc. The standard deviation of the summary scores provides an initial indication of the degree of conflict, whilst sensitivity tests using the model allow more detailed examinations. The case studies have demonstrated how practical policy conclusions can be drawn from the models applications. They also illustrate how past assessments using other methods can be replicated or audited using the PAM. The development work carried out to date demonstrates how the PAM can be operationalised within a reasonable time and budget constraint. Data needs are not a major constraint, nor are programming requirements. 0.4.4 The PAM is now in a state of development at which a more comprehensive version can be produced and used in an ex-ante context, alongside other, traditional assessment methods. Such an application, as part of a live planning process, would provide a further test and demonstration of the potential of the PAM. The PAM could also be applied to several projects, ex-post, within the same State, in order to derive the implied weights that have resulted in the actual decisions to proceed with or cancel projects. Such an exercise would highlight the consistency of policy-making and decision-making within a State (or region). Specific, detailed development is required in a number of areas, detailed in the main text. Within Tenassess, the Delphi research could be used to identify sets of normative weights for particular policy areas. The PAM could also be viewed of one of a

0.4.3

0.4.5

0.4.6

0.4.7

0-12

HALCROW FOX
family of policy-related models, for example, alongside the Task 36 Barrier Model. Together, such a system of models could be used, not only as assessment tools, but as means of exploring the broader technical, social and political implications of major transport projects.

0-13

HALCROW FOX

1.
1.1 1.1.1

Introduction
BACKGROUND This document represents the Final Report on Task 32 of the FP4 Strategic Transport Research Programme which aim to develop a policy assessment methodology for large scale transportation projects within Europe. The Task 32 research has been undertaken by the following partners of the Tenassess consortium: Halcrow Fox; NEI; University of Cologne; ICCR; PLANCO; INRETS; FACTUM; EXACTO; and DROMOS.

1.1.2

1.1.3

During the course of the project to date, the Task 32 research team have produced six internal project reports, together with a Modelling Module Progress Report [Deliverable R(2), Tenassess project team, April 1997, DGVII]. Two of these internal reports are attached to this report as a series of technical appendices. The Case Studies Internal Report (Internal Report D[J] - Results of Demonstration Studies, Tenassess Project Team, July 1997, DGVII) is a confidential document that cannot be reproduced at the current time within the scope of the public Final Report. This and the remaining three are cross-referenced to the relevant publications where necessary. A full list of the Task 32 Deliverables is attached to this report as Appendix A.

1.2 1.2.1

TERMS OF REFERENCE The general aims of the Tenassess project are outlined in the Technical Annex as follows: to provide a preliminary assessment methodology related to decisions on transport infrastructure investments and service evaluations; more specifically, the aim is to develop a methodology that could be utilised in the assessment of alternative TEN options and proposals on specific targets and issues related to the Common Transport Policy (CTP). and; To provide a comprehensive policy assessment of the European Common Transport Policy with a view to advancing forward recommendations that may assist its further development and implementation. Along these

1-1

HALCROW FOX
lines, to also deliver guidelines about the constructive use of science and research in the field of transport by policy makers.1 1.2.2 The first of these two aspects is the concern of Task 32 whereas the latter is the study material of the Task 36 section of the Tenassess project. In terms of the development of a preliminary policy assessment methodology, it was stated in the Technical Annex that the following ought to be noted: the development of a preliminary assessment methodology has to take into account a number of issues in the assessment methods used within Member States and across the Community as a whole, namely: (i) the inconsistency of treatment of road, rail and other transport infrastructure and service projects, (ii) the often different treatment of national as opposed to transnational transport objectives, (iii) the adoption of different evaluation methods, and within them, the inclusion of different sets of evaluation criteria and (iv) the adoption of different parameter values between adjacent states; the appraisal and assessment method will (i) build on current methods and practice, (ii) display a consistent approach between modes and regions, (iii) be transparent, easy to follow and open to inspection, (iv) be designed on a modular basis so that it can embrace all the relevant assessment criteria and can be applied at all stages of scheme appraisal, (v) include national and TEN requirements, and (vi) disaggregate the benefits to each group of traffic separately such that those generated from European traffic can be identified with respect to those that contribute only to local and national objectives.2

1.2.3

1.2.4

It is these questions that this report attempts to answer.

1.3 1.3.1

REPORT CONTENT Chapter 2 introduces the policy assessment methodology that has been developed for the Tenassess project in terms of the type of decision model that is required to answer the issues outlined in the preceding section. Chapter 3 describes two aspects of what has been termed the top-down part of the methodology. This chapter essentially deals with the identification, structuring and operationalising of policy objectives of different decision making bodies.

1.3.2

Policy Assessment of Trans-European Networks and Common Transport Policy, Technical Annex, Project TENASSESS, DGVII, (1996). Policy Assessment of Trans-European Networks and Common Transport Policy, Technical Annex, Project TENASSESS, DGVII, (1996).

1-2

HALCROW FOX
1.3.3 The next chapter describes the bottom-up approach which commences the analysis from the viewpoint of describing the impacts of strategic transportation projects. Chapter 5 then outlines how the methodology in Chapters 3 and 4 is operationalised into a working policy assessment tool. This test represents the principal output from the study. The Case Study testing phase of the project is described in Chapter 6. This analysis includes a series of worked examples of the Policy Assessment Model. Chapter 7 draws together the Task 32 research with an overview of the main findings and a discussion of the implications arising from the study to date. The Brief of Task 32 was to produce a preliminary assessment methodology. Bearing this in mind, Chapter 8 documents issues arising from the research work, together with a set of recommendations upon which action will be required before a comprehensive policy assessment methodology for large scale European transport projects can be developed.

1.3.4

1.3.5

1.3.6

1.3.7

1-3

HALCROW FOX

2.
2.1 2.1.1

Policy Assessment Methodology

THE DECISION MODEL Hill1 defines rational planning as A process for determining appropriate future action by utilising scarce resources in such a way as to maximise the expected attainment of a set of given ends. This definition provides the systemic framework for project appraisal, in which alternative courses of action are evaluated to identify which best achieve a set of given objectives. The implication of this framework is that the objectives form the evaluation criteria, the benchmarks, against which the performance of each option is measured. Objectives form a statement of need; evaluation measures the extent to which needs are likely to be fulfilled by each option. Hence objectives and evaluation form two sides of the same coin. This decision model reflects a standard, systems approach, to planning. The model is designed to ensure that projects remain focused upon their objectives, and that their success or failure is measured and expressed in terms of the particular needs driving the project. Figure 2.1 illustrates the classic decision model. Key aspects of this are: policies are formulated in response to perceived needs, based on specific problems (congestion, safety, etc.) or potential opportunities (regional development, reduced travel costs, etc.); policy objectives articulate the desired end-state; that is, the change that must be secured to fulfil the policy and exploit the opportunity or solve the specific problem. Objectives are of greatest value when they contain explicit targets and thresholds which specify success and failure; strategic plans take the broad policy objectives and produce a set of refined project objectives; they also provide an overview of the measures required to achieve the objectives; detailed proposals represent the options for action (new highway alignments; high speed rail proposals; etc.); Problems Opportunity

2.1.2

2.1.3

2.1.4

Perception of Need
1

Hill, M: A Goals Achievement Matrix for Evaluating Alternative Plans. Journal of the American Institute of Planners, Vol. 34, pp 19-29, 1968

2-1

HALCROW FOX

Socio-Political Influences Policy Formulation

Competing Needs

Policy Objectives

Strategic Plans

Detailed Proposals / Projects

Appraisal Methodology

Project Appraisal

Decision: Selection of Projects

Implementation of Projects

Figure 2.1: The Classic Decision Model an appraisal method links the alternative options with the policy objectives; that is, it identifies the extent to which options meet the objectives. The instrumental elements here are the measures of performance which form the central feature of an appraisal methodology; project appraisal applies the measures of performance to the alternatives and concludes on the extent to which each option meets the declared objectives; 2-2

HALCROW FOX

following selection of the preferred project the implementation commences.

2.1.5

This classic decision model recognises, fully, the role of socio-political influences in the process of project appraisal and selection. Such influences are often driven by competing and conflicting needs of other sectors of the economy (social welfare; regional development; finance; etc.). Indeed, it is quite possible and sometimes common, for socio-political considerations to over-ride the results of a technical appraisal. The fundamental role of the classic decision model is to make explicit the basis for selecting one particular project ahead of another and to demonstrate precisely where socio-political influences have over-ridden an analytical appraisal. To reiterate, the possible influence of socio-political factors in decision-making does not weaken the classic decision model; rather, it reinforces the need for it. The model makes explicit, either, how the selection of a project fulfils stated objectives; or where such a decision deviates from objectivity and submits to socio-political influences. In either case, the quality and integrity of decisionmaking is improved. The principal focus of the classic decision model is the appraisal method. This engineers the explicit linkage between objectives and project outputs. Central to the appraisal method are the measures of performance. These broad issues are the main subject of Task 32 of the Tenassess project.

2.1.6

2.1.7

2.2 2.2.1

ALTERNATIVE APPRAISAL METHODS The main principles guiding the development of appraisal methodology are as follows: policy orientated - the value system within the method must reflect the various policies to be incorporated within the appraisal; project focused - the method must address the desirability of major European transport projects; comprehensive - the method must be capable of addressing a large variety of issues and project impacts across several sectors; performance focused - policy objectives, the aspirations of policy making bodies, must form the central criteria by which the performance of projects is assessed; practical - the method is intended to be applied, ultimately, to a wide variety of transport projects across the EU; it must therefore be practical

2-3

HALCROW FOX
and impose realistic modest data needs on those using it. 2.2.2 Three techniques were considered as the core approach to this task. They differ in complexity and the extent to which non-quantifiable, subjective, judgements enter into their workings. They are respectively, cost-benefit analysis, the planning balance sheet and the goals achievement matrix method.

2.3 2.3.1

COST-BENEFIT ANALYSIS Cost-Benefit Analysis (CBA) derives from welfare economic theory and has the simple aim of finding the most efficient amongst several solutions (i.e. that which maximises net benefits to society). The analysis relies heavily upon quantifiable elements and the ability to value impacts in monetary terms. A number of other projects within the fourth framework are developing CBA techniques for major transport projects. The value system within which CBA operates has the aim of maximising social welfare. All impacts are expressed in utilitarian (social welfare) terms, with valuation of the impacts defined as either the resource cost (e.g. fuel, capital items) or as willingness to pay to secure a benefit (e.g. time saving). It is not intended to provide a detailed description of CBA here. A number of publications should be referred to for further detail, including Pearce and Nash2 and Ng 3. The method is widely, almost universally, used in project appraisal and is accepted by most governments, development institutions and funding bodies. Nevertheless, a CBA based method is not considered to fulfil the aims of this project. The principal factor acting against CBA in this context is that CBA is relatively narrow in its most common applications because of difficulties in quantifying and valuing a wide range of impacts according to a common cardinal unit (e.g. environmental, social and regional development impacts). CBA does not provide the explicit links between diverse policy objectives and project impacts. The rigorous requirements for quantification also reduce the breadth of applications. A less selective method therefore is required.

2.3.2

2.3.3

2.3.4

2.3.5

2.4 2.4.1

PLANNING BALANCE SHEET The Planning Balance Sheet (PBS), first proposed by Lichfield4, greatly extends traditional CBA by incorporating all the impacts implicit in alternative courses of action. The PBS overcomes one shortcoming of CBA in that it does not require the monetary valuation, or even the quantification, of impacts. Therefore it allows

Pearce, D.W. & Nash, C.A. The Social Appraisal of Projects, A Text in Cost-Benefit Analysis. 3 Ng, Y.K. Welfare Economics.

2-4

HALCROW FOX
the comprehensive inclusion of all relevant effects of a project within the appraisal. 2.4.2 The object of the PBS is to present a clear indication of (I) each impact; and (II) the groups upon which impacts fall within a set of project accounts - the balance sheet. Impacts, costs and benefits, are presented in monetary terms where possible, or quantified in other units, or presented descriptively. Whilst the PBS has a major advantage in terms of its breadth of coverage, the method's lack of objectivity is a significant constraint in the context of policy appraisal. For the tradition of PBS emphasises the role of planning judgement in weighing impacts, reflecting a single value system. This is often an over simplification which may negate the breadth of appraisal that the PBS is seeking to achieve. Thus, whilst PBS may be appropriate where there is a general consensus on values, goods or objectives, it has difficulty resolving resource allocation problems where there may be conflict within the planning or policy system. To illustrate this point, a project may have a major impact, say the loss of buildings of historic interest. However, a community may place little or no importance on the retention of such buildings (through its policy statement). Nevertheless, the PBS analyst may impose her own values on such a loss and include it in the appraisal, because there is no direct link to policies. The PBS method may be suitable for those project appraisals where a simple and consistent value system can be assumed. It is not considered suitable to address the complex policy issues inherent in major European transport projects. A more objective and democratic method is outlined below.

2.4.3

2.4.4

2.4.5

2.5

THE GOALS ACHIEVEMENT MATRIX Role in Appraisal

2.5.1

It is important to recognise that the role of any appraisal technique is to guide decision-makers: a GAM is not, by any means, the sole basis for decisions [Bruck et al5]. The type of GAM developed in this project (i.e. the Policy Assessment Model) is intended to advise decision-makers on the policy implications of major transport projects. The choice of project will be influenced by a number of other technical procedures (such as a CBA) and by reference to unspecified sociopolitical factors. The two over-riding roles of a GAM are: (i) to provide a channel of communication between professional planners and communities (through their

2.5.2

4 5

Litchfield, Cost Benefit Analysis in Plan Evaluation, Town Planning, 1964 Bruck et al, Evaluation of Alternative Transport Proposals. Journal of the American Institute of Planners 23, 1996

2-5

HALCROW FOX
political representatives); and (ii) to make this channel, and the information processed, explicit. 2.5.3 Figure 2.2 illustrates the basic role of appraisal in general and a GAM in particular in this process. It provides a clear link between projects and policy objectives, the focal point of which are the performance measures contained within the GAM.

2.6 2.6.1

COMPARISON OF ALTERNATIVE METHODS Whilst the previous section has suggested that the GAM is deemed the preferred tool for the purposes of the Tenassess project, similarities between this approach and, particularly, CBA do exist. To an extent, these similarities have already been discussed, but it is useful to reiterate them at this juncture.

2.6.2

2-6

HALCROW FOX

Transport Projects: Road Rail Air Water

Impacts: Economic Environmental Social Regional Financial Public Acceptance

Appraisal: Measures of Performance POLICY ASSESSMENT MODEL

Policy Objectives: Access Environment Development Efficiency

Policies: EU National Regional Local

Figure 2.2: The Role of the Policy Assessment Model

2-7

HALCROW FOX
2.6.3 In the Tenassess policy assessment model, there is no need, or indeed wish, to place a monetary value of project impacts, as the method leaves this valuation to the decision-maker to express through the weights that they attach to different policy areas. In practice, however, project impacts with monetary valuations do enter into the process wherever policy objectives deem them necessary. For example, a policy aimed at reducing the cost of transport to the user requires the policy maker to know by how much a certain project may actually reduce costs - these costs will include valued time savings and vehicle operating costs. Whilst the decision-maker then has to place a weight on the importance of this policy within all other policies, it does not represent a double valuation. The separation of citizens valuations of say travel time savings from the importance that society (through the political decision making process) places upon time savings relative to other project impacts is an important distinction to be made. The CBA valuation of individual impact areas is always done in isolation from other impacts - research into time valuation or accident valuations, for example, rarely if ever presents the information in a context of a much wider range of scenarios and project impacts. The GAM on the other hand, allows these trade-offs and inter-relationships to be explicitly examined - as all the information is in front of the decision-maker at the same point in time, and not aggregated into one overall bottom-line number. Altering the weights within a GAM analysis is akin to sensitivity testing of CBA results, although there is no reason why, in principle, that any CBA-derived project impacts used within the GAM cannot also be subjected to sensitivity testing - doing so may assist the decision-maker to further understand the implications of adopting a certain weighting set.

2.6.4

2.6.5

2.6.6

2.6.7

2.7

THE POLICY ASSESSMENT MODELS ROLE IN TRANSPORT PLANNING Figure 2.3 outlines the Policy Assessment Model in the context of transport planning. The key driving force can be seen to be the policies, which determine objectives, transport strategies and proposals, and the performance measures which link the two.

2.7.1

2-8

HALCROW FOX

Problems

Opportunities

Socio-political Aspirations

Policies

Policy Objectives

Transport Strategies

Objective Thresholds: definition of success definition of failure

Transport Proposals: infrastructure regulation pricing

Performance measures: Measurement systems for assessment of impacts upon objectives

Project Impacts: socio-economic financial public acceptance

Application of performance measures to impacts; Scaling of magnitude of impacts upon objectives

Framework of measurement and presentation of impacts

Weighting System: objectives impacts sectors Interpretation

POLICY ASSESSMENT MODEL

Sensitivity / Policy Option Testing

Figure 2.3: The Policy Assessment Model in Context with Transport Planning

2-9

HALCROW FOX

2.7.2

The process illustrated in Figure 2.3 corresponds closely to the classic decision model shown in Figure 2.1. This systematic approach to transport planning reflects the Commissions classical, policy-led approach, outlined in various EC transport planning documents6. Thus, the proposed development of such a tool within the project corresponds closely with the general, objective, approach to policy-making and transport planning practised by the Commission.

2.8
2.8.1

STUDY METHODOLOGY The subsections that follow outline in more detail how the theoretical aspects of a Goals Achievement Matrix have been transformed into an operational Policy Assessment Model for the Tenassess project. Figure 2.4 provides a basic outline of the structure of the proposed Policy Assessment Model, along with its linkages to other parts of the policy appraisal process. Key features are as follows:

2.8.2

The assumed starting point for both objectives and transport projects are the various national, regional and local policies. Projects should be designed to achieve particular policies, whilst objectives are the operational expression of policies. Transport projects under consideration include infrastructure projects (new rail lines, highway improvements, waterways, etc.); pricing proposals (e.g. road pricing); and regulatory proposals (de-regulation, liberalisation, border controls). A comprehensive range of impacts must be identified for the projects likely to be addressed by the methodology. These include economic, environmental, social, financial and regional development impacts. In addition, it is proposed to include the likely implications of projects for public opinion, as this can be a major determinant of whether a project progresses to implementation.

Commission of the European Communities, The Future Development of the Common Transport Policy - A global approach to the construction of a Community framework for sustainable mobility. Bulletin of the European Communities, Supplement 3/93.

2-10

HALCROW FOX
Policies

Major Transport Projects infrastructure pricing regulation

Policy Areas Environmental Transport Performance Harmonisation Liberalisation Safety Fair & Efficient Pricing Accessibility Traffic Avoidance Finance Policy Objectives

Project Impacts Economic Environmental Development Regional Financial Public Acceptance

Definitions of success and failure

Group Impact Framework categorisation of impacts summary of groups affected

Measures of Performance relating impacts to objectives

Performance Scale Cardinal scale for performance measures

Weighting system: Objectives Impacts Sectors

Results summary

Interpretation and conclusions

Figure 2.4: Structure of the Policy Assessment Model Impacts should be presented within a structured framework, a Groupimpact Framework (GIF). This identifies: the category of impact (economic, environmental, social, etc.); the group upon which each impact falls (user, non-user, operator); the method for measurement and forecasting of each impact;

2-11

HALCROW FOX

Where it is not possible to measure or forecast a particular impact, a qualitative index is developed. Policy areas are identified to structure and rationalise the diverse range of policies which must be considered within appraisals. These cover broad policy issues, such as the environment, pricing and safety. Policy areas are defined for each relevant level in the policy hierarchy (local, regional, national, and EU). Policy objectives are imputed from the publicised policies. Ideally, objectives will be stated explicitly within policy documents. Otherwise, the analysts must use judgement to define objectives. For each objective, a definition of success and failure must be produced. These definitions help to operationalise objectives by making explicit the point at which a project can be judged to have made a significant impact, either by supporting or constraining the objective. Measures of performance provide an explicit linkage between project impacts and objectives. They measure the extent to which an objective is achieved as a result of implementing a new project. These may be quantitative measures (e.g. changes in levels of atmospheric pollution; changes in modal shares) or qualitative indices (levels of harmonisation). A cardinal, numeric scale provides a common means of scoring each performance measure. A scale of +5 to -5 is used in this study. A weighting system allows decision-makers to apply their own values to various parts of the appraisal methodology. Weights may be applied to particular policy objectives (e.g. concerning safety); or to particular sectors (e.g. regional development; the environment); or to particular impacts (e.g. financial viability). The main feature of the weighting system is its transparency. Results are presented in a variety of ways, involving an array of scores against each policy area, at each level of the hierarchy; summed scores across all policy areas and hierarchical levels; and sensitivity tests involving the application of alternative weights. It is important that presentation and interpretation of results do not rest on single scores alone - there is no equivalent of CBA NPV or IRR . It should also be noted that, as with other technical decision aides, the final choice is taken by the decision maker after consideration of all sources of information.

2.8.3

The process, during the construction of the PAM, under which the analyst interprets policy into measures of performance is akin to that which takes place in CBA where social welfare components are interpreted in quantified terms. Both

2-12

HALCROW FOX
processes are very difficult and open to charges of bias and subjectivity. In CBA this is minimised through application of best practice to surveys and other techniques designed to 'reveal' social value systems. Equivalently bias in the 'revelation' of policy objectives and measure of performance adopted in the PAM is minimised by a thorough understanding and appreciation of policy, though this is made more difficult if policy is not well articulated. Ultimately, and crucially, bias is subject to sanction. For, as with best practice in CBA where assumptions are made explicit, the objectives and benchmarks used in PAM are transparent and can readily be altered by the policy maker. 2.8.4 As noted above, the measures of performance represent the heart of the tool. These consume the bulk of the technical design activities, and discussed in more detail in Appendix C (Internal Report D(G) - Measures of Performance, Tenassess Project Team, April 1997, DGVII). They are also discussed below in Chapter 4, especially in the context of where the Case Study testing are the focus of a separate study deliverable. Figure 2.5 gives a broad overview of how the tool developed for the Task 32 requirements of the Tenassess project divides into individual work packages. Subsequent chapters of this Main Report develop each aspect in turn, starting with the top-down approach, through the bottom-up approach and the model design stages and finally, to the Case Study testing and model refinement stages. As can be seen from Figure 2.5, the Policy Assessment Model, whilst at the heart of the Task 32 project, relies on inputs being provided to it from two angles policy objective data and project impact data. The construction of policy measures of performance brings these two aspects together to create the Policy Assessment Model in its entirety.

2.8.5

2.8.6

2.8.7

2-13

HALCROW FOX

Figure 2.5: Structure of Task 32

Topdown

Stage 1

Policies, impacts, objectives

Stage 3

Policy Assessment Model, measures of success & failure, weighting system Bottom -up
Stage 2

Stage 4

Stage 5

Case Studies
refinement & adjustment

Conclusions

feedback

Impact measurement

One case study

2.9

BENEFITS OF GOALS ACHIEVEMENT MATRICES IN TRANSPORT PLANNING In order to assess the full potential impact and benefits of the GAM to Tenassess in particular and transport planning in general, it is necessary to review a number of key characteristics of GAMs and a number of guidelines in their design and applications. These are outlined below. One of Several Assessment Tools

2.9.1

2.9.2

GAMs are one of several assessment techniques, all of which provide valid inputs to project appraisals. Figure 2.6 illustrates the GAM in the context of Cost Benefit Analysis and financial appraisal. Other forms of assessment (eg: environmental impact assessment) could also be added to this model. GAMs are not intended to provide a comprehensive assessment tool in isolation from other techniques (just as CBA and financial assessment should not claim comprehensibility). They do claim to provide a different perspective on a project and its viability to other assessment tools, stemming from their distinct value system, as outlined in the sections below.

2.9.3

2-14

SOCIO - POLITICAL SYSTEM

Value System

Finance

Welfare

Policy-focussed

Goals

Maximise Financial Returns

Maximise Economic Welfare

Achieve Policy Objectives

Appraisal Method

Costs

Revenues

Project Impacts

SocioEconomic Values

Project Impacts

Measures of Performance

Financial Rate of Return

Scores and Weights

FINANCIAL ASSESSMENT

COST-BENEFIT ANALYSIS

GOALS ACHIEVEMENT MATRIX

OUTPUTS

Financial Return to a Specified Body

Contribution to Social Welfare

Measure of General Support to Policies

Technical Assessment

Environmental Assessment

COMPREHENSIVE APPRAISAL

Figure 2.6 : Value Systems, Goals and Appraisal Methods

HALCROW FOX

2.9.4

In terms of how the GAM fits together with other forms of project appraisal tool, it is the opinion of the research team that the main role of the Tenassess Policy Assessment model is as, what has been termed, a screening tool. There are many projects and project variants that compete for scarce funding resources at the national and trans-national level. The tool developed allows a first-cut examination of which projects, or variants of any given project, best meet the aspirations and goals of decision-makers / society in general. Whilst it is recognised that the PAM requires a certain amount of data, much of this is generally available from national and regional sources. The principal project-related need is for a traffic forecasting model. This screening process allows projects that clearly do not conform to a reasonably wide set of goals and objectives to be removed from the decision-process before the costs of detailed project studies (where CBA and EIA have their role) are incurred. The GAM also has the potential to act as a monitoring tool within the overall appraisal process. The period from project inception to final design and feasibility can be very long winded. Given such long project lead times, it is entirely possible that policy objectives and preferences will evolve during the course of the project planning lifetime. In this context, the GAM approach can act as a monitor on projects - reassessment during the planning process can ensure that upon completion of full project evaluation and design, the scheme is still broadly in line with the aspirations of decision-making bodies.

2.9.5

2.9.6

2.9.7

Distinct Value System 2.9.8 The theoretical driver of each form of assessment is important here. CBA and financial assessment derive from in-built value systems (e.g. values of time and accident valuations), optimising socio-economic welfare and maximising financial returns respectively. These reflect two common measures of performance that are of relevance to most appraisals. However, these do not encompass all value systems or project objectives - even the German BVWP assessment framework (the most comprehensive CBA tool in existence within Europe) cannot address issues such a the public acceptability of projects. Hence, they cannot claim to be the only elements of a comprehensive appraisal framework.

2.9.9

2.9.10 GAMs reflect a value system based upon the policy environment; i.e. the policies dictate the values governing the appraisal. At a simple level, a good project fulfills the policy objectives of the particular socio-political environment, with economic and financial performance being of secondary significance. The French TGV network provides a good example of these processes in action, expensive projects that have been pursued for a much wider range of political objectives than the provision of an economically efficient rail network.

2-16

HALCROW FOX
2.9.11 Given the importance of policy objectives in the classic approach to transport planning (as practised throughout most of Europe) the logic and rationale behind a GAM is clear. Need for Flexibility 2.9.12 In view of the relationship between GAMs and policies and given the transient nature of many policies, there is a clear need for flexibility in the design of GAMs. GAMs must be capable of reflecting policy changes, or different emphases within the policy environment. This gives the GAM a distinct air of realism - an assessment method that reflects the transient nature of the world must be closer to reality than a static and inflexible process. 2.9.13 Allied to this flexibility, a GAM should be capable of handling temporal policy evolution. A well designed GAM should be capable of handling policy changes and changing emphasis on existing policies as time progresses. Need for Versatility 2.9.14 One advantage, and key requirement, of GAMs is that they are able to measure and trade-off impact in a wide variety of policy environments (finance, quality of life, regional development, traffic, etc.). The GAM must be capable of summarising a mix of impacts from across different sectors and allow practitioners to trade-off benefits and costs across sectors. This places a distinct need for some form of multi-criteria algorithm within the GAM. Transparency 2.9.15 One of the main merits of a GAM is that it compels decision-makers to be transparent about the rationale underlying their actions. If a GAM is used to inform a decision-making process, it makes clear the extent to which the decision supports or conflicts with policy and the extent to which weights have been applied to particular groups of impacts (eg: environmental impacts).

Focus on Interpretation 2.9.16 The use of sensitivity tests, different weighting systems and different project configurations all form essential aspects of the application of a GAM. Most importantly, it should not be viewed as a means of providing advice to decisionmakers based on a single, compound, output term. Figure 2.7 illustrates how the GAM is applied and interpretations made. Application Outside the Transport Sector 2.9.17 In principle, there is no reason why the approach to project assessment described in this chapter should be confined to the transportation sector - this is merely the focus of the Tenassess project.

2-17

HALCROW FOX
2.9.18 In all sectors of the economy, policies exist and project are designed to meet the objectives of these policies. The method is, therefore, directly transferable. 2.9.19 In terms of the method allowing cross-comparison between sectors, this is dependant upon the ability to adopt and operationalise a common set of measures of performance and also to find common ground in the area of policy objectives. By way of an example, the Private Finance Initiative (PFI) in the UK sees a set of policies and policy objectives that are directly relevant to a number of different sector (e.g. transport, prisons, hospitals). Whilst there are a core set of objectives that allow projects from different sectors to be compared against each other, there are many other objectives that are not directly comparable. For example, the objective of not making patients wait more than 18 months for an operation within the UK Health Service has absolutely no relevance to the transport sector. 2.9.20 In short, cross-sector comparison of projects is possible, but only to the extent that policy objectives allow.

2-18

PROJECT

IMPACTS

POLICY AREAS

POLICIES

MEASURES OF PERFORMANCE

VALUE SCORES Are impacts good or bad?

PERCEIVED IMPORTANCE OF POLICIES

WEIGHTING SYSTEM

UNWEIGHTED, OBJECTIVE, SCORES

WEIGHTED SCORES

THE P.A.M.

ANALYSIS AND CONCLUSIONS Do project impacts reinforce policies or detract from them? Do results change when we weight the most important policy areas? Are results robust or are they sensitive to weights? Is the final score comprised of similar scores for each policy area (low standard deviation) or is there a high variance in scores between policy areas (high standard deviation)?

Figure 2.7 : Applying the Policy Assessment Model

HALCROW FOX
Integration into the Broader Tenassess Project 2.9.21 Tenassess looks beyond the issue of project and policy appraisal to the broader relationships between projects, local and regional policies and the Common Transport Policy (CTP). The Policy Assessment Model - the GAM reported here provides one tool within Tenassess to examine these relationships. In particular, it provides a high-level sifting tool with which to carry out an initial assessment of the extent to which major transport projects (pricing and regulatory as well as infrastructure) support or conflict with policies. The PAM also provides an initial, measurable, indication of conflicts between different levels of the policy hierarchy. 2.9.22 The following chapters describe the PAM produced within Tenassess and explore further these potential contributions of GAMs to decision-making.

2-20

HALCROW FOX

3.

Top-Down Analysis - Policy Data and Measures of Performance

INTRODUCTION The Task 32 Policy Assessment Model has two starting points. From the "top", looking down, the policies affected by transport projects and, from the "bottom", looking up, the impacts of the transport projects themselves. This chapter review the "top-down", policy led approach. One of the main conclusions to emerge from this research exercise has been that the identification of explicit policy objectives that are expressed in measurable terms has proved virtually impossible, insofar as policy objectives tend to be expressed in qualitative terms rather than numerically. Additionally, transport policies are not mutually exclusive, hence considerable duplication can occur. For example, policies on achieving a certain mode split target are not a means to an end in their own right. Such policies may be in place to achieve reductions in environmental pollutants - but these environmental concerns are often as not expressed as policy options in their own right. Furthermore, the mode split required can be achieved through a variety of different means such as increasing the cost to the user of the more polluting modes, through physical restraint of road traffic or through subsidisation / concentration of investment resources on more environmentally friendly modes of travel. All of these tools are frequently expressed as policy areas in their own right. Some of the problems encountered in this exercise are summarised in the Austrian Country Report written by ICCR1. .the evaluation as such of specific measures is difficult because of the complexity of the issues involved and their non-linear incorporation into the time framework. Many goals such as modal shift, reduction of emission and energy consumption are to be achieved by a mix of various policies and effects become visible in the long term only. Specific goals, especially those related to environmental issues, are set within long-term frames explicitly. These components make it difficult to make exact statements about the success and failure of policies. Furthermore, in a hierarchy of objectives, the achievement of success at the lowest level does not necessarily mean that the higher-level objectives are also met with success. It is against this background that the current chapter is written. The subsequent sections document how transport policies have been structured and operationalised in terms of success and failure in order to examine how projects perform against policy targets. It is our firm belief that such an exercise as this is necessary if a meaningful policy assessment framework is to be constructed. Whilst it is acknowledged that
Tenassess Deliverable R(1) - Policy Issues (National / European), Tenassess Project Team, January 1997, DGVII.

3.1 3.1.1

3.1.2

3.1.3

3.1.4

3.1.5

3-1

HALCROW FOX
different policies interact in both the spatial and temporal context, this structuring exercise needs to be undertaken if the framework is to be operationalised in an informative manner. 3.1.6 In many instances, where explicit and quantifiable policy objectives have not been forthcoming, the study team have inferred from the text exactly what the policy is trying to achieve and constructed a measure of performance from this standpoint. It is our firm intention that this exercise should be made as transparent as possible in order to stimulate debate on exactly what decision-makers have in mind when they design policy. If our assumptions and inferences are deemed inappropriate, then we will have gone part of the way towards advancing our understanding, so long as the criticism is of a constructive nature - in other words, the critique provides us with a better measure or a more accurate set of measurable objectives. If for some reason, the targets defined for various policy objectives turn out to be either an under or overstatement of the actual situation, then the measure of performance will require adjusting accordingly. The Policy Assessment model has been designed with the full intention of allowing targets and benchmarks to be easily amended as more accurate, specific, or simply different policy targets values become apparent. The previous point suggests an interesting conclusion as to why, in so many cases, explicit policy targets are not forthcoming from policy documents. The formulation of policy targets is potentially such a dynamic, constantly evolving process that it contains inherent risk to the credibility of decision-making if exact values are to be specified at any one point in time. The alternative conclusion to draw from the lack of explicit policy targets is that decision-makers have little faith in the ability of the overall decision-making process to meet them. This latter conclusion also provides a role for the GAM, as outlined above, insofar as it stimulates the debate upon exactly what is meant when policy documents are specified.

3.1.7

3.1.8

3.1.9

3.2 3.2.1

THE POLICY STRUCTURING PROCESS Much of the analysis has been undertaken using the transport policy material from Austria and Germany which was the most complete and quantified material available at the time. The Case Studies have helped to identify where this approach has proved inadequate and amendments have been made to certain measures of performance to reflect different requirement in different areas of the European Union, particularly in areas away from the core region of north-western continental Europe. The Austrian policy data supplied by ICCR was taken as a template for the initial construction phase of policy objective measures of performance, it was noted that, from the data supplied, the provision of clear and explicit measures of success and failure of a policy was only evident in a very few instances. The research

3.2.2

3-2

HALCROW FOX
programme initially envisaged the manual construction of these indicators, along with measures of performance for each and every policy, for each and every country in Europe for which data has been supplied. 3.2.3 The first conclusion to be drawn from this exercise, was that, in the vast majority of cases, this constructed measure would be totally arbitrary in its end points and open to a good deal of criticism from either the ultimate user and/or from a theoretical robustness viewpoint. The first task in the analysis of the policy data structured the policies, as supplied, in accordance with their stated (verbalised) objectives. This exercise highlighted the fact that, in terms of objectives, many policies shared common goals and, more importantly, the policies could be grouped into nine broad categories into which they each would fit, albeit with some overlap of objectives between aggregate policy groupings. This exercise is reproduced in Table 3.1 below. Where these overlapping objectives occur, they are, in all instances, covered by another aggregate policy area. The conclusion of this exercise was that this broader grouping of policies provides a more transparent format for the construction of measures of performance.

3.2.4

3.2.5

3-3

HALCROW FOX
Table 3.1: Initial grouping of Austrian policy data into aggregate areas
OBJECTIVES environmental impacts quality of transport increase safety costs environmental impacts tonne kms/pax kms internalisation of externalities increase mobility increase system efficiency modal split quality increase safety costs environmental impacts competitiveness, tonne kms/pax kms capacity improve efficiency accessibility interchange costs income co-ordination of transport & regional policy land use employment tonne kms/pax kms promote combined transport reduce bottle necks increase mobility costs quality modal split environmental impacts improve efficiency environmental impacts quality increase safety costs mobility congestion costs labour quality improve competitiveness reduce public spending, lower tax burden improve efficiency increase safety environment modal split quality increase safety

POLICY AREA 1. Environment 2. Fair and efficient prices

3. Improve transport

4. Improve accessibility and regional development

5. Harmonisation and common market (intermodality, interconnectivity, interoperability)

6. Avoid traffic

7. Deregulation, privatisation, liberalisation

8. Restrict transport

9. Improve safety

Note: = change in 3.2.6 The next observation to come out of the analysis was the realisation that the list of objectives drawn up for the policies was almost identical in content to the list of identified project impacts [see Tenassess Deliverable D(E) - Impact Assessment Framework, Tenassess Project Team, January 1997, DGVII, reproduced as

3-4

HALCROW FOX
Appendix B of this report]. The objectives outlined in Table 3.1, for all the Austrian policies supplied, collapse into the following 13 areas: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 3.2.7 Reducing the negative environmental impact of transport; Influencing modal shift; Improving competitiveness of one or more than one mode; Improving the efficiency of the transport system; Reducing road transport; Increasing capacity to meet projected demand increases; Improving transport safety; Improving conditions for private finance in transport; Co-ordinating investments between modes (through transport plans); Improving service quality; Influencing regional planning and/or development; Increasing mobility; and Reducing the need to travel.

The perceived distinction between objectives, as currently defined, and project impacts is, in most cases, non-existent. This result gives a clear and direct link between policy objectives and project impacts. All that is required is consistency of measurement on either side. For each aggregate policy area, an overall aim exists. For example, the policies grouped under the Improve Transport category all have the aim of influencing mode split in a certain manner and those under Fair and Efficient Pricing heading all aim at influencing relative prices of transport modes. It is to these aims, rather than the policy specific objectives, that measures of success and failure have been constructed. This preserves the integrity of the methodology, in so far as the interpretation and inferences drawn by the research team are kept to a minimum. It also makes the construction of robust measures of performance a realistic proposition in the available time horizon. With this structure, the identified project impacts can be directly linked to the aggregate policy aims via measures of performance. The links are made explicit where individual policies have objectives that lie in more than one policy area (as in the Austrian example quoted above). As an example, Austria has very clear targets for reductions in environmental pollutants and this is explicitly stated as a policy in the supplied information. Many other policies, such as many of the proposed investment policy strategies (e.g. Improvements to the Danube to effect a road to water mode shift) have improving the environment as one of their ultimate aims. Thus, whilst one objective of the investment policy is to achieve mode shift (thus appearing in the Improve Transport policy area), another is to improve the environmental impacts of transport, which forms a subset of the overall national environmental policy targets (appearing in the Environmental policy area) - this is the linkage between two broad policy areas that will be shown in the revised framework. If a project does achieve the road to water mode shift, it will also go some way to achieving the environmental targets and will be scored

3.2.8

3.2.9

3-5

HALCROW FOX
accordingly in the two aggregate policy areas. This formulation goes a long way towards addressing concerns over inter-locking policies. 3.2.10 From the discussion above, it may seem possible to simplify the analysis by aggregating the policy areas - for example, policies aimed at influencing mode split will generally have a similar goal to policies aimed at reducing road transport. Aggregating policies in this manner would, however, lose the focus of the method, which is always on the stated policy objectives. It is only through the retention of this level of detail that the process remains meaningful. This disaggregate presentation may appear to introduce double counting issues into the PAM. Whilst potentially correct, this apparent double counting is (or should be) a reflection of a high level of emphasis in a particular sector / country / region on closely related policies. If the presentation reflects actual policy closely, the model is accurately representing policy issues - the double counting merely reflects emphasis by decision makers in certain areas. 3.2.11 One impact that does not correspond directly with a policy objective is that of investment cost. This is due to the fact that the magnitude of all other impacts and also of policy objectives are a function of the size of the investment cost. For any given project, there will be a maximum contribution towards policy aims that can be achieved for a given investment. The investment cost of a project is, therefore, presented as a separate item in the Policy Assessment Model (see Tenassess Internal Report D(H), Tenassess Project Team, April 1997, DGVII) below the measures of performance. This enables the decision-maker to examine how trading project cost (e.g. through different design specifications) affects the achievement or otherwise of policy goals. 3.2.12 It is possible that investment cost may become a policy area in its own right as well. Choices between alternative projects may, due to financial pressures, start to be directly linked to the cheapest way of achieving other objectives. This would be relevant when there are severe limited on funding available for the types of project in question. In these circumstances value for money is measured not merely in absolute terms but in relation to the size of investment. The assessment tool developed is sufficiently flexible to allow for the addition of other policy areas, as and when they come into existence. Whilst not explicitly incorporated within the Policy Assessment Model developed as part of this project, there is no reason in principle against the inclusion of investment cost in its own right.

3.3

IDENTIFICATION OF MEASURES OF PERFORMANCE

3-6

HALCROW FOX
3.3.1 The following subsections outline the measures of performance that have been developed for each of the identified policy areas. For each performance measure, the overall aims, definition of success and failure and mathematical specification of the measure are stated, together with any modifications/alterations that have been identified/incorporated as a result of the Case Study testing phase of the project and/or subsequent discussions with our consortium partners and the European Commission. For a fuller discussion of this aspect of the project, the reader is directed to R(2) Modelling Module Progress Report, Tenassess Project Team, April 1997, DGVII and Internal Report D(G) - Measures of Performance, Tenassess Project Team, April 1997, DGVII (reproduced in Appendix C of this report). In the latter, the full mathematical specification and justification for the adoption of the measures of performance operationalised within the Policy Assessment Model is documented. An additional point to note here is that each of the measures of performance discussed below can be viewed as a stand alone model in its own right. The mathematical models constructed during the Tenassess project represent the researchers view of how the achievement of a policy objective can be translated into a meaningful tool that measures the extent to which a given transport project contributes towards or detracts from this goal. The modular nature of the Policy Assessment Model is such that any one measure of performance can be substituted for an alternative measure, so long as the interlinkages between individual indicators are noted and taken into consideration. Environment 3.3.4 The ultimate aim of policies focused in this area is the reduction of atmospheric pollutants. In R(2) - Modelling Module Progress Report (reference as above), reduction in noise emissions was also included as a measure of performance. Following comments received during the Case Study model testing phase, the proposed noise component of the measure of performance has now been removed from the Policy Assessment Model. This decision was taken on the grounds that the meeting of noise legislation standards is built into project design as a mandatory requirement rather than as a policy option. The previous point is an important one, insofar as the Tenassess project makes an implicit distinction between laws and policies. Legislation effectively marks out the boundaries within which planning and appraisal of transport project operates. It would be an irrational decision-maker who designed a project which did not meet certain aspects of national or European legislation (e.g. on noise pollution). Attainment of policy objectives, on the other hand, is not a mandatory requirement of project planning. Policy objectives effectively represent a wish list of what various decision-makers would like to see happen in the future. They need not and, indeed, are often not, as previous Tenassess Deliverables (see D(1), reference as above) have noted, be mutually compatible - policy conflict is a

3.3.2

3.3.3

3.3.5

3-7

HALCROW FOX
common phenomenon. In short, they are targets which decision-makers are working towards rather than targets that have, in a legal sense, to be fulfilled. 3.3.6 Success (i.e. a score of +5 within the Policy Assessment Model) and failure (a score of -5) in the field of environmental pollutant reduction is a direct function of the targets adopted at different levels in the decision-making hierarchy - in many cases this is in line with European level policy, but some Member States adopt more stringent targets. The pollutants considered within the Policy Assessment Model are strategic rather than local pollutants (reflecting the strategic nature of the assessment tool) and their breadth is delimited by the individual pollutants covered in the IWW / INFRAS study2 - namely CO2, NOx, CxHy and particulates. Some countries do not have specific targets for these pollutants (in these instances, EC and UNECE targets have been adopted) and other have specific targets for other pollutants. This core set is, however, maintained throughout the Policy Assessment Model on the grounds that it provides a broad brush, cross-project, cross-border set of assessment criteria - it is not intended to replace detailed project analysis, including a full EIA. Targets are dynamic in the sense that the achievement of a given target is a function of where pollutant levels lie now and by how much they have to be reduced, year-on-year, in order to reach the specified target at a given point in the future. This is operationalised in the model with reference to observed and forecast background pollutant trend data - see chapter 5 for a fuller discussion of these aspects of the Policy Assessment Model. The percentage contribution of the project to the achievement of each pollutant target is calculated and scored accordingly. Individual scores are then combined to produce an overall project score in lie with the relative damage of each pollutant, as reported in the IWW / INFRAS study. Fair & Efficient Pricing 3.3.10 Policies under the heading fair & efficient pricing are all aimed at changing the relative prices of different modes, usually in an attempt to internalise the (negative) external costs of transport. It should be noted that this definition is not strictly in line with that adopted in the Commissions Green Paper3 on this subject, insofar as it is focused on the internalisation of externalities and does not consider the coverage of infrastructure costs. 3.3.11 Success has been defined within the Policy Assessment Model as the actual costs4 of a particular mode equal its true5 costs following the implementation of a

3.3.7

3.3.8

3.3.9

2 3

The External Effects of Transport, IWW / INFRAS, 1995. Towards Fair and Efficient Pricing in Transport, COM(95)691, European Commission. 4 Actual costs are defined as the out of pocket expenses associated with undertaking a particular journey - i.e. fare paid or fuel / operating costs.

3-8

HALCROW FOX
given project. Failure is defined as the situation where a project widens the gap between actual and true costs for a given mode of transport. The neutral point (a score of zero) is when the project has no impact upon cost internalisation or the relative prices of transport modes. 3.3.12 Following the Case Studies, a special case for this policy area in the context of toll motorways has been incorporated. In these instances, the toll charge itself may internalise some of the external costs of road transport - i.e. if the toll more than offsets the benefits to the user in terms time and operating cost savings. Thus, whilst the aim of the project is not to influence relative modal costs (there is often only one mode in the study corridor), some degree of external cost internalisation may be achievable. Improving a Particular Mode of Transport 3.3.13 Whilst the previous policy area was aimed at influencing the relative prices of different mode of transport, this policy area is directly aimed at boosting the mode share of a specified mode - through pricing or other means such as direct investment. 3.3.14 Success for the policy area rests upon the existence of explicit modal split targets achievement of the desired mode split within the corridor in question denotes success, a worsening of the mode split (i.e. away from target percentages) denotes failure and no change represents zero. 3.3.15 In many countries, explicit mode split targets do not exist and an implicit assumption has been made that the aim of policy in this area is to stem the growth in road based passenger and freight traffic that can be observed in every Member State of the European Community. On this basis, success represents a reversal of the long-term trend towards increased road modal share and failure represents a continuation or worsening (i.e. increased growth in the modal share of road transport) of existing trends. 3.3.16 Input data requirements are the before and after traffic volumes by mode, together with either explicit modal split targets, or implicit targets derived from background data trends. Accessibility & Regional Development

The true costs of a transport mode are defined as the sum of actual (monetary) costs and external costs, with the external costs (by mode) being derived from values contained in the IWW / INFRAS study report.

3-9

HALCROW FOX
3.3.17 Policies aimed at general economic growth and / or regional development are grouped under this Tenassess policy area. It implicitly also incorporates many of the Commissions aspirations in the field of regional policy, such as economic and social cohesion - the interpretation here being that the policy area implies that certain regions of the Community should be encouraged to grow at a faster rate than others in order to minimise regional inequities in income and wealth distribution. 3.3.18 The measure of performance initially developed to examine this issue was based on the public capital hypothesis of Aschauer insofar as the project economic rate of return (compared to some benchmark values derived from World Bank data) was deemed an appropriate tool for the purposes of this study. 3.3.19 The role of infrastructure in stimulating economic growth is a controversial field. Many years of research in the field of regional science has not proved a conclusive link - there is a strong school of thought that suggest that transport infrastructure provision may indeed work against peripheral /disadvantaged regions. It is acknowledged that the measure developed here is only one potential way of analysing regional development impacts. As with all impact measuring tools utilised in the Tenassess project, this indicator is merely used to show how the method can be operationalised and not a statement of the best impact measurement tool. 3.3.20 The Case Study testing phase of the project has, however, indicated that the measure of performance originally developed did not adequately fulfil the needs of projects in all areas of the Community, particularly in the less developed regions. 3.3.21 The use of the project economic rate of return as a proxy for economic growth tends to overstate the regional accessibility benefits of projects in the highly developed European core (high values of time and large traffic volumes will tend to give projects strong rates of return) relative to projects in less developed regions (lower values of time and low initial traffic volumes serving to produce low economic rates of return). The purpose of many projects in less developed regions is to stimulate development that does not already exist - in other words, a poor (relative to the specified benchmarks) economic rate of return does not necessarily imply poor development potential. 3.3.22 A two part indicator has, therefore, been developed - one part is the original ERR criterion, whilst the second looks at average forecast travel time reductions between towns and cities in the project corridor. Both are compared to benchmark values that suggest how a project should perform if it is to foster economic growth and accessibility improvements. A variable weighting factor can be applied to each component to examine the impact of placing more emphasis on the ERR criterion or on the travel time reduction criterion. 3.3.23 The reduction in average travel time component is a form of accessibility indicator. It is used in preference to indices that are derived from a series of journey times because of its simplicity (i.e. it needs less data and uses a model

3-10

HALCROW FOX
which feeds other measures of performance), and because it is well suited to infrastructure projects that tend to be of a linear nature. Alternative accessibility measures based on more than journey time, for example populations within certain journey time catchments, tend to suffer from the same bias as that of ERR, discussed above. They are also highly sensitive to planning forecasts, which are particularly uncertain in areas of low existing development. 3.3.24 Whilst this accessibility index in conjunction with the ERR has proved a robust measure of performance during the Case Study testing phase, we are mindful that other solutions to this and, indeed, other measures of performance may well exist. This would be a fruitful area for further research. 3.3.25 One suggestion for the accessibility and regional development policy area is some form of cohesion index which relates the forecast change in regional GDP stemming from the project to changes in the regions ranking in the EU GDP league tables. One potential problem with this indicator, which is an issue that equally applies to any proposed measure of performance, is that the requisite project impact data may not always be available. Estimation of likely impacts of a transport infrastructure project upon regional GDP are a particularly difficult aspect of project benefits to quantify. In all instances, a balance needs to be struck between the depth and quality of the measure of performance and the availability of the project impact data that is required in order to operationalise it. Harmonisation & Common Market 3.3.26 This policy issue is of most relevance to the European Commission level within the decision making hierarchy and is broadly aimed at increasing European integration by stimulating intra-EC movements of both goods and citizens. 3.3.27 In terms of the measure of performance developed within the Tenassess Policy Assessment Model, both goods and passenger movements are considered - by focusing on changes in total vehicle kilometres. 3.3.28 The inputs to the measure are changes in total national and total international vehicle kilometres (in percentage terms) induced by the project. The benchmark against which the measure is compared is the impact of the integration of Portugal and Spain into the European Community during the 1980s where annual international traffic grew at a rate of nearly 40%, compared to a European average increase in traffic movements of 3.6% per annum.

3-11

HALCROW FOX
3.3.29 The accession of countries to the European Community should be viewed as the historic maximum that can be achieved in terms of boosting cross-border flows relative to national ones - a link that improves access between two existing Member States will never have such an effect as other barriers to cross-border movement were also removed such as customs arrangement, different legislation and formalities. This rate of increase is, therefore, viewed as the maximum that is possibly achievable (a score of +5). The European average is viewed as the norm (a score of 0). Adopting this rationale allows the benchmarks to be applied across all the countries of the European Union. Avoiding Traffic/Reducing the Need to Travel 3.3.30 Traffic avoidance and policies aimed at reducing the need to travel are in a sense very long term policy aims which can be achieved through a variety of transport and, in particular, land-use planning policy instruments. The central aim is to effectively break the link between economic growth and traffic growth that is a feature inherent in the vast majority of developed nations around the world. Put another way, the key variable is the level of transport intensity within the economy of a given country. 3.3.31 In order to measure traffic intensity, a common measurement must be chosen for passenger and freight traffic alike - this has been defined as a transport unit. Following the methodology utilised in the IWW / INFRAS study6, one transport unit (TU) equals one passenger or 200kg of freight. Transport intensity has then defined as the relationship between TUs in the economy and GDP. 3.3.32 From this definition, a picture has been constructed of transport intensity trends in different EU Member States and for the EU as a whole. The analysis also deals with either increasing or decreasing long terms trends in transport intensity. In the former case, success is defined as stabilising or reducing the long-term trend, failure as an increase in the long term trend and the neutral point being where a particular project has no influence on the trend. For the case of decreasing longterm transport intensity, success is a greater reduction in intensity than the longterm trend, zero as in line with the trend and failure any increase in transport intensity (relative to the trend) following the implementation of a project. 3.3.33 The case study analysis of this policy area tends to suggest that individual projects are rarely of such a scale to influence overall transport intensity to any significant degree. This tends to fit with the essentially long-term nature of the policy area it is only through the concerted combination of a large number of individual projects (transportation initiatives combined with complementary land-use planning projects) that a reduction in transport intensities will be realised. Having said this, it is an important area of policy in many countries which merits its presentation within the Policy Assessment Model. Reducing the Price of Transport to the User

Reference as before.

3-12

HALCROW FOX
3.3.34 This policy area represents the traditional rationale for investment in transport infrastructure - to relieve bottlenecks, alleviate congestion, increase speeds - all of which provide time and operating cost savings to the user of the system and, consequently, benefits to society as a whole. 3.3.35 In terms of the measure of performance developed for the Tenassess project, a similarly traditional route has been followed insofar as the cost-benefit ratio is used as the building block for the analysis. 3.3.36 Whilst the bulk of the benefits associated with transportation projects (in developed countries at least) are comprised of time and operating costs savings, other benefits are evident, such as environmental, developmental and safety. Whilst important, these benefits should not be part of the calculation used for this measure of performance - we are essentially focusing on the private benefits to individuals rather than collective benefits for society as a whole or non-users. 3.3.37 In the analysis undertaken by University of Cologne, the assessment of the efficiency of a transport project has been derived from the German Federal Transport Investment Plan which is considered to be the most comprehensive costbenefit analysis framework package currently in use in Europe. Savings in economic costs (i.e. savings to the user) generally account for around 73% of total scheme benefits. Comparing this figure to the annualised scheme investment costs, a measure of performance has been created - using the current German costbenefit decision criterion (of over 3, given existing funding constraints) as the decision rule. 3.3.38 Success is defined as a project meeting this decision rule whereas failure is a result of 0% - i.e. scheme benefits are insufficient to produce anywhere near the required ratio of benefits to costs. Restriction of (Road) Transport 3.3.39 In a number of European countries, physical restriction of road transport or its restriction through pricing or regulatory initiatives is becoming an important area of transportation policy. 3.3.40 This policy area is operationalised into a measure of performance by comparing the forecast impacts of the project on road traffic growth with the long-term trends witnessed in the country/countries of relevance to the project. 3.3.41 Success is defined in terms of the project achieving the stated target objectives either the specified reduction in total road vehicle kilometres or the specified reduction in the rate of increase of road vehicle kilometres. Conversely, failure is an increase in road traffic away from the stated objectives. Transport Safety

3-13

HALCROW FOX
3.3.42 This policy area covers a collection of individual policies that are all aimed at reducing the numbers of traffic related deaths and injuries. 3.3.43 Most European countries have explicit targets for road safety improvements. Success is defined as the achievement of these aims (in percentage terms within the project corridor) following the implementation of the project and failure is an increase in accident rates. Targets can be specified in one of two ways. Either a reduction in the absolute numbers of accidents can be defined or a reduction in the accident rate can be specified - as long as the targets are consistent with the project impact input data, there is no need for any change to the measure of performance specifications. Deregulation, Privatisation and Liberalisation 3.3.44 The policies grouped under this heading are in many senses a very diverse set of policies. In the context of the measure of performance developed here, the broad aim is to expand the set of private sector opportunities in order to reduce the burden of transport projects on the public purse. It is, however, acknowledged that policies aimed at deregulation and liberalisation entail far more than the increased involvement of the private sector in project sponsorship and / or operation. The argument employed here is that projects cannot be designed to improve the degree of deregulation and liberalisation - the two terms are effectively end states insofar as something is either deregulated, partially deregulated or regulated - an alternative project design will not change this situation. The same is true of liberalisation policies. 3.3.45 Following from the discussion above, the critical indicator for private sector participation in project design, construction, ownership and operation is essentially the financial return that the project will generate for the private sponsor - as measured by the Internal Rate of Return (IRR)7. The IRR in this context is the narrowest possible measure, insofar as it should consider only the direct project costs (construction, maintenance and operating cost changes) and revenues - no wider set of costs and benefits are considered in this private sector financial decision. A minimum IRR has been specified as a benchmark against which to test projects. This benchmark (estimated at between 12% and 20%) is based upon the experience of Halcrow Fox in the field of private finance initiatives within the European Union and in other locations around the world. Given the potential funding support opportunities available from the European governmental institutions for strategic transport projects in Europe, we have adopted the lower end of this range as the benchmark for the Policy Assessment Model.

The required IRR is, in effect, a catch all number which summarises a number of private sector concerns regarding the project. These are discussed in more detail in the section on Financial Acceptance or projects.

3-14

HALCROW FOX
3.3.46 A project is, therefore, successful (+5) in the model if the IRR is greater than twice the specified minimum - in this instance, the +5 score is achieved at IRRs of 24% or more. Failure (-5), in terms of private sector interest, is achieved at IRRs of less than 12%. 3.3.47 The potential availability of public funds, in one form or another, tends to complicate this picture of private sector involvement in transport service / infrastructure provision. In order to address these issues during the Case Study phase of the Tenassess project, project IRRs both with and without funding have been prepared. The basic principle to be adopted has, however, been to report the IRR of the project in its own right - this approach has the benefit of clearly identifying a deficiency in this respect and focuses the attention of the decision maker on a potential problem if private finance is viewed as a critical aspect of the projects case. 3.3.48 It has also been drawn to the attention of the research team that the 12% benchmark is not uniformly applicable across all Member States. The adopted value is intended to represent a default value (as is the case with all the benchmarks contained within the Policy Assessment Model) which can, if required, be amended by the user to reflect local conditions and assumptions. Financial Acceptance 3.3.49 Whilst it is acknowledged that transport projects generally have wider benefits associated with them than those that are captured in a purely financial project appraisal, pressure on public finances throughout Europe are making financial project constraints a major issue in project assessment. 3.3.50 Concept such as public/private partnerships (PPP) from Commission policy and Private Finance Initiatives (PFI) as witnessed in the UK, tend to suggest that project design is already being affected by the desire / necessity to fund projects from purely private financial sources or via a mixture of public and private sector sources. 3.3.51 The measure of performance being developed to assess the financial acceptability of transport projects essentially addresses issues that are crucial if private finance is to be attracted in sufficient volumes to advance the scheme in this manner. It in effect expands upon the IRR assessment criterion discussed in preceding sections of this chapter. 3.3.52 Whilst not a substitute for a full financial acceptability study8, a measure of performance on a scale of -5 to +5 will give an insight into whether the project under assessment is liable to encounter significant problems in this area of project decision-making. Clearly, cut-off points are likely to be implied within the range used, indicating a finite ability to distinguish between similar options. Other
8

The Transport Research APAS Strategic Research volume on Financing Models for new transport infrastructure DGVII (1996) provides a comprehensive analysis of the state of the art in this field.

3-15

HALCROW FOX
assessment methods would provide a finer level of resolution, in conjunction with the PAM. 3.3.53 The measure of performance would ideally be constructed around the following areas of information: Project Net Present Value (NPV); Project Internal Rate of Return (IRR); Returns on Equity (ROE); Debt Service Coverage Ratio; Debt to Equity (Gearing) Ratio; Distribution of financial impacts; and Financial risk and uncertainty surrounding the project.

3.3.54 It was not envisaged that the Tenassess project would undertake full-scale financial acceptability assessments as part of the Case Study assessment, as many of the projects under discussion are to be or have been funded through more traditional forms of project finance. Additionally, backers of private finance projects are very sensitive to the release of this sensitive information. As such, the methodology is presented with a view to it being incorporated into the early stages of new project appraisal procedures. 3.3.55 This measure of performance is presented with a view to its utilisation in future project assessments, alongside the other measures of performance being developed. Whilst a sensitive area of project decision-making and planning, it is one that should not be ignored during the developmental phases of strategic projects. 3.3.56 From the above, it is evident that the IRR criterion appears in more than one measure of performance - i.e. it is also used in the deregulation measure of performance. As has been stressed elsewhere, this does not necessarily present a problem for the proposed approach - as long as the measures of performance are understood and transparent, the user always has the option of weighting down individual policy areas. Public Acceptance 3.3.57 Whilst financial acceptability can be crucial to the advancement of a project, many financially and socio-economically sound projects can be delayed, modified or, indeed, abandoned through lack of public acceptance.

3-16

HALCROW FOX
3.3.58 The decisions about large infrastructure project implementations is the task of politicians and of decision makers. "To decide" means that all relevant parameters are considered. There are, on the one hand, more technical parameters - economic ones, geographic ones, network related ones, etc. - and there are more subjective ones, concerning life quality and needs of different population groups who are affected by the project or who are expected to use the infrastructure project. The latter variables can be summarised under the head-line "acceptance aspects". 3.3.59 Criteria for measuring acceptance are on the one hand technical data of the planned project (route, position of a new terminal, track, etc.). On the other hand, subjective aspects may become relevant. What level of acceptance does a planned project meet? What attitudes do different individuals or groups have towards planned new projects? Often different societal groups have different and sometimes contradictory interests. Often, political or community objectives are inconsistent with individual interests. 3.3.60 Individual interests can aggregate to become group interests, and in this case they may become a political power. It has shown recently that interest groups involved in connection with the planned implementation of new large scale projects can disturb or obstruct implementation in a decisive manner. 3.3.61 As with issues surrounding project financial acceptability, a full analysis of public acceptability concerns lies outside the scope of the Tenassess project. Having said this, a measure of performance (on a scale of -5 to +5) will be constructed based on a series of sample questions that can give an early indication of the likelihood of problems in this area. 3.3.62 For a fuller description of this type of analysis, the reader is directed to Internal Report D(E) - Impact Assessment Framework, Tenassess Project Team, January 1997 (see Appendix B) and Internal Report D (I) - Methodology Case Studies, Tenassess Project Team, April 1997, DGVII. 3.3.63 Table 3.2 summarises the main benchmarks used within the measures of performance outlined in this chapter, drawing on examples where necessary.

3-17

HALCROW FOX

Table 3.2: Policy Areas and Key Benchmarks - A Summary Policy Area
Environment Fair & Efficient Pricing

Benchmarks
No benchmark, targets derived from policy (e.g. Rio). Pollutants combined in line with IWW/INFRAS proportions IWW/INFRAS External Costs of Transport, by mode and by country in ECU / 1000 tonne kms / pax kms Historic mode split trends / specific targets World Bank typical economic rates of return (RoR); travel times savings use comparable projects as benchmarks EU average cross-border traffic growth (3.6%) and Spanish / Portuguese growth (39.6%) Historic traffic intensity trends Average time savings as % of total benefits - 73.1% (Germany). Annual cost % - 2.8% National road traffic projections National accident projections / trends IRR of 12% - minimum required for private sector involvement Various financial indicators (NPV, IRR, ROE, debt service coverage ratio, gearing ratio) Descriptive statement

Boundary Between Success and Failure (i.e. Score of 0)

0% change in emissions, according to weighted average of pollutants used No change in relative cost distortions for each mode Change in mode split in line with trends E.g. 25% RoR and 12.5% reduction in travel time (road projects) Cross-border growth in traffic relative to internal growth is 103.6% Traffic intensity stays in line with trends User benefits per annum are 2.05% of total investment cost In line with trends Target meet, e.g. 24% reduction in accidents by 2000 IRR of 12% N/A N/A

Benchmark or Target Used For Maximum Score (5)

E.g. target of emission reductions per annum: CO2 -0.1%, CxHy -11%, NOz -5%, particulates -0.4% (EU targets) Ratio of user cost to social cost is the same for each modes - so choice between modes is efficient E.g.: no change in mode split over times E.g. 50% RoR and 25% reduction in travel time (road projects) Cross-border growth in traffic relative to internal growth is 139.6% E.g. Road traffic increases by more than 250% (Portugal) User benefits per annum > 6.15% of total investment costs No target Double target of 24% reduction in accidents by year 2000 Twice minimum IRR N/A N/A

Improve Transport Accessibility & Regional Development Harmonisation & Common Market Traffic Avoidance Reduce price of transport Restrict road transport Improve Safety Deregulation, privatisation & liberalisation Financial acceptability Public acceptability

3-18

HALCROW FOX

4.

Bottom-Up Analysis - Ex-Post Evaluations, Review of Predictive Models and The Group Impact Framework
INTRODUCTION This chapter describes how the various impacts of transport projects are to be identified and measured. It describes the available modelling tools for forecasting and measuring the impacts of major projects. These tools are used to operationalise the measures of performance defined in the preceding chapter. According to the Technical Annex, the bottom-up approach consists of three individual sub-tasks: A group impact framework for potential IETN; Ex-post evaluations of IETN proposals against original predictions of performance; and A review and development of predictive models.

4.1 4.1.1

4.1.2

4.1.3

Each of these sub-tasks has been reported upon in an individual Internal Report (see Appendix A for a full list). The latter two tasks essentially provide background material on the state of the art in transport modelling tools and the accuracy of forecasting tools that have been used in the past to predict the impacts of major transportation projects. The former, on the other hand, is an integral component of the Policy Assessment Model development. As such, only the former is discussed within this Final Report, with the interested reader directed to the relevant Internal Report as noted in Appendix A.

4.1.4

4.2 4.2.1

EX-POST EVALUATIONS An important starting point for the Tenassess project analysis is the types and quality of project evaluations that exist at the current time - whilst a policy assessment model is being developed as part of Task 32, it will utilise existing project assessment tools in order to provide estimates of the relevant project impacts. An important aspect of this task is an initial assessment of how well projects actually perform against predictions - ex-post project evaluations fulfil this role. The most striking feature to come out of this research is the distinct lack of detailed ex-post / ex-ante comparisons that have been undertaken. This is a point raised more than once in the documents that we have reviewed and is probably best summarised by the following quotation from a Danish Transport Council report:

4.2.2

4-1

HALCROW FOX
...it is remarkable how little data and research is available world-wide that would help to answer two basic questions, (i) whether such (major transport) projects have had the intended effects and, (ii) how the actual rate of return on such projects compares to the projected rate of return.1 4.2.3 A substantial volume of transport literature exists in the field of ex-ante evaluations and feasibility studies and, to a lesser degree, in the areas of ex-post descriptive studies of the observed situation following project implementation. Little information has been found that actually compares the two situations. Bearing this situation in mind, the analysis concentrated on a series of in-depth exante and ex-post case studies that have been identified. In the main, the projects described form part of the strategic European transport network and, where they do not, can be considered as strategic at the national level, if not at the EU level. Whilst limited in number, they do provide a useful insight into the problems that have befallen many large scale projects that have been implemented in the past few years. The following project appraisals were analysed: the M25 London Orbital Motorway; a selection of other strategic UK road projects; the Amsterdam Ring Road; the French TGV network; the Channel Tunnel; and the resund and Great Belt Fixed Links. Given the variety of projects covered within this paper, and their clear fundamental differences, it was difficult to generalise when examining the accuracy of forecasts, or how ex-post situations generally compare economically to the pre-construction state. Naturally, the forecasting procedures used in different types of project are often designed and implemented with the individual characteristics of such projects in mind. However, some important conclusions could be drawn from the analysis. With large infrastructure projects, the Danish Transport Council2 state that: Cost overruns of 50 to 100% in real terms are common; overruns above 100% are not uncommon; Traffic forecasts that are off by 20 to 70% compared with actual development are common; and Forecasts of project viability are often over-optimistic to a degree that such forecasts correspond poorly with actual viability.

4.2.4

4.2.5

4.2.6

4.2.7

Fehmarn Belt: Issues of Accountability - Lessons and Recommendations Regarding Appraisal of a Fixed Link Across Fehmarn Belt, May, Report no.95.03. Danish Transport Council, 1995 Fehmarn Belt: Issues of Accountability - Lessons and Recommendations Regarding Appraisal of a Fixed Link Across Fehmarn Belt, May, Report no.95.03. Danish Transport Council, 1995

4-2

HALCROW FOX
4.2.8 The DTC note that cost overruns are, as a phenomenon, not merely confined to the public sector. The Channel Tunnel project and its financial problems bear testament to this, although these particular cost overruns may have been worse if the project was funded by the public sector. They also state that in their analysis of traffic forecasting with respect to the Channel Tunnel, the relationship between traffic forecasts and share prices for the Tunnel in the past is very strong. For transport infrastructure projects, a key point is that the credibility of forecasts relies, to an extent, on the institutional arrangements that form the forecasts. If finance from the private sector is to play an important role in the implementation of projects in the future, then to attract such finance, potential investors must see that the forecasts presented to them are based on a sound economic analysis, and are free from manipulation. Otherwise, the credibility of such forecasts will be in doubt which could, in turn, lead to future private funding difficulties.

4.2.9

4.2.10 The case of the resund project highlights how the presentation of project appraisals might be influenced in such a way that it is more likely that a project gets the go-ahead. Once projects of this sort of size are sufficiently implemented, they are generally completed, regardless of the original appraisals presented. Again, for transport infrastructure projects, the credibility of such forecasts in the future will be affected by the publics perception of the institutional arrangements that form these forecasts. The DTC illustrate how accountability is a crucial issue here. 4.2.11 Regarding traffic forecasting on roads, the earlier analysis of the M25 and other UK road schemes revealed some interesting insights into the need for analysis of potential induced traffic in strategic route appraisal. It was concluded that the very nature of Trans-European Road Networks means that it is important to consider the phenomenon of induced traffic. 4.2.12 The analysis of the Amsterdam Ring Road showed that forecasting should also consider other effects such as large return to the peak shifts in car traffic. Here, according to SACTRA3, an appreciable proportion of traffic flow on the new connection was not matched by equivalent reductions on other routes. Different types of road scheme require forecasting procedures that are, to some extent, designed for the scheme concerned. Other, more homogenous, schemes of a simpler nature can use a more generalised approach. 4.2.13 The important point with respect to roads is the question of whether forecasting errors, that is the disparity between ex-ante forecasts and ex-post out-turns, are random errors or are biased in some way, (say because of induced traffic effects leading towards a tendency towards under prediction). This issue is unresolved if projects are overly aggregated since the reasons for over prediction may not be the same as the reasons for under prediction. Progress towards quantifying induced effects in forecasting procedures is now being made, as illustrated in the
3

SACTRA is a UK governmental advisory committee - Standing Advisory Committee on Trunk Road Assessment.

4-3

HALCROW FOX
September 1996 PTRC paper on Pan-European Transport Issues Volume 23 (1996) of Transportation. and in

4.2.14 More generally, progress is also being made towards reducing the random error element in road forecasting procedures, through transparent moves towards examining ex-post data in comparison to ex-ante predictions. Again, the frequent presentation of results to the public in future may generate incentives towards reducing such forecasting errors, in terms of improving ex-ante forecasts. 4.2.15 The examination of the TGV-PSE project, though not an ex-ante / ex-post comparison in terms of forecasts versus actual outturns, does provide a useful insight into how the correct mix of service characteristics and fulfilling passenger needs can lead to the successful operation of a new project. It should be noted, however, that this particular TGV service was more successful than the TGVAtlantique service that followed. The inauguration of the TGV-PSE service caused large inter-modal substitution effects from car and plane to the train, and also generated a large volume of ridership that would of otherwise not have been present in the absence of the new service. The TGV-PSE project provides an example of how a project can pay for itself. 4.2.16 It is clear therefore that, for reasons of accountability and to progress towards an efficient allocation of resources in the context of future Trans-European Network projects, that frameworks are put into place that will provide an impetus for such ex-ante and ex-post comparisons to be made. The case studies identified in this paper enabled a detailed discussion of the accuracy of forecasting procedures with respect to the large infrastructure projects cited. It is hoped that in future, such analysis will be possible on a more general basis, for a wide range of European transport projects. 4.3 THE GROUP IMPACT FRAMEWORK

Introduction
4.3.1 The Group Impact Framework (GIF) has been designed to manage the measurement and prediction of impacts. The GIF summarises the impacts of projects, the groups on which they fall (from here onwards, this is termed incidence groups) and, most importantly, the techniques required to quantify them. As such, it provides the means of deriving measures of performance for the PAM. The research has been designed to draw together existing methods for measuring impacts. It has not been the intention of designing new measurement techniques. The absence of a suitable measurement method may thus be viewed as a constraint upon the PAM. The following sub-sections describe the PAM. Impacts

4.3.2

4-4

HALCROW FOX
4.3.3 The GIF primarily administrates impacts of larger infrastructure projects on the defined incidence groups. The framework distinguishes three major groupings of impacts: 4.3.4 socio-economic impacts such as transport resource and mobility impacts; public acceptance; finance.

The economic evaluation, through the socio-economic impacts, shows whether a project is feasible or not from the economic point of view. It is entirely possible that an economically sound project may not, however, be acceptable in a broader sense of the word. Public and political acceptance (public opinion/pressure) or lack of financial viability can stop a project with good economic returns. Incidence Groups

4.3.5

When considering possible impacts of infrastructure it is important to consider which groups are involved, as different groups may be afflicted or involved in different ways, and there also might be grounds for conflict between groups. In the GIF, all impacts are related to all relevant incidence groups and thus conflict between groups is explicitly internalised within the framework. In the GIF the following incidence groups are distinguished: Users; the users of the new infrastructure. Operators; the potential service operators. Providers; the potential (non-governmental) providers of the new infrastructure. Non-users; persons that are not potential users of the new infrastructure, but are directly faced with the impacts of it. Government and Society; Government: granter of subsidies, possible provider of infrastructure; Society: payer of taxes, bearer of externalities.

4.3.6

4.3.7

It is helpful to group Government and Society together because democratically elected governments are elected to represent society, and the perspectives of the two groups often coincide.

Functional Relationships 4.3.8 Within the framework five incidence groups and three major sets of impacts will be distinguished, i.e. socio-economic impacts, public acceptance and financial impacts. Table 4.1 gives a very schematic overview of the functional relations between impacts and incidence groups. These linkages may only represent one view of the way in which infrastructure projects impact upon different groups in society but they allow the subsequent work packages to be operationalised. There is no reason why, in principle, other groupings cannot be considered if they are deemed 4-5

4.3.9

HALCROW FOX
of relevance in certain situations. The overriding principle must be that any linkage that is made must be justifiable in terms of measuring the impact upon the impactee in a logical and coherent manner.

4-6

HALCROW FOX
Table 4.1: The Group Impact Framework (schematic) - Functional Relations between Incidence Groups and Impacts
Impacts Users Socio economic impacts Investment costs Economic costs Safety Quality Employment Environment Regional development Social Impact Public acceptance Financial viability Service Operators Incidence Groups Infrastructure providers Non-users Society & Government

Principles 4.3.10 In this section four principles will be formulated that should apply to each ticked group impact cell in the framework, as illustrated in Table 4.1: simplicity: at this stage of development of the Group Impact Framework the approach to the group impact cells will be simple and pragmatic. When the framework is used in later feasibility studies the simple straight-forward approach to the cells is replaced by in-depth studies; consistency at all levels of disaggregation: the approach to the group impact cells will be such that the framework is applicable at all regional and policy levels in a consistent manner, but also consistent in methodology between assessments of different (alternatives of) infrastructure projects; applicable throughout the planning process: the approach to the framework will make it applicable at varying levels of ambition concerning the level of detail, in accordance with the stage in the planning process; no weighting applied in the framework, all impacts are equally important: the framework only administrates impacts, serving as input to the GAM. In the GIF impacts are not combined or compared and are thus not weighted do not need to be valued.

4-7

HALCROW FOX
Socio-Economic Impacts 4.3.11 Within the Group Impact Framework, as can be seen in the first block in table 4.1, the following socio-economic impacts are distinguished: investment/maintenance costs of infrastructure; economic costs; safety; quality; employment; environment; regional planning; and social impact.

4.3.12 Some of these impacts are said to be efficiency impacts, like the economic costs, safety and quality. Regional planning and social impact are regarded to be equitable, i.e. distribution between regions and incidence groups. Employment and environment have efficiency and distributive implications. 4.3.13 The focus in discussing each impact is the contribution of new infrastructure to transport related policies on a EU scale. This scale and the chosen general framework methodology lead to presenting minimum requirements for impact measurement. This makes the framework broadly applicable and data request minimal. In addition to this, starting points will be mentioned for more detailed approaches, possibly to be applied in the case studies. 4.3.14 Each impact group is described briefly, below. All are outlined in greater detail in the Appendix volume. Investment/Maintenance Costs of Infrastructure 4.3.15 The investment category is very important due to the fact that most of the other socio-economic impacts, but also public acceptance and financial viability, are connected with the investment sum. There is a trade-off between the invested sum in new infrastructure and the extent and quality of other impacts. For example, by tunnelling part of a new railway near some populated areas the investment costs will be higher, but: travel time may be lower because of less congestion; quality in terms of security or even ambience may be higher; environmental impact, in terms of noise pollution, land-loss and landscape value, may be very positive; public acceptance may improve.

4.3.16 Besides the initial investment in infrastructure, its maintenance is an important issue. Maintenance ensures the extent and quality of impacts in the long term, and thus enables the long term effects.

4-8

HALCROW FOX
4.3.17 Through integrating investment as an impact category, the GIF tables to assess impacts of different project alternatives, and to compare the capital cost and associated impacts with public and financial acceptance. Assessing different alternatives of a project implies running the GIF as many times as is necessary. Functional Relationships Impact to Incidence Groups 4.3.18 The investment cost impact of an infrastructure project is directly related to the service operators and infrastructure providers. 4.3.19 The infrastructure providers are affected, in economic terms, only by the investment sum4. This category enables the framework to deal with the trade-off between investment sum and associated impacts, and thus to assess different investment-impact alternatives of the same project. 4.3.20 The service operator is affected, in the economic sense, by the investment through the subsequent maintenance costs. The maintenance costs will be approached by life-cycle replacement costs. Life-cycle is defined as the period during which the whole (physical) infrastructure is replaced through yearly maintenance. The total life-cycle replacement costs are defined as the present value of all maintenance expenditures in the economic lifetime period.

Method/Model for Measurement 4.3.21 The investment sum should be known from project planning. The life-cycle replacement costs can be derived through relating the investment sum to the expected economic lifetime of the new infrastructure, and thus deriving a yearly percentage of investment costs as expected maintenance costs. As there will be no discounting, the present value of the maintenance expenditures equals its sum, and thus investment costs. Economic Costs 4.3.22 The economic costs, as defined here, impact of infrastructure projects will be approached through travel time and user (operating) costs impacts. This is a narrower definition of economic costs that is usually adopted in project appraisal, but reflects the project impacts that are defined in this category. Other areas of cost that are sometimes classed as economic costs, such as accidents and environmental damage, are defined outside this category in order to more closely reflect the linkages of project impacts to policy objectives. Travel Time 4.3.23 In traditional cost-benefit analysis travel time savings are generally one of the most important benefit components of an infrastructure project. Within the GIF this cannot be determined ex-ante, weights to impacts are not given, but obviously travel time is still an important impact of infrastructure. Travel time is approached
4

All financial aspects and impacts of the investment are dealt with in the financial viability block

4-9

HALCROW FOX
as a short term impact. Changes in travel time occur immediately after opening the new infrastructure, and this change in factor costs implies potential long term effects for example in employment levels or choice of house location. These indirect impacts are handled elsewhere. 4.3.24 Main components of travel time are: off-vehicle time; access and egress time; waiting time; transfer time; in-vehicle time (including congestion).

4.3.25 The precise definition of total travel time thus depends on the characteristics of the trip: mode(s) involved; amount of segments (per mode); access and egress characteristics.

User Costs 4.3.26 User costs can be defined as the operating costs for the user of the infrastructure, consisting of fixed costs, associated with (possible) ownership of transport mode, and variable costs, associated with use of the transport mode. This classification already illustrates the mode dependency of user costs, as consequence user costs are approached further per mode, for passenger and freight transport. Safety 4.3.27 Safety is dependent on a large set of variables, but especially on traffic volume and speed behaviour. The general approach to safety in the GIF concentrates on traffic volume, additional attention will be paid to speed behaviour at the end of the paragraph. Functional Relationships Impact to Incidence Groups 4.3.28 Safety as an impact of infrastructure has relations to the users and the service operators of the infrastructure. Users are directly affected by accidents. Service operators are affected directly and indirectly - directly when employees of the operating company are involved in accidents and indirectly when safety issues in general have repercussions on operator income through decreasing use of the infrastructure it operates. Given the strategic nature of the assessment framework being developed, only the direct impact are considered. Method/Model for Measurement 4.3.29 The framework must assess the contribution of the new infrastructure to policies concerning safety. Safety expressed in fatalities, injured and damage is considered to be linear related to kilometre volumes. In passenger transport this is passenger

4-10

HALCROW FOX
kilometres, as a combination of vehicle kilometres as a measure of traffic intensity and number of people per car as a measure of numbers affected, in freight transport tonne kilometres. The linear relationship applies within unchanged safety policies. Safety is also considered to be mode dependent. Quality 4.3.30 Quality of infrastructure is another important component of mode choice in passenger and freight transport. Three elements of quality can be identified: comfort; reliability; security; ambience.

4.3.31 Comfort relates to passenger transport and says something about the convenience of the journey and the facilities available on board, entities that are expressed in qualitative terms. There is evidence for increasing weight attached to comfort by passengers as total travel distance or time increases. Comfort is also a subjective experience, and although it might be possible to define a common denominator for quality, it still will be qualitative of nature. Thus, comfort will be approached qualitatively. 4.3.32 Reliability, defined as the measure to which set time schedules are reached, is also an important quality aspect. in both passenger and freight transport. Reliability obviously is closely related to congestion. There is evidence for high experienced reliability for short distance train trips, and lower for road trips, and converging reliabilities across modes on longer distances. 4.3.33 Quality is directly dependent on the investment in infrastructure, thus within the framework there is a trade-off between, among others, quality which has an impact on users and investment which has an impact on service operators and infrastructure providers. The financial part of this trade-off is dealt with in the finance block - a higher quality specification for a project will be reflected in higher investment costs. 4.3.34 Security relates to both passenger and freight transport, and might be the most important of the three quality elements when it comes to mode choice. Security could be approached in a quantitative way, for example by defining the percentage of trips considered secure by passengers or freighters through interviews, or by measurement with accidents in relation to transport volume as a security indicator. To apply this approach on EU scale would require an enormous effort, especially because the framework must administrate changes. The GIF proposes a general approach, applicable at all geographical levels and levels of ambition. In this line of thought security will also be approached in qualitative terms. 4.3.35 A much more difficult to approach entity is ambience, but it certainly has to be reckoned with. One could describe ambience as being related to the image of a transport mode, a part of consumers perception of the infrastructure or mode

4-11

HALCROW FOX
which falls beyond comfort. It seems obvious that ambience is difficult to quantify, and also will be approached in qualitative terms. 4.3.36 Concluding from the above, the three elements of quality identified above all will be approached qualitatively. The impact of new infrastructure on quality can then for example be expressed in terms like: a positive (+) change in comfort; a very negative (--) change in ambience; a very positive change (++) in security; no quality change (). 4.3.37 The question remains whether quality has to be assessed on its three elements or as a whole, and if it is approached on the elements how to aggregate to total quality change.

4-12

HALCROW FOX
Units of Expression Output 4.3.38 Quality will be approached qualitatively, and changes in quality can be expressed as: very positive positive neutral negative very negative (++) (+) () (-) (--) or 5 3 0 -3 -5

Method/Model for Measurement 4.3.39 Quality as total, and possibly its elements comfort, security and ambience, will be approached qualitatively. Defining a good method for measurement therefore is difficult, and will always be based on subjective judgement. 4.3.40 The change in quality, as impact of new infrastructure, must be scored on a scale like proposed above. This could be total quality or its three elements. If quality is approached on the elements the question remains how to aggregate to total quality change. The three elements are rather heterogeneous, leading to the conclusion that where possible they should be considered separately. It is difficult to hold that all elements are equally important, but it is impossible to attach objective and relevant weights to the elements on which to aggregate in all instances. The straight forward approach chosen is to consider all equally important, and thus, to attach equal weights to all three elements. Employment 4.3.41 The employment impacts of new strategic infrastructure investments are potentially of utmost importance, as employment is a very dominant policy issue throughout Europe, at all policy levels. The paragraphs below document one possible approach to examining and measuring this impact. What follows may, therefore, be supplemented and amended if a more suitable (in terms of analysing observed policy objectives) method of analysis is deemed necessary. Method/model for measurement 4.3.42 The measurement of the employment impact of larger infrastructure projects will be approached very pragmatically at this stage, through:

4-13

HALCROW FOX
mode specific investment (IM). change in added value through the investment; labour productivity changes;

4.3.43 In mathematical format the change in employment can be approached through:

I M added value employment added value labour productivity per ECU changes
4.3.44 Mode specific investment, through added value per invested ECU figures leads to total added value, and through the added value to changes in employment, taking into account labour productivity changes. 4.3.45 In terms of computing these project impacts, as suitable regional economic model is required - the German BVWP provides some interesting examples of potential approaches to the forecasting of employment impacts. Depending upon the focus of policy objectives, the forecast change in employment can include either the temporary increase in employment during construction or the projected longer term employment effects created by the accessibility enhancements conveyed by the project. Environment 4.3.46 The environmental impact of new infrastructure has a variety of aspects, some of them connected with the infrastructure itself and some with the accompanying (de/increase in) traffic volume. 4.3.47 Directly linked to the infrastructure itself are: land-loss; endangering of eco-systems; loss of (perceived) environmental value.

4.3.48 Land-loss due to infrastructure is obvious. For all modes for which new infrastructure is build some surface area will have to be available and in most cases this area has actual or possible alternative use. 4.3.49 The endangering of eco-systems and the loss of (perceived) environmental value are related to cutting across the landscape by the new infrastructure. Eco-systems that are disturbed because of for example physical splitting by a railway or road might seriously endangered5. The loss of environmental value, which is obviously
5

Although physical splitting of feeding and breeding areas of wild animals endangers locally the species involved, all kind of solutions have been raised by infrastructure builders. In the Netherlands several successful examples of wild animals tunnels and wild animals crossovers exist. In co-operation with experts wild animals are trained

4-14

HALCROW FOX
linked to individual/human perception, is also related to the cutting across the landscape. When looking out of the window or walking in the country people prefer the sight of green fields and woods to railways, highways or airports. 4.3.50 These impacts related to the physical presence of new infrastructure are hardly quantifiable and will not be directly integrated into the GIF in this block. As stated above the value attached to these impacts depends on the perception of individuals involved. From this viewpoint assessment of this impact is integrated in the public acceptance block of the GIF, which is discussed below. Through the public acceptance block one might come to the conclusion that some project alternative is of a more marketable nature to the public than an other because it relieves some environmental burden. The extra costs involved for example should be confronted to public pressure, and possibly to financial viability in both cases. In short, the magnitude of certain environmental impacts may be traded against both the financial viability (and investment) impact categories in order to increase the likelihood of public acceptance. 4.3.51 The environmental impacts of new infrastructure which will be considered here are those related to traffic volume, primarily: noise; air pollution.

4.3.52 Total noise due to traffic is a complex relation between, for road traffic for example, traffic intensity and composition, driving behaviour, average speed, structure of soil and type and amount of preventive measures. The impact of noise is also linked to the location of the new infrastructure, especially the proximity to housing areas. 4.3.53 A more pragmatic approach to noise would be:

noise = 3 log ( vehicle kilometres)

4.3.54 Taking into account that the relation between noise and vehicle kilometres is logarithmic, and that the diffusion is some third power function. Whilst not a replacement for a detailed project noise impact study, for the purposes of the assessment being performed by the project methodology, it is probably sufficiently robust to give an initial indication of a schemes impact in this area. 4.3.55 Air pollution, expressed as emissions of pollutants, depends primarily on driving behaviour, and thus amount of cold starts, average speed, speed changes, congestion, type, age and technical condition of car, type of fuel, type and amount of preventive measures. In the GIF only CO2 and SO2 emissions are considered although other strategic indicators can be incorporated later if deemed important.

with reasonable success to use these tunnels and crossovers between feeding and breeding areas instead of crossing railways and roads.

4-15

HALCROW FOX
4.3.56 For other modes, estimates based upon energy / fuel consumption will apply in manner akin to that described above. 4.3.57 Integrating all the factors that are necessary to accurately determine changes in environmental impact goes far beyond the actual ambition level of the GIF and the assessment framework to be subsequently developed. At a more pragmatic strategic level, it can be assumed noise and air pollution in all events increase with traffic volume. Method/model for measurement 4.3.58 There is a variety of very complicated models for assessment of noise and air pollution, especially concerning the dispersion. Alongside these models, various more pragmatic approaches have been developed to assess the impact of noise and air pollution, like: decrease in land/house value in near proximity infrastructure; costs of preventive measures; percentage of inhabitants exposed to noise levels in excess of some defined limit; noise contours, number of affected people within several defined noiseclasses.

4.3.59 Within the GIF noise and air pollution will be directly related to traffic volume through key figures based on mode/trip characteristics and traffic composition relevant for the actual scale of analysis. Besides this, modal split changes will be taken explicitly into account, as the pollution levels may differ enormously between modes. Regional planning 4.3.60 In the pragmatic approach chosen for the GIF at this stage, regional planning impacts of new larger infrastructure projects will be considered in a qualitative manner. 4.3.61 The general impact of infrastructure on regional aspects is strongly connected to the concept of accessibility. New infrastructure is assumed to improve accessibility to opportunities for all segments in society, for example accessibility of: regions (and people) in general; labour markets and employment opportunities; potential markets for goods (consumers) and supplies.

4.3.62 Thus the regional impact depends on measures of accessibility, aspects to be taken into account would be for example: size of output markets; size of input markets;

4-16

HALCROW FOX
interregional distances; cost of labour; availability of labour; urbanisation; regional policy.

4.3.63 Regional planning impacts of infrastructure projects are considered to be: long term impacts following from changes in factor costs; distributive of nature; impacts related to land-use factors.

4.3.64 In the GIF is supposed that there is a clear distinction between short and long term impacts of infrastructure. Short term impacts lead to changes in factor costs, which changes induce the long term impacts, like regional development. 4.3.65 The regional impact of infrastructure is considered to be distributive of nature. New infrastructure induces changes in allocation of activities and reaches of markets, and thus redistributes opportunities between regions and groups. 4.3.66 The regional impacts of infrastructure relate to land-use factors, through: urbanisation or ruralisation; changes in allocation of activities.

4.3.67 New infrastructure, when assuming that it improves accessibility, increases the radius within which the interaction between for example working and living, producing and selling takes place. Through this increase the relative attractiveness of areas will change, and people or firms might choose to relocate housing, working, producing or selling.

Units of Expression Output 4.3.68 Potential regional development will be approached qualitatively, and changes in development will be expressed as: very positive positive neutral negative very negative (++) (+) () (-) (--) or 5 3 0 -3 -5

Method/Model for Measurement 4.3.69 Various methods and approaches are available for assessing some of the elements of regional development. For example the change in attractiveness of a region for firm establishment can be approximated through the change in land price of

4-17

HALCROW FOX
business areas in that region. The same approach could be applied to housing areas. 4.3.70 At this stage of development of the GIF the method for measurement will be very pragmatic and straight-forward. Based on changes in projected GDP growth per region6 and expert opinion on overall regional development, the impact of the infrastructure will be scored in accordance with the scheme above. In later stages of development, and possibly in the case studies within this project, this approach can be replaced by more in-depth studies. Social impact 4.3.71 The social impact of infrastructure is, comparable to regional development, closely related to the concept of accessibility. The accessibility of and within regions is a prime condition for social integration. 4.3.72 Different social impacts can be identified: social cohesion; European integration; income distribution; integration of minorities.

4.3.73 Social cohesion is about integration between social groups within regions. European integration deals with the cohesion or integration between regions or countries. 4.3.74 Income distribution and minority (e.g. ethnic, disadvantaged) issues can be seen as special cases of social cohesion, but are mentioned apart because they are often addressed in special policy objectives. Units of Expression Output 4.3.75 Social impact will be approached qualitatively, and changes will be expressed as being: very positive positive neutral negative very negative (++) (+) () (-) (--) or 5 3 0 -3 -5

Method/Model for Measurement 4.3.76 Assessment of social impact changes due to infrastructure projects includes all 4 integration elements specified above.

Some form of regional economic model will be needed in order to assess such changes.

4-18

HALCROW FOX
4.3.77 Usually social cohesion is measured through income distribution, the impact of a project through changes in the distribution. This is a useful approach and GDP per capita changes can be estimated at the regional level. 4.3.78 European policies also aim at social integration between regions, thus this should be taken into account in the GIF. This integration could be approached through changes in the number of interregional job occupation. 4.3.79 A similar approach can be applied to the integration of minorities, by assessing the impact of a project on the number and quality of jobs occupied by policy target groups. 4.3.80 Combination of these impacts, thus income distribution changes and demographic impacts, implies a qualitative scoring mechanism. By their very nature, many of these impacts will be specific to individual project and a detailed analysis of the potential social impacts will be required. Even at the EC level, many social impacts will only be relevant for projects in certain geographic areas. Public acceptance Functional relationship impact to incidence groups 4.3.81 The acceptance impact is first of all related to non-user and user groups. 4.3.82 Non-users are those whom a new product is not made for. They may in one or the other way be affected, or not affected at all. The probability of reactance (= lack of acceptance and preparedness for reactance) is especially high among non-user groups that have no direct advantages to expect from a certain project. They are no customers, they "do not need" the planned object. Particularly when other people decide about things that might affect ones own life (this happens often, but not often as well visibly as in connection with, e.g., a new railway track, or a container terminal, or the like) and no advantages are derived from a project, this may lead to lack of acceptance and as a further consequence to resistance. 4.3.83 The degree of affectedness depends, among others, on the individual's subjective perception. It sounds trivial, but the less the effects are felt, the lower the degree of affectedness is experienced. 4.3.84 In order to ensure a successful implementation of a project and to guarantee the use of it, reasons for making, or making no use of services offered have to be detected. If the project does not meet interests of the target groups, the use will not be in the planned sense. Returns will be lower than expected, public budgets will be strained. Method/Model for Measurement 4.3.85 Acceptance cannot be measured directly. To get an idea of the actual acceptance situation the concept of "acceptance" has to be operationalised. This has to be done, on the one hand, by comparing in which way product characteristics correspond with needs and interests of the specific target groups, on the other hand

4-19

HALCROW FOX
which image qualities are connected to the project and the ones who are responsible for it. 4.3.86 By means of information policy (studying literature, expert interviews) researchers carrying out measuring acceptance should examine which product characteristics ( = technical data of the project) could cause problems, e.g., which characteristics meet with lack of acceptance. The results of these analyses will help to obtain a general view of the actual situation for the researcher and will be the basis on which questionnaires for interviews will be developed. Depending on: which target groups have been identified which larger infrastructure projects are going to be built where the larger infrastructure projects are going to be built.

Financial Acceptance 4.3.87 Three aspects of financial acceptance have been identified: an assessment of financial viability; and an analysis of the distribution of financial benefits; an assessment of risk.

These are summarised below. Financial Viability 4.3.88 An investment should yield a stream of returns over future periods. Whether a project is financially viable depends upon these cumulated financial receipts. The Net Present Value (NPV) of a project is one measure of financial feasibility. If the Net Present Value of a project is positive, then it is financially viable, otherwise it is not. 4.3.89 The Internal Rate of Return (IRR) of a project is the discount rate (r*) at which its Net Present Value equals zero. 4.3.90 The Return on Equity (ROE) is the likely return offered by a project to shareholders in a venture. Such returns are in the form of the potential flow of dividends that such shareholders are likely to benefit from. Given that shareholders provide equity for the implementation of a project, the magnitude and probability of a sufficient return must compare favourably with alternative investments for the providers of these inputs of finance. If a project does not look attractive enough to potential shareholders, then it is unlikely that it will attract the finance for it to be implemented.

4-20

HALCROW FOX
4.3.91 The Debt Service Coverage Ratio is the ratio of the cash-flows available every year from the project to service debts incurred in implementing a project, to the sum of the annual debt repayment and interest charges. This ratio therefore shows the ability of the annual cash flows generated by a project to service the debts incurred by it. As a measure of the security offered to lenders, the Debt Service Coverage Ratio is also an indicator of a projects financial viability. 4.3.92 The Debt to Equity Ratio is the ratio of the sum of outstanding long-term debts, to the equity at the end of each year. This is a measure of the attractiveness to lenders in terms of security. A low ratio indicates that equity financing is important relative to debt, revealing that project sponsors are highly committed to a particular project. However, a higher emphasis on equity financing means that more cash-flows have to be distributed to shareholders when dividends are to be paid, or that available cash-flows must be spread over more shareholders. Distribution of Financial Impacts 4.3.93 The distribution of financial impacts amongst various groups and stakeholders can, under certain circumstances, dictate the overall viability of a project irrespective of the economic or financial returns. This 'equity' effect is becoming increasingly relevant as private sector participation within project design, promotion and funding grows. In short, some projects require that all key stakeholders benefit to a certain extent if they are to proceed. 4.3.94 The need for financial equity within a project largely reflects the possibility that schemes may be blocked where key stakeholders are likely to lose financially. In this sense, financial equity is a project constraint - a condition which must be overcome if the scheme is to proceed. In terms of the Group Impact Framework, financial equity is treated as an impact, that is, a consequence of a project. Ideally, successful projects should seek to optimise this impact by ensuring that financial benefits are spread widely amongst all key parties. Conditions for Appraisal of Financial Equity 4.3.95 Key tasks in the determination of financial equity are as follows: Identification - identification of the affected parties; Classification - as to whether each party is a potential 'promoter' or 'constraint'; Measure of financial performance - specification of how the financial impact upon each party is to be defined; Prediction - forecasting of the scale of impact upon each party.

4-21

HALCROW FOX
4.3.96 Identification requires that all potential stakeholders in the project be identified. These are likely to include the potential operator(s), financiers, planning authorities and developers. Classification requires that stakeholders be separated out into those most likely to promote the project (operators, financiers); and those who impose constraints - conditions which must not be violated (planning authorities, competitors). 4.3.97 The recommended measures of financial performance are: cost : revenue ratio (discounted at a nominal rate, say 10%, for each party); a simple measure of profitability; rate of return, by stakeholder (i.e.: return on stakeholders investment); and description of other, intangible, impacts (publicity, effects on related interests, etc.).

4.3.98 Prediction requires that the above performance measures are forecast over the life of the project. 4.3.99 Finally, the results of the analysis must be interpreted and presented in order to assess the overall level of financial fairness of the project. The most straightforward summary measure is the threshold percentage net gainers from the project, where a score of 100% indicates that all stakeholders benefit to some extent; and 0% indicates that no party gains. A more sophisticated system allows particular parties to be weighted, however, at this preliminary stage of framework development, simplicity is favoured. Table 4.2 summarises the presentation of the assessment of financial equity. Table 4.2:
Stakeholder Cost Revenue Ratio Stakeholders Rate Of Return Description Of Effects Sum Summary Measure (%)

Financial Equity Summary Table

1 2 3...

Financial Risk in Strategic Transport Projects 4.3.100 Risk and uncertainty are major constraints upon the successful implementation of projects. The long-term nature of transport projects, through both a long construction phase when revenue is not being generated, to long pay-back periods for project finance, both serve to introduce elements of risk and uncertainty. Uncertainty in Traffic Forecasts 4.3.101 Traffic forecasts produced from transport models that are used to predict likely usage of potential transport projects are based on numerous assumptions and predictions regarding future economic and policy developments. Any variation in one or more of these assumptions or predictions is likely to alter the forecasts and consequently the viability of the project. Variations may concern any one of a number of factors including different rates of economic growth, different levels

4-22

HALCROW FOX
and mix of land use activity and different assumptions regarding the availability of competing transport schemes, to name just a few. 4.3.102 If the future could be predicted with absolute certainty, it would be possible to define traffic levels associated with particular socio-economic or development scenarios at specific points in time, thus providing an accurate picture of potential revenues at all points in time in the future. As this is clearly not the case, even with the best estimates of future travel patterns, based on sophisticated land use and transport modelling techniques, the future cannot be predicted with certainty. Risk Analysis 4.3.103 As uncertainty cannot be avoided in project assessment, decisions can be greatly informed by quantifying risk, in order to identify a range in which the most likely traffic levels will fall. Risk analysis essentially builds a range of uncertainty around some central set traffic (and revenue) forecasts, in order to assess the likely levels of risk that will be associated with a particular project. These procedures take into account a number of causes of traffic level variation and, for specified years in the future, produce a probability distribution which indicates the complete range of possible outcomes and the level of uncertainty associated with this. 4.3.104 The results of a risk analysis are used to design risk management measures within a project. These include setting the IRR or ROE to reflect levels of risk, or changes to the project design. The greatest source of risk and uncertainty generally concerns revenue projections, and allowance for risk therefore focuses on the revenue side of projects. Summary and Conclusions 4.3.105 The GIF administrates (structures and analyses) impacts of larger infrastructure projects on specific incidence groups. 4.3.106 The focus of the GIF is on a clear and explicit assessment methodology rather than on detailed analysis of any one impact. The guiding principle is that all impacts, at this stage, are equal. Weighting of specific impacts is left as an exercise for the user of the Policy Assessment Model. 4.3.107 The GIF is sufficiently flexible and dynamic to deal with constantly changing policies. Incidence groups, impacts or impact relationships can be augmented, altered or updated as and when required. 4.3.108 The GIF impacts, although not necessarily the impact measurement tools, are building blocks for the measures of performance of infrastructure projects on policy objectives.

4-23

HALCROW FOX
4.3.109 The GIF consists of 3 blocks of impacts, socio-economic, public acceptance and finance. Public acceptance is a new concept within assessment. This area is potentially of central importance to the future planning and successful implementation of new transport infrastructure projects. 4.3.110 Economic and financial impacts are strictly separated. This reflects the increasing importance of making projects financially attractive in order to involve the private sector. 4.3.111 EU wide common treatment of impacts - in other words, within the GIF, geographic variations in impacts are not identified. These issues are left for specific Case Studies to identify. 4.3.112 No monetary values are attached to impacts. 4.3.113 All impacts, all incidence groups, all modes, freight and passenger traffic fit into a single interrelated framework system. 4.3.114 The approach to the group impact cells is simple and straight-forward in order to maintain clarity and make the impacts explicit. 4.3.115 The approach to the group impact cells is consistent in methodology between assessments and between geographical and policy levels within assessments. 4.3.116 The approach to the group impact cells is free from value judgement, no weighting is applied. 4.3.117 The approach to the group impact cells is applicable to different stages of the infrastructure project planning process. 4.3.118 For each group impact cell, what should be measured, what should be forecast (scenarios), the units of expression and the time scale needs to be identified. 4.3.119 Double counting may be present but, in those cases where this occurs, it will be made explicit. In such an assessment framework, double counting is not a problem in so far as we are scoring projects against individual policy objectives and analysing the results in a disaggregate manner. If the policies identified require the utilisation if similar impacts more than once then this is a valid exercise. In this sense, the method is more akin to multi-criteria analysis insofar as the disaggregation removes double counting and, more importantly, the method highlights where similar or inter-related project impacts have been utilised.

4-24

HALCROW FOX

Table 4.3: Master Table

Investment/maintenance Functional relationships to incidence groups User costs Safety Quality Employment

Service operators Infrastructure providers

Users

Users Service operators

Users

Society

Relevant modes Long term versus short term Units of expression output

All modes Short term Monetary terms

All modes Short term

All modes Short term

All modes Short term Qualitative scores (very positive to very negative)

Not relevant Long term Number of jobs

Travel time in minutes User costs in monetary terms

Number of fatalities Number of injured Damage to freight in monetary units Per passenger/ tonkilometre

Method/model for measurement

Investment and life-cycle replacement costs

See relevant sections

Linear relationship between traffic volume and number of injured and fatalities

Scoring mechanism

Key figures, pragmatic model

Measurability of the impact Scenarios

Good No specific scenario information needed

Good Traffic volume growth

High

Low Not relevant

Reasonable

Policies affecting safety Technological developments affecting safety

Added value per invested ECU Added value per job Labour productivity development

Traffic volumes

Minimum data requirements

Investment costs Expected lifetime

See relevant sections

Traffic volume for all modes Number of injured and fatalities for all modes Number of injured and fatalities of operating personnel

Expert judgement

Change in added value through the investment Labour productivity changes Mode specific investment

Environment Regional planning

Relevant scenario data

Public Acceptance Financial Acceptance

Social Impact

4-25

HALCROW FOX
Functional relationships to incidence groups

Non-users Society

Society

Society

Non-users Users

Users Service operators Infrastructure providers Society

Relevant modes Long term versus short term Units of expression output

All modes Short term impact

All modes Long term impact Qualitative scores (very positive to very negative)

All modes Long term impact Qualitative scores (very positive to very negative)

All modes Not applicable Qualitative scores(very positive to very negative)

All modes Short-term or long term

Noise in decibel per vehicle kilometre Air pollution CO2 and SO2 emissions per vehicle kilometre

Financial NPV IRR Cost/revenue .. disaggregated by impact group

Method/model for measurement

Through key figures, traffic volume and modal split changes.

Pragmatic, expert opinion.

Scoring mechanism on Income distribution and demographic analysis

Questionnaire

Financial model, disaggregated

Measurability of the impact Scenarios

Good

Low Technological development scenario per mode GDP growth per region

Low GDP scenarios

Good Not applicable

Reasonable GDP scenarios

Developments in composition of fleet scenario per mode

Minimum data requirements

Traffic volume scenario A set of key figures on noise an air pollution Traffic volume Modal split changes Scenario information GDP and scenario information

Income distribution figures Scenario information Demographic information

Technical data of the project

Costs, disaggregated Revenue, disaggregated GDP scenarios

4-26

HALCROW FOX

5.
5.1 5.1.1

Operationalising the Policy Assessment Model

INTRODUCTION This chapter essentially outlines how the measures of performance and project impacts are physically brought together within what has been termed the Tenassess Policy Assessment Model. It commences with the general structure of the model as developed by the Tenassess research team and then goes onto to discuss the policy weighting system that has been developed and the background data that is required to make all the measures of performance function in the manner intended. We also demonstrate how a measure of performance is constructed from first principles (i.e. from transforming a verbal policy objective into a operationalised mathematical formula), using an example from the IC5/IC25 motorway case study in Portugal. The actual physical form of the Policy Assessment model spreadsheet is outlined in Appendix D, through a series of figures that demonstrate the functioning of the Excel spreadsheet.

5.1.2

5.1.3

5.1.4

5.2 5.2.1

MODEL STRUCTURE The functioning of the Tenassess Policy Assessment Model has been outlined in a schematic manner previously, in Figure 2.3. The policies of the EU and each member state reviewed as part of the project have been grouped into the eleven policy areas discussed in Chapter 3. For each policy area a measure of performance has been derived, with policy targets that wherever possible refer to up to date policy information directly from the country in question. For some countries, the existence of quantifiable measures of policy performance (i.e. what constitutes success and failure of policy in a measurable format) has been difficult to identify. For this reason, the greatest level of detail within the model is reserved for those countries where detailed Case Studies are being undertaken. In these locations, project resources allow for more detailed analysis and interpretation exercise to be undertaken. The derivation of the measures of performance for the nine main policy areas (i.e. excluding financial and public acceptance) was based on detailed policy data from Germany and Austria. The Case Study testing phase has identified a number of modifications (as discussed in Chapter 3) to this initial specification. As such, the model is now able to adequate address policy issues at all levels (i.e. European, National and Regional) of the decision-making hierarchy.

5.2.2

5.2.3

5.2.4

5-1

HALCROW FOX
5.2.5 In some instances, additional (country specific) policy areas are being incorporated in order to address issues that may be peculiar to certain Member States within the Community and also policy issues that may only be of immediate relevance at the level of the European Commission itself. The final Master Policy Table within the model will, therefore, as far is possible, incorporate operationalised measures of performance for policy areas in most EC countries. When the model is run, only the policies of the countries relevant to the project being assessed are copied forward to the data entry phases of the model, in order to reduce the amount of information on display at any one stage. The user is then prompted to enter the project impact data in a format that is compatible with the policy measures of performance. Running the model then scores the project against the policy area objectives to produce a set of disaggregate scores for each policy area.

5.2.6

5.2.7

5.2.8

5.2.9

5.2.10 At this stage, the user is prompted to enter a set of weights (see below) that best reflect their preferences in terms of ranking the importance of the different policies areas. It is also possible to rank the policies of different actors within the decision-making hierarchy with different weights as well say, in order to reflect the greater influence of national policy. At any one stage, up to five different weighting systems can be specified in order to test the sensitivity of the results to the weights attached to different policy areas or actors in the decision-making hierarchy. 5.2.11 Running the next stage of the model produces a weighted set of scores for the project. The user can then interactively return to earlier stages to examine further policy weighting options, different project impacts (say, from alternative route alignments) and/or the sensitivity of any one project impact on the overall result.

5-2

HALCROW FOX
5.3 WEIGHTING SYSTEMS The Importance of Policies 5.3.1 Weights allow decision-makers to express preferences and values within the analysis outlined above. They also enable the sensitivity of the results to changes in the scoring system to be assessed, or to changes in the perceived importance of different objectives. Weights are used to multiply the scores (performance measures) allocated to each objective within the Policy Assessment Model. They are, essentially, a measure of importance. Hence, the score (+5 to -5) indicates the extent to which a policy objective is furthered or constrained by a particular project, as estimated by the measure of performance; the weight indicates the importance of the policy objective. The product of these two factors indicates the value score for that particular performance measure. This value score is carried forward to the final interpretation stage. The concept of the importance of policies can be applied in several ways: importance of one or more specific policy areas (environment, safety, etc.); importance of a particular Member States policies (Germany, Finland, etc.); importance of policies at a particular level of the hierarchy (local, regional, national, EU).

5.3.2

5.3.3

5.3.4

The single, over-riding principle governing the design of a weighting system is that it be transparent. In particular, that the application of weights be visible for all to inspect and question, and that it be open to audit. The proposed weighting system is based upon the above concept of importance of policies. Three parts to the system are proposed, concerning specific policy areas; policies of particular States; and policies at particular hierarchical levels. Weights will be applied by the decision-maker. The default weight will accord the same level of importance to all policies and between different levels in the decision-making hierarchy. The process for applying weights is a manual addition to the processes of the model. The procedures are relatively simple to undertake and can be quickly implemented. Hence, sensitivity tests can be undertaken with relative ease by varying weights.

5.3.5

5.3.6

5.3.7

5-3

HALCROW FOX
5.3.8 The weighting system adopted so far comprises a decimal scale of 0.0 to 3.0. This could be re-specified as a percentage scale, though this would create considerably larger scores for which there does not seem to be any particular advantage. It should be stressed at this point that the adoption of a scoring system from -5 to +5 and a weighting system of 0 to 3 is only one way of expressing the output. The scoring scale corresponds broadly with commonly used ranking system such as very dissatisfied, dissatisfied, indifferent, satisfied and very satisfied. The weighting system has the advantage of keeping the resultant numbers to a small overall size and also allows for an irrelevant policy coupled with negligible project impact to score zero in the model. It also allows the number 1 to be the unweighted scores, as multiplying score by one reproduces the score.

5.3.9

5.3.10 In short, the scores and weighting scales adopted do, in our opinion, provide one of the most transparent ways to represent the information. 5.3.11 The scores signify the following: 0.0 - no importance attached to the policy. This will result in a score of zero attached to the policies in question. 3.0 - very high importance attached to the policy, which is accorded maximum importance in the analysis. 2.0 - moderate importance attached to the policy. 1.0 - the default weight.

5.3.12 If no weights are applied, a default weight of 1.0 is given across all policies and also between different levels in the hierarchy and between different geographic bodies. 5.3.13 The weights specified in the model are then applied to the project scores in the following manner. 5.3.14 In order to facilitate comparison between the base (unweighted) scores and the alternative policy scenarios and sensitivity tests, the concept of a standardised score has been adopted. The standardised score is basically the raw score (i.e. between +5 and -5) that is produced by the measure of performance, multiplied by the default weight of one divided by the number of policy areas. 5.3.15 If a project scores +5 against a particular policy objective and there are 15 different policy objectives, the standardised score would be 1/15 * 5 or 0.33. What this process does is merely factors all the raw scores by the same number to produce an identical distribution - i.e. the relative magnitude of the scores are unchanged, although the resultant number is different.

5-4

HALCROW FOX
5.3.16 The reason for undertaking this intermediate step is merely to allow subsequent differential weighting systems to be compared to the base (unweighted) result. 5.3.17 This weighting system allows the user of the spreadsheet to apply different levels of emphasis to certain policy areas without the need to go through a lengthy series of pairwise comparisons. For example, if a particular policy objective was deemed to be twice as important as all the others, applying a weight of two rather than one would produce a standardised score of 2/16 * 5 = 0.625, with all other objectives have a score of 1/16 * score. In other words, the denominator is always the sum of the individual weights. 5.3.18 If one policy area is of utmost importance, the decision-maker can place a higher weight on it (in the system adopted here, this would be +3.0) leaving all other policy areas at the default weight. In the example quoted above, a score of +5 would become 3/17 * 5 = 0.88 whereas all other scores would become 1/17 * score. In other words, the important policy becomes a large number and the less important policy areas become smaller numbers. 5.3.19 Whilst the distribution of the policy scores is the most informative type of exercise when using a tool such as the Policy Assessment Model, the summed weighted scores can, in themselves, provide interesting insights into how the relative importance of different policies can aid the identification of politically robust projects. This type of analysis is of particular benefit when the tool is being used to screen a large number of projects. 5.3.20 In general, high weights placed on negative individual scores tend to make negative overall scores more negative and positive ones less positive. The reverse is true of high weights on positive scores. Whilst this process at its simplest level is a trivial exercise, it becomes a powerful analysis tool when used to test individual policy weights within a complex set of policy objectives. It also highlights the benefits of adopting a simple standardised weighting system such as the one outline above, insofar as the summed scores are internally consistent. Summary 5.3.21 Clearly, the application of weights to policies is a subjective activity. This should not be regarded as a weakness of the Policy Assessment Model; rather, it represents a key strength. All appraisals carry an element of subjectivity, but in many cases this is not clear to the onlooker. 5.3.22 The benefit of the weighting system within the model is the transparency of the process. The method compels the user to be explicit about the size of weight, where it is applied and how it is interpreted. This makes for a totally open and honest approach to appraisal. 5.3.23 Whilst the weighting process as defined does not require an extensive set of pairwise comparisons, it does require some degree of consensus between different decision-makers as to the values to be placed on different weights. The Tenassess Task 36 methodology being developed in parallel to Task 32 has been undertaking

5-5

HALCROW FOX
Delphi analysis to understand policy conflicts within European decision-making. This style of analysis would seem to be one of the most promising ways in which consensus building can be achieved.

5.4 5.4.1

BACKGROUND DATA The final component of the Policy Assessment Model is the background data that has been collected as part of the model development phases. Such information is required if the model is to be dynamic in its structure: projects may not reach completion until several years in the future; policy is often expressed as reaching a certain level by a certain date; and historic background data is required in order to operationalise several of the measures of performance described in Chapter 3.

5.4.2

5.4.3

A wide variety of historic and forecast data has been collected from a variety of sources (although EUROSTAT data has been the single largest source). Where necessary, this data has been extrapolated (linearly in the absence of more robust estimates) to 2010. One of the recommendations that the Tenassess project would make to the Commission is that other Fourth Framework projects (e.g. SCENARIOS, EUNET and STREAM) could provide a better source of forecast data in the areas necessary to operationalise the Tenassess measures of performance.

5.4.4

5.5 5.5.1

MEASURES OF PERFORMANCE - AN EXAMPLE In order to fully understand the process involved in deriving a measure of performance that operationalises a policy objective, the following example from the IC5/IC25 Portuguese case study is presented. The accident rates on the Portuguese road network is currently one of the worst in Europe. A policy target of the Portuguese government is, therefore to reduce the accident rate per 106 veh x Km on the road network (national) - obtaining a decrease of 24% between 1995 and year 2000; Most of the accidents observed occur on the existing network, so new motorway schemes, with their grade separated junctions and segregated carriageways are expected to deliver reductions way in excess of the 24% overall target, as accident rates on existing roads will continue to be relatively high. Successful achievement (+5) has, therefore, been defined as a project achieving a decrease of the rate of accidents greater then twice the average decrease planned (2 x 24% = 48%) in the corridor that it serves. Failures has been defined as a decrease smaller (twice as small, at least) then the average.

5.5.2

5.5.3

5.5.4

5-6

HALCROW FOX
5.5.5 The measure of performance is, therefore, defined as: Formula: score = -5 (if x 0,5) = (10*x) -10 (if 1 x > 0,5) = (5*x) -5 (if 2 x > 1) = 5 (if x > 2)

where x =

decrease of the rate of accidents in the projects corridor average decrease planned in the network (1995 / 2000)

5.5.6

This formulation is basically saying that if x, which is the ratio of the project induced decrease in accident rates to the average target reduction in the accident rate, is equal to 2 or more (i.e. project rate = 2 * accident rate) then a score of +5 is recorded. If the ratio lies between 1 and 2, a positive score between 0 and 5 is recorded. If the ratio lies between 0.5 and 1 (i.e. below the average), a negative score between -5 and 0 is reported. If the ratio is less than 0.5 (i.e. the project reduces the accident rate by less than half the average target), a score of -5 is recorded. To operationalise this measure of performance, the minimum inputs are: rate of accidents (in Portugal) per type of road; average decrease planned for the network (1995/2000); distribution of the traffic per type of road, in the projects corridor (before and after the construction of the project).

5.5.7

5.5.8

The project impacts and background data identified are as follows: estimates of temporal changes in accident rates stemming from improved vehicle design - i.e. improved technologies on their own will improve safety towards the targets; traffic distribution (by road type) in study corridor before and after introduction of IC5/IC25 motorway, including induced traffic;

5.5.9

This information is then used to produce the following estimates: X1 - accidents in corridor before project (1995) = 4.21 / day; X2 - accidents in corridor after project (2000), redistribution only = 3.49 /day for Scenario A and 3.47 / day for Scenario B; X3 - accidents due to induced traffic (2000) = 0.07 / day for Scenario A and 0.08 for Scenario B.

5.5.10 These changes translate into the following accident reduction rates in the corridor over the period 1995-2000:

5-7

HALCROW FOX
Scenario A = 15.98% Scenario B = 16.35% Target reduction rate = 24%

5.5.11 The ratio of the actual forecast reduction to the planned reduction gives the scores for the project: -3.34 for Scenario A and -3.19 for Scenario B. 5.5.12 These results show that, whilst the IC5/IC25 does bring about significant reductions in accidents, the figures achieved do not perform well against the benchmark target - hence the negative scores.

5.6 5.6.1

APPLICATION OF WEIGHTINGS - AN EXAMPLE The following example uses results from the Decin-Praha-Breclav Rail Upgrade in the Czech Republic. It demonstrates the transition from raw scores to weighted scores and more aggregated results. The figures used are shown in Table 5.1. In each case the project is assessed from the perspectives of three key national states: the Czech Republic, Austria and Germany. The raw scores are those derived from measures of performance. In several cases they are the same for the three national states. The final row is the sum of these scores for which a weighting has been applied. In this case the sum has been divided by 60 = 12 policy areas multiplied by a maximum score of 5, so ensuring that the overall score is within the range -1 to +1. The raw scores indicate that the Czech Republic should not be in favour of the project, but the other two states should be. However the scores take no account of the states policy priorities. Some test weighting are shown, which seek to reflect policy priorities. All three countries give policy areas of 'accessibility and regional development' and 'harmonisation and common market' high priority. The weighted scores are the product of the raw score and the test weight, factored so that the test weights do not effect the absolute size, just the size relative to the unweighted score and scores for other policy areas. For example the 'raw scores' implicitly have weights of one for each policy area, and the sum of the weights is 12, because there are 12 policy areas. The sum of the weights for the Czech Republic are 13.5. Therefore the weighted scores of the Czech Republic are factored by 12/13.5, so that the raw and weighted scores are comparable. Therefore the weighted score for the Czech Republic for the Environment policy area is -4.4. This is the product of -3.3 and 1.5 and 12 and 1/13.5. In practice some other adjustments were made to try to account for policy priorities, including the incorporation of further policy areas and adjusting the measure of performance for 'accessibility and regional development', as discussed in sections 3.3.17 to 3.3.25.

5.6.2

5.6.3

5.6.4

5.6.5

5-8

HALCROW FOX

Table 5.1: PAM Example of Scores and Weightings - Czech Rail Upgrade Raw Score Austria
-1.9 0.1 0.9 -0.9 5 -5 5 3.3 0 -5 0 5 +0.11

Policy Area
Environment Fair & Efficient Pricing Improve Transport Accessibility & Regional Development Harmonisation & Common Market Traffic Avoidance Reduce price of transport Restrict road transport Improve Safety Deregulation, privatisation & liberalisation Financial acceptability Public acceptability Overall Score

Czech Republic
-3.3 0.19 5 -0.9 5 -5 -0.7 3.3 0 -5 -5 4 -0.04

Germany
-2.2 0.1 1.1 -0.8 5 -5 5 3.3 0 -5 0 5 +0.11

Czech Republic
1.5 0 0 3 3 0 1.5 0 0 1.5 1.5 1.5 N/A

Weights Austria
1.5 1.5 1.5 3 3 1.5 1.5 1.5 1.5 1.5 1.5 1.5 N/A

Germany
1.5 1.5 1.5 3 3 1.5 1.5 1.5 1.5 1.5 1.5 1.5 N/A

Czech Republic
-4.4 0 0 -2.3 13.3 0 -0.9 0 0 -6.7 -6.7 5.3 -0.04

Weighted Score Austria

-1.6 0.1 0.8 -1.5 8.6 -4.3 4.3 2.9 0 -4.3 0 4.3 +0.15

Germany
-1.8 0.1 0.9 -1.3 8.6 -4.3 4.3 2.9 0 -4.3 0 4.3 +0.15

5-9

HALCROW FOX

6.
6.1 6.1.1

Testing the Policy Assessment Model

INTRODUCTION Six case studies have been undertaken as part of the testing and refinement of the Policy Assessment Model. The projects are as follows: IC5 / IC25 motorway, Portugal; Igoumenitsa - Volos / Lamia motorway, Greece; TGV Nord High Speed Railway, France; Decin-Prague-Breclav Rail Corridor Upgrade, Czech Republic; Twente-Mittelland Kanal, Germany; and Amsterdam Ring Road, The Netherlands

6.1.2

The selection of these projects has been driven by several factors which are discussed in more detail in Internal Report D(I) - Methodology Case Studies, Tenassess Project Team, April 1997, SGVII: adequate modal coverage in the context of Trans-European Transport Networks; adequate geographic coverage to reflect projects in all areas of the Community; adequate demonstration of the functioning of the Policy Assessment Model; the ability to assess the impacts of projects at various stages of the planning process, ranging from early feasibility stage right through to expost project appraisal; a degree of compatibility and overlap with the case studies being undertaken as part of Task 36; adequate coverage of a wide range of identified policy issues; and coverage of Trans-European Transport Network scale projects that also address issues of pricing and regulation.

6-1

HALCROW FOX

6.2

THE IC5/IC25 TOLL MOTORWAY - GUIMARES / CHAVES BORDER, PORTUGAL The IC5/IC25 is classified as a complementary itinerary, linking Guimares to the Chaves frontier. This itinerary has two possible corridors: corridor A linking Guimares to Cabeceiras de Basto and then shifting to the Northeast, directly to Chaves, in an extension of 102 Kms; and corridor B linking Guimares to Cabeceiras de Basto and to Vila Pouca de Aguiar, in a transversal West / East axis, and then going up to Chaves, in an extension of 109 Kms.

6.2.1

6.2.2

6.2.3

The two corridors are common between Guimares and Cabeceiras de Basto, and between Chaves and the Spanish frontier. Between Cabeceiras de Basto and Chaves, corridor A crosses an undeveloped region, with no significant traffic demand at present, while corridor B follows the EN206 corridor to V. P. Aguiar, and then the EN2 corridor to Chaves. In this alternative, the IC25 is common to the IC5 between Cabeceiras de Basto and V. P. Aguiar, and common to the IP3 between V. P. Aguiar and Chaves (frontier), while corridor A will represent an independent (from IC5 and IP3) itinerary, thus implying in global terms an additional1 network extension of 46 Km. This itinerary is presently planned as tolled motorway, although the initial study (1995) has been done considering both tolled and untolled alternatives. As can be seen in Figure 6.1, the IC5 / IC25 will be included in a package of tolled motorways (in the North region) that will be soon concessioned to a private group for construction / operation.

6.2.4

6.2.5

As in this alternative the IC5 between Cab. Basto / V. P. Aguiar and the IP3 between V. P. Aguiar / Chaves will also be constructed;

6-2

HALCROW FOX

6-3

HALCROW FOX

6.2.6

The motorway will comprehend 10 interchanges (11 interchanges in corridor B) and includes a section of 6 Kms, between Guimares and Pa Vieira interchanges, which is classified as IP9 / IC5, and a section of 19 Kms between Chaves and the Spanish frontier, classified as IP3 / IC25. The global cost of the motorway is estimated as follows (1995 ECU prices): Tolled 506 x 103 ECU 535 x 103 ECU Non Tolled 4572 x 103 ECU 527 x 103 ECU

corridor A corridor B 6.2.7

The IC5 / IC25 is a set of two itineraries, with some common sections: IC5 P. Varzim (IC1) - Famalico - Guimares - Fafe - Cab. Basto - V. P. Aguiar - Mura - V. Flor (IP2)

IC25 corridor A IC24 - Felgueiras - (Fafe - Cab. Basto) - Chaves (IP3); corridor B IC24 - Felgueiras - (Fafe - Cab. Basto - V. P. Aguiar) - Chaves (IP3). Note: inside brackets are the common sections, IC5 / IC25; 6.2.8 While the IC5 can be considered as having functions appropriated to a complementary itinerary, the IC25 was not initially planned, and appeared basically to allow a good connection between the Porto metropolitan area and the Spanish frontier, in the direction of central Europe. In fact, the 1985 Highway Plan considered that this connection should be given by the IP4 (Porto - Bragana), but two major reasons (a posteriori) have made it necessary to find a better alternative: the Spanish Highway Plan privileges the Chaves frontier (Verin), with a direct link to a motorway network that represents the shortest route to Europe; the IP4 has a bottleneck near Porto, because of two unidirectional long tunnels, which makes it very difficult to increase its capacity in the future (having already a traffic demand that causes some capacity problems).

6.2.9

6.2.10 The IC25, as an alternative route to the IP4, and apart from the good connection between Porto and Europe, also has the advantage of diverting a significant part of the existent traffic demand in the IP4, thus enlarging its capacity reserve. It will also relieve the traffic demand on the A3 (Porto / Braga) which also suffers from capacity problems.

as non tolled motorway the IC5 / IC25 (corridor A) will have a reduced cross section, although with two lanes per direction, in the 46 Km link between Cab. Basto and Chaves;

6-4

HALCROW FOX
6.2.11 The motorway in question (IC5 / IC25, between Guimares and the Chaves frontier) was also a response to strong regional aspirations, in terms of the requirements of the Chaves3 region and its need to improve road links with Porto. 6.2.12 In both aspects (international connection and regional links to Porto), corridor A appeared to be the most favourable, both to the Portuguese National Highway Authority (JAE4) and to regional highway and local authorities (CCRN5 and Chaves Municipality), although with the need to add 46 kms of motorway to the network, as noted above. 6.2.13 However, corridor B showed itself as being the best solution in all technical aspects studied (traffic demand, economic viability and environmental impact) and now looks the most probable solution to be implemented. 6.2.14 This project, as all other transport infrastructural projects in Portugal, is seen as being a favourable option to allow economic development of the country, and its regions. In fact, Portugal has not yet completed the basic preliminary highway network (IPs and ICs), and there are few doubts that this is a necessary and beneficial step towards a successful and balanced development. Environmental doubts, and consequent conflicts of interest between different groups, have only appeared, so far, in very particular cases, and when these impacts were considered significant (as in the Tagus second crossing, or in the IP1 / IC4, in the Algarve), but even then, the question was of choosing a different alternative route, and not of giving up the project. 6.2.15 As already noted this motorway (IC5 / IC25) will be tolled, and concessioned to a private group. This concession will include other motorways in the North region, in order to make an attractive package for the private sector, as traffic demand in this motorway will not be sufficient to generate a good IRR on its own. Summary Findings

6.2.16 The analysis of the IC5/IC25 motorway raises some interesting issues regarding
the development of the Policy Assessment Model, notably that several policy areas included in the initial analysis are not relevant to the Portuguese situation and that some additional policy areas are required in order to fully explain the situation. To reflect the latter, two additional measures of performance have been constructed.

6.2.17 In particular, issues such as traffic avoidance and restriction of road based traffic
are not applicable to the Portuguese situation where development aims (with their associated traffic growth) are central to the policy debate. It is only in the urban centres of Lisboa and Porto where traffic congestion issues are deemed to be of any real concern.

3 4 5

a very important socio-economic centre in the North east; Junta Autnoma de Estradas; Comisso de Coordenao da Regio Norte;

6-5

HALCROW FOX
6.2.18 The developmental aspects of the project also tend to make it perform poorly against traditional project performance indicators, as the initial traffic volumes on the link are forecast to be low - implying poor financial and socio-economic viability. Whilst this is not seen as a critical problem for the project, the Portuguese government recognises the importance of creating a project that can be pushed forward by the private sector and, to this end, the IC5/IC25 will be bundled together with a series of other toll motorway links in northern Portugal in order to present a better financial outcome. 6.2.19 In order to adequately address the developmental benefits of the project, the Accessibility and Regional Development measure of performance developed in the project has been amended to take account of reductions in travel time between towns and cities within the projects sphere of influence. This addresses to an extent the shortcomings of a indicator based purely on the economic rate of return of a project - the low initial traffic volumes may not be a reliable indicator of the potential for future economic development that the project may convey. 6.2.20 Elsewhere in this report, the uncertainty about the magnitude of such impacts and that a causal linkage between investment in transport infrastructure and regional development does in fact exist exists, has been discussed. The approach adopted here (using project ERR and travel time savings) suggests that a link from transport infrastructure to development does exist. If further research does, however, suggest that this assumption is flawed, an alternative measure of performance, based upon a different set of project impact criteria may be required. 6.2.21 Many of the policy issues that are high on the agenda in more densely populated and developed regions of northern Europe are not applicable in less developed regions such as rural north-eastern Portugal. To address this problem, the weights attached to policy areas in the Policy Assessment Model can be adjusted accordingly to reflect the particular characteristics of such regions. 6.2.22 Additional policy areas may need to be developed (as has been the case here) in order to accurately reflect the policy situation in any given project corridor. It should, however, be borne in mind that the technical specification of a policy area measure of performance can have crucial bearing on the perceived performance of a project - here witnessed by the IC5/IC25 running in the wrong direction to fulfil the aims of some of the Portuguese specific policy areas. 6.2.23 A number of alternative policy scenarios were tested as part of the model testing phase - the base case assessment (with all policy areas afforded equal weighting) produces a moderate negative score of -1.56 is generated.

See section 5.3 for a discussion of how the weighted value is constructed.

6-6

HALCROW FOX
6.2.24 Figure 6.1a shows the unweighted base case results for the IC5 / IC25 motorway in terms of how it scores against national Portuguese policy objectives. The definitions of the policy areas are in the footnote at the bottom of this page7. Figure 6.1a: Scoring of IC5 / IC25 against national Portuguese policy objectives
National Policy scores
5 Score 0 -5 Policy

P14

P15

ACC

HAR

RPR

ENV

AVD

RST

DPL

6.2.25 As can be seen from these results, the project, on the whole, scores poorly except in the areas of accessibility, harmonisation and public acceptability. 6.2.26 Applying a zero weighting to those policy areas deemed to be of little relevance to transport planning decisions in Portugal (namely traffic avoidance, traffic restriction and improving non-road based transport) serves to reduce the overall negative score associated with the project. 6.2.27 Extending this analysis further and placing a greater emphasis (a weight of 3) on policy areas concerned with the general enhancement of mobility, produces a small positive score for the project when assessed against the policy objectives of the Portuguese national government. This type of manipulation of policy weights thus starts to build a picture of the policy decisions which push certain types of project forward. 6.2.28 Similarly, if the structure of policies at the European level is weighted more towards issues surrounding integration and regional development and away from policies concerned with the reduction of the negative impacts associated with transportation projects, the overall project score improves further.

6.3 6.3.1

THE IGOUMENITSA - VOLOS/LAMIA MOTORWAY, GREECE This new corridor is located at the regions of Thessaly and Sterea Hellas. More specifically, the corridor consists of two 2X2 lanes closed motorway branches, with a reversed Y-shape. Its single northern end is the Panagia interchange, located at the north-west border of Thessaly region, providing connection with the Egnatia motorway. The two branches ends are located in the vicinity of Lamia, providing connection of western and central Thessaly with PATHE towards Athens and southern Greece, and Larisa, providing connection of western and

ENV=environment, FEP=fair & efficient pricing, IMP=improve transport, ACC=accessibility & regional development, HAR=harmonisation & common market, AVD=traffic avoidance, RPR=reduce the price of transport, RST=restrict (road) transport, SAF=safety, DPL=deregulation, privatisation & liberalisation, FNA=financial acceptability, PBA=public acceptance and P14-16 are policies specific to Portugal

6-7

PBA

FNA

FEP

SAF

IMP

P16

HALCROW FOX
central Thessaly with PATHE towards the port of Volos and the rest of northern Greece. 6.3.2 Various alignment schemes have been investigated, the four most favourable of which are presented in the attached map, all of which have a common section between Panagia interchange (connection with Egnatia motorway) and Kalambaka. Four alternatives have been analysed in terms of reconnaissance alignment. These are highlighted in Figure 6.2. Alternative 1 Red Alignment 6.3.3 The route consists of a branch towards Lamia, running at a north-south direction, very close to the mountainous area of Pindos, west of the cities of Trikala and Karditsa, and a branch towards Volos, at an east-west direction, ending at the area of Girtoni north of Larissa, providing connection with Volos via the Girtoni Velestino section of PATHE. Alternative 2 Yellow Alignment 6.3.4 The route consists of a branch towards Lamia, running at a north-south direction, across flat terrain areas (farmlands), east of the cities of Trikala and Karditsa, and a branch towards Volos, at an east-west direction, starting at the area of Karditsa Palamas and ending at the area of Nikea south of Larissa, providing connection with Volos via the Nikea - Velestino section of PATHE.

6-8

HALCROW FOX

6-9

HALCROW FOX

Alternative 3 Green Alignment 6.3.5 The route consists of a branch towards Lamia in a north-south direction, running further to the east of the previous alternative across flat terrain (farmland), east of the cities of Trikala and Karditsa, and a branch towards Volos, at an east-west direction, starting at the area of Doxaras and ending at the area of Moschochori south of Larissa, providing connection with Volos via the Moschochori Velestino section of PATHE. Alternative 4 Blue Alignment 6.3.6 The route consists of a branch towards Lamia, as described in Alternative 1, and a branch towards Volos, at an east-west direction, from the area of Kallithiro that ends at the area of Velestino, on the Velestino - Volos existing road. Within the currently completed feasibility study, the first two alternative schemes have been selected for further investigation, in terms of detailed socio-economic feasibility and environmental impact assessment (A stage preliminary approval of alignment). This selection has been based on a preliminary concise analysis of their spatial characteristics and the existence along the route of schemes 3 and 4 of low level areas that become flooded in the winter, along with extensive irrigation networks crossed. These features imply the need to construct of significant number of structures and bridges, thus extending the level of disruption to human activities and implying higher total cost. It is schemes 1 and 2 which have been assessed as part of the present case study. Summary Findings 6.3.10 The Igoumenitsa - Volos/Lamia motorway project exhibits many of the characteristics that were evident in the IC5/IC25 study of the preceding section. It has proved necessary to construct an additional measure of performance to analysis the impact of the project upon potential archaeological sites - a policy objective which is specific to the Greek environment. 6.3.11 The low (by traditional European standards) social rate of return associated with the project is not a barrier to its progression given the substantial perceived regional development benefits that the project will bring. Indeed, an economic rate of return of 5% is considered to be satisfactory in Greece. 6.3.12 The importance of the further development of the strategic Greek road network is explicitly stated in the Strategic Plan for the Development of Transport Infrastructure - Greece 2010 policy document. In this report, road investment is seen as the key to the furtherance of regional development, accessibility and international connectivity aims.

6.3.7

6.3.8

6.3.9

6-10

HALCROW FOX
6.3.13 The base case result for the project produces a small positive score in aggregate for Alternative 2 (Alternative 1 produces a small negative score due to a less robust cost-benefit result). 6.3.14 The situation in Greece surrounding policy preference is similar in many ways to that of Portugal, with a heavy emphasis placed on the regional developmental aspects of projects and little or no weight given to the more restrictive areas of policy noted in the preceding section. 6.3.15 Increasing the weights attached to accessibility, user cost savings, regional development and safety issues and reducing the weight to zero on traffic avoidance and traffic restriction produces positive overall scores for both route alignments. 6.3.16 If a greater emphasis is placed upon what has been termed European objectives, a conflicting picture emerges. The project scores well against development and integration objectives, but poorly against environmental and efficient pricing objectives. This raises an important issue, insofar as it must be recognised that, under the vast majority of cases, all policy objectives are not of equal importance. Moreover, general policy is often tailored to very specific geographic circumstances. This is evident in the green priorities policy scenario that was analysed as part of the Igoumenitsa - Volos/Lamia motorway case study - in the main, environmental policy objectives are inconsistent with large scale investment in road infrastructure, but then environmental considerations are not the drivers behind such projects. 6.3.17 Figure 6.2a below shows how the base case results (scenario 1) for Greek national policy objectives compare against the green priorities (scenario 4) and green (Greek)8 priorities (scenario 5).

A subset of what has been termed green priorities elsewhere during the Case Study testing phase.

6-11

HALCROW FOX

Figure 6.2a: Comparison of three different policy scenarios of Greek motorway project
Weighted (Standardised) Score by scenario - National policy
1 0.5 Score
P14 P15 ACC HAR RPR ENV AVD RST DPL PBA FNA FEP SAF IMP P16

0 -0.5 -1

scenario 1 scenario 4 scenario 5

Policy

6.3.18 The standardised score is basically the unweighted score (between -5 and +5) multiplied by the standardised weight, as discussed in section 5.3. The analysis shows how more positive project performance can be achieved by manipulating the weighting of individual policy areas.

6.4 6.4.1

TGV NORD, FRANCE This section focuses on the TGV Nord High Speed Rail project in northern France, completed in 1993. It is unusual in terms of the Tenassess case studies insofar as the available data allows both an ex-post and ex-ante assessment of the project within the context of the Policy Assessment Model. Ex-ante and ex-post evaluations, comparing an evaluation of the chosen scenario, undertaken by SNCF in 1987, and the actual impacts of TGV Nord, obtained from appropriate authorities have been studied. Alignment alternative evaluations, comparing four of the different route options examined in an influential document known as the `Rudeau Report. All evaluations have been based on an important hypothesis adopted by SNCF and the `Rudeau reports of 1987. This hypothesis considers that the impact of TGV Nord should be examined against the scenario where the Channel Tunnel is operational with through services running between the UK and Northern Europe, employing existing, but upgraded infrastructure and new, but lower specification rolling stock. The TGV Nord links Paris (pop 8 million) with Lille (pop 1 million), Calais and the Channel Tunnel. The line was opened to passengers in 1993, at a cost of Ff. 18.5 billion, (approximately 3 billion ECU - 1993 values), funded by both National and Regional Governments. The rail link is the third 'Ligne Grande Vitesse' (LGV) to be constructed in France, after TGV Sud-Est (Paris-Lyon, serving the Alps and the Mediterranean ) and TGV Atlantique (Paris-Le Mans, serving Bordeaux and Nantes).

6.4.2

6.4.3

6.4.4

6-12

HALCROW FOX
National Context 6.4.5 The TGV Nord is 333 Kilometres in length, with seamless new infrastructure extending from the outskirts of Paris, through the centre of Lille, and through to Calais and the Channel Tunnel. Between Paris and Lille, a LGV spur extends from the mainline to connect the town of Arras and the larger towns of Lens and Bthune via the existing network. A large part of the new line, between Paris and Lille, parallels the existing A1 autoroute. Dedicated new TGV stations have been constructed at Gare Picard (a park-and-ride station on the A1 autoroute, mid-point between Paris and Lille, serving the region of Picardie), Euralille (a new station, also acting as a key interchange node for other European high-speed services and national TGV services) and Calais Frthun. Similar to the previous two TGV projects, the TGV Nord is a dedicated highspeed-railway, (i.e. only TGV services operate along its length) permitting a consistent operating speed of 300 kph and allowing a consequentially high service frequency. Significant time savings can be made compared to services provided by the conventional railway network, as shown in Table 6.1, below. Regional Connectivity 6.4.7 The TGV Nord not only benefits those towns served by a new TGV station, but also connects other towns in the Pas-de-Calais region, employing links between the LGV and the conventional network. In this manner, the towns of Lens, Bthune and Hazebrouck, are served by direct TGV services to Paris, connecting to the LGV south of Arras. Similarly, Douai, Valenciennes, Dunkerque and Tourcoing are served directly. Most of these towns have population catchments of between 200,000 and 400,000. However, some of these towns are served with only a few direct TGV services per day, therefore, benefits are arguably more psychological than economic and have been provided for complex political objectives. Furthermore, a new high-speed-rail 'Paris-bypass,' connects the TGV Nord to the TGV Sud-Est and TGV Atlantique, with new TGV stations at Disneyland Paris and Roissy-Charles de Gaulle Airport. This link permits though TGV services from Lille and beyond to Nantes, Bordeaux, Lyon and Marseille. Table 6.1: Link Paris - Lille Paris - Arras Paris - Calais Lille - Roissy Charles de Gaulle Airport * Lille - Lyon* * Via TGV 'Paris Bypass' Travel Time Savings, National Context Before TGV Nord 1h59 1h25 2h48 2h30 minimum, via centre of Paris 4h23 After TGV Nord 1h00 0h50 1h45 0h53 2h53

6.4.6

6.4.8

6-13

HALCROW FOX

European Context 6.4.9 The decision to build the TGV Nord was spawned largely on the back of the decision to build a fixed rail link between the UK and mainland Europe. However, the TGV Nord also provides a crucial first step towards creating what was historically known as 'PKAB' (Paris, Kln, Amsterdam, Bruxelles) and later 'PKABL'. In this respect the TGV Nord represents the first true TEN rail project which will significantly promote transit between member states.

6.4.10 Travel time savings between Paris and London and Paris and Bruxelles allow international services to compete effectively with the airlines. The section between Lille and the Channel Tunnel is also employed by London - Bruxelles Eurostar services. Furthermore, as has been seen by the effect Eurostar, fierce inter-modal competition, in the absence of a truly de-regulated airline environment, has helped to significantly reduce average fares. 6.4.11 Completion of the Belgian LGV between Lille and Bruxelles (to open 1998), and the British Channel Tunnel Rail Link (to open 2002) will reduce trans -state travel times even further. Dedicated high-speed-rail links between Bruxelles and Amsterdam and Kln are planned to be complete early in the next century. However, TGV services now operate between Paris and Amsterdam, via Bruxelles, employing the TGV Nord. Table 6.2, below demonstrates the international travel time savings between principal European Cities, comparing the base case without TGV Nord, the inauguration of TGV Nord, and the completion of the PKABL network. Table 6.2: Travel Time Savings, European Context Link Before TGV After TGV Nord (1992) Nord (1996) Paris - London Paris - Bruxelles Paris - Amsterdam Paris - Kln Bruxelles - London 5h12 2h25 5h23 5h00 4h59 3h00 1h58 4h46 5h00 3h15

After PKABL Completion (2005) 2h30 * 1h20 ** 2h50 3h05 2h30 / 2h00 ***

All are fastest times * 2002, after opening of British Channel Tunnel Rail Link ** 1998, after opening of Belgian LGV ***1998, 2h30 after opening of Belgian LGV, 2002, 2h00 after opening of British Channel Tunnel Rail Link 6.4.12 Significant `generalised time benefits also accrue from the provision of such high-speed-rail services, relative to the air alternative. The components of such benefits include relatively less onerous access, egress and `at-terminal-time components, improved quality of `in-vehicle-time, and a significant enhancement of total `air + rail capacity. Where high-speed-rail services are provided on routes with limited airline competition, airline fares are likely to fall.

6-14

HALCROW FOX
Decision Making Process 6.4.13 The decision making process, leading from the projects conception, the examination of alternative options, through to the determination of the chosen alignment involved significant conflicts and motives which are invaluable to the study of TGV Nord in this context. The project was subject to political interest within the French National government, other nearby European nations and, significantly, in the two northern French regions of Picardie and Pas-de-Calais. 6.4.14 During the process leading to the start of construction, several alignment alternatives were examined, leading to significant national and regional political interactions. Primarily, four route options, as shown in Figure 6.3, were identified: Route option `A: Western alignment, via Amiens, splitting south of Bthune, providing thereof, two separate alignments, on towards the Channel Tunnel the other towards Euralille (the central Lille station). Scenario `A: Alternative western alignment, via both Amiens and Euralille, without split alignments. Scenario `B: (chosen route option) Central TGV alignment, paralleling the A1 motorway corridor, via Euralille, and incorporating spur to Arras. Scenario `C: Eastern Alignment via St. Quentin and Euralille.

6.4.15 There were other alignments based on the above route options. Route options `B and `C were studied with alignments via Roissy Charles-de-Gaulle airport. Initially, route option `B was studied with three alternative approaches to Lille. The `Sud de Lille option, where Paris-London services split from the main TGV Nord North-South Axis at Arras, with a direct alignment towards Bruxelles, bypassing Lille to the East. Lille bound services would be obliged to use the traditional network, using the existing `Lille Flandres station. This option would form a `triangle of new infrastructure to the south-west of Lille. The `Plaine de Flandres option, without a split in infrastructure at Arras, but with the Bruxelles and London bound infrastructures splitting to the south of Lille. Again, Lille bound services would be obliged to use the traditional network, using the existing `Lille Flandres station. The `Plaine de Flandres (chosen) option, but with the new infrastructure passing through the centre of Lille, and TGV services using a new dedicated station at Lille.

6-15

HALCROW FOX

6-16

HALCROW FOX
The Influence of the City of Lille 6.4.16 The city of Lille sought to ensure that the TGV Nord alignment would pass directly through the centre of Lille, involving the construction of a new, dedicated TGV station in proximity to, in their opinion, an area of immense development potential. SNCF and the national government did not initially support the provision of a central station and TGV alignment through Lille. This was due to significant civil engineering constraints, imposing high costs onto the project. Furthermore, it was anticipated that a 10 minute delay would be imposed on ParisLondon traffic, significantly affecting revenues. 6.4.17 The decision to allow TGV Nord to pass through the centre of Lille, was therefore governed by negotiations between SNCF and the regional government,. It was agreed that all additional infrastructure costs above those required to link the Channel Tunnel and Bruxelles should be paid for by the regional government of Pas-de-Calais. A significant proportion of these costs were incurred by tunnelling through Lille, and the provision of Euralille station, rather than bypassing the centre of Lille and having TGVs terminating at the existing Lille Flandres station. The excess cost of approximately 1 Billion Francs was divided between the French state, the region and the city of Lille itself. The region was not liable for the loss of revenue incurred by the 10 minute increased journey time between Paris and London. Irrigation of TGV Nord Services in the Pas-de-Calais Region 6.4.18 In order to provide regional political consensus towards the expenditure of a direct alignment through the centre of Lille, it was proposed that through TGV Nord services would be provided to smaller towns such as Dunkerque, and Valenciennes. The region was also accountable for the provision of these largely non-economic services. 6.4.19 Irrigation of services serves to benefit the whole region, therefore, the region was amenable to contributing capital towards infrastructure through the centre of Lille. Although such irrigating services are not economic, they provided consensus to the whole region, enabling inter-regional harmony, supporting the Euralille project, which, it has been suggested, provides considerable economic benefit to the Lille conurbation. Without the direct TGV services to smaller towns in the region, Euralille may not have been supported within the region, creating a loss of economic potential to the city of Lille. Final Route Alignment Choice: Conflicts between the Pas-de-Calais and Picardie Regions

6-17

HALCROW FOX
6.4.20 Following negotiations with SNCF, it was decided that Lille would be provided with a TGV Nord alignment through the centre of the town. However, further studies (notably the Rudeau Report) centred on the choice between alignments `A, `A, B and C, with alignments A and C benefiting the region of Picardie. 6.4.21 No regional consensus was evident in the efforts Picardie undertook towards ensuring that the TGV Nord alignment traversed the region. Internal regional conflicts between the Picardie towns of St. Quentin (which would be served by the eastern alignment proposal `C) and Amiens (which would be served by the western alignment proposal `A) rendered such interregional harmony impossible. Combined with the aim of the central government to maximise the regional acceptability of the project, such conflicts damaged the prospects of the Picardie region towards ensuring either of alignments `A or `C. Trade-offs are therefore manifest on both regional and national political levels between regional development objectives and regional equality. 6.4.22 Following the `Rudeau Report, the decision was made to construct TGV Nord along route option `B. This was identified as the most economic option (aside from regional politics), involving a direct alignment to Lille, with infrastructure provided parallel to the A1 motorway corridor. Concessions and Compensation 6.4.23 As a small token to the region of Picardie, a regional TGV Nord station was provided mid-point between Paris and Lille along the A1 motorway. This was provided as a Park-and-Ride station, serving towns without a TGV services. Amiens was assured that further studies would be undertaken to explore the future possibility of a second TGV Nord alignment via the city. 6.4.24 Following the commencement of TGV Nord services, many national and international services which had previously passed through towns in the region of Picardie would no longer be provided. In order to minimise any further conflict between the Picardie region and the central Government, it was agreed that the original service frequency along the traditional network would be maintained.

6-18

HALCROW FOX
Summary Findings 6.4.25 The TGV Nord case study provides an interesting test of the Policy Assessment Model in terms of appraising an ex-ante and ex-post set of project data, together with different ex-ante route alignment forecasts. 6.4.26 Additional French specific policy objectives regarding promotion of French technical ingenuity and regional acceptability have been included in order to ensure adequate coverage of French transport policy issues. 6.4.27 The ex-ante and ex-post assessments of the model produce very different scores within the Policy Assessment Model, with the ex-ante results in all respects bar one being more optimistic. These differences are largely due to the overly optimistic demand forecasts contained within the 1987 SNCF /Rudeau reports, taken in conjunction with the underestimation of projects costs by some 400 MECU. 6.4.28 Changing policy preferences over the past 10 years have, to an extent, diluted the overall score associated with the TGV Nord project. Whilst the project still scores positively, the magnitude of the scores against individual areas of policy is, in many instances, diminished. One of the most important areas where performance is not as envisaged is in the bottom-line financial indicators, where the project has served to worsen the overall financial position of SNCF. 6.4.29 The ex-post and ex-ante model results are presented in Figure 6.3a below. This clearly indicates the issues discussed above.

6.4.30 The results derived from the Policy Assessment Model support the actual choice of route for the project, if the caveat of the (non-strategic and hence nonmodelled) environmental concerns associated with route C (and the associated cost increases to circumnavigate these issues) is taken into consideration. 6.4.31 The main mode choice decision in the corridor associated with TGV (and the Channel Tunnel) is between air and rail. Many of the measures of performance within the Policy Assessment Model are, however, associated with road to public transport policy issues. This is a direct reflection of the policy preferences expressed in the policy documents that have been reviewed as part of previous Tenassess Deliverables. It should not be forgotten that pressures on air transport capacity within Europe are becoming an ever more important issue on the political agenda. A modified version of the Policy Assessment Model may be required in order to fully address the implications of these issues.

Figure 6.3a: Comparison of unweighted ex-post and ex-ante scores for the TGV Nord project

6-19

HALCROW FOX

Ex-Ante (1987) Raw Score - National policy

5 3 Score 1 -1 -3 -5 Policy
ENV FEP IMP ACC HAR AVD RPR RST SAF DPL FNA PBA P14 P15 P16

Ex-Post (1996) Raw Score - National policy

5 3 Score 1 -1 -3 -5 Policy
ENV FEP IMP ACC HAR AVD RPR RST SAF DPL FNA PBA P14 P15 P16

6.5 6.5.1

DECIN-PRAHA-BRECLAV RAIL UPGRADE, CZECH REPUBLIC The project involves up-grading 453 km of existing infrastructure in order to allow faster running speeds, employing new, tilting rolling stock. The line forms the central segment of the proposed Central European Axis from Berlin (pop. 3.4 million) to Wien (pop. 1.5 million) via Dresden (pop. 0.5 million ) and Praha (pop. 1.2 million). (see Figure 6.4) The Berlin - Praha - Wien corridor has been given top priority within an accord, signed in June 1995, between the Czech, German and Austrian Transport Ministers. The project is likely to be completed by the end of the year 2000 and is seen as a key to greater integration between western and central Europe.

6.5.2

6-20

HALCROW FOX

6-21

HALCROW FOX
Financial Structuring 6.5.3 In 1994, the total investment cost of upgrading the central segment within the Czech Republic was estimated to be Kc30.7 bn (ECU 880m), of which Kc6.0 bn will be invested in new rolling stock. Funding has been secured by central Government, and a variety of external sources, including the European Investment bank (ECU 125m) and the European Bank for Reconstruction and Development (ECU 42.6m). Export-Import Bank and four other Japanese banks are providing a ECU100m loan to the Czech national railway company, 'Ceske Drahy' (CD). The EU's PHARE programme was to provide ECU 350m between 1995 and 2000 for the upgrading of Czech Infrastructure, including the Berlin - Praha - Wien main line. However, there are indications which suggest that cost overruns in the Czech Republic are likely to be in the order of 30% to 60%. Upgrading Approach 6.5.4 The nature of the provision of infrastructure for this project is quite distinct from that of the TGV Nord. The decision was made to upgrade the existing infrastructure for high speed running (160 kph), rather than to construct a new, high specification alignment. The onerous terrain of the corridor, as well as the economic circumstances of the Czech Republic with respect to a Western European country such as France, makes the construction of a brand new railway prohibitively too expensive. Moreover, the upgrading of the traditional network allows benefits to freight traffic, which can also use the new infrastructure. In order to obtain service speeds of 160 kph, it has been decided to replace much of the existing fixed equipment. The project will entail track relaying, curve realignments, new signalling, telecommunications and train control, and the electrification of the remaining 18 % of the corridor for which trains are currently diesel-powered. However, crucial for the attainment of higher speeds, particularly where onerous terrain restricts expensive re-alignment, will be the provision of tilting passenger rolling stock. National Context 6.5.6 The Czech Republic has a total population of 10.3 million. The Decin - Praha Breclav corridor serves a total potential market of approximately 2 million inhabitants. As well as the capital, Praha, the region of Northern Bohemia is traversed, serving towns with a total population of 190,000. To the east of Praha, the corridor passes through Mid and East Bohemia, providing access to towns such as Pardubice, Hradec Kralove and Ceske Budejovice, with a total market catchment of 200,000 inhabitants. However, after Praha, the town of most significant size, served by the route is Brno.

6.5.5

6-22

HALCROW FOX
6.5.7 The town of Brno is located in the region of South Moravia, in proximity to the Austrian border, and has a population of 370,000. With the provision of upgraded infrastructure, and new tilting trains, the journey time between Praha and Brno will be reduced from the current 3h06 to 2h14. Service frequencies will also be improved significantly. Although the corridor upgrading will primarily provide benefits to passenger services, freight services will also benefit. The Czech government is keen to minimise the mode shift of freight traffic towards the roads as the country transforms towards a western market economy. Moreover, the rail-freight business in the Czech Republic is potentially highly profitable, due to the geographic location of the country with respect to central Europe. European Context 6.5.9 For the complete Berlin - Praha - Wien corridor, the section between Berlin and Dresden is being re-built to provide a service speed of 200 km/h. The aim is to cut Berlin - Praha journey times from 4h40 to 3h00 and Praha - Wien from 4h50 to 3h30. Further Projects 6.5.10 The corridor under study is the first in a proposed incremental upgrading of the Czech railway network, and is referred to as 'Corridor 1.' Three other corridors have been identified for upgrading to 160 km/h, with the provision of tilting trains. The accord signed in June 1995, between the German, Austrian and Czech Transport Ministers also paved the way for the upgrading of the Praha - Nrnberg and Praha - Linz corridors. These form part of corridors 3 and 4 respectively. Corridor 3, forming an east - west axis is also proposed to link the main Praha Nrnberg corridor with Mnchen, and to extend corridor 1 eastwards towards Ostrava and the Polish border. Corridor 2 is proposed to link Breclav, near the Austrian border, on the alignment of corridor 1, with corridor 3 at Prerov, thus improving access to the north-eastern town of Ostrava and enhancing cross-border traffic between Austria and Poland. Summary Findings 6.5.11 This case study illustrates a number of important points concerning uses and merits of the Policy Assessment Model. These include: the importance of applying weights to key, prioritised policy areas; the value of considering policies across state frontiers and the effect of this on the result of an appraisal; potential conflicts between western European policies set in the context of congested networks, concerning traffic management, and the policies of developing/transitional economies, which are growth focused; the ability of the Policy Assessment Model to focus on key policy issues.

6.5.8

6-23

HALCROW FOX

6.5.12 In general, the unweighted analysis does not produce favourable (ie: supportive) results. This largely corresponds with the poor results of the economic and financial analyses. Indeed, the low/negative unweighted scores within the Policy Assessment Model are driven by traditional traffic volume and cost/benefit data. 6.5.13 The application of weights to two, key, Czech policy areas (Accessibility and Regional Development and Harmonisation and the Common Market) results in a net positive score and illustrates how the project could be justified in a highly focused, development-led policy context. Such focus is not uncommon in developing and transitional economies, where planning decisions are heavily influenced by a desire for growth and integration with other trading partners. By contrast, more mature, stable economies tend to be increasingly concerned by the intricate trade-offs between development, transport efficiency and the negative impacts of transport investment. 6.5.14 Given that the Corridor 1 upgrading is being implemented, it appears that the implied weights identified by this case study are indeed of relevance in the Czech Republic. An opportunity exists to turn this methodology around and conclude that project appraisals in the Czech Republic should, as a matter of course, apply weights to the policy areas: Accessibility and Regional Development and Harmonisation and the Common Market. That is, use of this method (i.e. the Policy Assessment Model) in an ex-post context enables standard weights to be determined for particular projects in particular states. 6.5.15 The value of considering policies from other states is also illustrated by the case study. Consideration of the policy benefits to Germany and Austria significantly increases the value of the project. This raises a number of issues, including whether Germany and Austria should contribute financially to the project in return for these benefits. In the absence of such a contribution, they could be accused of exploiting a free rider status. 6.5.16 Finally, perhaps the main conclusion to draw from the Czech case study is the value of the Policy Assessment Model in focusing a projects appraisal on diverse, international policy issues, and away from simple economic and financial evaluations. Moreover, it demonstrates that different recommendations can emerge when such a broader perspective is adopted.

6.6 6.6.1

TWENTE-MITTELLAND KANAL, GERMANY - THE NETHERLANDS The planned waterway should - as an additional part of the European waterwaynetwork - contribute to the elimination of certain bottlenecks as well as to the reduction of the overall transportation times. It will connect the Twente-Kanal in the Netherlands and the German Dortmund-Ems-Kanal a few kilometres north of the junction of the Mittelland-Kanal. Besides the existing connections via Rhine and Ems it would be a further connection between the two countries national waterway-networks.

6-24

HALCROW FOX
6.6.2 The project represents a 90 kilometre short cut on the existing route, which takes a large U-shaped route to the south, although the existing route is almost totally upgraded to current optimum design characteristics (in terms of depth, etc.). Traffic originating from Rotterdam currently has to travel some distance upstream on the Rhine at 8 kph (vs. 16 kph downstream). This implies that there are significantly larger time savings to be obtained from the Dutch side as opposed to traffic originating the German side. The canal will have little regional development impact - the main project impacts will be time and operating cost savings to long distance barge traffic. The existing route has several locks to negotiate (an average delay of one hour is considered reasonable per lock) whereas the TMK will only have two, although the connecting point at the German end has been specified as being to the north of Rheine which entails the negotiation of two further locks before entering the Mittelland Kanal itself. The project, although only at initial feasibility stage, has been ranked as very high priority by the Dutch as it substantially improves the hinterland connections of Rotterdam. The main downstream traffic on the Rhine is coal and iron ore whereas the upstream traffic (which would benefit most from this project) is container traffic - there is a heavy Dutch influence in this industry. PLANCO have undertaken a study of lock traffic on the German/Austrian border and found that 40% of the traffic was of Dutch origin - this gives some indication of the size of the industry concerned. The TMK connection also gives strategic access to Berlin and other eastern areas of Germany. The apparent one-sided nature of project promotion is not so surprising. The poor CBA score of under 1.0 arrived at by the German planning authorities is a direct reflection of the lower strategic benefits, lower time savings and lower operating cost savings for traffic originating in Germany - given the ability to achieve speeds of 16 kph for nearly half of the distance on the existing U-shaped route which is 90 kms longer in terms of distance. Thus, overall, the relatively high benefits to Dutch traffic, when combined with the low benefits to German traffic do not sum to a level which offsets the costs of the project to a high enough degree. The previous sections also highlights an interesting point regarding the spatial distribution of project benefits. Whilst economic assessments have been undertaken from both a Dutch and a German perspective, the results are reported for the project as a whole, which hides many potentially interesting features. The perceived benefits to traffic originating in The Netherlands are wholly dependent upon the whole project being in place. The costs of the project are to be divided by the number of kilometres in each country, rather than in relation to who is getting the bulk of project benefits. This is a critical issue to cross-border projects that is often a barrier to progress.

6.6.3

6.6.4

6.6.5

6.6.6

6.6.7

6-25

HALCROW FOX
6.6.8 An important factor in favour of the project is its location in a border region. The map of the waterway networks in this part of Europe clearly shows a missing section that crosses the Dutch-German border. This, as is the case in many instances of missing links, is a consequence of past national policies focusing on the development and completion of the national network as a priority over cross border links, which would serve as better routes when looking at the geography in a regional context. The benefits to the region, as opposed to the economies of The Netherlands and Germany respectively, are, therefore, likely to be significantly greater. It is possible that an East-West link such as this could make inroads into the hinterland of the port of Hamburg. In terms of container traffic to the Central European hinterland, the current ranking is Rotterdam, Hamburg, Bremerhaven then Antwerp. The construction of the Twente-Mittelland will tend to increase the advantages of Rotterdam over Hamburg and Bremerhaven. A study conducted by PLANCO for the city of Bremen analysing the impact of the (E-W) Betuwe freight railway link did show some cost advantage switching to Rotterdam, so some similar shifting in response to the TMK link might be expected. Summary Findings 6.6.10 The TMK is the only inland waterway project to have been tested with the Policy Assessment Model and is also the only project for which the main market is freight transport. It also represents a missing link project between The Netherlands and Germany - an alternative route does exist but this is not a direct link. 6.6.11 The policy areas against which the case study projects have been assessed have been constructed around actual policy as reported in the Tenassess Country Reports9. It would appear that most of the identified policies tend to focus on passenger travel and, where they exist, of rail and road freight10 rather than inland waterway traffic. 6.6.12 Inland waterway traffic is an important freight transport mode in a number of countries but does not, it seems, feature strongly in policy documents emanating from national level sources. 6.6.13 It is also possible the case that the Policy Assessment Model in its present form does present an accurate representation of the impact that waterway projects have on policy objectives - namely a small core area centred around the socio-economic viability of the project. This suggests that the scope of the policy assessment should be geared towards the project under consideration. If the impacts only impinge upon a small number of policies, a cut down version of the Policy

6.6.9

10

Tenassess Internal Report D(K) - Country Reports, Tenassess Project Team, January 1997, DGVII. See for example the recent Commission communications on Trans European Rail Freight Freeways [COM(97) 242 final] and Intermodality and Intermodal Freight Transport in the European Union [COM(97) 243 final].

6-26

HALCROW FOX
Assessment Model may well be a more appropriate tool, in order to focus the decision makers attention on the relevant issues. 6.6.14 The project scores do not vary much between the German and Dutch policy areas. This may seem an unusual conclusion given the discussions regarding the distribution of project impacts (i.e. focused around time and operating cost savings to vessels travelling in a west to easterly direction). It is, however, a consequence of the manner in which project impacts are reported - for the project as a whole, in line with traditional transportation evaluation practices. We would argue that a disaggregation of impacts between the two countries would highlight a very uneven distribution, especially as the costs of the project, if constructed, are to be divided upon the basis of geographic distance rather than distribution of project benefits.

6.7 6.7.1

THE AMSTERDAM RING ROAD, THE NETHERLANDS The Ring Road (A10) as a whole is 32 kilometres in length and interchanges with 15 city roads and 5 national highways. The road is characterised by numerous bridges and flyovers, two road tunnels and two train tunnels. The whole Ring Road is dual carriageway, with the Coentunnel being dual 2 lane. The remainder is at least dual three lane, with four lane sections in parts. The most expensive section to be constructed was the Zeeburger shore connection, Zeeburger tunnel and Zeeburger bridge, which connect the eastern and northern sections of the Ring Road. Total cost of this section was 250m Guilders. The road to the west of the Coentunnel was already in use during the 1960s. The remaining sections to be constructed are illustrated in the diagram below and have been subdivided into six distinct sections. Between Europaboulevard (7) and Johan Blookerweg (6), a large number of flyovers and bridges occur over the Amstel River. Between 1987 and 1989, a second bridge over the Amstel was constructed, with a railway bridge sandwiched between the two road carriageways. The section between Johan Blookerweg (6) and Middenweg (5) was built between 1987 and 1993, with traffic being handled at 3 levels in some places. A moveable bridge over the Weespertrekvaart forms part of the Gooise Knoop - a three level traffic interchange with two national highways and a water crossing. The period 1988 - 1989 saw the completion of the Ring Road between Middenweg (5) and the Amsterdam-Rijn Channel (4). Between the Amsterdam-Rijn Channel (4) and Zuiderzeeweg (3) are located the Zeeburgertunnel and Zeeburgerbridge, completed in 1989 and 1990 respectively. All hazardous traffic is transported over the bridge, whichever direction it is moving in. This was the final link to be completed in September 1990.

6.7.2

6.7.3

6.7.4

6.7.5

6.7.6

6.7.7

6.7.8

6-27

HALCROW FOX

6-28

HALCROW FOX

Alkmaar A8 Coentunnel

2 3

IJ

4 7 Den Haag A4 A9 Utrecht A2 Gooise knoop 6 A9 5 Zeeburger connection Amersfoort A1

Figure 6.5: Schematic Diagram of Amsterdam Ring Road

6.7.9 The section between Noorhollands kanaal (2) and the A8 to Alkmaar (1) was opened to traffic in 1988.

6.7.10 During the construction phases of the Ring Road, several problems had to be surmounted. Financial setbacks were a continuing problems - in the 1970s several unfavourable choices were made concerning the Ring Road due to pressure on central government finances. In 1981 the collection of road tax was extremely low, which put the completion of the highway in doubt for some time. 6.7.11 Technical problems were encountered due to the clay, sand and peat based earth in the Amsterdam region. When the western part of the Ring Road opened in 1975, it passed close to many houses. Once completed, traffic levels were such as to create noise problems for nearby residents, necessitating expensive sound proofing to adjacent properties and subsequent alignment changes to later sections of the road. 6.7.12 When finally completed in 1991, Amsterdam was freed from some of its worst bottlenecks and traffic jams - the A9 (widened as an interim measure prior to completion of the Ring Road) was freed of congestion, traffic on the N10 (city

6-29

HALCROW FOX
road) heading north was much reduced and to the west of the Coentunnel, the traffic jams became shorter and rat running through surrounding districts was reduced. 6.7.13 It has, however, become clear that the Ring Road, so long in the planning and construction process, has been overtaken by time and congestion now regularly occurs on certain sections during peak hours. The construction of a half outer circle is already taking place, aimed at providing more capacity for through traffic on the national highways. The opening date for this Westrand road is planned for 1998, although a second Coentunnel will be required. Summary Findings 6.7.14 The assessment of the Amsterdam Ring Road with the Policy Assessment Model highlights some features that appear to be typical of road building projects, particularly in the more industrial countries of northern Europe. 6.7.15 The assessment is typified by a series of large negative and large positive scores (see Figure 6.5 below) which indicate that there is significant policy conflict if all policies that are of relevance in a particular country are considered in the analysis. Figure 6.5a: Unweighted Dutch national scores for the Amsterdam Ring Road
Raw Score - Dutch national policy
5 Score 0 -5 Policy

P14

P15

ACC

HAR

RPR

ENV

AVD

RST

DPL

6.7.16 This apparent conflict may also suggest that a road such as the Amsterdam Ring Road was constructed in order to satisfy no more than a very narrow range of policy objectives, such as time savings and general congestion relief and the associated boost that this will convey for the Amsterdam region in terms of accessibility enhancement. 6.7.17 The alternative explanation for this policy conflict is that Dutch transport policy has changed since the construction of the Amsterdam Ring Road. Whilst completed in 1990, the project planning dates back many years prior to this date, to a time when the political agenda regarding transportation was very different. 6.7.18 As has been noted previously, a Ring Road only functions as intended once fully complete. To not complete the final links would not be a logical course of action, even if the policy agenda has changed markedly.

6-30

PBA

FNA

FEP

SAF

IMP

P16

HALCROW FOX
6.7.19 Whether or not projects such as this would be constructed from scratch today, under the new, more sustainable transport policy agendas that exist throughout much of Europe, is open to question. 6.7.20 If a Dutch priorities policy scenario is constructed, based upon statements of current Dutch thinking in the field of transport, the overall (ex-post) project score worsens markedly. This result conceivably indicates that a project planned many years ago is no longer in line with Dutch transport policy - once the project was started, there is little point not completing it. On the other hand, it may simply indicate, as noted above, that the driving forces behind the progression of the project may have been based on a very narrow set of criteria (e.g. congestion relief), rather than on any wider policy agenda. 6.7.21 If this line of thinking is pursued, a policy scenario that reflects these narrower aims has been constructed, the project total score does improve markedly, although not sufficiently to produce an overall positive score for the project when assessed present day policy objectives. 6.7.22 Table 6.3 outlines the overall (standardised) scores for each case study, together with a series of alternative scenario results. Table 6.3: Summary Case Study Results Case Study Base Case

Green EU objectives policies IC5 / IC25 Motorway -1.49 -1.67 -1.18 Igoumenitsa Motorway -0.28 -1.25 +0.51 TGV Nord (Ex-post) +1.01 TGV Nord (Ex-ante) +2.23 Czech Rail Upgrade +0.18 Amsterdam Ring Road -0.54 -2.13 -1.10 Note: no overall scores were calculated for the Twente Canal and, the two rail projects did not have the scenario analysis undertaken.

6-31

HALCROW FOX

6.8 6.8.1

REFINEMENTS TO THE POLICY ASSESSMENT MODEL The central purpose of the Tenassess Task 32 case studies was to test the functioning of the Policy Assessment Model developed in earlier phases of the work. The brief for the case studies (outlined in Internal Report D(I) Methodology Case Studies, reference as before) was to ensure that the Policy Assessment Model was capable of handling a wide range of transportation issues that are pertinent at the European level, was capable of ensuring adequate treatment of any relevant mode and that the approach was able to deal with all policy issues that may impact upon any given project. The measures of performance developed have, on the whole, performed well during the case study testing phase. A number of modifications have, however, been required. The initial specification of the Policy Assessment Model envisaged an all encompassing set of 9 core measures of performance into which all aspects of transport-related policy would fit. As the Case Studies progressed, it became increasingly clear that individual countries often required additional specific measures of performance in order to reflect peculiar circumstances. In all cases, this does not invalidate the core set of performance measures, although not all are relevant in all countries. The measure of performance that operationalises the accessibility and regional development policy area has been subdivided into two components. It now encompasses a reduction in average travel time component as well as an economic rate of return criterion. This change stems from the need to accurately represent the development potential that projects in the less developed parts of the Community convey. An ERR based measure of performance tends to underestimate the potential impacts as the project may traverse a relatively unpopulated region with the aim of fostering development here. Additionally, values of time (time savings are the main project benefit of many projects) in these areas may be low. Linked to the preceding point, the relative weight applied to the two components of the accessibility indicator is related to the location of the project. In densely populated areas of northern Europe, the ERR component is more relevant (for TGV Nord, all emphasis was placed upon this element), whereas the travel time element is more relevant in other areas (a 50/50 split was adopted for the Portuguese case study). A number of benchmarks exist within the Policy Assessment Model. One conclusion to be derived from the case studies is that these benchmarks are not common across all European countries. It needs to be borne in mind that these benchmarks may not be accepted norms everywhere throughout Europe and at every level in the decision-making hierarchy. The model can address such issues through the initial checking of background data and assumptions which the user is instructed to carry out. Alternatively, as a further extension of the models

6.8.2

6.8.3

6.8.4

6.8.5

6.8.6

6-32

HALCROW FOX
structure, the model itself could be made interactive in order to allow the user to choose the benchmark and sensitivity test the implications of their choice. 6.8.7 As noted above, the initial construction of the measures of performance largely focused on road / public transport issues, in line with findings of much of the policy objective analysis. An amendment to the mathematical functioning of a number of the measures was required for the TGV Nord assessment in order to adequately handle issues of rail versus air modal competition.

6-33

HALCROW FOX

7.
7.1 7.1.1

Discussion and Summary Findings

INTRODUCTION The discussion points in this chapter have been structured around four core areas: the principles arising from the use of a type of Goals Achievement Matrix for policy appraisal; issues that have arisen in relation to transport policies in different countries; the performance of the Policy Assessment Model itself; and issues surrounding the transport project impact data requirements without which the Policy Assessment Model cannot function.

7.2 7.2.1

PRINCIPLES OF A GOALS ACHIEVEMENT MATRIX The initial conclusion from this part of the Tenassess project is that a GAM is an appropriate policy assessment tool. The PAM put forward here appears to fulfil a variety of criteria, both in terms of the Commissions needs and the internal consistency of the method. These criteria are summarised below. Flexibility

7.2.2

The PAM is amenable to a wide variety of policies, at a range of hierarchical levels (local, regional, national, etc). Whilst initially designed with nine policy areas, these have been expanded to 14 and more can be added with relative ease. Versatility

7.2.3

The versatility of the PAM is illustrated by its ability to assess policies from a variety of sectors: transport, the environment, regional planning, finance, etc. It has been possible to create measures of performance across a broad range of sectors, which, whilst possible not finalised, at least serve to demonstrate the viability of the method.

7-1

HALCROW FOX

Transparency 7.2.4 Transparency is, perhaps, the key attribute of the method. Whilst acknowledging the risk that any method that allows users to weight impacts is open to abuse, the PAM compels all weighting to be made explicit and hence open to audit and justification. The PAM also records and displays all inputs and output scores, hence allowing a clear audit trail through all internal procedures. Interpretation 7.2.6 It has been stressed on several occasions that the PAM is designed to be used as an analytical tool and a means of generating understanding about the relationship between policies and projects. On the basis of such understanding, the user can identify the direct impacts of major transport projects. It must be stressed again that it is not intended as a means of producing a single answer or a single numeric summary of a projects impacts. The output from the PAM is intended to be a series of scores, for different scenarios and weights, from which a general view of the impact upon policies, and the robustness of a project, can be formed. Input to Comprehensive Assessments 7.2.7 The PAM is designed to be complementary to other forms of assessment (economic, financial, etc). It is not intended to be all embracing or to identify all impacts, in all sectors, representing all value systems. In this sense, it supports existing assessment methods and provides a more complete analysis of major project impacts. Having discussed these broad principles, more detailed findings are outlined below.

7.2.5

7.2.8

7.3

POLICY RELATED ISSUES The Ongoing Evolution of the Method

7.3.1

The initial specification of the Policy Assessment Model grouped policies into 13 broad families. The specification of these policy areas was based upon an analysis of the transport related policies of Austria and Germany, which represented the most complete Country Reports1 available at the time of the models development.

Reported upon in Tenassess Deliverable D(1) - see earlier reference.

7-2

HALCROW FOX
7.3.2 In several of the case studies undertaken it has proved necessary to add additional, country-specific policy areas to certain national policy hierarchies. For example, a specific French regional accessibility criteria was deemed necessary in order to reflect a very explicit French policy goal. Similarly, additional policy areas were required for the Portuguese case study to reflect some specific Portuguese developmental objectives. This point illustrates the methodological nature of this project. It is not claimed that the Policy Assessment Model as described in this Final Report is the final version of the appraisal tool. Considerable evolutionary development is still required. Nevertheless, the research work conducted within the Task 32 area does illustrate the potential value of this approach. In other instances, some of the original policy areas proved irrelevant to the policy debate - for example, traffic avoidance and traffic restraint policies do not feature at all in the policy debate that currently exists in Greece or Portugal. As such, this reflects a difference in policy objectives between northern and southern/eastern European countries, but it also reflects a perceived difference between objectives in the large northern European countries when compared to the smaller ones. In the more densely populated regions of northern Europe, issues surrounding the negative side effects of road transport tend to dominate the policy debate. In southern, eastern and other peripheral regions, economic growth, with the associated increases in (largely road) traffic, is the most important issue on the political agenda. The observed difference between large and small northern European countries is that infrastructure investment project are still seen as being an important solution to transport problems in those countries where geographic space does not constrain its construction. This conclusion supports the findings of earlier sections of the project which examined national transport policies throughout Europe in far more detail than has been done here. The Czech case study provides another interesting angle on this policy issues debate, insofar as the project is being promoted on grounds of economic growth, but also to improve the acceptability of the Czech Republic as a new member of the European Community. The core / periphery policy divide identified in the preceding sections has several implications for individual areas of policy. The most transparent of these is the treatment of the environmental externalities associated with all forms of transport. The relevance of these issues to the policy debate is a direct function of the level of development of the basic transportation network. Where this minimum level of infrastructure is lacking, environmental concerns (at the strategic level) tend to be well down the policy hierarchy. In the more highly developed region and countries of the European Union, where the base network is well developed, the negative aspects of transport are far higher up the political agenda. The Weighting System

7.3.3

7.3.4

7.3.5

7.3.6

7.3.7

7.3.8

7-3

HALCROW FOX

7.3.9

These disparities in the relevance of different policy issues leads to the question of whether one should consider the impacts of a project on all the policy areas identified within the Policy Assessment Model, or just against a subset of policies.

7.3.10 These differences can be reflected through the weighting system that is built into the model, insofar as policy areas that are of no importance can be given a weight of zero and those central to the decision making process in a particular environment can be given higher weights. As part of the most of the case studies, a series of policy scenarios have been constructed in order to analyse the impact of the adoption of different bundles of policies, with different levels of importance attached to individual policies. This allows the decision-maker to reflect those issues that are relevant in their own particular environment. Policy Conflicts 7.3.11 If the policies of the European Commission are taken in their entirety, a number of potential policy conflicts present themselves. The recognition of several broad thrusts to Community level policy does, however, explain these perceived conflicts. For example, policies aimed at economic growth and social cohesion are, in the main, focused in peripheral regions, whereas the more restrictive polices of say, fair and efficient pricing and sustainable mobility are aimed largely at the core of the Union. In this, Commission policy is very much in step with the picture that emerges from an assessment of national or regional policies. 7.3.12 The bulk of the transport related policies identified during the production of the country reports tended to focus on the interaction of road and public transport, particularly the desire in many countries to achieve a more sustainable balance between road and public transport modes. 7.3.13 In the sphere of Trans European Networks, high speed rails main competitor is, in many cases, air transport rather than the private road transport sector. The Policy Assessment Model has been modified to examine these rail / air issues in the case of the TGV Nord case study. In terms of policy conclusions, these issues do not appear to be well articulated in the policy documents of many countries. Infrastructure, Pricing and Regulation 7.3.14 Whilst the six case studies undertaken are all transport infrastructure investment projects, pricing and regulatory initiatives can be assessed in an identical manner, insofar as the Policy Assessment Model operates by scoring the impacts of any project against policy objectives. Additionally, many pricing and regulatory issues are already operationalised within the Policy Assessment Model in terms of the measures of performance that operationalise policy objectives. The case studies in themselves bring out some pricing issues such as prices in the Czech Republic generally being too low to justify the capital costs of high speed rail and Portuguese values of time being too low allow high toll charges that are required to make the projects financially viable.

7-4

HALCROW FOX
7.3.15 As part of the initial policy structuring process, a clear distinction between laws and policies has been observed. It is an important task to make this distinction, insofar as the Tenassess project is concerned with the latter rather than the former. The dividing line that has been adopted for the purposes of the development of the Tenassess Policy Assessment Model is the recognition that laws are a mandatory requirement that all proposed transport projects must satisfy. In other words, there is no room for trading project impacts against the legal framework - the laws of the European Community and the Member States essentially form the boundaries within which the Policy Assessment Model must operate. This is not to suggest that conflicts between different laws do not exist but rather stipulates that it is not the purpose of a policy research project to address these issues. 7.3.16 Policies, unlike legal requirements, represent a desired end state in a given area of society. In an ideal world, all policies would be achieved in their entirety but there is no legal compulsion to achieve them. It is this distinction that defines the subject matter of the Tenassess project. 7.3.17 It should also be noted that the Task 36 component of the Tenassess study is examining the impact of policies on CTP.

7.4 7.4.1

PERFORMANCE OF THE POLICY ASSESSMENT MODEL The Policy Assessment Model clearly highlights conflict within the policy hierarchy. The standard deviation of the individual policy area scores, together with the graphical presentation of the results, gives a clear indication of the degree to which projects are in line with policy objectives or in conflict with them. In terms of policy scores, road projects tend to have a high standard deviation and rail projects a low one. In general, rail projects tend to score positively across the entire range of policy areas whereas road projects tend to score very highly in some areas and poorly in others. This finding would tend to support the hypothesis that the current policy thrust in many European countries is towards rail and away from road projects. Whilst most of the case study results have been described in terms of overall scores, these numbers should not be interpreted as giving a definitive answer on project performance. It is only through extensive use of the method on a large number of different projects (similar projects in the same country or across many projects in many countries) that the total scores start to become meaningful in their own right. With a limited number of studies available for comparison, the total score should be interpreted as an indication of the most likely outcome, rather than the definitive answer. The measures of performance defined within the Policy Assessment Model, and subsequent amendments thereof, perform their function with a good degree of reliability. Several suggestions have been received from other partners within the Tenassess consortium for alternative measures of performance for certain areas of policy (e.g. a Cohesion Index based analysis of the regional development

7.4.2

7.4.3

7.4.4

7-5

HALCROW FOX
benefits of projects, in which the change in the ranking of regions on the European GDP ladder drives the score). Whilst the potential of such measures is recognised, one of the guiding principles in the design of the model has been the need to reconcile the robustness of measures of performance and the availability of the required project impact data to the correct degree of quality and accuracy. In the example mentioned above, reliable estimates of the GDP impact of projects is a controversial area of impact measurement and none of the projects analysed during the Case Study phase reported this information as a matter or course. As such, we believe that the performance measures that exist within the model represent a reasonable balance between these two opposing requirements. 7.4.5 The ex-post / ex-ante project analysis for TGV Nord highlights how the model can show how changing policy objectives or optimistic project performance figures can produce a very different result. Following from the previous point, the ex-ante / ex-post analysis of TGV Nord has highlighted that the model results essentially confirm the reasons for the projects progression through the planning process to construction. The model also performs well in predicting the preferred route option. The Czech case study highlights that the incorporation of the policy objectives of other countries whom are affected by the project can have a deciding impact upon the project score. One of the most important issues here would appear to be: if the project benefits are felt abroad, can any of the project costs be recouped from the beneficiary state? One point that should be stressed regarding the Policy Assessment Model is that the tool is not intended as a replacement for a full, detailed project assessment. Many of the project impacts that feed into the model are derived from more detailed aspects of a project assessment such as the social cost-benefit analysis and the environmental impact assessment. In a sense the Policy Assessment Model can be viewed as a high-level screening tool for projects - many projects can be analysed and sensitivity tested against a range of different policy scenarios in order to test the robustness of projects to a series of future policy environments. Through extensive sensitivity testing, the decision maker can analyse how sensitive the overall results are to variations in project impacts, different weighting of policy areas and even different specifications of the policy objectives in terms of amended measures of performance.

7.4.6

7.4.7

7.4.8

7.4.9

7.4.10 As noted in the preceding chapter, the benchmarks adopted for several of the measures of performance have required amendment during the Case Study testing phase for several different circumstances. As with all aspects of the model, it is up to the user to check that the benchmark values are applicable to their particular environment. Changes are possible, as long as the reasons behind such amendments are understand, along with the implications of doing so.

7-6

HALCROW FOX
7.4.11 In the context of the case studies, the weights attached to policy areas are specified as an input to the testing process, based upon expert judgement as to the relative importance of different areas. It is however possible to use the Policy Assessment Model to derive the weights through an iterative process of testing a large number of different projects in a country and then adjusting the weights until the model produces the correct result in terms of project prioritisation. The Delphi process can be viewed as the type of exercise through which a deeper understanding of the relevant policy weights can be gained.

7-7

HALCROW FOX

7.5 7.5.1

POLICY ASSESSMENT MODEL DATA REQUIREMENTS The assessment of a mere six case studies has necessitated the collation of a large amount of project specific and background data. The latter is required in order to operationalise many of the policy measures of performance which are, by their very nature, of a temporal nature. The quality of background data2 varies enormously across the European Union and very little forecast data exists. Given that many policy objectives specify the attainment of a certain level or value by a given date in the future, this forecast data is of great importance to the assessment process. It is hoped that other projects within the Fourth Framework Research programme will be able to advise on a wider range of robust forecasting data for the areas required by the model. The Policy Assessment Model also specifies that it should receive data in a certain format. The conclusion from the testing work done to date is that an assessment of this type should start with a detailed modelling exercise in order to produce the requisite material in the correct format. Post-processing of existing project impact data and estimation of missing components is not a trivial task. This conclusion has implications for the scope and depth of project appraisal undertaken in different countries, as reported in other areas of European research3. The definition of the project corridor and competing projects is crucial to the adequate analysis of the project. The Twente-Mittelland project highlights these issues through its competitor project the Betuwe rail line. Analysis of these conflicts, interactions and policy preferences for each scheme is crucial to an adequate understanding of the decision making process. As the Twente case study has show (this conclusion would seem particularly applicable to cross-border projects), the distribution of impacts is of central importance to the understanding of the policy objectives that lie behind a project and its chances of success.

7.5.2

7.5.3

7.5.4

7.5.5

EUROSTAT sources have provided the bulk of the background data material used within the Policy Assessment Model. See for example, the APAS report commissioned by DGVII on Methodologies for Impact Assessment.

7-8

HALCROW FOX

8.
8.1 8.1.1

Further Research Areas

POTENTIAL RESEARCH AREAS Many of the measures of performance contained within the Policy Assessment Model, rely on historical and forecast trends in a variety of areas, such as accident rates, GDP and traffic growth and modal split evolution. The Tenassess research team have attempted to source relevant forecast data, but in many cases such forecasts do not exist at the level of all Member States. In these instances, linear extrapolations of past trends have been constructed. Whilst it is acknowledged that such an approach is less than ideal, with the time and resources available to the team, no other viable solution existed. It is our belief that other projects still ongoing within the Fourth Framework Programme (e.g. SCENARIOS, POSSUM, STREAMS and EUNET) will ultimately be able to provide the robust macro-economic and transport-related data which is required to accurately forecasts the requisite variable into the future. Whilst during the Case Study testing phase of the project we have attempted to demonstrate these procedures, the understanding of the model would be greatly enhanced by a far more thorough testing of a range of projects and project variants. Such a level of analysis, which could be achieved through say a review of the projects financed under the Cohesion Fund, would provide a much more detailed understanding of the process involved in project selection and which areas of policy or impact are driving project implementation decisions. Following from the above point, the derivation of country specific policy weights, as discussed in the preceding chapter, could also be a by-product of such an analysis. This procedure essentially implies using the Policy Assessment Model in an inverted manner - one aims to construct a set of policy weights for each Member State so that the model correctly predicts the project selection decision i.e. whether to proceed or not to proceed and if so, with which project variant. The project impact data normally available from a typical European project assessment is often not in the requisite format to adequately handle issues related to achievement of policy objectives. The Twente-Mittelland Canal provides a very good example of this point, insofar as the benefits of the projects are very unevenly distributed - which explains to an extent the reasons behind the strong Dutch support for the project and the perceived indifference from a German perspective. Running both sets of project impacts (i.e. under German and Dutch appraisal rules) through the Policy Assessment Model produces a not dissimilar overall project result. This seemingly counter intuitive result is explained by both sets of project impact data being presented in aggregate form - this effectively ensures that the distribution of project benefits is not visible within the appraisal. Funding of the project, if it was to proceed, had been agreed along strict geographic lines

8.1.2

8.1.3

8.1.4

8.1.5

8.1.6

8-1

HALCROW FOX
(i.e. the Dutch paying for the stretch of canal on Dutch soil and vice versa for Germany), with little regard for the project benefit distribution. Logic would suggest that if the benefits to the Dutch are higher than for the Germans then the Dutch should bear a disproportionate share of the project costs. 8.1.7 The Policy Assessment Model is, therefore, only a good as the project impacts that are fed into it. Given the importance of impact distribution to policy assessment at different political and spatial levels, attention should be paid to the identification of suitable impact modelling tools which provide this type of presentation.

8.2 8.2.1

LINKS TO OTHER PARTS OF TENASSESS Other parts of Tenassess involve a detailed examination of the decision-making process. A Delphi panel is being formed, whilst case studies are being analysed using the Barrier Model. Such activities could be directed to assessing the weights that should be applied by various decision-making bodies within the PAM. That is, they could identify, for different bodies, the normative, or implied, degrees of importance perceived for each policy area within the PAM. This would provide an initial set of weights for a base-case application of the PAM. Perhaps of greatest interest is the potential for applying the PAM in association with the Barrier Model, not just as an assessment tool, but as a means of exploring the technical, social and political implications of major transport projects for different levels of the policy hierarchy. In theory, this could be extended to other models, beyond the Tenassess project, thus forming a system of models to support policy makers and planners in the development of a wide range of infrastructure, pricing and regulatory initiatives within the European transport sector.

8.2.2

8-2

REFERENCES Bruck et al (1996) "Evaluation of Alternative Transport Proposals". American Institute of Planners, Vol 23 (USA). Journal of the

Commission of the European Communities (1993) "The Future Development of the Common Transport Policy - A global approach to the construction of a Community framework for sustainable mobility". Bulletin of the European Communities, Supplement, 3/93. Danish Transport Council (1995) "Issues of Accountability - Lessons and Recommendations Regarding Appraisal of a Fixed Link Across Fehmarn Belt". Report no.95.03 (Denmark). European Commission (1995) "Towards Fair and Efficient Pricing in Transport " COM(95)691 Final. Green Paper. European Commission (1997) "Trans European Rail Freight Freeways" COM(97)242. European Commission (1997) "Intermodality and Intermodal Freight Transport in the European Union" COM(97)243.

Hill, M (1968) "A Goals Achievement Matrix for Evaluating Alternative Plans" Journal of the American Institute of Planners, Vol. 34, pp 19 - 29 (USA).

IWW (Universitat Karlsruh Institue fuer Wirtschaftspolitik und Wirtschaftsforschung, Germany) and INFRAS AG (Switzerland) (1995) "The External Effects of Transport". Jones, I., Zamini, H. and Reehal, R. "Financing models for new transport infrastructure" Transport Research APAS Strategic Research VII - 20. European Commission Directorate General Transport. Office for Official Publications of the European Communities (Luxembourg).

Litchfield, N (1964) "Cost Benefit Analysis in Plan Evaluation". Town Planning (UK).

REFERENCES CONT./

Marcial Echenique & Partners, LT-Consultants, Trasporti e Territorio SRL, and Marcial Echenique y Compania SA (1996) "Methodologies for transport impact assessment" Transport Research APAS Strategic Research VII - 21. European Commission Directorate General Transport. Office for Official Publications of the European Communities (Luxembourg). Ng, Y.K. (1983) "Welfare Economics: Introduction and Development of Basic Concepts". Macmillan (UK) Pearce, D.W. and Nash, C.A. (1981) "The Social Appraisal of Projects - A Text in Cost Benefit Analysis" Macmillan (UK). Standing Advisory Committee on Trunk Road Assessment (SACTRA) (1994) "Trunk Roads and the Generation of Traffic". HMSO (UK).