THE PBMR RISK MANAGEMENT SYSTEM - A CASE STUDY

Dr. Greg Ker-Fox

1

Summary The risk management system implemented on the PBMR project is presented. The system comprises a method of establishing the context, defining the risk, its attributes and subjectively estimating significance. The risk is then impacted against the project cost and duration. Significant risks require a mitigation plan, linked to the project plan. These risks are reviewed periodically to indicate mitigation progress and changing risk status. Finally the information is compiled in a monthly project report. The structured risk, captured in the risk register, is combined with the unstructured risk in the development of risk-based project plans. Monte Carlo simulation analysis provides insight into the drivers of risk and reliability, as well as motivating confidence levels for project contingency and indicating the likelihood of achieving cardinal milestones. Keywords: project risk management, risk management system, risk analysis 1. Introduction

Corporate governance reports [7] have developed specific requirements with regards to a demonstrable system of risk management. As a consequence, companies are required to understand the need for defining and implementing formal risk policy, structured risk management procedures, and to have a common risk management system applied throughout the organization. PBMR (Pebble Bed Modular Reactor) is a high temperature gas cooled reactor currently under development in South Africa. The current demonstration phase of the programmes development, in addition to facing the normal challenges for capital projects, also contains a large first of a kind engineering component. These project characteristics, together with the requirements from investor organisations, have necessitated the development and implementation of the PBMR Risk Management System (RMS). 2. Objectives of the PBMR Risk Management System

The PBMR RMS is implemented to support the development of risk-based project plans and the management of the PBMR projects, including: The identification of risk and opportunity, within the project scope. Development, implementation and monitoring control strategies to mitigate risk. Systems for communicating risk information within the organisation. A risk-based project summary schedule and cost estimate. Analysis of schedule and cost risk to determine the probability of project success as well as motivating management contingencies. These risk profiles are presented to potential investors and serve to support the business case model. 3. Corporate Governance

The King Committee on Corporate Governance in March 2002 Published the King Report on Corporate Governance for South Africa (referred to as King II) [7], which includes four new

Dr. Ker-Fox is a Risk Manager with Murray and Roberts Engineering Solutions, currently under secondment to PBMR as the Risk Manager: Projects.

International Cost Engineering Council's 4th World Congress, Cape Town, April 2004

1 of 7

chapters on risk management. King II clearly assigns responsibility for the process of risk management, and the need for a risk management system to the board; The total process of risk management, which includes a related system of internal controls, is the responsibility of the board. The board has the responsibility to ensure that the company has implemented an effective ongoing process to identify risk, measure its potential impact against a broad set of assumptions, and then activate what is necessary to proactively manage these risks. To ensure that the project risk management system would be concurrent with a risk management system developed for addressing higher level organisational and strategic issues, the PBMR risk management plan was developed from two main references; The Project Management Body of Knowledge [5], which provides a concise description of the process, indicating interfaces with other project management processes. The Australia New Zeeland Risk Management Standard [2], a generic risk management standard which can be tailored to any risk management application, including corporate and project risk management. 4. Overview of the PBMR Risk Management Process

An overview of the risk management process is given in Figure 1 on the following page. The process is described briefly below. Summary schedules and cost estimates are generated at an appropriate level of resolution. The estimated financial requirements, developed to correspond with the summary schedule activities, are described in terms of capital and time dependent costs. Minimum and maximum ranges for the capital costs, time variant costs and activity durations describe uncertainties and unstructured risk. Following the contextual definition of the risk system, purpose and scope of the RMS, risks are identified during workshops and a risk owner is nominated. Their underlying cause, effect, assumptions, decisions, and other attributes are defined and the likelihood of occurrence and impact given occurrence subjectively measured. The monetary or duration effects of the risks are impacted against the costed summary schedule. A risk analysis is conducted utilising Monte Carlo simulation. The results are evaluated, indicating the main risk drivers and critical sensitivities. This information is applied to establish the required management contingency to achieve the desired level of reliability. The sensitive schedule activities and cost items are communicated to the project team, and attention focused on reducing the associated uncertainty and risk impact. Risk assessment reports are compiled and distributed to the relevant stakeholders. Risk mitigation plans are developed and implemented by the risk owners. The effectiveness of the control strategies, status of the identified risks and any new risks are monitored and used to update the risk register. Risk mitigation reports are distributed monthly. The monitoring and control is an iterative process, performed throughout the project or until the risk has been closed out or realised.

International Cost Engineering Council's 4th World Congress, Cape Town, April 2004

2 of 7

Project Management Team

Work Pack Definition

Context Definition

Risk Workshops

Summary Schedule

Summary Cost

Risk Identification

Risk Impact Model

Definition and Measurement

Risk Owner

Contingency and Sensitivity Adjustments

Analysis & Evaluation

Risk Manager
Report Communication

Mitigation Plan Development and Implementation

Risk Owner

Monitor Control Strategies and Risk Status

Risk Manager

Figure 1: PBMR Risk Management Process 5. The risk management system

A software application was developed to support the risk management process described above. The system is divided into Context definition, Risk Identification, Risk Impact, Mitigation Planning, Review and Communication. These sub-categories are discussed briefly below:

International Cost Engineering Council's 4th World Congress, Cape Town, April 2004

3 of 7

5.1

Context definition

Context definition establishes the dimensions against which the risks will be quantified, assigning numerical values to levels of consequence as well as setting thresholds for qualifying risk status and response. The Low, Medium or High risk status is derived from the highest of the dimensions expected values (the product of likelihood and impact). The status indicates the response requirements for mitigation planning and review frequency. As an example, low risks are registered and reviewed quarterly, with no requirement for a mitigation plan. High risks however need a scheduled and resourced mitigation plan, with monthly reviews. 5.2 Risk Definition

Following the identification of risk, either as part of formal workshops or on an ad hoc basis, the risk is registered. This includes assigning a unique reference number, risk name and short description, the underlying cause, and effect given occurrence as well as recording associated assumptions and key decisions. The risk is placed within the project work breakdown structure and assigned a risk owner, responsible for risk mitigation and reporting. The period during which the risk is active as well as a resolution deadline are provided by the risk owner. Risk attributes, like project phase, risk category and performance areas affected are assigned to the risk. The risk is then subjectively quantified, by indicating the likelihood of occurrence and impact given occurrence in the dimensions defined in the context definition. The PBMR project dimensions are project cost, duration and design stability. The risk is assigned a status, based on the highest expected value, measured against predefined thresholds, with corresponding response and review frequency. 5.3 Risk Impact

Risk Impacting attempts to more objectively quantify the consequence of the risk to the project. This is done by identifying one or more risk carrying activities in the schedule which would be affected by the realization of the risk. Each activity is then assigned a minimum, expected and maximum possible impact on the duration and cost. This information is utilised in analysing the risk-based project plans discussed under section 6. 5.4 Mitigation Planning

Risks with a status requiring the development of mitigation plans must complete this section. It involves a description of the necessary mitigation strategies, together with responsible persons, link to the schedule and work management system and whether the actions require approval or can be absorbed into the project baseline. 5.5 Review

Review is a key element in the risk management process and one which can easily be overlooked. The review process requires the risk owner to report progress against each mitigation strategy, as well as indicating any change in the risk status, either due to updated likelihood or impact. 5.6 Communication

Monthly progress reports are generated which describe the contents of the risk register, including number of active, closed out, realised, low, medium and high risks. Graphs can also be compiled from the WBS and attributes assigned, indicating risk trends in the projects. A Cumulative Risk with Mitigation profile is produced from the expected values and resolution deadlines for all the

International Cost Engineering Council's 4th World Congress, Cape Town, April 2004

4 of 7

risks. This provides a baseline against which the progress with mitigation and changing risk status can be measured. This baseline is reviewed periodically, to reflect new risks. An example of this risk profile is given in Figure 2

Baseline Reviewed Baseline Mitigation Progress

Cumulative Impact (EV)

Time
Figure 2 Cumulative risk profile indicating baseline and actual progress with risk mitigation 6. Quantitative risk analysis

Risk analysis provides a rational means of processing the information captured in the summary schedule, cost estimate and risk register. Insight can be developed into drivers of risk and reliability and the relative importance of the different sources of uncertainty. This is often difficult to achieve intuitively, particularly highly complex projects. Capturing risk and uncertainty transforms the project plan, which is initially deterministic, to a probabilistic one. In the initial plan, costs and durations are described as single point estimates. The risk-based plan however reflects the probabilistic nature of the project plan, where the duration and cost of every activity is described by a probability distribution. Furthermore, both the occurrence and the consequence of structured risk are modelled probabilistically. The effect of increased activity duration (where there is an associated time related cost) will be reflected in the cost distribution. 6.1 Risk-based project plan

A summary schedule is constructed at a manageable meaningful level of detail. Leads and lags represent logic equivalent to the detail. Costs are allocated to these activities, indicating that proportion which is time based (rate per unit time). Unstructured risk describes the uncertainty associated with the cost and duration estimates (see Section 6.2). Structured risk is defined in the risk register (see Section 6.3) and assigned to the relevant risk carrying activities. The costed schedule with the corresponding unstructured risk together with the impacted structured risk produces the risk-based project plan.

International Cost Engineering Council's 4th World Congress, Cape Town, April 2004

5 of 7

6.2

Unstructured Risk

Unstructured risk describes the uncertainty associated with the estimated cost and activity durations in the project plan. It was initially found that simply asking the work owners, planners and estimators to provide upper and lower ranges for their estimates did not produce a credible input to the analysis process. This was largely due to the qualification of the estimate for a given scope. To remedy this, the estimating conditions for cost were categorised as follows: Maturity of the design on which the estimate was based (concept, basic or detailed) Estimating method employed (provisional, calculated, informal quote and binding quote) The estimators view of the estimate value (conservative, reasonable or aggressive) Familiarity with the technology (novel or known technology) These inputs are used to qualify ranges for the estimates, based on the suggested ranges for Class 1 to 4 estimates [1]. This method produced results which were more in line with the perceptions of the senior project managers. In the absence of a better process to estimate schedule uncertainty, the project engineers provide upper and lower limits for the activity durations. A similar method to the one described above for cost could be developed for estimating schedule uncertainty. 6.3 Structured Risk

Identified risk captured in the risk register is referred to as structured risk. This is captured in the plan by creating activities to represent the risk with a corresponding probability of occurrence. These risks are then linked to the risk carrying activities, with the relevant duration or cost impact. This is done by a series of IF statements, conditioned on the occurrence of the relevant risk. A manageable summary schedule facilitates the process of impacting the structured risk. 6.4 Monte Carlo Simulation

Monte Carlo simulation is applied to analyse the likelihood that a project will be successful, for a given budget and the probability of achieving the cardinal milestones (see Figure 3). A further useful output from Monte Carlo Simulation is the proportion of the simulations that a particular activity spent on the critical path, and the sensitivity of the milestones to the underlying activities (see Figure 4). A number of software packages are available to conduct Monte Carlo simulations of costed schedules, which include; Predict! Risk Analyser [6], @Risk for MS Projects [3] and Pertmaster Project Risk [4].
Distribution for Fuel load/Finish
0.16
X <=19/03/2008 5% X <=01/04/2008 95%

Regression Sensitivity for Fuel load/Finish

RU Support systems functio.../Duration (Dist.18) 0.667

0.14

Mean=25/03/2008

Fuel load/Duration (Dist.32)

0.482

0.12

Load Graphite/Duration (Di.../Duration (Dist.4)

0.47

0.1

RU Maintainability verific.../Duration (Dist.20)

0.312

0.08

PLICS Dry & Vacuum/Duratio.../Duration (Dist.12)

0.256

0.06

Software V&V/Duration (Dis.../Duration (Dist.22)

0.145

0.04

Helium Leak Test/Duration .../Duration (Dist.16)

0.118

0.02

Install Indexer/Duration (.../Duration (Dist.6) -1 -0.8 -0.6 -0.4 -0.2 0

0.11 0.2 0.4 0.6 0.8 1

0 3/17/08

3/24/08

3/31/08

4/7/08

Std b Coefficients

Fig. 3 Probability distribution for a cardinal milestone

Fig 4. Sensitivity of milestones to activity variability

International Cost Engineering Council's 4th World Congress, Cape Town, April 2004

6 of 7

7.

Challenges to implementing a formal risk management system

The implementation of a risk management system is not without its challenges. As a relatively new field in project management, there are very few robust integrated systems to support the collection, processing and distribution of risk information. The formal identification, mitigation and monitoring work can be perceived to be non-productive, particularly if the risk owners are already addressing the issues. There can be a conflict of interest, if disclosure reflects badly on the risk owner. The author experienced a poor perception of quantitative risk analysis, which was largely due to perceived mathematical complexity and the quality of the input information. 8. Conclusions

Risk management systems can be successfully implemented, not only in terms of compliance with governance requirements, but also to promote project performance. It provides a structured means of focussing management attention on the key issues that can affect performance. It also provides a formal and auditable system for capturing and controlling risk, which in turn develops investor confidence in terms of project delivery. 9. Acknowledgements

The author wishes to thank Dr. Dave Wimpey for his contribution to the PBMR Risk Management Plan and mentorship, and Mr. John Maddalena for his support in developing a more credible method for estimating unstructured cost risk. 10. References

[1] Cost Engineering Association of Southern Africa (2003), Classes of Estimates, www.ceasa.org.za [2] Joint Technical Committee OB/7 (1999), Australia / New Zeeland Standard : Risk Management AS/NZS 4360:1999, Standards Association of Australia, Strathford [3] Palisade Corporation (2003), @Risk for MS Projects, http://www.palisade.com/ [4] Pertmaster Limited(2003), Pertmaster Project Risk, http://www.pertmaster.com/ [5] Project Management Institute (2000), A Guide to the Project Management Body of Knowledge (PMBOK Guide), Pennsylvania, pp 127-144 [6] Risk Decisions Management Solutions (2002), Predict! Risk Analyser, http://www.riskdecisions.com/ [7] The Institute of Directors in Southern Africa (2002), King Report on Corporate Governance for South Africa, Johannesburg, pp 73-81

International Cost Engineering Council's 4th World Congress, Cape Town, April 2004

7 of 7