Risk In Systems Acquisition

CSE300 - Advanced Software Engineering

University of Sunderland 2005

Material supplied By Helen M Edwards Professor of Software Engineering School of Computing and Technology

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

1
Session 2

Overview
What is Risk Risks from the Literature Dealing with Risk (Risk Management) The Generic Risk Management Lifecycle Overview of A Risk Method
Riskit: for systems development

Prior Reading - "Software Development Risk: Opportunity, Not Problem" by Roger Van Scoy
University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

2
Session 2

Risk Definitions
The possibility of incurring misfortune or loss. "Chance of a loss or other event on which a claim may be filed. The possibility of suffering loss, injury, disadvantage or destruction.

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

3
Session 2

Technical Definitions
The possibility of suffering loss: for risk to be understandable it must be expressed clearly. Such a statement must include
A description of the current conditions that may lead to the loss A description of the loss.
From Higuera, R.P., Gluch, D.P., Dorofee, A.J., Murphy, R.L., Walker, J.A., & Williams, R.C. An Introduction to Team Risk Management. (CMU/SEI-94-SR-01) Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pa., May 1994.
University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

4
Session 2

Technical Definitions
The possibility of loss, the loss itself or any characteristic, object or action that is associated with that possibility.
A loss is defined as an outcome that falls short of what was expected.
Expectations are held/defined by stakeholders.
From Kontio,J. The Riskit Method for Software Risk Management, version 1.00, CS-TR-3782, 1997. Computer Science Technical Reports. University of Maryland. College Park, MD.

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

5
Session 2

Risk
Risk is inextricably linked with the future.
what is: the likelihood? cost/benefit of the outcomes of what-if scenarios? Dealing with: uncertainty and decision-making:
based on existing data of past experiences, expectations and (often) prejudices.

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

6
Session 2

Risk Definition
An Aphorism: You cant control what you cant measure. (Attributed to Lord Kelvin) So: in any project each risk needs to be defined and expressed in terms of:
Likelihood of occurrence (probability) Consequence of occurrence (impact).

These measures need to be monitored and evaluated

appropriate data needs to be collected at the appropriate time.

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

7
Session 2

Risk and Time.

Risk = Probability x Impact. 1 But 0.9 0.8 both the probability and 0.7 0.6 the impact of an event 0.5 0.4 happening can change 0.3 over time. 0.2
0.1 0 1st Qtr 2nd Qtr 3rd Qtr 4th Qtr risk probability impact

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

8
Session 2

Typical Risks
The next few slides show typical risks that have been identified and published in the literature:
These are often the basis for formal repository lists. These are all from a systems development perspective

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

9
Session 2

Boehms Top 10 risk items

People
Personnel shortfalls

External risks
Shortfalls in externally produced components Shortfalls in externally performed tasks

Resources
Unrealistic schedules and budgets

Requirements
Developing wrong functions Developing wrong UI Gold plating Changing requirements
University of Sunderland

Technology risks
Real-time performance inadequacies Straining CS [computing] capabilities

IEEE Software, January 1991.

CSE300 ADVANCED SOFTWARE ENGINEERING
10
Session 2

Common Risks from Keil et al

Identified by experienced software project managers from the USA, Hong Kong and Finland.

In order of perceived importance, these factors are:

1. lack of top management commitment to the project 6. changing scope/ objections 7. lack of required knowledge or 2. failure to gain user commitment skills in the project personnel 3. misunderstanding the requirements 4. lack of adequate user involvement 8. lack of frozen requirements 9. introduction of new technology

10. insufficient or inappropriate staffing

11. conflict between user departments. 5. failure to manage end user expectations Framework for identifying software project risks Communications of the ACM, vol 41 (11)
University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

11
Session 2

Moynihans risks/concerns
(From 14 experienced systems developer managers in Ireland: developing systems for other companies)
1. Clients understanding of the requirements/problem to be solved (12) 2. Seniority & commitment of the project patron/owner (9) 3. Level of IT competence, experience of the customers (9) 4. Need to integrate/interface with other systems (9) 5. Scale/coordination complexity of the project (need to share resources, subcontract, etc) (8) 6. Where project control resides (developer v client v third parties) (8) 7. Level of change to be experienced by the client (to procedures, workflow, structures, etc) (7) 8. The need to satisfy multiple groups of disparate users versus the need to satisfy one group of similar users (7) 9. Who we will be working through: users versus the IT department, individuals versus committees (7) 10. Developers familiarity with platform/environment/methods (7)
University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

12
Session 2

Moynihans risks
11. Developers previous experience with the application (6) 12. Level of enthusiasm/support/"energy" for the project in the clients organization (5) 13. Logical complexity of the application (5) 14. Ease of solution validation (e.g. possibility of prototyping) (4) 15. Clients willingness/capability to handle implementation (3) 16. Freedom of choice of platform/development environment (3) 17. Criticality/reversibility of the new system roll-out (2) 18. Maturity of the technology to be used (2) 19. Developers knowledge of country/culture/language (2) 20. Stability of the clients business environment (2) 21. Developers knowledge of clients business sector (2)

IEEE Software 14(3) pp35-41

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

13
Session 2

Risk Assessment & Mgt. General Framework

Another Aphorism: If you fail to plan, you plan to fail. (Attributed to Tom Gilb)

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

14
Session 2

Example Method

Riskit: a method for systems development. Reference: Kontio,J. The Riskit Method for
Software Risk Management, version 1.00, CS-TR3782, 1997. Computer Science Technical Reports. University of Maryland. College Park, MD.

Kontio's ref: downloadable from: http://www.sbl.tkk.fi/jkontio/riskittr.pdf

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

15
Session 2

Riskit
Aimed at systems development.

All stakeholders within a project should have input.

Difficult to evaluate risks quantitatively:
therefore need to cater for qualitative risk assessment.

Participant communication is essential.

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

16
Session 2

The Riskit Method and its Life Cycle

The focus in typical risk methods

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

17
Session 2

Components of Each Step

Description: Entry Criteria Input: Output: Methods and tools: Responsibility: Resources: Exit Criteria:
University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

18
Session 2

The Riskit Method and its Life Cycle

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

19
Session 2

Techniques in Riskit
Steps Risk management mandate definition Goal review Risk identification Techniques Risk management mandate template. Goal-Question-Metric (GQM), Stakeholder-goal priority table, goal definition template. Brainstorming techniques Goal and stakeholder driven identification approaches Meeting aids Interviews Riskit analysis graph Multiple criteria decision making tools (such as AHP) Riskit Pareto ranking technique Riskit element review Riskit controlling action taxonomy NA Organisation measurement program or database
20
Session 2

Risk analysis

Risk control planning Risk control Risk monitoring

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

Summary of Techniques
Techniques strategic Goal-Question-Metric (GQM) Brainstorming techniques Goal and stakeholder driven identification Meeting aids Interviews Riskit analysis graph Multiple criteria decision making tools Riskit Pareto ranking technique Riskit element review Riskit controlling action taxonomy Organisation measurement program or Database * Features detailed Riskit defined * * * (e.g. AHP) generic

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

21
Session 2

Goal and Stakeholder Driven Identification

Stakeholders Goals Deliver within timescale Deliver all required features Have fully trained staff Deliver within budget Sales Manager Priority: 3 1 2 3 4 Project Manager Priority: 2 2 1 4 3 Customer Priority: 1 1 3 4 2

A simple technique to rank the identified goals within each stakeholder. N.B. Comparison cannot be made across rows (or goals) Different stakeholders may be of different priority/ importance in the project .
University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

22
Session 2

Risk Analysis Graph

(example from the Riskit Manual)

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

23
Session 2

Risk Analysis Graph

Each path through the graph is a scenario
Scenario 1

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

24
Session 2

Risk Scenario Ranking: Using Pareto-efficient sets

Risk Scenario Almost Very likely Utility loss certain. (0.9) (0.75) Catastrophic Critical Severe Minimal Scenario1 Scenario 4 Scenario 5 Scenario 7 Scenario2

probability
Likely (0.6). Scenario3 Scenario 6 Almost impossible (0.1)

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

25
Session 2

Riskit controlling action taxonomy.

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

26
Session 2

Prior Reading
Prior to the next session, Session 3 Software Measurement, students MUST read the Notes entitled Measurement Prior Reading

University of Sunderland

CSE300 ADVANCED SOFTWARE ENGINEERING

27
Session 2