SE 4353

Risk Analysis and Management for

Engineering Systems
Dr. Bryan M. O’Halloran

Systems Engineering department

Naval Postgraduate School

Week 1: Basic Concepts of Risk and

Uncertainty, Risk Analysis Process
 Topics
• Definition of RISK

• Risk vs. Uncertainty

• Risk Analysis Process

• Risk Assessment

• Qualitative vs. quantitative risk

• Probabilistic Risk Assessment (PRA)

• Risk Management

• Risk Communication
 Risk: What is it?
Class exercise:
Define risk!
Define uncertainty!
 Risk vs. Uncertainty
• Risk and Uncertainty very commonly used interchangeably

• Definitions from Decision Theory and Statistics (Doug Hubbard):

• Uncertainty: a lack of certainty; a state of having limited knowledge

about an event, making it impossible to exactly describe existing state
or future outcome

• Measurement of Uncertainty: a set of possible states or outcomes where

probabilities are assigned to each possible state or outcome
 Risk vs. Uncertainty
• Definitions from Decision Theory and Statistics (Doug Hubbard):

• Risk: a state of uncertainty where some possible outcomes have an

undesired effect of significant loss

• Measurement of Risk: a set of measured uncertainties where some

possible outcomes are losses, and the magnitude of those losses
 Risk: What is it?
• Risk

• Key components:
• Likelihood (aka probability, or, uncertainty)

• Consequence (losses in dollars, work hours, human lives, etc.)

• RISK = (Probability of the event) x (Consequence of the event)

 Types of Uncertainty
• Uncertainty might be only due to a lack of knowledge about
obtainable facts which can be remedied

• e.g.,You might be uncertain about how a new rocket design will

work, but you might be able to remove this uncertainty with
further analysis and testing

• Uncertainty might be more intrinsic and hence unavoidable

• e.g., Inherent variations in manufactured products (tolerances)

• Uncertainty of a measurement given in error bars

• Found by repeating the measurement enough times until you

obtain a good estimate of the standard deviation of the values
 Risk: Where? When?
Class exercise:
• Engineering: Where do you expect to find risks/uncertainty?
 Risk vs. Uncertainty
• Example: Learning to live with rain
• If you do not know whether it will rain tomorrow, you have a
state of uncertainty

• If you apply probabilities to the possible outcomes using weather

forecasts, you have quantified the uncertainty

• If you have quantified your uncertainty as a 90% chance of

sunshine, and are planning a major outdoor event for tomorrow,
then you have risk since there is a 10% chance that it will be rain,
which is undesirable

• If there is potential of monetary loss due to the cancellation of

the event in case of rain, then you have quantified the risk (e.g., a
10% chance of loosing $50,000)
 Risk: Why do we care?
Consequence

Risk
 Risk in Different Phases of the
Product Lifecycle

• Design

• Manufacturing

• Operations
 Types of Risk Considerations
Technical Cost

Schedule
 Risk Analysis Process
Class exercise:
Write down ideal risk analysis process
Identify, assess, manage/mitigate, communicate risks!
 Risk Analysis Process
• In most cases, risks cannot be avoided, and in some cases, it is part
of the business model and a natural component of technology
development
• It is important to:
• Identify risks (proactive, continuous, and EARLY!)
• Analyze risks: levels, likelihood, consequence
• Assess options for improvement
• Develop plan to manage, mitigate risks, and recover from failures
due to risks
• Goal: Treat risk as just another design parameter!
• Design stage: RISK BASED (INFORMED) DESIGN
 Risk Analysis Process
• Risk assessment:
• Identifying sources of potential harm and assessing the likelihood
that harm will occur and the associated consequences
• e.g., FMECA, hazard analyses, PRA, etc.
• Risk management:
• Evaluating which risks, that were identified, require management,
and choosing/implementing the plans or actions required to control
or mitigate those risks
• e.g., risk register, “burn-down” plan, etc.
• Risk communication:
• Creating an open dialogue between the various stakeholders
(customers and engineers), risk assessors (analysts), and managers to
then actively inform all the other processes involved
 Risk Analysis Process
Class exercise:
Provide examples of “qualitative” and “quantitative” risk
assessments
 Risk Assessment
• Qualitative:
• lists of risks, discussions about risks, brainstorming sessions to
improve risks, lessons learned databases, prior knowledge,
broad categorization of risks (example on next slide), ...

• Quantitative:
• simple rankings, occurrence, probabilities, performance
degradations, ...
 Qualitative Risk Assessment
High

HIGH
X

Likelihood
Significant

Moderate MODERATE
Likelihood of Failure - Technology Dependence
High Significant Moderate Minor Low Minor
Dependent on
new technologies
Dependent on
new technologies
Dependent on
innovative use
Minor
modification of
No new
technology -- LOW
that are not yet that are in of existing existing systems are
funded development technologies technology off-the-shelf Low

Low Minor Moderate Significant High

Technical Consequence of Failure

High Significant Moderate Minor Low
Consequence
Major Significant Reduction in Minor reduction Little or no

Fever Charts
degradation in degradation in technical in technical impact on
technical technical performance with performance program
performance that performance limited impact on with little or no objectives
could jeopardize with a major program impact on
program success impact on objectives program
program objectives
objectives
 “Quantifying” qualitative risks
• Simple rankings

• Fever charts with numbers for likelihood and consequence (1-5)

• FMEA/FMECAs with numbers (Risk Priority Number)

• Probabilities based on past experience, expert judgment...

 Quan./Qual. Risk Assessment

Class exercise:
What are some advantages/disadvantages with quantitative
risk assessment?
When should quantitative assessments of risk be used?
 Quan./Qual. Risk Assessment
• Qualitative

• Advantage: quick, simple, does not require data

• Disadvantage: not accurate (results have limited usefulness)

• Quantitative

• Advantage: accurate (results become very useful)

• Disadvantage: requires data, takes significant effort to develop

analysis

• General consensus

• Qualitative Risk Assessment is best to use early in design where

limited knowledge/data is available, then

• Quantitative Risk Assessment is best to use when knowledge/data

becomes available
 Quantitative Risk Assessment
• “Risk Sensitive” Industries:

• Ex: nuclear power (and other industrial style facilities), aerospace, Oil
and gas, automotive, etc.

• Possible failure of a complex series of engineered systems can result

in highly undesirable outcomes

• Required to manage risk in a highly quantitative way

• The usual measure of risk for these types of events:

RISK = (Probability of the event) x (Consequence)
 Quantitative Risk Assessment
• Measuring engineering risk is often very difficult

• Probability is assessed using case studies

• Frequency of past similar events

• What to do in new designs, technologies, environments?

• Rare failures are hard to estimate (see “Normal Accidents” book)

• Huge problem with weapon systems! - Data sets are small

• Public scrutiny: tax dollars, human lives

• Astronauts understand and accept the risk of space flight

• Benefits of space flight/exploration outweigh the risks!

• Requires deep knowledge and lots and lots of analyses!

 Probabilistic Risk Assessment
• Probabilistic Risk Assessment (PRA) - brief introduction

• For “risk sensitive” industries:

• PRA: a systematic and comprehensive methodology to

evaluate risks associated with complex engineered
technological systems (e.g., airliners, nuclear power plants,
spacecraft)

• Accepted use by US government agencies, regulatory agencies

to enhance safety without applying undue conservatism

• Answers 3 basic questions:

• What can go wrong (initiating events)?

• What and how severe are the potential detriments?

• How likely are they to occur (probabilities or frequencies)?

Probabilistic Risk Assessment
@10k ft.
 Probabilistic Risk Assessment
• What can go wrong (initiating events)?

• Use of technical knowledge of possible causes

• Use of FMEA to focus on most important initiators

• Top-level hazards (safety)

• e.g., Inadvertent activation of the booster

• What and how severe are the potential detriments?

• Development of event (or accident) scenarios

• Deterministic analyses (thermal, fluid, structural) to describe the

phenomena that can occur along the path of event scenario
 Probabilistic Risk Assessment
• How likely are they to occur?

• Probabilities or frequencies of events

• Discuss this with RBDs

• Boolean logic methods for model development (ETA & FTA)

• Probabilistic & statistical methods to quantify model

 Risk Management

Class exercise:
Define risk management
 Risk Management
• Risk Planning: What is
the program’s risk
management process?
• Risk Identification: What
can go wrong?
• Risk Analysis
(Assessment): What are
the likelihood and
consequence of the risk?
• Risk Handling: Should the
risk be accepted, avoided,
transferred, or mitigated?
• Risk Monitoring: How
has the risk changed?
 Risk Management
• Risk Handling Strategy: prioritize to handle the risks with the
greatest loss & probability of occurrence first

• Rank order based on risk value, e.g., RPN

• This desire is driven by

the safety community

• Safety order of
precedence (right slide)

• In practice: very difficult

to balance between high
prob/low loss and low
prob/high loss
 Risk Management
• Risk avoidance: do not perform activity that has high risk

• Losing out on potential gain, not flying, not going to space?

• Risk reduction: implement methods to reduce severity of loss

• E.g., sprinklers to put out fire to reduce risk of loss; developing software
incrementally; prototyping

• Risk retention: accepting loss when it occurs

• Viable strategy for small risks; the cost of insuring against risk greater
than the total losses sustained

• Risk transfer: causing another party to accept/share the risk

• Insurance, liability

• Outsourcing SW development, manufacturing, customer support

 Your Design Projects
• When researching the system for your risk case study,
look specifically for how the organization choses to
manage each risks:
Risk avoidance, reduction, retention, transfer?
 Risk Communication
• Communicating risk via…

• Meetings, charts, brainstorming sessions, reviews, reports,

informal discussions

• How effective do you think this is in the real world?

• Think about recent events, failures...how much was due to lack

of communication or miscommunication about the potential
risks?

• e.g., Challenger o-ring failure

• Numbers help...but are they realistic? Do they mean anything?

 Airport Example
Class exercise:
List the steps in the process you go through at the airport starting
when you arrive (assuming you’ve already booked tickets, but haven’t
checked in to your flight). Things to consider:

• baggage

• security

• transportation within airport

• boarding the airplane

• etc.
 Airport Example
Class exercise:
Perform risk analysis on the day-to-day operation of an airport.
• List potential events/steps (~20)

• List failures/issues that could occur (10)

• Estimate qualitative consequence (1

low, 5 high)

Likelihood
• Estimate qualitative likelihood (1 low,
5 high)

• Place on a 5 by 5 chart (likelihood vs.

consequence)
Consequence
• What are sources of uncertainty?
 Airport Example
• Events: Check bag, check in for flight, get screened through sec., etc.

• Failure: flight delayed, lost ticket, computer systems down, baggage

doesn’t arrive

• Consequence: miss subsequent flight or arrive late, can’t travel and

have to reschedule, can’t travel or have to resort to physical tickets,
can’t change clothes and have to buy new clothes.

• Estimate qualitative consequence (1 low, 5 high)

• [3, 4, 4, 2]

• Estimate qualitative likelihood (1 low, 5 high)

• [1, 2, 1, 3]

• Sources of uncertainty: flight arrival time, time you arrive at terminal,

 Airport Example
Failure Consequence consequence likelihood

flight delayed miss subsequent flight or arrive late 3 1

lost ticket can’t travel and have to reschedule 4 2

can’t travel or have to resort to physical

computer systems down 4 1
tickets
can’t change clothes and have to buy new
baggage doesn’t arrive 2 3
clothes

likelihood =100%-83.72%
=14.28%
likelihood of 1 is [0%-20%], thus
the value is 1
 Airport Example
• Place on a 5 by 5 chart (likelihood vs. consequence)

• WWDD?

• What Would a
Design Do with
Likelihood

these numbers?

• Prioritize them top

to bottom, then get
to work.

Consequence
 Airport Example
• Key takeaway. Understanding the system is the key to analyzing it.