Software Risk Analysis

Software analysis goal

 The goal of requirement analysis phase is answer to question: What
software must do (and with what constraints)?
 The goal of software analysis phase is answer to question: how system
should work?
 The goal of software development phase is answer to question: how
system should be implemented?
Object oriented analysis
Goal of object oriented analysis
The goal of object oriented analysis is to prepare model, that will
represent software according to the user needs.

Steps
1. Agree with the customer basic requirements of the system
2. Identify classes
3. Prepare class hierarchy
4. Describe relationships between objects
5. Prepare object behavior model
6. Repeat steps 1-5 until complete model is finished
Software analysis phase
Software engineering elements that are used during
analysis
phase:
 Notations for model record,
 Methods of model preparation,
 Tools for easy use of notations and methods.
SOFTWARE DEVELOPMENT LIFE CYCLE
Traditional terminology
 The asset, or object of the protection efforts, can be a system component, data, or even a
complete system.
 Risk, the probability that an asset will suffer an event of a given negative impact, is determined
from various factors: the ease of executing an attack, the attacker’s motivation and resources, a
system’s existing vulnerabilities, and the cost or impact in a particular business context.
 The threat, or danger source, is invariably the danger a malicious agent poses and that agent’s
motivations (financial gain, prestige, and so on). Threats manifest themselves as direct attacks
on system security.
 A vulnerability is a defect or weakness in system security procedure, design, implementation, or
internal control that an attacker can compromise. It can exist in one or more of the components
making up a system, even if those components aren’t necessarily involved with security
functionality. A given system’s vulnerability data are usually compiled from a combination of OS-
and application-level vulnerability test results, code reviews, and higher-level architectural
reviews. Software vulnerabilities come in two basic flavors: flaws (design-level problems) or
bugs (implementation-level problems). Automated scanners tend to focus on bugs, since human
expertise is required for uncovering flaws.
Traditional terminology
 Countermeasures or safeguards are the management, operational,
and technical controls prescribed for an information system that, taken
together, adequately protect the system’s confidentiality, integrity,
and availability as well as its information. For every risk, a designer
can put controls in place that either prevent or (at a minimum)
detect the risk when it triggers.
 The impact on the organization, were the risk to be realized, can
be monetary or tied to reputation, or it might result in the breach of
a law, regulation, or contract. Without a quantification of
impact, technical vulnerability is hard to handle—especially when it
comes to mitigation activities.
 Probability is the likelihood that a given event will be triggered. It
is often expressed as a percentile, although in most cases,
probability calculation is extremely rough.
Software Risk Management
 The Top Ten Software Risk Items
Risk Item
1. Personnel Shortfalls
2. Unrealistic schedules
and budgets
3. COTS; external components
4. Requirements mismatch;
gold plating
5. User interface mismatch
Risk Management Techniques
Staffing with top talent; key personnel
agreements; incentives; team-building; training;
tailoring process to skill mix; peer reviews
Business case analysis; design to cost; incremental
development; software reuse; requirements descoping;
adding more budget and schedule
Qualification testing; benchmarking; prototyping;
reference checking; compatibility analysis; vendor
analysis; evolution support analysis
Stakeholder win-win negotiation; business case
analysis; mission analysis; ops-concept formulation;
user surveys; prototyping; early users’ manual;
design/develop to cost
Prototyping; scenarios; user characterization
(functionality, style, workload)
 The Top Ten Software Risk Items

6. Architecture, performance, Architecture tradeoff analysis and review boards;

quality simulation; benchmarking; modeling; prototyping;
instrumentation; tuning

7. Requirements changes High change threshold; information

hiding; incremental development (defer
changes to later increments)

8. Legacy software Design recovery; phaseout options analysis;

wrappers/mediators; restructuring

9. Externally-performed Reference checking; pre-award audits;

tasks award-fee contracts; competitive design
or prototyping; team-building

10. Straining Computer Technical analysis; cost-benefit analysis;

Science capabilities prototyping; reference checking
Risk Exposure
Calculates the effective current cost of the risk and can
be used to identify and prioritize risk.
Risk Reduction Leverage
Calculates the value for the ROI for countermeasures and
can be used to prioritize possible countermeasures.
Risk Exposure Factors
Unsatisfactory Outcome (UO) Prob Loss Risk Exposure
A. S/ W error kills experiment 3-5 10 30 - 50
B. S/ W error loses key data
3-5 8 24 - 40
C. Fault tolerance features cause unacceptable
performance 4-8 7 28 - 56
D. Monitoring software reports unsafe condition
5 9 45
as safe
E. Monitoring software reports safe condition 5 3 15
as unsafe
F. Hardware delay causes schedule overrun 6 4 24
G. Data reduction software errors cause extra 8
8 1
work
H. Poor user interface causes inefficient 6 5 30
operation
1 7 7
I. Processor memory insufficient
J. Software loses derived data 2 2 4
 Prioritizing Risks: Risk Exposure
Risk Exposure - (Probability) (Loss of Utility)

High Check Major

Utility - Risk
Risk Probability Loss Estimate

Check
Little Probability
Low Risk Estimate

Low High
Loss of Utility
Risk Exposure Factors
Risk Reduction Leverage (RRL)

RRL - RE BEFORE - RE AFTER

RISK REDUCTION COST

· Spacecraft Example
LONG DURATION FAILURE MODE
TEST TESTS
LOSS (UO) $20M $20M
PROB (UO) B 0.2 0.2
RE B $4M $4M
PROB (UO) 0.05 0.07
A $1.4M
REA $1M
COST $2M $0.26M
4-1 4- 1.4
RRL = 1.5 = 10
2 0.26
Risk Reduction Leverage (RRL)
For Failure Mode Test
For Long Duration Test
Loss (UO) = $20 M
Loss (UO) = $20 M
Probabilty (UO)b = 2%
Probabilty (UO)b = 2%
RE = 20,000,000 * 0.02
RE = 20,000,000 * 0.02
= $ 4,000,000
= $ 4,000,000
Probabilty (UO)b = 7%
Probabilty (UO)b = 5%
RE = 20,000,000 * 0.07
RE = 20,000,000 * 0.05
= $ 1,400,000
= $ 1,000,000
Cost = $1,000,000
Cost = $ 1,000,000 Failure mode test has a
higher result, meaning it
RRL = 4-1.4/0.26
RRL = 4-1/2 is more cost effective
= 10
= 1.5 compared to the long
duration test.
EXAMPLE OF A RISK CALCULATION
 One classic risk-analysis method expresses risk as a financial loss, or
ANNUALIZED LOSS EXPECTANCY, based on the following equation:
ALE = SLE × ARO

SLE is the single loss expectancy

ARO is the annualized rate of occurrence (or the predicted frequency of a loss
event happening)
EXAMPLE OF A RISK CALCULATION
The middle- and back-office procedures will catch and negate any malicious
transaction such that the loss associated with the event is simply the cost of
backing out of the trade. Cost is $150 for any such event, with an occurrence of
100 per year.

SLE=$150
ARO = 100 per year

ALE = 150 x 100

= $ 15,000 annual lost