Defect Prevention Training

Defect Prevention Training
Induction – Sep 2007
Protection notice / Copyright notice

Version 2.0
Introduction
Defect Prevention is a process of improving quality and
productivity by preventing the injection of defects into a
software work product.
Definition: “…an activity of continuous institutionalized

learning during which common causes of errors in work
products are systematically identified and process changes
eliminating those causes are made.”
[Eickelmann]
SEI has identified ‘Causal Analysis and Resolution’ as Level 5

PA of CMMI

Page 2 Sep-07 P&Q For Internal Use Only
Objectives
Course Objective
To enable participants understand and apply defect prevention
concepts
Defect Prevention Objectives

 Identify and analyze the causes of defects& Reduction in
number of defect categories
 Reduction in the extent of defect escape between phases
 Reduction in frequency of common defects
 Improvement in PCB values

Contents
Defects and Bugs (Examples)

Origin of Defects
Classification of Defects
Defect Management
Defect Detection
Defect Prevention Cycle

Objectives of Defect Prevention
• Establish practice of Root Cause Analysis within projects for Analysis

of Identified Defects
• Identify critical processes as part of root cause analysis
• Set goals for improving critical process (shift mean and narrow
variation)
• Reduce most frequent type of defects such as “ not following coding
guidelines”
• Analyze opportunities for improvement by conducting escape analysis.
• Use defect distribution data to drive process improvement activities
• Spread lessons learnt - Team Meetings, SEPG, Process Database

Responsibility
Project team is responsible for the DP activities pertaining to

the project life cycle activities &
Project Manager (at project level)
Project Quality Manager (at project level)
P&Q (at Org level)
SEPG (at Org level)

Defects and Bugs

Defects and Bugs - Example
The Atlas-Agena spacecraft, destined for Venus, had to be blown up

during launch because it became unstable about 90 miles up.
(Malfunctioning rockets have to be destroyed to avoid crashes in
populated areas). A missing hyphen in the flight plan resulted in the
loss of the $18.5 million US spacecraft.

Defects and Bugs - Example
On January 15, 1990, 114 switching computers in the AT&T telephone

network crashed because of a software flaw. 65 million subscribers were
unable to use their phones. The problem arose when a switching
computer in New York crashed, sending out a digital “out of service”
message to nearby computers. Normally, other switches would route
traffic around the disabled computer. However, a misplaced “break” in
a C statement caused the nearby computers to go down as well. For the
next 9 hours, the switches went down, rebooted themselves, and came
back up, only to go back down immediately.

Effect of Software Error
•unreasonable added cost

•lost time and effort
•inconvenience and annoyance
•death

Could these incidences of software
errors been prevented?
YES!

Product and Process Defects
PRODUCT PROCESS
Artifacts created Complete set of activities
Definition during the life cycle needed to transform user
of the project. requirements to a product.
Product Defect are Process Defects are related

related to to tasks/activities:
requirements : Non-adherence to standards
Types of Defects
Functional and
Non-Functional Poor Documentation
Strategy for Handling Defect Removal/ Schedule overrun

Elimination Defect Prevention
Training related
Defect
Product defect is always a result of Process Defect
Process defect is like a potential carrier of disease

Stages of a Software Cycle
Proposal Requirements Design Coding Testing Maintenance
Reviews
White Box
Black Box
Stress/Load
Problem
Reports/CRs
Defect Prevention - Feedback and Process adjustments

Defect Analysis and Process Improvement

Origin of Defects

Human Errors
Types of Errors
Omission More than 80% of

software errors are
Ignorance
human
Commission
Typography
Knowledge
Information
External
Translation Errors
NEEDED TOLD URS DESIGN BUILD
TRANSLATION ERRORS Detected Bugs

Hidden Bugs
A requirement is often stated in terms of a solution

Focus on solution may hide the real requirement
The mismatch between the solution desired and the
real requirement leads to translation errors
Design Errors
Errors that affect data integrity
Types of Errors Errors that alter correctly stored data
Incorrect algorithm used to compute a value
Mitigation
of design
errors Checklist
Some examples:
•Does each module in the system design exist in detailed design?

•Are all assumptions explicitly stated? Are they acceptable?
•Have the exceptional conditions been handled?
•Are all data formats consistent with the system design?
•Are the loop termination conditions properly specified?
Coding Errors
Exception handling
Incorrect Algorithm A programming error
alters a program’s
Missing Functionality ability, in a negative
Language pitfalls sense, to completely
and effectively meet
Memory release
the user’s requirement.
Omitted program sections

Testing Errors
Failure to notice a problem

Misreading the screen
Failure to execute a planned test
Failure to use the most ‘promising’ test cases.
Ignoring programmers suggestions
Corrupt data file used
Incorrect test cases
Concentration on trivial
Failure to report
Objectives of Defect Prevention
140
120
100 Ideal
80 Acceptable
60 Costly
40 Disaster
20
0
w
w
rt
w
ev
g
in
o
ie
ie
ie
ep
ev
ev
st
ev
Te
R
od
R
R
m
S
n
FS
C
ig
R
le
es
ob
D
Pr
Defect Detection as early as possible

❑ High proportion defect reported by customer - Unacceptable
❑ More defect detected in testing – Quality at High cost
❑ Maximize defect detection during reviews- Quality at right

Page 20 cost
Sep-07 P&Q For Internal Use Only
So where should the focus be?
On Proactive Defect Prevention

On Early Defect Detection
On Usage of Past Experience

Defect Prevention

Defect Management Rules
Barry Boehm
•Fixing downstream is costly…
Victor Basili
•Rework eats away resources...
•Pareto Rule - About 80% of the available rework comes from 20%
of the defects
•Another Pareto Rule - About 80% of the defects come from 20% of
the modules (and about half the modules are defect free)
•Peer Reviews catch 60% of the defects
•Perspective-based reviews catch 35% more defects than non-
directed reviews (use of checklists)
•Disciplined personal practices can reduce defect introduction rates
by up to 75%
•About 40-50% of user programs enter with trivial defects

Activities Performed during DP –
Organization Level
• Defect Prevention Plan is prepared at SBU-level (by

TC/Business Partner) – contains lessons learnt and
improvement actions
• SPI (Software Process Improvement) Plan is an overall Plan

for the Organization and bears reference to DP Plan as well
• DP Plan is shared with SEPG members and PQMs
• Lesson’s Learnt from SBU is disseminated through SEPG
• Improvement Action are tracked and DP Plan is updated

accordingly
Activities Performed during DP –
Project Level
•Kick-off/Start PES meeting –
•Goal setting,
• Identification of critical processes
• Incorporation of Lessons Learnt from previous projects as
preventive measures
• Phase-wise Defect Distribution goal setting
• Defect reporting – Reviews & Testing
• Root Cause Analysis
• Action Implementation
• Information Dissemination
Root Cause Analysis
• Causal Analysis should be conducted as early as possible after a
defect is found
• Evaluate all errors (team effort) - from LOD and test results.
• All team members learn from the errors of other team members
• Prepare Pareto Chart for common causes and identify special causes
 Special causes of defects are focused on defects, which cause
high impact to the project.
 Common causes of defects are focused on high frequency defects
that occur often in the project or across projects.
• Perform Fishbone Analysis & escape analysis
• Create an action list against root causes
• Compare the results of the tasks performed with the goals set in the
kickoff meeting
• Record Corrective and Preventive Actions (in PPR/PQR, RCA report)
• Disseminate information to team - Project Meetings
• Disseminate information across projects - SEPG

Causal Analysis Cycle
Reviews/
Testing Classify Defects
Test Logs (Type, Injected &
LODs Detected Phase)
Identify Top
Application 80% Defects for RCA
And select all high

Preventive Feedback
impact defects
Prj RCA Arrive at Root Cause Perform Fishbone Analysis

Report And Action List Using Potential Causes

Pareto Chart
What is a Pareto Chart

Bar chart arranged in descending order
Bars on the left are more important than those on the right
Separates the “vital few” from the “trivial many”
Uses of Pareto Chart

Breaks a big problem into smaller pieces
Identifies most significant factors (80-20 rule)
Shows where to focus efforts
Allows better use of limited resources

Pareto Chart - Exercise
Participants to discuss possible Code Review Defects
Classify the defects under different categories/types and

assign a number of defects against each
Prepare a Pareto Chart using Excel to focus on the most

significant defects (80-20 rule)

Page 30
Number of Defects
0
1
2
3
4
5
6
7
8
Logical error
Redundant code
Sep-07
Remove debugging comments
Comments not exhaustive
Implementation error
P&Q
Functionality missing
Pareto Chart (contd.)
Code readability
Header incomplete
Pareto Chart - Code Review
Wrong parameters passed
Naming convention not followed

0
20
40
60
80
100
120
% of
Defects
Percentage
No. of defects

For Internal Use Only
Cause & Effect Diagram - Fishbone
What is a Cause & Effect Diagram?

A graphic tool that helps identify, sort and
display possible causes of a problem or quality
characteristic
Benefits of CED
Determination of root causes
Encourages group participation
Indicates possible causes of variation
Uses a orderly, easy-to-read format
Identifies areas for collecting data
Cause & Effect Diagram
• Decide the “Effect” to examine

• Identify the main categories
• Identify as many causes or factors as possible and attach them
as sub branches of the major branches
• Identify increasingly more detailed levels of causes by asking a
series of why questions
• Look for causes that appear repeatedly. These may be root
causes
• Identify and circle the causes that we can take action on

Cause & Effect Diagram
Method Machinery
Wrong Carburetor needs
gears used adjustment
Driven too
Under inflated
fast
tires
Poor
Mileage
Poor Wrong Octane
Maintenance gas used
Poor Improper
Driving habits Lubrication
Man Materials
Exercise on RCA
Make Groups
Assign a PM to each group
Brainstorm and prepare a cause and effect/fishbone

analysis
Present the result
(20 mins)

Defect Estimation
Phases Defect Distribution Proposed Goal

Requirement 7% 9%
Design 14% 16%
Coding & UT 49%
IT/ST 27%
44%
AT 3% 29%
2%
AT Requirement
3% 7% Design
IT/ST
27%
14% - Use Historical Data
- Focus on Business Objectives
and Process Improvement
Coding & UT - Set more Challenging Goals
49%

Applicability of RCA
•Defects
•Customer Feedback
•Non-conformance (NC)
•Process Capability Baselines
•Major Issues (that impact cost, quality,
schedule)

Example
DEFECTS AT END OF CODING & UT
People Process/ Measurement

Standards
Not Trained/
Inadequate resources Assets not Data not
Guidelines not available adequate
Guidelines not updated
followed
Coding & UT
Not aware Not adequate

Not Communicated
Long Overdue Tool not inspected
Not Available
Support/ Hardware/Software/
Guidance Tools

Common Defect Types
Defect Types Examples
Database design/modeling error, functionality

Function/Class/Object error is one that affects not working, etc
significant capability, end-user interfaces,
product interfaces, interface with hardware
architecture, or global data structure(s) and
should require a formal design change.
Assignment error indicates a few lines of code, Oversight during coding, initialization of
such as the initialization of control blocks or data parameters/variables, incorrect setting of
structure. variables, java script validation, etc
Interface/Messages corresponds to errors in
interacting with other components, modules or
device drivers via macros, call statements,
control blocks or parameter lists.
Checking addresses program logic, which has
failed to properly validate data and values Incorrect validation, missing validation, error
before they are used. handling, return value not checked
Timing/serialization errors are those, which are
corrected by improved management of shared
and real-time resources.
Build/package/merge describe errors that
occur due to mistakes in library systems,
management of changes, or version control. Incorrect packaging, Setup problem, etc
Common Defect Types (contd.)
Defect Types Examples
Documentation errors can affect both

publications and maintenance notes. Unclear specifications, standards not followed,
redundant code, GUI errors, incorrect
description, ambiguous description, etc
Algorithm errors include efficiency or
correctness problems that affect the task and
can be fixed by (re)implementing an algorithm or
local data-structure without the need for Hard coded values used, data type mismatch,
requesting a design change. etc
External Environment errors that occur due to Test data, test drivers, other tool defects,
factors that are outside the application scope. support system, concurrent work, inherited
from previous release, third party software
dependency, etc.
Performance errors affect the performance of Memory not released, Web session timeout
the system. not handled properly, browser cache related
problems, etc
Database errors are related to errors in Integrity constraint violated, SQL statements
database or scripts. not tuned, Error in SQL statement, etc
Trivial/Minor
Typo/minor errors in documentation,
rephrasing, extra information in document, etc

Common Root Causes
Major Defect Categories (from Fishbone) Root Cause
Support/Guidance (e.g. Management Support, Training, etc)

Handover (Change Coordination)
Inadequate training (QMS, Defect Prevention, technical)
Process/Standards Guidelines/Standards/Procedures not updated
Inadequate Process for Handling Requirements/Design
Change in Requirements/Design
People Breakdown of communications
Lack of knowledge (domain/system/tool)
Oversight
Hardware/Software/Tools Configuration related problem
Inadequate tools
Measurements Incorrect analysis of data

Escape Analysis
(Requirements defects getting slipped to next phases of Life cycle)

Code
Phase Detected Requirements
Design Review UT IT ST AT Total
Phase Injected
Requirements 10 9 1 1 2 3 26
Design 16 3 1 1 2 23
Code review 24 20 16 6 66
UT 0
IT 2 2
ST 3 3
AT 0
Total 10 25 28 22 21 14 0 120
Defects Found during Slippage (from
Code Review 28 Requirements to
Defects slipped from Design = 9/19) 47%
Coding (not considering DRE of Requirements
those injected in IT & Phase (10/19) 53%
ST) 52 Slippage (from Coding
Defects slipped into Phase to subsequent
Coding Phase 4 phases = 52/80) 65%
TOTAL (Coding + DRE of Code Review =
subsequent Phases) 80 28/80) 35%
Root Cause & Action Planning
Root Causes of Defects for <Month_1>, <Phase_1> <Month_n>, <Phase_n>
Special Causes:
(Root causes of high impact defects)
Common Causes:
(Root causes of high occurrence
defects)
Expected Actual
Implementation Monitoring Associated Impact Person
Action Plan Priority Date of Status Date of
Technique technique Risk on PCB * Responsible
Closure Closure
Team to be Workshop every

PM to ensure
given training Friday by each Defects in Schedule 20th June
that training High PM Open
on Domain team member in software Slippage 2004
is held
Knowledge turn

Some Lessons Learnt
Integration Testing should be scheduled so that the core

1
modules are initially tested.
2 Client should be given overview for SISL's P&Q processes
Client responsibilities should be clearly communicated in

3
the beginning of the project.
Design documents should contain all the necessary
4
validations to avoid validation error
5 Checklist should be used to avoid GUI errors
6 Rigorous unit testing to be done to avoid logical errors.
Basic level review and testing should be done at
7 developer's level before handing over the code to the
testers and reviewers.
8 Test cases should be formed with test data.

Recap…
Defects and Bugs (Examples)

Origin of Defects
Classification of Defects
Defect Management
Defect Detection
Defect Prevention Cycle

Any Questions?

Thank You


Defect Prevention Training

Загружено:

Сведения о документе

Исходное описание:

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Defect Prevention Training

Загружено:

Авторское право:

Доступные форматы

Defect Prevention Training

Induction – Sep 2007

Protection notice / Copyright notice

Definition: “…an activity of continuous institutionalized

SEI has identified ‘Causal Analysis and Resolution’ as Level 5

Protection notice / Copyright notice

Defect Prevention Objectives

Protection notice / Copyright notice

Defects and Bugs (Examples)

Protection notice / Copyright notice

• Establish practice of Root Cause Analysis within projects for Analysis

• Use defect distribution data to drive process improvement activities

• Spread lessons learnt - Team Meetings, SEPG, Process Database

Protection notice / Copyright notice

Project team is responsible for the DP activities pertaining to

Protection notice / Copyright notice

Protection notice / Copyright notice

The Atlas-Agena spacecraft, destined for Venus, had to be blown up

Protection notice / Copyright notice

On January 15, 1990, 114 switching computers in the AT&T telephone

Protection notice / Copyright notice

•unreasonable added cost

Protection notice / Copyright notice

Protection notice / Copyright notice

Product Defect are Process Defects are related

Strategy for Handling Defect Removal/ Schedule overrun

Product defect is always a result of Process Defect

Process defect is like a potential carrier of disease

Proposal Requirements Design Coding Testing Maintenance

Defect Prevention - Feedback and Process adjustments

Protection notice / Copyright notice

Protection notice / Copyright notice

Omission More than 80% of

NEEDED TOLD URS DESIGN BUILD

TRANSLATION ERRORS Detected Bugs

A requirement is often stated in terms of a solution

•Does each module in the system design exist in detailed design?

Protection notice / Copyright notice

Failure to notice a problem

Defect Detection as early as possible

❑ More defect detected in testing – Quality at High cost

❑ Maximize defect detection during reviews- Quality at right

On Proactive Defect Prevention

On Usage of Past Experience

Protection notice / Copyright notice

Protection notice / Copyright notice

Protection notice / Copyright notice

• Defect Prevention Plan is prepared at SBU-level (by

• SPI (Software Process Improvement) Plan is an overall Plan

• DP Plan is shared with SEPG members and PQMs

• Lesson’s Learnt from SBU is disseminated through SEPG

• Improvement Action are tracked and DP Plan is updated

• Defect reporting – Reviews & Testing

• Root Cause Analysis

Protection notice / Copyright notice

And select all high

Prj RCA Arrive at Root Cause Perform Fishbone Analysis

Protection notice / Copyright notice

What is a Pareto Chart

Uses of Pareto Chart

Protection notice / Copyright notice

Participants to discuss possible Code Review Defects

Classify the defects under different categories/types and