Optimize Your Defect Lifecycle

- Philosophy of Resolution -
November 24, 2005 – Trevor Atkins
http://www.qalabs.com

Copyright ©2005 QA Labs Inc.

All Rights Reserved.
Economic Argument for Testing

Copyright ©2005 QA Labs Inc.

All Rights Reserved.
 NIST Report

• According to a study commissioned by the

National Institute of Standards and
Technology (NIST) of the Department of
Commerce:
– Software errors cost the U.S. economy an
estimated
$59.5 billion annually, or about 0.6 percent of
the gross domestic product
– About one-third of these costs, or an estimated
$22.2 billion, could be eliminated by an
improved testing infrastructure
– 80 percent of the software
Copyright development costs
©2005 QA Labs Inc.

of a typical projectAll Rights

areReserved.
spent on identifying and
 Costs of (Poor) Quality

• Costs associated with preventing,

finding, and correcting defective work
– can be of the order of 20% - 40% of
sales
– Total Cost of Quality = Prevention +
Appraisal + Internal Failure + External
Failure

http://www.badsoftware.com/qualcost.ht
Copyright ©2005 QA Labs Inc.
m All Rights Reserved.
 The Challenge of Testing

• Appraisal Costs are associated with

activities designed to find quality
problems, such as code inspections
and any type of testing.
• How do you make testing more
effective?
– …especially more cost effective?
– …get more done with less?
Copyright ©2005 QA Labs Inc.
All Rights Reserved.
 Tools Aren’t Silver Bullets

• Tools need to be supported by manual

tasks and workflows first, not the other
way around
– Planning,
– Requirements management,
– CM, or
– Testing
• Consider how well the tool fits into your
current processes, whether those
processes are formal or informal.
Copyright ©2005 QA Labs Inc.
All Rights Reserved.
 The Defect Lifecycle

• One of the foundation processes in

any company that produces software
• Describes how Testing and
Development interact around an
issue or defect report
• Typically a focus on how setting and
tracking of the severity and/or
priority
Copyright ©2005 QA Labs Inc.
All Rights Reserved.
 You Have Found A Bug!

• Subsequent to detecting a quality

problem are the costs associated with
what happens next:
– How that quality report is handled,
– how it is investigated, and
– how it is determined to be resolved.

Copyright ©2005 QA Labs Inc.

All Rights Reserved.
Deciding Product Behaviour

Product Project
Management Management

Test

Business /
Requirements Development
Analysts

Copyright ©2005 QA Labs Inc.

All Rights Reserved.
 Attributes of a Defect Report
• Status • Environment

• Assigned To • Resolution

• Priority • Opened Version

• Severity • Opened By

• Functional Area • Opened Date

• Feature • Related Test case(s) or

Requirement(s)
• How found
• History or Audit Trail
• Type
Copyright ©2005 QA Labs Inc.
All Rights Reserved.
 Using Defect Resolution

• All defects must be resolved at one

time or another
Defect is not fixed

Start Open Resolved Closed End

Defect Reported Defect fixed Defect confirmed fixed

Deferred

Defect to be addressed later

Copyright ©2005 QA Labs Inc.

All Rights Reserved.
 Resolution Philosophy

• Capture much of the underlying

ideology of the defect resolution
process.
• Obtain a more detailed picture of the
defect counts
• An improved analysis of that data for
more accurate and useful metrics.

Copyright ©2005 QA Labs Inc.

All Rights Reserved.
 Defect Resolutions
• Fixed: the programmer says it's fixed. Check it.
• Cannot Reproduce: The programmer can't make the
failure happen. Add details, and notify the programmer.
Also known as Not Repro.
• Deferred: It's a bug, but it will be fixed later.
• As Designed: The program works as it's supposed to. Also
known as By Design.
• Need Info: The programmer needs more info about the
bug.
• Duplicate: This is a repeat of another bug report (cross
reference it on this report.)
• Withdrawn: The person who reported this bug is
withdrawing the report.
Copyright ©2005 QA Labs Inc.
Cem Kaner: "How To Win
All Rights Friends, And Stomp Bugs"
Reserved.
 Process Implications

• An effective defect lifecycle ensures

that defects aren’t investigated for
the wrong reasons and that the
correct defects are logged.

Copyright ©2005 QA Labs Inc.

All Rights Reserved.
 Process Implications (cont.)
Product Management
Business Analysts

Spec Issue
By Design
Deferred

Fixed
Resolved For Re-Testing
Third Party

Not Repro
Need Info
Withdrawn

Testers

Copyright ©2005 QA Labs Inc.

All Rights Reserved.
 More Information

• Newsletter #33 – April 2005

– http://www.qalabs.com/resources/

• Software Test & Performance

magazine
– Volume 2 – Issue 10, November 2005

Copyright ©2005 QA Labs Inc.

All Rights Reserved.
End of Presentation

http://www.qalabs.com

Copyright ©2005 QA Labs Inc.

All Rights Reserved.