Академический Документы
Профессиональный Документы
Культура Документы
9 March, 2006
Presenter:
Donovan Bean
Agenda
Vocabulary
Why Do We Test Software?
Multiple roles of the Software Tester
Vocabulary
Working definitions:
Producer’s view – Quality of product
meets requirements
Customer’s view – Quality of product “fit
for use”/ meets customer’s needs
Quality Assurance vs. Quality Control
Preventive methods Detective methods
Planned, systematic activities necessary to Process where product quality compared to
provide adequate confidence that products applicable standards with action taken
and services will conform to specified upon detection of nonconformance.
requirements and meet user needs
Staff function, responsible for implementing Line function where work is done within a
the quality policy defined through the process to ensure that the work product
development and continuous improvement conforms to standards and requirements
of software development process
Activities that establish and evaluate the Activities that focus on identifying defects in
processes that produce products the actual products produced.
Measures processes to identify Activities begin at the start of the software
weaknesses, then correct these development process wit reviews of
weaknesses to continually improve the requirements, and continue until all
process application testing is complete
Relates to all products or services that will Relates to specific product or service
ever be created by a process
Helps establish processes and Verifies whether specific attributes are or
measurement programs to evaluate are not in a specific product or service
processes
Identifies weaknesses in processes and Identifies defects for the primary purpose of
improves them correcting defects
Responsibility of management Responsibility of team/worker
Quality Assurance vs. Quality Control (cont.)
QA = QC over QC because it
evaluates whether QC is working
QA staff don’t do QC except to
validate that the QC process is
working
The Cost of Quality
All costs above producing the product
right the first time.
Quantifies total cost of prevention,
appraisal & production of software
Includes costs of assuring delivered
product meets established quality
goals
Includes costs of preventing,
identifying & correcting defects
The Cost of Quality (cont.)
Prevention Costs
Establishing methods & procedures
Training
Tool acquisition
Quality planning
Costs incurred prior to build
Appraisal Costs
Comparing completed product to requirements
Inspections
Testing
Reviews
Costs incurred after build but prior to ship or moved to
production
Failure Costs
Costs associated with delivering a defective product to user or
deploying to production
Repairing products to meet requirements
Cost of operating faulty product
Damage caused by faulty product
Help desk operations
The Cost of Quality (cont.)
Identification and correction of defects account for
a majority of the Cost of Quality
Production costs can be minimized by focusing on
preventing defects
Eliminate rework and build inspection into the
process to attain the goal of optimizing the
production process
Continuous testing applied to the development
process can reduce the cost of quality
The IT QA group is responsible for identifying,
quantifying and developing programs that
minimize prevention, appraisal & failure costs
Software Quality Factors
Attributes of a system that pose a business risk if
desired and absent
When defining your testing scope, risk factors are
the basis and/or objective of your testing efforts
Testing software quality factors are the objectives
of many of these tests
Improving the quality of the software product is the
objective of applying these quality factors to a
software development program
Once determined, the software quality factors
need to be communicated to the development
team, documented. The CBOK recommends
conducting a briefing to communicate the inclusion
of the quality factors
Software Quality Factors (cont.)
Correctness
Reliability
Efficiency
Integrity
Usability
Maintainability
Testability
Flexibility
Portability
Reusability
Interoperability
Deming definition:
“Action taken on a stable system in
response to variation within statistical
control, in an effort to compensate for
this variation – the result of which will
inevitably increase the variation and will
increase cost from here on out”
Process variation is expected and is
not a reason for tampering
Reducing the Frequency of Defects
in Software Development
DOD’s attempt to make software
development more effective relying on
research from Carnegie Mellon University’s
Software Engineering Institute (SEI)
resulted in the Capability Maturity Model
Defines five levels of maturity as a process
moves from level 1 to 5, the variability in
the process is reduced
For a level 5 organization, the cost
difference of producing a function point of
logic is 100 times that of the cost for a level
1 organization
Five Levels of Process Maturity in a
Quality Management Environment
Multiple Roles of the Software Tester
People Relationships
Scope of Testing
When Should Testing Occur?
How the Test Plan Should be
Developed
Testing Constraints
People Relationships
Attitudes that shape a negative view of testing and testers:
Testers hold up implementation
Give them less time and reduce the chance of finding defects
Testers as debuggers
Testers to blame for defects found in production
Testers don’t need training, only developers do
Variables that affect the testing process:
The development process
Software risk
Customer/user participation
Testing process
Tester’s skill set
Use of tools
Testing budget
Resource constraints
People side of software testing has been ignored over the more process
related issues like test planning, test tools, defect tracking etc.
People Challenges
Training
Relationship building
Tool use
Getting managers to understand testing
Communicating to users about testing
Making time for testing
Testing “over the wall” software
Trying to hit a moving target
Fighting a lose-lose situation
Having to say “no”
Essential Testing Skills
Test planning
Automated and manual tool use
Test execution
Defect management
Risk analysis
Test measurement
Designing a test environment
Designing effective test cases
Solid testing vernacular
Understand what to test
Understand who does what types of tests
Understand when to test
Understand how to test
Understand when to stop testing
Broadened Scope of a Tester
Test to determine whether or not the user’s
needs have been met regardless of the
specifications
Find defects early in the software
development process where they’re
cheaper to fix
Remove defects of all types prior to going
into production
Identify weaknesses in the development
process so the processes can be improved
and thus mature the process
When Should Testing Occur?
Too often testing after coding is the only verification technique used
to determine the adequacy of a system
If testing constrained to a single phase at the later stages of
development, testing can consume 50% of the development budget
An error found toward the end of the life cycle must be paid for 4
times:
1st – develop erroneously
2nd – test to detect the error
3rd – wrong specs, coding and docs replaced with correct versions
4th – system must be retested
Incorporating testing at each phase lowers cost and increases
quality
2/3rds of all detected system errors attributed to errors in design
phase see Table 5. Life Cycle Verification Activities (pp. 68) for the
recommended testing process
When Should Testing Occur?
(cont.)
Table 5. Life Cycle Verification Activities (pp. 68):
When Should Testing Occur?
(cont.)
Activities that should be performed at each
phase of the life cycle:
Analyze the structures produced for internal
testability and adequacy
Generate test sets based on the structure of the
phase
Additional steps for design and
programming:
Determine that the structures are consistent
with structures produced during previous
phases
Refine or redefine test sets generated earlier
Developing the Test Plan
Figure 9. Example of a High-Level Test Plan (pp.
Select and rank test 71)
objectives
Identify the system
development phases
Identify the business
risks associated with
the system under
development
Place risks in the
matrix
Testing Constraints
Budget and schedule
Incomplete statement of work
Changes in technology (pp. 75)
Limited tester skills
Software risks (pp. 77)
Software defects
Software design defects
Data defects
Finding defects (why hard to find, and
circumstances causing defects pp. 79)