Surviving the Top Ten Challenges of Software Testing

Introduction and Basis of

Research

Surviving the Top Ten These findings are the results of

surveying over 1,000 testers from all
Challenges of Software major industries and geographic
regions.
Testing Research was conducted over a three
Randall W. Rice, CQA, CSTE, FLMI year period.
Rice Consulting Services, Inc.
405-692-7331
www.riceconsulting.com
Solving Your Testing Problems
RCS RCS
© 1997, Rice Consulting Services, Inc.
2

Major Finding How Testing Tests Testers

Most testing problems are not technical, but human in Many testers wear two faces:
nature. Tester One
Attempts at solving testing problems in most - The person that understands how
companies are aimed at technical approaches. to test software, is good at finding
defects, and enjoys testing
- Tester one is generally happy
when only performing tests.

RCS RCS
3 4

Challenge Ten: Is Testing a

The “Other” Tester Natural-born Instinct?
Tester Two Central Issue - Lack of
- The person that must fight the education in testing
political battles that surround The large majority of testers
testing
have never had formal training
- Tester two is generally unhappy
when dealing with political issues.
in software testing.
Many testers and their
management are unaware of
training sources for testing.

RCS RCS
5 6
Surviving the Top Ten Challenges of Software Testing

Challenge Nine: Whose Side

Solutions for Training are Testers On?
Obtain formal training in testing
techniques Central Issue - Lack of
Certification teamwork
- CSTE (Certified Software “Us versus them” mentality
Test Engineer) Seen most often in
Attending conferences organizations with
Reading books and articles independent test teams.

RCS RCS
7 8

Solutions to the Teamwork Challenge Eight: Testing

Challenge Without Tools
The goal is to get to “Us and Central Issue - Lack of
them” understanding of the usefulness of
Each person on the team can test tools
have a role in testing Without upper management
- Developers: unit and support, it is difficult to acquire test
structural testing tools.
- Testers: independent testing A process is needed for the tools.
- Users: business-oriented People should be trained in the use
testing of test tools.
- Management: to support People must accept and apply the
testing activities tools.
RCS RCS
9 10

Solutions for Acquiring and Challenge Seven: Managers

Using Test Tools Just Don’t Understand Testing
Identify a “champion” for obtaining
test tools. Central Issue - Management is not
fully aware of what is required for
Base the case for test tools in costs
effective testing.
vs. benefits.
Management is often focused on
Have a basic testing process in
the product, not the process.
place.
This leads to lack of management
Train people in tool usage.
support and involvement for
Measure the benefits. testing.

RCS RCS
11 12
Surviving the Top Ten Challenges of Software Testing
Solutions to Educating
Management in Testing Issues
Cultural change is needed.
Focus your message to
management on:
- reducing the cost of rework
- meeting the project schedule
The benefits of testing must relate
to these two things to be
persuasive.
RCS
13
Solutions to Identifying and
Involving the Customer in Testing
Involve the customer and users
throughout the project by
performing reviews and
inspections.
Include users on the system test
team.
Perform user acceptance
testing.
Understand the difference
between the customer and
users.
RCS
15
Solutions to the Time Crunch
Base schedules and
estimates on measurable
testing activities.
- Scripts to be executed
- Cases to be tested
- Requirements to be tested
Have contingency plans for
schedule slippage.
Integrate automated testing
tools to the project.
RCS
17
Challenge Six: Would the Real
Customer Please Stand Up?
Central Issue #1 - A lack of
understanding of who the
customer really is. We must
understand the difference
between:
- Customers
- Users
Central Issue #2 - Lack of
customer involvement in
testing.
RCS
14
Challenge Five: Too Much
Work - Too Little Time
Central Issue - Arbitrary
schedules and estimates do
not allow enough time for
testing.
Realism: There is never
enough time to test every
possible combinations of
test conditions.
RCS
16
Challenge Four: Overcoming
Throwing Stuff Over the Wall
Central Issue - Developers
depending on testers to find
bugs.
Seen most often when
independent test teams are
used in an organization.
RCS
18
Surviving the Top Ten Challenges of Software Testing

Solutions to Overcoming Challenge Three: Hitting a

Throwing Stuff Over the Wall
Developers must take
ownership and responsibility for
the quality of their work.
Quality control is most effective
when performed at the point of
creation.
Train developers to become
excellent testers.
Get management support for
developer responsibility for
quality.
RCS
19
Moving Target
Central Issue - Software and
systems that undergo rapid
and/or constant change
Software development
techniques such as Rapid
Application Development can
produce a new version of the
software very quickly.
You can’t perform a four-week
test every day manually.
RCS
20

Solutions for Hitting a Moving Challenge Two: Fighting a

Target Lose-Lose Situation
The testing process must Central Issue - Testers are
accommodate change. often the “bad guys” when they
Focus on testable find defects, yet if the software
requirements. goes live with problems, testers
are blamed for the defects.
Use automated testing
tools. Since independent testers have
Manage the rate and no control over the software
degree of change. development process, they
cannot guarantee software
quality.

RCS RCS
21 22

Solutions for Fighting a Lose- Challenge One: Having to Say

Lose Situation “No”
The perception of testing must Central Issue - Testers are
change. often the bearer of bad news.
- Testers are paid to find Problem: Management doesn’t
defects like to hear bad news.
- Each defect found is one
more the customer or user
will not find
Testers are not to blame for
bottlenecks. It is management’s
responsibility to have an
efficient process.
RCS RCS
23 24
Surviving the Top Ten Challenges of Software Testing

Solutions for Having to Say Summary - Applying the

“No”
Most responsibility is on
management to:
- have a quality software
development process in place.
- have contingency plan in place in
case of problems.
- understand that testing is only an
evaluation activity.
- accept the honest facts.
Keep the test results objective
Lessons Learned
Most testing problems seem to
be human in nature.
“Quality is everyone’s job, but it’s
management’s responsibility.”
Dr. Deming.
There are three key parts of the
testing picture: Process, People
and Tools.

RCS RCS
25 26