Introduction to Software

Testing
 What is Software Testing?
Software testing is an activity to
check whether the actual results
match the expected results and to
ensure that the software system is
defect free.
It involves execution of a software
component or system component to
evaluate one or more properties of
interest.
 Several definitions:
Testing is the process of establishing
confidence that a program or system
does what it is supposed to. by
Hetzel 1973
 by Myers 1979
Testing is the process of executing a
program or system with the intent of
finding errors.
 by Hetzel 1983
Testing is any activity aimed at
evaluating an attribute or capability
of a program or system and
determining that it meets its required
results.
 Software Testing
One of very important software development phases

- A software process based on well-defined software

quality control and testing standards, testing
methods, strategy, test criteria, and tools.

- Engineers perform all types of software testing

activities to perform a software
test process.

- The last quality checking point for software on its

production line
 Who does Software Testing
Test manager
Software Test Engineers and Testers
Independent Test Group
Development Engineers
Quality Assurance Group and
Engineers
 Test manager
manage and control a software test
project
supervise test engineers
define and specify a test plan
Software Test Engineers and Testers

Define test cases,

Write test specifications
Run tests
 Development Engineers
Only perform unit tests and
integration tests
 Quality Assurance Group and
Engineers
Perform system testing
Define software testing standards
and quality control process
Software Testing Scope

Software Testing Software Testing Software Testing

Management Methods Strategies

Configuration Software Testing Software Test

Management Process Models

Software Problem Software Testing Software Test

Management Tools Criteria
 Software Testing Activities
Test Planning
Test Design and Specification
Test Set up
Test Operation and Execution
Test Result Analysis and Reporting
Problem Reporting
Test Management and Measurement
Test Automation
Test Configuration Management
 Test Planning
Define a software test plan by
specifying
A test schedule for a test process
and its activities, as well as
assignment
Test requirements and items
Test strategy and supporting tools
 Test Design and
Specification
Conduct software design based well-
defined test generation methods.
Specify test cases to achieve a
targeted test coverage
 Test Set up
Testing Tools and Environment Set-up
Test Suite Set-up
 Test Operation and
Execution
Run test cases manually
or automatically
 Test Result Analysis and
Reporting
Report software testing results and
Conduct test result analysis
 Problem Reporting
Report program errors using a
systematic solution.
Test Management and Measurement

Manage software testing activities,

Control testing schedule,
Measure testing complexity and
Calculate cost
 Test Automation
Define and develop software test
tools
Adopt and use software test tools
Write software test scripts and
facility
 Test Configuration
Management
Manage and maintain different versions of
software test suites,
test environment and tools,
and documents for various product
versions
 Verification and Validation
Verification --> refers to the set of
activities that ensure that software
correctly implements a specific
function.
Validation -> refers to a different
set of activities that ensure that the
software that has been built is
traceable to customer requirements.
 Verification and Validation
Verification: Are we building the
product right?
Validation: Are we building the right
product?
 Verification
Verification addresses the concern:
"Are you building it right?"
Verification takes place first and
includes the checking for
documentation, code, etc.
Done by developers.
 Validation
Validation addresses the concern:
"Are you building the right thing?
Validation occurs after verification
and mainly involves the checking of
the overall product.
Done by testers
 Software Quality Factors
Functionality (exterior quality)
Engineering (interior quality)
Adaptability (future qualities)
- Flexibility, reusability,
maintainability
 Functionality (exterior
quality)
Correctness
Reliability
Usability
Integrity
 Engineering (interior
quality)
Efficiency
Testability
Documentation
Structure
 Adaptability (future
qualities)
Flexibility
Reusability
Maintainability
 Software Testing Principles
Principle #1: Complete testing is impossible.
Principle #2: Software testing is not simple.
Reasons:
Quality testing requires testers to
understand a system/product completely
Quality testing needs adequate test set, and
efficient testing methods
A very tight schedule and lack of test tools.
 Software Testing Principles
Principle #3: Testing is risk-
based.
Principle #4: Testing must be
planned.
Principle #5: Testing requires
independence.
 Software Testing Principles
Principle #6: Quality software testing
depends on:
Good understanding of software
products and related domain
application
Cost-effective testing
methodology, coverage, test
methods, and tools.
Good engineers with creativity,
and solid software testing
 Software Testing Myths
We can test a program completely. In
other words, we test a program
exhaustively.
We can find all program errors as
long as test engineers do a good job.
We can test a program by trying all
possible inputs and states of a
program.
 Software Testing Myths
A good test suite must include a great
number of test cases.
Good test cases always are complicated
ones.
Software test automation can replace
test engineers to perform good software
testing.
Software testing is simple and easy.
Anyone can do it. No training is needed.
 Software Testing Limits
Due to the testing time limit, it is
impossible to achieve total confidence.
We can never be sure the
specifications are 100% correct.
We can never be certain that a testing
system (or tool) is correct.
No testing tools can copy with every
software program.
 Software Testing Limits
Tester engineers never be sure that
they completely understand a
software product.
We never have enough resources to
perform software testing.
We can never be certain that we
achieve 100% adequate software
testing.
Why is Software Testing Important?

Testing is important because

software bugs could be expensive or
even dangerous.
Software bugs can potentially cause
monetary and human loss, history is
full of such examples
 First
In April 2015, Bloomberg terminal in
London crashed due to software
glitch affected more than 300,000
traders on financial markets.
 Second
Nissan cars have to recall over 1
million cars from the market due to
software failure in the airbag sensory
detectors. There has been reported
two accident due to this software
failure.
 Third
Starbucks was forced to close about
60 percent of stores in the U.S and
Canada due to software failure in its
POS system. At one point store
served coffee for free as they unable
to process the transaction.
 Fourth
Some of the Amazons third party
retailers saw their product price is
reduced to 1p due to a software
glitch. They were left with heavy
losses.
 Fifth
In 2015 fighter plane F-35 fell victim
to a software bug, making it unable
to detect targets correctly.
China Airlines Airbus A300 crashed
due to a software bug on April 26,
1994, killing 264 innocent live
 Sixth
In 1985, Canada's Therac-25
radiation therapy machine
malfunctioned due to software bug
and delivered lethal radiation doses
to patients, leaving 3 people dead
and critically injuring 3 others.
 Seventh
In April of 1999, a software bug
caused the failure of a $1.2 billion
military satellite launch, the costliest
accident in history
 Types of Software Testing

Functional Testing
Non-Functional Testing or
Performance Testing
Maintenance (Regression and
Maintenance)
 Functional Testing
Unit Testing
Integration Testing
UAT ( User Acceptance Testing)
Localization
Globalization
Interoperability
 Non-Functional Testing or
Performance Testing
Performance
Endurance
Load
Volume
Scalability
Usability
 Maintenance (Regression and
Maintenance)
Regression
Maintenance
 Actually, More than 100 Testing
Types
Acceptance Testing
Accessibility Testing
Active Testing
....
White box Testing
Workflow Testing
 When to Start Testing?
An early start to testing reduces the cost
and time to rework and produce error-free
software that is delivered to the client
It also depends on the development model
that is being used.
in the Waterfall model, formal testing is
conducted in the testing phase; but in the
incremental model, testing is performed at
the end of every increment/iteration and
the whole application is tested at the end
 When to Stop Testing?
Testing Deadlines
Completion of test case execution
Completion of functional and code
coverage to a certain point
Bug rate falls below a certain level
and no high-priority bugs are
identified
Management decision