|


|

   

1. Introduction to Software Testing

2. Building Software Testing Strategy
3. Establishing a Software Testing Methodology
4. Determining Your Software Testing
Techniques
|

 
v Testing is nothing but Debugging
v Testing is not the job of the programmer

v Testing is done to prove that software

³works´
v Testing is done to prove that software
doesn¶t ³work´
v Testing activities start only after the coding is

complete
v Testing is not a creative Task
  

 Testing is a structured process of

uncovering the defects in a system
 Testing is an integral part of SDLC
 Testing can be started early in the life
cycle
 The purpose of testing is to have all the
bugs known before release
 
Software testing is a vital part of the software
lifecycle. To understand its role, it is instructive to
review the definition of software testing in the
literature.
Among alternative definitions of testing are the
following:
‰
   
  
  
 

 
   

 

 



    
 


    
 

  
 

‰
 

‰  

  

  


 


      
  





 
  
 
 


 
 
 ‰

‰
   

    




   ‰
 

· Software testing is defined as 'the execution

of a program to find its faults'.

· A successful test is one that finds a defect.

· Besides finding faults, a software can also

be tested for performance, safety, fault·
tolerance or security.
 
 o matter how well software has been
designed and coded, it will inevitably still
contain defects
 Testing is the process of executing a program
with the intent of finding faults
 A ³successful´ test is one that finds errors, not
one that doesn¶t find errors
 Testing can ³prove´ the presence of faults
(bugs), but can not ³prove´ their absence
 !

A defect is a variance from a desired product

attribute. There are two categories of defects:
± variance from product specifications
± variance from customer/user expectation

Defects generally fall into one of the following three

categories:
± Wrong
± Missing
± Extra
 !"
 
 While the defect is a flaw in the software system, it
has no impact until it affects the user/customer and
the operational system.

 A defect that causes an error in operation or

negatively impacts a user/customer is called a failure.
 |# 

Subjective ± as it depends on customer

satisfaction

 Bug free
 delivered on time
 within budget
 meets requirements
 maintainable
° |
|
 $%|$|&
· A risk is a condition that can result in a loss.
· The development and installation of a computer
system introduces risks into the organization.
· We cannot eliminate risks, but we can mitigate them.
·
ne of the most effective methods to reduce computer
system strategic risk is testing.

Types of strategic risk:

† Incorrect results will be produced

† Unauthorized transactions will be accepted by the
system.
$%|$|&

Continued . . .
† Computer file integrity will be lost.
† Processing cannot be reconstructed.
† Continuity of processing will be lost.
† Service provided to the user will degrade to an
unacceptable level.
† Security of the system will be compromised.
† Processing will not comply with organizational
policy or governmental regulation.
 $%|$|&
Continued . . .

† „esults of the system will be unreliable.

† System will be difficult to use.
† Programs will be unmaintainable.
† System will not be portable to other hardware and
software.
† System will not be able to interconnect with other
computer systems.
† Performance level will be unacceptable.
† System will be difficult to operate.
$%|$|&
Continued . . .
· An effective approach to testing is to identify and
evaluate the risks in a computer system.

· Those risks deemed important to reduce

become the areas for testing.

· A decision can be made as to how much risk is

acceptable and then a test plan designed to
achieve that goal.
 $

 Too little testing is a crime · Too much

testing is a sin.
 a
àà à à à à 
 aà
  
  
   
  

àààààà
 



   



 
 
àààààà   a
 
 a
 
   
à
à
à
àààààààà
à à
 à

à
à à   à
à à
à à
à à
 à 
à
à ààààààààààààààààààààààààààà
à
à  à à
à à
à à
à  à
à à
 à  
à
 à
à à
à à
à à
 
à à à  
à
à  à à
à  à à
à à
à à
 ààààààààààààààààààààà à   àà àààààà ààààààààààààààààààààààààààààà
à
 à à à
à
à
 |%%

„
   
     
 
 

?

 

 


· All too often, testing after coding is the only

verification technique used to determine the adequacy
of the system.

· When testing is constrained to a single phase and

confined to the later stages of development, severe
consequences can develop.
 |%%
· It is usual to hear of testing consuming 50% of
the development budget.

· All errors are costly, but the later in the life cycle
that the error discovery is made, the more costly
the error.

· An error discovered in the latter parts of the life

cycle must be paid for four different times.
|%%

 „„„
!"

##$%#
„„„
!&"

m

 




  

  #' 

· The recommended testing process is presented in

previous diagram as a life cycle chart showing the
verification activities for each phase.
· The success of conducting verification throughout the
development cycle depends upon the existence of
clearly defined and stated products at each development
stage.

· The recommended test process involves testing in

every phase of the life cycle. Life cycle verification
activities table
 |%%
$(%$)*# )*# +„(##+

„equirements Upon validation to determine that the defined  Determine verification approach
requirements meet the needs of the  Determine adequacy of
requirements
organization.
 enerate functional test data
 Determine consistency of design
with requirements

Design
n verification to ensure that the design  Determine adequacy of design

accomplish the defined requirements.  enerate structural and functional
test data
 Determine consistency with
design

Program (build /
n verification to ensure that the programs  Determine adequacy of

accomplish the defined requirements. implementation
construction)  enerate structural and functional
test data for programs

Test
n inspection to determine that the  Test application system

implemented system meets the system
specification.

Installation
n inspection to determine that the  Place tested system into

implemented system meets the system production
specification.
Maintenance „etesting to determine that the changes work  Modify and retest
and that the unchanged portion continues
to work.
 Test Strategy
 The objective of testing is to reduce the risks
inherent in computer systems.

 The strategy must address the risks and present

a process that can reduce those risks.

 The system concerns or risks then establish the

objectives for the test process.

 The two components of the testing strategy are

the test factors and the test phase.
Test factor:
The risk or issue that needs to be addressed as
part of the test strategy. The strategy will select
those factors that need to be addressed in the
testing of a specific application system.

Test phase:
The phase of the systems development life cycle
in which testing will occur.
 ot all test factors will be applicable to all
software systems.
 The development team will need to select and
rank the test factors for the specific software
system being developed.
nce selected and
ranked, the strategy for testing will be partially
defined.
 The test phase will vary based on the testing
methodology used.
 Ex: The test phases in a traditional waterfall
lifecycle methodology will be much different from
the phases in a „apid Application Development
methodology.
 Test Factors
† Correctness
† File Integrity
† Authorization
† Audit Trail
† Continuity of processing
† Service levels
† Access control
† Compliance
† „eliability
† Ease of use
 Test Factors
Continued . . .

† Maintainability
† Portability
† Coupling
† Performance
† Ease of
peration
 Developing a Test Strategy
A generic test strategy is illustrated in the figure
which is a test_factor/test_phase matrix. However,
this strategy will need to be customized for any
specific software system.
Four steps must be followed to develop a customized
test strategy.
 Select and rank test factors
 Identify the system development phases
 Identify the business risks associated with the
system under development
 Place risks in the matrix
 |'
TEST
PHASE

TEST
FACT
„S
(high to low)

(
„,
 
 
 Testing Methodology
The testing methodolgy we propose
incorporates both testing strategy and testing
tactics. The tactics add the test plans, test
criteria, testing techniques, and testing tools
used in validating and verifying the software
system under development.
 |
 
 |
 

The testing methodology is the means by

which the test strategy is achieved.
Testers Job.
What Are You Testing For?

 ariance from specifications

 ariance from what is desired
 Testers need to identify both types of
defect.
Why Are Defects Hard to Find?

 ot Looking.
 Looking, but not seeing.
The four testing tactics of validation,
verification, functional test, and structural
test, which are the bread and butter of
testing can be separated into two groups:

(1) validation and verification

(2) functional and structural testing
 ww 

erification :
Did we build the right system?

alidation :
Did we build the system right?³
$%|$w'$% 
+„(# )„(„ % -)$## $+„#$
-#)$
„equirements reviews Developers, Users The study & discussion „eviewed statement of
of the computer requirements, ready to
system reqs to ensure be translated into
they meet stated user system design.
needs & are feasible
Design „eviews Developers The study & discussion System design, ready
of the computer to be translated into
system design to computer programs,
ensure it will support hardware configs,
the system docs, and training.
requirements
Code Walkthroughs Developers An informal analysis of Computer software
the program source ready for testing or
code to find defects more detailed
and verify coding inspections by the
techniques. developer.

Code Inspections Developers, Subject A formal analysis of the Computer software

Matter Experts program source code ready for testing by the
to find defects as developer.
defined by meeting
computer design
specifications.
$%|$w '$% 
+#$ # )„(„ % -)$## $+„#$
-#)$
Unit Testing Developers The testing of a single Software unit ready for
program, module, or unit testing with other system
of code. alidates that the component, such as other
unit performs as designed. units, hardware, docs, or
users.
Integrated Testing Developers, Testers The testing of related Portions of the system
programs, modules, or ready for testing with other
units of code. alidates portions of the system.
that multiple parts of the
system interact according
to the system design.

System Testing Developers, Testers The testing of an entire A tested computer system,
computer system. This based on what was
kind of testing can include specified to be developed
functional and structural or purchased.
testing, such as stress
testing. alidates the
system reqs.
User Acceptance Testers, Users The testing of a computer A tested computer system,
Testing system or parts of a based on user needs.
computer system to make
sure it will work in the
system regardless of what
the system requirements
indicate.

 | 

What is Functional Testing?

What is Structural Testing?


 
Advantages :
· Simulates actual system usage.
· Makes no system structure assumptions.

Disadvantages :
· Potential of missing logical errors in software.
· Possibility of redundant testing.
 | 

Advantages :
· Software¶s structure logic can be tested.
· Areas not covered in functional testing can be
tested.

Disadvantages :
· Doesn¶t ensure that software meets user
requirements.
· Its tests may not mimic real·world situations.
!" %
 
Following are considerations to convert the
developed test strategy, into test tactics or test
plan that will be followed in executing the day·to·
day testing.

1. Acquire and study the test strategy

2. Determine the type of development Project.
3. Determine the type of software system.
4. Determine the project scope.
!" %
 
Continued . . .

5. Identify the tactical risks.

6. Determine when testing should occur.
7. Build the system test plan.
8. Build the unit test plan.
!$(|
)
 !$(|
)
Common testing techniques used in evaluating
computerized applications are divided into three
categories:

(1) System structural testing techniques

(2) System functional techniques
(3) Unit testing techniques.

 We would also see the interrelationship between

the test factors and the techniques in order to
illustrate the purpose for which each of the
described techniques is most valuable.
 %

1. Structural s functional testing

2. Dynamic s static testing
3. Manual s automatic testing
| w
 
Structural Testing
Properties of TEST SET derived from the
program's internal structure.

Functional Testing
Properties of TEST SET derived from a
description of the program's function.
| w
 
Structural Testing
Structural analysis based test sets tend to
uncover errors that occur during coding of the
program.

Functional Testing
Functional analysis·based test sets tend to
uncover errors that occur in implementing
requirements or design specifications.
| w
 
Structural Testing
Structural testing ensures sufficient testing of the
implementation of a function.

It is used primarily during the coding phase,

structural analysis should be used in all phases
of the life cycle where the software is
represented formally in some algorithmic,
design, or requirements language.
| w
 
Continued . . .
The intent of structural testing is to assess the
implementation by finding test data that will force
sufficient coverage of the structures present in
the implemented application.

Structural testing evaluates both that all aspects

of the structure have been tested and that the
structure is sound. Determining that all tasks
through a structure are tested is a difficult
process and one requiring extensive test data.
However, determining if the structure functions
properly is a test task that is more easily
accomplished.
| w
 
Functional Testing
Functional testing ensures that the requirements
are properly satisfied by the application system.
The functions are those tasks that the system is
designed to accomplish.

Functional testing is not concerned with how

processing occurs, but rather, with the results of
processing.
 !$w|
TEST METH
DS can be classified into Dynamic
and Static techniques.

Dynamic Testing
Dynamic analysis requires that the program be
executed and hence involves the traditional
notion of program testing.
i.e. The program is run on some test cases and
the results of the program's performance are
examined to check whether the program
operated as expected.
 !$w|
Static Testing
Static analysis does not usually involve actual
program execution. Common static analysis
techniques include such tasks as syntax
checking.
 !$w|
Dynamic Testing
Exhaustive Testing:
It is a Dynamic analysis technique.
It is infinite and infeasible.

Static Testing ± Carried out during initial phases

of SDLC

Dynamic Testing ± Carried out during Later

phases of SDLC.
 !$w|
· Find criteria for choosing representative test
cases from the total population of test
conditions.

· ormally a ë
  
ë
 
ë

 is used, selecting a reasonable subset of
test conditions to provide a high probability that
the system will perform correctly in an
operational status.
 w$
· Manual techniques are performed by people,
and automated techniques by the computer.

· This classification is made on the basis of

whether the method is a manual one, such as
structured walkthrough or code inspections, or
whether the method is automated.
 w$
· The more automated the developmental
process, the easier it becomes to automate the
test process.

·
n the other hand, the greater reliance on

people to analyze, document, and develop
computer system manually, the more it becomes
necessary to test manually.
 ?
 $(#„

? 


„ $)*#
   
 

 (%„„#

%„..„#$ %(.#$

$%)(


$*/. .*/. $*/.

%# # # %# # %#

$ $ $
* * *

$#.#$ $#.#$ $#.#$ $#.#$ $#.#$ $#.#$
„ „ „ „ „ „
#.# #.# #.# #.# #.# #.#
 | |$
)
· Structural system testing is designed to verify
that the developed system and programs work.

· The objective is to ensure that the product

designed is structurally sound and will function
correctly.

· It attempts to determine that the technology has

been used properly and that when all the
component parts are assembled they function as
a cohesive unit.
 | |$
)
Continued . . .

· The structural system testing techniques provide

the facility for determining that the implemented
configuration and its inter·relationship of parts
functions so that they can perform the intended
tasks.

· The techniques are not designed to ensure that

the application system is functionally correct, but
rather, that it is structurally sound.
*/. „) -#)$
Determine system performs Sufficient disk space
ST„ESS with expected volumes allocated
Communication lines
adequate
System achieves desired level Transaction turnaround time
EXECUTI
of proficiency adequate
Software/hardware use
optimized
System can be returned to an Induce failure
„EC
E„Y operational status after a failure Evaluate adequacy of
backup data
System can be executed in a Determine systems can run

PE„ATI
S normal operational status using document
JCL adequate
System is developed in Standards followed
C
MPLIA CE accordance with standards and Documentation complete
procedures
System is protected in Access denied
SECU„ITY accordance with importance to Procedures in place
organization

 |$
)
· Functional system testing is designed to ensure
that the system requirements and specifications
are achieved.

· The process normally involves creating test

conditions for use in evaluating the correctness
of the application.
 */. „) -#)$
„/.„ System performs as
specified
„„ erifies that anything
unchanged still performs
correctly
„„„*# $ Errors can be prevented or
detected, & then corrected

Prove system requirements
Compliance to policies,
regulations
Unchanged system segments
function
Unchanged manual
procedures correct
Error introduced into test
Errors re·entered

#.#$.))„ The people·computer Manual procedures developed

interaction works People trained

„% Data is correctly passed Intersystem parameters

from system to system changed
Intersystem docs updated
„$ Controls reduce system risk File reconciliation procedures
to an acceptable level work
Manual controls in place
)#„#$$$
ld & new system are run & 
ld & new system can
the results compared to reconcile
detect unplanned difference 
perational status of old
system maintained
)0
?

· A Team