Software Testing

What is Testing
It has been said that "approximately 50% of the elapsed time and over 50% of the total cost are expended in testing a program or system being developed. In the simplest terms, Software Testing amounts to observing the execution of a software system to validate whether it behaves as intended and identify potential malfunctions. [IEEE]

What is Testing
Testing is the process of executing a program or system with the intent of finding errors.
Myers Art of Software Testing

Testing is the process of establishing confidence that a program or system does what it is supposed to.
Hetzel 1973

What is Testing
Software Testing is the Process of Exercising or evaluating a system by manual or automated means to verify that it satisfies specified requirements or to identify differences between expected and actual results. IEEE Standard Glossary 1983

Practitioners' Views of Software Testing

Checking programs against specifications Finding bugs in programs Determining user acceptability Ensuring that a system is ready for use Gaining confidence that it works Showing that a program performs correctly Demonstrating that errors are not present Understanding the limits of performance Learning what a system is not able to do Evaluating the capabilities of a system Verifying documentation Convincing oneself that the job is finished and so forth.

Insights from a Venn Diagram

Specification faults of omission What if there are specified behaviors that have not been programmed? Program faults of commission what if there are programmed (implemented) behaviors that have not been specified?

Test Cases

A very good view of testing is that it is the determination of the extent of program behavior that is both specified and implemented What can a tester do to make the region where these sets all intersect be as large as possible? How the test cases in the set T are identified?.

Exploring Software Testing

Good testing involves much more than just running the program a few times to see whether it works. Thorough analysis of the program helps us to test more systematically and more effectively.

For example, consider a program that add two numbers, entered by the user. Each number is of one or two digits. The program shall read the inputs and print the sum. Press <enter> after each number.

Exploring Software Testing

#include<stdio.h> void Main() { int a, b, c; scanf(%2d,&a); scanf(%2d,&b); c=a+b; printf(%2d\n,c); }

First cycle of Testing

Step I Add 2 & 3 5 will be displayed

Problem Reports After First Test

The program worked in the sense that it accepted 2 & 3, and returned 5. But it still has problems. Design Errors:- No On screen Instructions. How do we know what to do? Instructions could easily be displayed on the screen. The errors are described on problem report form.

TESTS OF VALID INPUT

Sr. No. Test Case Expected Results Notes

1.
2.

99 + 99
-99 +(-99)

198
-198

Largest pair of numbers the program can add The specification did not say we cannot use negative numbers.

TESTS OF VALID INPUT Contd

3. 99 + (-14) 85 A large first number might affect the programs interpretation of the second Check addition of a negative to a positive number. Large second numbers effect on first. 9 is largest one digit number Programs often fail on zero. 0 should be tested in first and second entry position

4. 5. 6. 7. 8. 9.

-38 + 99 56 + 99 9+9 0+0 0 + 23 -78 + 0

61 155 18 0 23 -78

Summarize thoughts about the program

The communication style of the program is very terse. The program does not deal with two digit negative numbers. The largest sum that it can handle is 198 and the smallest is 0. The program accepts any character as valid input, until we press <enter>. If we type anything that is not a digit it gives a garbage value.

Remember: A best tester is not the one who finds the most bugs

or who embarrasses the most programmers. The best

tester is the one who gets the most bugs fixed.

Software Testing Self Assessment Test

The program reads three integer values. The three values are interpreted as representing the lengths of the sides of a triangle. The program prints a message that states whether the triangle is Scalene, Isosceles, or Equilateral Write the set of Test Cases i.e. specific sets of data that you feel would adequately test this program

Software Testing Self Assessment Test

Do you have a test case that represents a valid scalene triangle ? Test cases such as 1,2,3 and 2,5,10 is not correct Do you have a test case that represents a valid equilateral triangle Do you have a test case that represents a valid isosceles triangle ? Do you have at least 3 test cases that represent valid isosceles triangles such that you have tried all three permutations of two equal sides e.g. 3,3,4 3,4,3 and 4,3,3

Software Testing Self Assessment Test

Do you have a test case in which one side has a zero values? Do you have a test case in which one side has a negative value? Do you have a test case with three integers greater than zero such that the sum of two of the numbers is equal to the third? (That is, if the program said that 1,2,3 represents a scalene triangle, it would contain a bug.) Do you have at least three test cases in above category such that you have tried all three permutations where the length of one side is equal to the sum of the lengths of the other two sides (e.g. 1,2,3; 1,3,2; and 3,1,2)? Do you have a test case with three integers greater than zero such that the sum of two of the numbers is less than the third (e.g. 1,2,4 or 12, 15, 30)?

Software Testing Self Assessment Test

Do you have a least three test cases in above category such that you have tried all three permutations (e.g., 1,2,4; 1,4,2; and 4,1,2)? Do you have a test case in which all sides are 0 (I.e., 0,0,0)? Do you have at least one test case specifying no integer values? Do you have at least one test case specifying the wrong number of values (e.g., two, rather than three, integers)? For each test case, did you specify the expected output from the program in addition to the input values.

Sample Test Case

Assuming Integer Greater than 0 and less than or equal to 200 Test Case
1 2 3 4 5

X
100 100 100 100 1

Y
100 100 100 1 1

Z
1 2 200 100 1

Expected Output Isosceles Isosceles Not a triangle Isosceles Equilateral

6
7 8 9

1
2 100 100

2
199 199 200

100
200 200 2

Not a triangle
Scalene Scalene Not a triangle

10
11 12

100
100 199

200
2 100

200
200 100

Isosceles Scalene Isosceles

Conclusion
When one tests a program, one wants to add some value to the program. Adding value means raising the quality or reliability of the program. Raising the reliability of the program means finding and removing errors. Therefore, one should not test the program to show that it works; rather, one should start with the assumption that the program contains errors and then test the program to find as many of the errors as possible. Does testing implies that it is destructive process of trying to find the errors ?

Conclusion - What Testing Shows

errors requirements conformance
performance

an indication of quality

Conclusion
"Bugs lurk in corners and congregate at boundaries
Boris Beizer

OBJECTIVE CRITERIA

to uncover errors in a complete manner

CONSTRAINT with a minimum of effort and time

Papers to Study
J.A. Whittaker, What is software testing And why is it so hard?, IEEE software, January / February, 7077, 2000. J.L. LIONS et. al., Report of the enquiry board constituted by Director General of ESA for the identification of the causes of failure, www.esrin.esa.it, July 19, Paris, 1996.

Definitions

Error
People make errors. A good synonym is mistake. When people make mistakes while coding, we call these mistakes bugs.

Errors tend to propagate; a requirements error may be magnified during design and amplified still more during coding.

Error
Typographical error

Misreading of a specification

Misunderstanding of functionality of a module

Fault
A fault is the result of an error. It is more precise to say that a fault is the representation of an error, where representation is the mode of expression, such as
narrative text dataflow diagrams hierarchy charts source code, and so on.

Failure
A failure occurs when a fault executes. A particular fault may cause different failures, depending on how it has been exercised.

Incident
When a failure occurs, it may or may not be readily apparent to the users. An incident is the symptom(s) associated with a failure that alerts the user to the occurrence of a failure

Test
A test is the act of exercising software with test cases. A Test has two distinct goals
To find failures or To demonstrate correct execution

Test Case
A test case has an identity and is associated with a program behavior. It has a set of inputs and a list of expected outputs.

Inputs

Preconditions
Outputs Post Conditions

Actual Inputs

Actual Outputs

Typical Test Case

Test Case ID: Purpose: Preconditions: Inputs: Expected Outputs: Post conditions Execution History

Date

Result

Version

Run By

Testing Life Cycle

Error Fix Fault Resolution

error

Requirement Specification Fault Design Fault

Error

Error

Fault Isolation

Coding Fault Testing

Fault Classification Incident

Testing Life Cycle

The first three phases are Putting Bugs in The testing phase is Finding Bugs The last three phases are Getting Bugs OUT The Fault Resolution Step is another opportunity for errors

Verification and Validation

Verification the process of determining whether or not the product of given phase fulfill the spec. from the previous phase Validation the process of evaluating the software at the end of the development to ensure the compliance with the requirements

Classification of V&V Activities

Most of the V&V activities can be classified as static or dynamic Static
Reviews program proving code reading

Dynamic Testing by Executing the Code

What is Testing
Software Testing is the Process of Exercising or evaluating a system by manual or automated means to verify that it satisfies specified requirements or to identify differences between expected and actual results. IEEE Standard Glossary 1983

Testing Principles
Testing Shows Presence of Defects
Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness.

Exhaustive Testing Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risks and priorities to is Impossible
focus testing efforts.

Early Testing

Testing activities should start as early as possible in the software or system development life cycle and should be focused on defined objectives.

Testing Principles
Defect Clustering
A small number of modules contain most of the defects discovered during prerelease testing or show the most operational failures. If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new bugs. Tb overcome this 'pesticide paradox', the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.

Pesticide Paradox

Testing is done differently in different contexts. For example, safety-critical Testing is the Context Dependent software is tested differently from an . e-commerce site.

Absence of Error Fallacy

Finding and fixing defects does not help if the system built is unusable and does not fulfill the users' needs and expectations

Why Testing is So Hard

To plan and execute test, software tester must consider
The software and the function it computes The inputs and how they can be combined The environment in which the software will eventually operate This difficult, time-consuming process requires technical sophistication and proper planning.

Why Testing is So Hard

As a new tester, You might believe that you can approach a piece of software, fully test it, find all the bugs, and assure that the software is perfect. Unfortunately, this isnt possible even with the simplest program. Why?

 It is difficult, due to following four reasons: 1. The number of possible input is very large. 2. The number of possible output is very large 3. The number of paths through the software is very large 4. The software specification is very subjective

Why Testing is So Hard

 How many test cases does it take to fully cover, or exercise, the source code? There are 2n 1 ways, where n is the number of conditions in the statement If the n is moderately large, imagine the number of test cases required to test the program For example, to test a clock application, How many test cases are required to test different times in a day?
12 hours x 60 minutes x 60 seconds x 2 am/pm for a total of 86,400 different input values. Thats just the valid values

Why Testing is So Hard

Limitations of Testing
We want to test everything before giving the software to the customers. What do we understand, when we say everything? We may expect one, two or all of the following when we refer to everything:
(i) Execute every statement of the program (ii) Execute every true and false conditions (iii) Execute every condition of a decision node (iv) Execute every possible path (v) Execute the program with all valid inputs (vi) Execute the program with all invalid inputs

Limitations of Testing
It is impossible to achieving these six objectives due to time and resource constraints as We may achieve few of them with limited coverage. If we do any compromise, we may miss a bug. Hence Everything is impossible and we have to settle for less than everything in the real life situations.

Functional Testing

Functional Testing/ Black Box Testing

Black Box Testing also called as Behavioral testing or Functional Testing An approach to testing where the program is considered as a black-box The program test cases are based on the system specification or the functional requirements of the software It is based on the view that any program can be considered to be a function that maps from its input domain to values in its output range Test planning can begin early in the Software process

Functional Testing / Black Box Testing

Input Test Data

System Under Test

Output Test Data

Black-Box Testing

requirements

output

input

events

Example: Lock and Key System

Functionality Features of a lock Feature of a key Action performed What you need to know to use It is made of metal, has a hole provision to lock, has a facility to insert It is made of metal and created to fit into a particular locks keyhole. Key inserted and turned clockwise to lock Key inserted and turned anticlockwise to unlock Locked Unlocked Key turned clockwise or anticlockwise Locked Unlocked

State

Inputs Expected outcome

Why Black Box Testing

Black box testing helps in the overall functional verification of the system under test. It helps in identifying any incomplete, inconsistent requirement as well as any issue involved when the system is tested as a complete entity. Black box testing address the stated requirements as well as implied requirements.

Functional Testing/Black Box Testing

Techniques Boundary Value testing Equivalence Class testing Decision Tables

Boundary Value Testing

How to use knowledge of the functional nature of a program to identify test case for the program Boundary Value testing has focused on the input domain, but help to develop range-base test cases Test values, sizes or quantities near the design limits Errors tend to occur near the extreme values of inputs Robustness: How does the software react when boundaries are exceeded?

Single Fault Assumption

The Single Fault assumption from reliability states: Failures are rarely the result of the simultaneous occurrence of two (or more) faults.

Boundary Value Testing

The basic idea of boundary value analysis is to use input variable values at their
minimum, just above the minimum, at a nominal value, just below the maximum, and at the maximum y
d

c a b x

Fig: Test Cases for Variables x and y

Case 1 : The Triangle Problem

The triangle program accepts three integers a,b and c, as input. These are taken to be sides of a triangle. The integers a,b, and c must satisfy the following condition :
C1: C2: C3: C4: C5: C6: 1 x 100 1 y 100 1 z 100 x y+z y x+z z x+y

Case 1: Triangle Problem Boundary Value Test Cases

Case 2 : The PrevDate Function

PrevDate is a function of three variables: month, date and year. It returns the date of the day before the input date. The month, date, and year variables have integer values subject to these conditions: C1 1 month 12 C2 1 day 31 C3 1900 day 2025

Case 2: The Previous Date Function Boundary Value Analysis Test cases

Limitations of Boundary Value Analysis

Boundary Value Analysis works well when the program to be tested is a function of several independent variable that represent bounded physical quantities. A quick look at the boundary value analysis test cases for PrevDate shows them to be inadequate. For example, Very little stress occurs on February and on leap years. The real problem here is that interesting dependencies exist among the month, day and year variable. Boundary Value Analysis test cases are rudimentary because they are obtained with very little insight and imagination

Adhoc Testing
Testing carried out using no recognized test case design technique. It occurs when the tester uses domain knowledge experience and information about soft spots to derive test cases Mostly intuitive and least uniform. Dependent on the abilities of the tester Also known as Special value testing/hacking, /outof-box testing,/ guerilla testing

Equivalence Class Testing

Equivalence Class Testing

The use of Equivalence Classes as the basis for Functional Testing has two motivations : We would like to have a sense of complete testing We would hope to avoid redundancy

Equivalence Class Testing

If we expect the same result from two test case, we consider them equivalent.
A group of test cases forms an equivalent class if :

They all test the same thing If one test catches a bug, the others probably will too.

If one test does not catch a bug, the others probably would not either.

Equivalent Classes
Important aspect of equivalence classes is that they form a partition of a set, where partition refers to a collection of mutually disjoint subsets when the union is the entire set. This suggest two things :
Entire set is represented provides a form of Completeness Disjointedness ensures a form of non-redundancy

The idea is to identify test cases by using one element from each equivalence class.

Equivalence Class Testing

Valid input Valid inputs System under test Outputs

Input domain

Output domain

Fig. : Equivalence partitioning

Most of the time, equivalence class testing classes of the input domain. However, equivalence classes should also be for output domain. Hence, we should equivalence classes based on input and domain.

defines
defined design output

Equivalence Class Testing

X2

b c

X1
d

Case 1: The Triangle Problem Equivalent Class Testing Output domain equivalence classes are:
O1={<x,y,z>: Equilateral triangle with sides x,y,z} O2={<x,y,z>: Isosceles triangle with sides x,y,z} O3={<x,y,z>: Scalene triangle with sides x,y,z} O4={<x,y,z>: Not a triangle with sides x,y,z}

Case 1: The Triangle Problem Weak Normal Equivalent Class Testing

Test Case WN1

WN2 WN3 WN4

a b c Expected Output 5 5 5 Equilateral

2 2 3 Isosceles 3 4 5 Scalene 4 1 2 Not a Triangle

Equivalence Class Testing

Equivalence Class Testing is clearly sensitive to the equivalence relation used to define classes. If we base equivalence classes on the input domain, we obtain a richer set of test cases

Equivalence Class Test

Equivalence class testing is appropriate when input data is defined in terms of intervals and sets of discrete values. Equivalence class testing is strengthened by a hybrid approach with boundary values testing. Strong equivalence class testing makes a presumption that the variables are independent and the corresponding multiplication of test cases raises issues of redundancy. If any dependencies occur, they will often generate error test cases.

Decision Table Based Testing

DECISION TABLE
A decision table is a matrix of rows and columns, rather than a tree, that shows conditions and actions. Decision rules, included in a decision table, state what procedure to follow when certain condition exist. Thus, we can call it a policy specification tool Decision Tables are most effective when problem is one of selecting a single decision alternative or set of alternatives out of many possibilities

DECISION TABLE
The decision table technique is indicated for the application characterized by any of the following prominent if-then-else logic Logical relationships among input variables High Cyclomatic complexity Cause-and-effect relationships between inputs and outputs

DECISION TABLE
DECISION TABLE QUADRANTS

QUADRANT I CONDITION STUB CONDITION STATEMENTS

QUADRANT III CONDITION ENTRY GRID Y or N

QUADRANT II ACTION STUB ACTION STATEMENTS

QUADRANT IV ACTION ENTIRY GRID X or nothing

DECISION TABLE
The decision table on the next slide describes the actions taken in handling payments from patients in a medical clinic. The actions taken depend on whether the patient has medical insurance and, if so, which type. Two types of insurance coverage are identified:
basic health insurance (condition 1) and social health insurance (condition 2). . Four rules relate the combinations of conditions 1 and 2

Three different actions:

the patient must pay the cost of the office call, but no other charges; the patient pays none of the charges; or the patient pays the full cost of treatment (office call and other charges).

Example
CONDITIONS DECISION RULES 1 2 3 4 Y N Y N N X X X X Y Y N

C1 Patient has basic health insurance C2 Patient has social health insurance A1 Pay amount of office call A2 Pay nothing A3 Pay full amount for services

DECISION TABLE
TITLE : The title of the decision table should clearly explain the purpose of the table CONDITION Statements : Then statements must be clear in their meaning and concise in their content RULES Each column of condition grid and action grid entries is called a rule. A rule is simply a statement that under a give set of condition, specific action must follow Number of Rules : The number of Rules which exist for a problem can be determined arithmetically as follows ; 1 Count the number of condition statements ( in this case 3) 2. Use the number arrived at in step 1 as the exponent of the number 2 3. Compute the number of rules 23 = 8

Decision Table for the Triangle Problem

Conditions c1: a, b, c from a triangle? c2: a=b? c3: a=c? c4: b =c? Actions a1: Not a triangle a2: Scalene a3: Isosceles a4: Equilateral a5: Impossible

N ---X

Decision Rules YY Y Y Y Y Y YY Y Y NNN YY NNY Y N YNY NY NY

Y N N N

X X X X X X X X

Refined Decision Table for the Triangle Problem

Conditions c1: a<b+c? c2: b<a+c? c3: c<a+b? c4: a=b? c5: a=c? c6: b=c? Actions a1: Not a triangle a2: Scalene a3: Isosceles a4: Equilateral a5: Impossible

F ------

T F -----

Decisions Rules T T T T T T T T T T T T T T F T T T T T T -- T T T T F F -- T T F F T T -- T F T F T F

T T T F F T

T T T F F F

X X X X X X X X X X X

Test Cases for Triangle Problem

Case ID
DT1 DT2 DT3 DT4 DT5 DT6 DT7 DT8 DT9 DT10 DT11

a
4 1 1 5 ? ? 2 ? 2 3 3

b
1 4 2 5 ? ? 2 ? 3 2 4

c
2 2 4 5 ? ? 3 ? 2 2 5

Expected O/P
Not a Triangle Not a Triangle Not a Triangle Equilateral Impossible Impossible Isosceles Impossible Isosceles Isosceles Scalene

Decision Table Based Testing

Number of Test Cases High

Low

Sophistication

Boundary Value

Equivalence class

Decision table

Decision Table Based Testing

High

Effort to identify Test Cases

Low

Sophistication

Boundary Value

Equivalence class

Decision table

Structural Testing

Structural Testing
Also called: White Box Testing Based on Source Code Can Be dynamic or static Necessary because:
Part of code not fully exercised. Section of code may be surplus to the requirements. Errors may be missed by functional requirements.

The Challenge of Testing

A major challenge in testing is to determine a good starting set of test cases that:
Eliminate redundant testing Provide adequate test coverage Allow more effective testing Make the most of limited testing resources

Problem: Too Many Paths There are typically many possible paths between the entry and exit of a typical software program. Every decision doubles the number of potential paths, every case statement multiplies the number of potential paths by the number of cases and every loop multiplies the number of potential paths by the number of different iteration values possible for the loop. Complete path coverage of even a simple unit is extremely difficult.

The Challenge Given that testing, like all other development activities has a limited amount of resources (time, people, equipment), the challenge is to select a set of test cases that is most likely to identify as many different potential defects as possible within those limits. To do this, we must eliminate as much redundancy as possible from the testing process while still insuring adequate test coverage.

Path Testing
Group of test Techniques This type of testing involves:
Generating a set of paths that will cover every branch in the program. Finding the set of test cases that will execute every path in this set of program paths.

Steps Involved in Path testing

Generate flow graph of a program. Calculate Cyclomatic Complexity Identify independent paths

Cyclomatic Complexity
Cyclomatic complexity is used to find the number of independent path through a program

This provides us the upper bound for the number of

test that must be conducted to ensure that all statements have been executed at least once and every condition have been executed on its true and false side. An independent path is any path through the program that introduce at least new set of processing statements or a new conditions.

Cyclomatic Complexity
Complexity is computed in one of three ways. 1. The number of regions of the flow graph correspond to the Cyclomatic complexity. 2. Cyclomatic complexity, V(G), for a flow graph, G, is defined as V(G) = E - N + 2 Where E is the number of flow graph edges, N is the number of flow graph nodes. 3. Cyclomatic complexity, V(G), for a flow graph, G, is also defined as V(G) = P + 1 where P is the number of predicate nodes contained in the flow graph G.

Cyclomatic Complexity
Calculate the Cyclomatic Complexity for the above figure and identify the independent path. 1. The number of regions = 5 2. Cyclomatic complexity, V(G), for a flow graph, G, is defined as V(G) = E - N + 2 = 10 7 + 2 = 5 3. Cyclomatic complexity, V(G), for a flow graph, G, is also defined as V(G) = P + 1 = 4+1 = 5

Cyclomatic Complexity

Cyclomatic Complexity

Risk Evaluation

1-10

a simple program, without much risk

11-20

more complex, moderate risk

21-50

complex, high risk program

greater than 50

untestable program (very high risk)

Practice Exercise

Sample Problem Draw the flow graph for the given problem? Derive the independent paths Compute the cyclomatic complexity Prepare Test Cases

procedure average TYPE value [1..100] IS SCALAR ARRAY TYPE average, total_input, total.valid, minimum maximum, sum IS SCALAR TYPE i is INTEGER i=1; Total _input = total _valid = 0 Sum = 0 DO WHILE value[ i] <> -999 AND total_input <100 increment total_input by 1; IF value[ i] >= minimum AND value[ i] <= maximum THEN increment total_valid by 1; sum = sum+ value[ i] ELSE skip endif increment I by 1; ENDDO IF total_valid > 0; THEN average = sum /total_valid; ELSE average = -999 ENDIF

procedure average TYPE value [1..100] IS SCALAR ARRAY TYPE average, total_input, total.valid, minimum maximum, sum IS SCALAR TYPE i is INTEGER i=1; Total _input = total _valid = 0 Sum = 0 DO WHILE value[ i] <> -999 AND total_input <100 3 4 increment total_input by 1; IF value[ i] >= minimum AND value[ i] <= maximum 6 THEN increment total_valid by 1; 5 sum = sum+ value[ i] 7 ELSE skip endif increment I by 1; 8 ENDDO 9 > 0; IF total_valid THEN average = sum /total_valid; 10 ELSE average = -999 11 ENDIF 12 13

1.

Flow Graph for the Procedure Average

1 2 3 4 10 12 13 8 11 5 6 7

Deriving Test Cases

2. Determine the Cyclomatic complexity of the resultant flow graph.

V(G) = 6 regions V(G) = 17 edges - 13 nodes + 2 = 6

Deriving Test Cases 3. Determine a basis set of linearly independent paths.

The value of V(G) provides the number of linearly independent paths through the program control structure. In the case of procedure average, we expect to specify six paths:

path 1: path2: path3: path4: Path5: path6:

1-2-10-11-13 1-2-10-12-13 1-2-3-10-11-13 1-2-3-4-5-8-9-2-. 1-2-3-4-5-6-8-9-2- 1-2-3-4-5-6-7-8-9-2-...

Deriving Test Cases 3. Determine a basis set of linearly independent paths.

The value of V(G) provides the number of linearly independent paths through the program control structure. In the case of procedure average, we expect to specify six paths:

path 1: path2: path3: path4: Path5: path6:

1-2-10-11-13 1-2-10-12-13 1-2-3-10-11-13 1-2-3-4-5-8-9-2-. 1-2-3-4-5-6-8-9-2- 1-2-3-4-5-6-7-8-9-2-...

Deriving Test Cases

Prepare test cases that will force execution of each path in the basis set. Data should be chosen so that conditions at the predicate nodes are appropriately set as each path is tested. Test cases that satisfy the basis set just described are Path 1 test case: value(k) = valid input, where k<i for 2 i 100 value(i) = -999 where 2 i 100 Expected results: correct average based on k values and proper totals Note: Path 1 cannot be tested stand-alone but must be tested as part of path 4, 5 and 6 tests.

Deriving Test Cases

Path 2 test case: value(1) = -999 Expected results: Average = -999, other totals at initial values. Path 3 test case: Attempt to process 101 or more values First 100 values should be valid Expected results: Same as test case 1. Path 4 test case: value(i) = Valid input where i < 100 value (k) < minimum where k < i Expected results: Correct average based on k values and proper totals.

Deriving Test Cases

Path 5 test case: Value(i) = valid input where i < 100 Value(k) > maximum where k <=i Expected results: Correct average based on n values and proper totals Path 6 test case: Value(i) = Valid input where i < 100 Expected results: Correct average based on n values and proper totals.

Data Flow Testing

Data Flow Testing

Data-flow Testing uses the control flow graph to explore the unreasonable things that can happen to data Consideration of data-flow anomalies leads to test path selection strategies that fill the gaps between complete path testing and branch and statement testing.

Data Flow Testing: What is it

Data-flow Testing is the name given to a family of test strategies based on selecting paths through the programs control flow in order to explore sequences of events related to the status of data objects. For example, pick enough paths to assure that every data object has been initialized prior to use or that all defined objects have been used for something

Data Flow Testing

Define / Reference Anomalies: 1. A variable that is defined but never used (referenced). 2. A variable that is used but never defined. 3. A variable that is defined twice before it is used. These anomalies can be discovered by static analysis.

Data Flow Testing

Limitations of Static Analysis : Arrays Arrays are problematic in that the array is defined or killed as a single object, but reference is to specific locations within the array. Array pointers are usually dynamically calculated, so theres no way to do a static analysis to validate the pointer value. And even if the pointer is within bounds, how do we know that the specific array element accessed has been initialized ? In many languages, dynamically allocated arrays contain garbage unless explicitly initialized and therefore, -u anomalies are possible.

Data Flow Testing

Limitations of Static Analysis : Records and Pointers : In many applications we create files and their names dynamically and theres no way to determine, without execution, whether such objects are in the proper state on a given path or, for that matter, whether they exist at all

Dynamic Analysis Dynamic analysis is done on the fly as the program is being executed and is based on intermediate values that result from the program execution

Data Object State and Usage

Data objects can be created, killed and/or used. Data Objects can be used in two distinct ways : In a calculation as part of a control flow predicate
d-defined, created, initialized etc. k-killed, undefined, released u- used for something c-used in calculation p-used in predicate

Data Object State and Usage

Defined : An object is defined explicitly when it appears in a data declaration or implicitly when it appears on the left-hand side of an assignment statement. Defined can also be used to mean that a file has been opened, a dynamically allocated object has been allocated, something is pushed onto the stack etc.

Data Object State and Usage

Killed or Undefined : An object is killed or undefined when it is released or otherwise made unavailable, or when its contents are no longer known with certitude. For example, the loop control variable is undefined when the loop is exited. An assignment operator statement can simultaneously kill and (re)define.

Data Object State and Usage

Usage : A variable is used for computation (c) when it appears on the right-hand side of an assignment statement, as a pointer, as part of pointer calculation, a file record is read or written and so on It is used in a predicate (p) when it appears directly in a predicate, but also implicitly as the control variable of a loop

Data Object State and Usage

There are nine possible two-letter combinations for d,k and u. Some are bugs, some are suspicious and some are okay. dd probably harmless but suspicious. Why define the object twice without an intervening usage ? dk probably a bug. Why define the object without using it ? du the normal case. The object is defined, then used kd normal situation. An object is killed, then redefined.

kk harmless but probably buggy. Did you want to be sure it was really killed ? ku- a bug. The object doesnt exist in the sense that its values is undefined or indeterminate ud usually not a bug because the language permits reassignment at almost any time. uk normal situation uu normal situation

Data Object State and Usage

Data Object State and Usage

In addition to the above two letter situations, there are six single-letter situations. Leading dash is used to mean that nothing of interest (d,k,u) occurs prior to the action noted and trailing dash to mean that nothing happens after the point of interest to the exit -k : possibly anomalous, we are killing a variable that does not exist. -d : This is just the first definition along this path -u : possible anomalous k- : not anomalous

Data Object State and Usage

d- : possibly anomalous. The variable was defined u- : not anomalous. The variable was used but not killed on this path.