Академический Документы
Профессиональный Документы
Культура Документы
Overview
By
Some slides are in courtesy of Dr. Paul Ammann and Dr. Jeff Offutt.
Contents
Testing
Concepts
Unit
Testing
Integration Testing
Object-Oriented Testing
Black-box Testing
White-box Testing
Model-Driven
Test Design
Criteria-Based
Test Design
Testing Concepts
Objective of Testing
Testing
Testing
Software
Software
Failures
The
The
A Concrete Example
Fault: Should
start searching
at 0, not 1
public static int numZero (int [ ] arr)
Test 1
{ // Effects: If arr is null throw
[ 2, 7, 0 ]
NullPointerException
Expected:
// else return the number of occurrences of
1
0 in arr
Actual: 1
Error: i is 1, not
int count = 0;
Test 2
0,
on
the
first
for (int i = 1; i < arr.length; i++)
[ 0, 2, 7 ]
iteration
{
Expected:
Failure:
none
if (arr [ i ] == 0)
1
{
Error: i is 1, not 0
Actual: 0
count++;
Error propagates to the variable
}
count
}
Failure: count is 0 at the return
return count;
statement
}
7
destruct on maiden
flight (64-bit to 16-bit
conversion: about
370 million $ lost)
Performance
Indication of
Quality
10
Developer
Understands the
system,
but will test "gently,
and
testing is driven by
"delivery"
Independent Tester
Must learn about the
system, but will attempt to
break it, and testing is
driven by quality
11
Not
If
Planning
V Model
System
Engineering
Analysi
s
Desig
n
Codin
g
System test
Validation test
Integration test
Unit test
15
15
Unit Testing
Concept
16
Unit Testing
What
to test
Module
Interface
Local Data Structure
Boundary Conditions
Independent Paths
Error-Handling Paths
Module
Module
Test
Test Case
Case
17
Unit Testing
Test
Driver
Stub
A
18
Unit Testing
Unit
Testing Environment
Module
Module
Stub
Stub
Test
Test
Driver
Driver
Stub
Stub
Test Results
Test
Test Case
Case
Module Interface
Local Data Structure
Boundary Conditions
Independent Paths
Error-Handling Paths
19
Unit Testing
Unit
20
Integration Testing
Concept
Once
Integration
21
Integration Testing
The
Incremental
Integration
The
Top-Down Integration
Bottom-Up Integration
22
Integration Testing
Top-Down
Testing
Modules
Bottom-Up
Testing
Begins
23
Integration Testing
Top-Down
M1,
M2, M5, M8
M6
M3,
M
M11
M7
M4
Top-Down
Testing
with Breath-First
M
M22
M
M33
M4
M1
M2,
M3, M4
M5, M6, M7
M8
M
M55
M
M66
M7
M
M88
24
C
As new modules are integrated,
some subset of tests is re-run.
25
Bottom-Up Integration
A
B
C
D
Cluster
26
testing
Focus is on software requirements
System
testing
Focus is on system integration
Alpha/Beta
testing
Focus is on customer usage
Recovery
testing
Forces the software to fail in a variety of ways and verifies that recovery is prop
erly performed
Security
testing
Verifies that protection mechanisms built into a system will, in fact, protect it fr
om improper penetration
Stress
testing
Executes a system in a manner that demands resources in abnormal quantity, fr
equency, or volume
Performance
Testing
Test the run-time performance of software within the context of an integrated
system
27
Testing
Conducted
Beta
Testing
Conducted
rs.
Developers generally are not present.
28
Exhaustive Testing
Exhaustive
Testing is impossible.
Testing
Example>
Application with 10 input fields where each can have size possible values
(number of combinations: 6 10)
If it takes 1 millisecond to run each test case, it will take about 2 years to c
omplete testing this application.
29
Selective Testing
Selected path
30
Black-Box Testing
Requirements
Output
Input
Events
31
Black-Box Testing
Concept
It
32
Black-Box Testing
Key
Problem
To
In
n
ai
Input
Input test
test Data
Data
Ie
Inputs causing
anomalous behavior
System
System
Outputs which reveal
the presence of defects
e
g
n
a
R
Oe
Output
Output test
test Results
Results
33
Black-Box Testing
Equivalence
Partitioning
Input
Identify
34
In
p
u
ts
Black-Box Testing
ion
t
i
rt
a
P
O
u
tp
ts
System
System
35
Black-Box Testing
Test
Choose
36
Input Values
11
10
9999
10000
50000
100000
99999
Test
White-box Testing
Concept
A
38
White-box Testing
Path
Testing
39
White-box Testing
Flow
Graph Representations
if-then-else
loop-while
case-of
40
White-box Testing
void
Binary_Search (elem key, elem*t, int size, boolean &found, int &L) {
int bott, top, mid;
bott = 0;
top = size - 1;
L = (top + bott) / 2;
if (T[L] == key)
found = true;
2
else
found = false;
3
while (bott <= top && !found) {
mod = (top + bott) /2;
if (T[mid] == key) {
4
5
found = true;
L = mid;
6
} else
if (T[mid] < key)
8
bott = mid + 1;
else
bott = mid - 1;
} // while
}
41
White-box Testing
Independent
Program Path
5
6
7
8
42
43
Like
ss
The model is an abstract structure
44
45
46
RIPR Model
Reachabilit
y
Infection
Propagatio
n
Revealabili
ty
Test
Reache
s
Fault
Infect
s
Incorre
ct
Progra
m
State
Reveal
s
Test
Oracle
s
48
main Class P
Class A
Class B
method mA1()
method mB1()
method mA2()
method mB2()
Acceptance
testing : Is the
software
acceptable to the
user?
System testing :
Test the overall
functionality of
the system
Integration
testing : Test
how modules
interact with
each other
Module testing
(developer
testing) : Test
each class, file,
module,
Unit
testing
component
(developer
testing) : Test
each unit
49
Class A
Inter-class
testing : Test
multiple classes
together
Class B
method mA1()
method mB1()
method mA2()
method mB2()
Intra-class testing
:
Test an entire
class as
sequences of calls
Inter-method
testing : Test
pairs of methods
in the same class
Intra-method
testing : Test each
method
individually
50
Coverage Criteria
Even
ems
Coverage
Source,
Make
Gives
shed
52
Test
3. Logic expressions
4. Syntax
descriptions
53
54
Most
mathematical
Most technically challenging
55
vities
1.
2.
3.
4.
Test Design
Test Automation
Test Execution
Test Evaluation
1.a) Criteria-based
1.b) Human-based
Each
56
1.
knowledge of :
Discrete
math
Programming
Testing
Requires much of a traditional CS degree
This is intellectually stimulating, rewarding,
and challen
ging
Test design is analogous to software architecture on th
e development side
Using people who are not qualified to design tests is a s
ure way to get ineffective tests
57
1.
ons
Requires knowledge of :
Domain,
Requires
almost no traditional CS
ging
2. Test
Automation
Programming
59
3.
Test Execution
Interns
Employees
If,
Test
4.
Test Evaluation
Usually
62
test
requirements
test
requirements
software
artifact
refined
requirements /
test specs
DESIGN
ABSTRACTION
LEVEL
IMPLEMENTATION
ABSTRACTION
LEVEL
pass /
fail
test
results
input
values
test
scripts
test
cases
63
4
return i
64
Example (cont.)
Graph
Abstract version
1
2
3
5
6 requirements
Edges
for Edge-Pair
12
Coverage
23
1. [1, 2, 3]
32
2. [1, 2, 5]
34
3. [2, 3, 4]
25
Initial Node: 1 4. [2, 3, 2]
5. [3, 2, 3]
Final Nodes:
6. [3, 2, 5]
4, 5
Test Paths
[1, 2, 5]
Find values
[1,
2,
3,
2,
5]
4
[1, 2, 3, 2, 3,
4]
65
66
es
Unit,
67
67
68
Source of Structures
These
This
The
1. Input Domain
Characterization
(sets)
A: {0, 1, >1}
B: {600, 700, 800}
C: {swe, cs, isa, infs}
2. Graphs
3. Logical Expressions
4. Syntactic Structures
(grammars)
if (x > y)
z = x - y;
else
z = 2 * x;
70
Lemon
Pistachio
Cantaloupe
Pear
Tangerine
Apricot
Colors :
1. Yellow (Lemon,
Apricot)
2. Green (Pistachio)
3. Orange (Cantaloupe,
Tangerine)
4. White (Pear)
Coverage
Given a set of test requirements (TR) for
coverage criterion , a test set (T) satisfies
the coverage if and only if for every test
requirement tr in TR, there is at least one
test t in T such that t satisfies tr
Infeasible
Most
tice
73
Coverage Level
The ratio of the number of test
requirements satisfied by T to the size
of TR
T2
ts
74
Criteria
Must
Examples
The
75
Advantages of
Criteria-Based Test Design
76
2.
3.
77
ntensive
Formal
More
Greater
Criteria
Graphs
Logic
Applied to
Applied
to
Source
Specs
Source
Specs
Design
Syntax
Use cases
Applied
to
FSMs
DNF
Source
Models
Integ
Input
79