Вы находитесь на странице: 1из 29

CmpSci 5/621 Fall06

10/17/06

CMPSCI 521/621 - Lecture 11


Regression Testing Wrapup Intro to OOS

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Reading Assignments
Marick Brian, "Notes on Object-Oriented Testing: Part 1: Fault-Based Test Design," available at http://www.testing.com/writings/1-fault.htm Marick Brian, "Testing FoundationsNotes on Object-Oriented Testing: Part 2: Scenario-Based Test Design," available at http://www.testing.com/writings/2-scen.htm Binder, Robert V. "Testing Object-Oriented Systems: a Status Report," American Programmer, v 7, n 4, April 1994. 22-28 available at http://www.rbsc.com/pages/ootstat.html A. Orso. Integration Testing of Object-Oriented Software. PhD thesis, Politecnico di Milano, Milano, Italy, 1998 http://wwwstatic.cc.gatech.edu/~orso/papers/orso.thesis.pdf Harrold, M. J., McGregor, J. D., and Fitzpatrick, K. J. 1992. Incremental testing of object-oriented class structures. In Proceedings of the 14th international Conference on Software Engineering (Melbourne, Australia, May 11 - 15, 1992) Ugo A. Buy and Alessandro Orso and Mauro Pezze,"Automated Testing of Classes," Proceedings of the International Symposium on Software Testing and Analysis,2000 H. Y. Chen, T. H.Tse, F. T.Chan, and T. Y.Chen,"In black and white: an integrated approach to class-level testing of object-oriented programs," ACM Transactions on Software Engineering and Methodology, 7(3):250{295, July 1998 R.-K. Doong and P. G. Frankl, "The ASTOOT approach to testing object-oriented programs.," ACM Transactions on Software Engineering and Methodology, 3(2):101{130, April 1994 C. D. Turner and D. J. Robson. The state-based testing of object-oriented programs. In International Conference on Software Maintenance, pages 302{310,Mont eal, Quebec (Canada), September 1993. IEEE Society Press.
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

CmpSci 5/621 Fall06

10/17/06

Announcements

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Outcomes - Regression Testing


Do, Rothermel, & Kinneer Rest-all and random are suboptimal strategies Block & Method Coverage similar and better than Retest-all, but mixed wrt random Feedback provides greater improvement, but not modification Graves, Harrold, Kim, Porter, and Rothermel, Regression testing is usually not more effective than the original test set For minimization, random, and test-all, the -larger the test suite size the better the fault detection -- improvement diminishes as the % gets higher Safe test suite size averaged 60% of original, but only performed slightly better than random(75%) Chen, Rosenblum, Vo Results depended on where a change was made Changes to leaf components required few test cases Changes to root node required all test cases
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

CmpSci 5/621 Fall06

10/17/06

Another experiment
Ostrand, Weyuker, Bell AT&T Studied release data over several years Two real systems from AT&T Inventory control, 17 releases over 4 years, full lifecycle info Provisioning System, 9 releases, over 2 years, post unit testing info only Used the data to develop a model to predict where the errors would be in a new release Could predict which 20% of the files would have 80% of the bugs Used a simple prediction model (LOC) and still could predict where 75% of the bugs would be found

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Observations
Could not use the error severity levels for predictions because they were unreliable Value was often political, not technical About 3/4 of the faults were found during unit testing LOCS & # files increased with each release Fault density (faults/KLOCS) decreased with each release

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

CmpSci 5/621 Fall06

10/17/06

Prediction Model
Used a negative binomial regression model LOC and the file's change status were the strongest individual predictors in the model Number of changes since the last release did not improve the model

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Results

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

CmpSci 5/621 Fall06

10/17/06

Results File Prediction => 80% of the faults

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Results
LOC+ almost as good a predictor as the full model

Prediction only using faults found after unit testing

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

CmpSci 5/621 Fall06

10/17/06

Provisioning System

Prediction by Model

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Conclusion from Study


Applied to two programs with similar results Appears to generalize Post-unit testing information was adequate for prediction LOC and change info was not as good as the more complicated prediction algorithm but it was pretty good

Remember: Not a safe technique

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

CmpSci 5/621 Fall06

10/17/06

Regression testing Conclusion


Regression testing a serious problem for some systems Want to reduce test suite size but not fault detection Need prediction models to select test cases Prediction models Safe selection techniques might still return too many test cases
Need to combine with priority techniques

Simple selection techniques, such as LOC and change info might be sufficient

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Regression Testing
Determine the set of safe test cases This can be run as a background job Cost is basically irrelevant Select from these safe test cases % selected depends on resources available Rerun selected test cases
Cost of executing test cases Cost of evaluating the results
Can often be automated

Select new test cases to exercise new functionality

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

CmpSci 5/621 Fall06

10/17/06

Testing Object Oriented Systems


Do OO systems make testing harder or easier? Does code reuse lead to test reuse? Do we need to change existing techniques? If so, how? Do we need to develop new techniques?

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Object Oriented Programming


Supports abstract data types (ADTs) Information hiding Encapsulation Supports inheritance Change to a parent type is reflected in the children Supports reuse Subtype or Subclass
Subclass- reuse implementation information Subtype-child type must be a legal member of the parent type

Supports dynamic binding/polymorphism

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

CmpSci 5/621 Fall06

10/17/06

What can we use?


possible to apply traditional testing techniques to object oriented problems, but object-orientation characteristics introduce concerns system testing in OOS, e.g., problems related to robustness, security, memory leaks analogous to traditional systems, and no special approach is required what about unit and integration testing? six critical features have to be considered: information hiding, shadow invocations, polymorphism and dynamic binding,conversions, inheritance, and genericity.

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Unit Testing
in procedural programming the basic component is the subroutine and the testing method for such component is input/output based in object-oriented programming, the basic component is represented by a class, composed of a data structure and a set of operations objects are run-time instances of classes the data structure defines the state of the object taht is modified by the class operations (methods) correctness of an operation is based not only on the input/output relation, but also on the initial and resulting state of the object in general the data structure is not directly accessible, but can only be accessed using the class public operations thus, encapsulation and information hiding make it difficult to check what happens inside an object during testing
Adapted from A. Orsos PhD thesis
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

CmpSci 5/621 Fall06

10/17/06

Information Hiding
class Watcher{ private: ... int Current _Value; int Last_ Value; int Status; void check_ pressure(); void alarm( int); public: void start(); } void check_ pressure { ... Last_ Value = Current_ Value; Current_ Value = reactor. temperature; if Current_ Value > NORMAL if Current_ Value - Last_ Value > 20 if Status == 2 alarm( 3); // start critical situation alarm else{ Status = 2; // set status to warning level alarm( 2); // send warning signal } else ... // other possible situations }

value produced by method check_pressure depends on class Watcher(variable Last_Value) failures due to incorrect values of Last_Value can be revealed only by tests that have control on that variable
example by Mauro Pezze & Michael Young 1998

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Encapsulation & Information Hiding


encapsulation is the converse of visibility, which in the worst case means that objects can be difficult, or even impossible to test encapsulation and information hiding raise the following main problems: identifying which is the basic component to test, and how to select test data for exercising it the private state is observable only through class methods (thus relying on the tested software) possibility of defining classes which cannot be instantiated, e.g., abstract classes, generic classes, and interfaces how to get at the concrete state of an object?
Adapted from A. Orsos PhD thesis
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

10

CmpSci 5/621 Fall06

10/17/06

Breaking the Encapsulation


Use the abstraction inspect state via access methods Use features of the languages C++ friend Ada95 Child Unit Instrument the code provide built-in or inherited state reporting methods, e.g.,hidden functions, to examine the state modify code and thus may alter the behavior Use low level probes or debug tools to manually inspect

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Encapsulation
proof-of-correctness techniques a proved method could be excused from testing to bootstrap testing of other methods state reporting methods tend to be small and simple, they should be relatively easy to prove examine sequences of events t. create ( ); t. push (item); t. pop( ) = t. create ( ) need to be able to define what are equivalent sequences and need to determine equal states state is implicitly inspected by comparing related behavior

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

11

CmpSci 5/621 Fall06

10/17/06

Shadow invocations
operations automatically or implicitly invoked no explicit invocation of these operations in the program, e.g.: constructors destructors casting operators class Foo { ... Foo(Foo& f); void bar(Foo f) { public: Foo(); Foo f1; Foo(); f1=~Foo(); f1=f; Foo(Foo& f); ~Foo(); } ~Foo(); ~Foo(); ... }
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Polymorphism
in procedural programming languages, procedure calls are bound statically, i.e.,the code associated to a call is known at link time polymorphism refers to the capability for a program entity to dynamically change its type at run-time each possible binding of a polymorphic component requires a separate set of test cases many server classes may need to be integrated before a client class can be tested e.g., t.insert would need to be tested for Table and UniqueTable may be hard to determine all such bindings complicates integration planning and testing
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

12

CmpSci 5/621 Fall06

10/17/06

Example
Shape

triangle

square

pentagon

...

void resize( ) { ... data = polygon.area; ... }

Which implementation of area is actually called?

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Dynamic Binding
can lead to messages being sent to the wrong object overriding changes the semantics of a method and can fool the clients of the class to which it belongs sub-classing is not inherently sub-typing, thus dynamic binding on an erroneous hierarchical chain can produce undesirable results polymorphism and late binding introduce undecidability concerns in program-based testing

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

13

CmpSci 5/621 Fall06

10/17/06

Dynamic Binding
Message(X,W)

A
is_a

X F Y

W Z

Would have to consider A sends (,) to D where (,) = (X,W) or (X,Z) or (Y,W) or (Y, Z) Would have to consider B sends (,) to D Etc.

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Address Dynamic Binding Problem


all of the possible bindings should be exercised, but the exhaustive testing of all possible cominations may be impractical Use static analysis (data flow analysis) to determine possible bindings and to reduce combinatorial explosion in the number of possible combinations of polymorphic calls In most systems, the average number of possible bindings is 2

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

14

CmpSci 5/621 Fall06

10/17/06

Summary
Program based testing in the presence of polymorphism may become infeasible,due to the combinatorial number of cases to be tested. New definition of coverage are required to cope with the testing of operations on a polymorphic object The creation of test sets to cover all possible calls to a polymorphic operation can not be achieved with traditional approaches. The presence of polymorphic parameters introduces additional problems for the creation of test cases. Interactions among specic polymorphic invocations along a particular execution path may lead to problems during integration. Such interactionshave to be considered and a way must be dened for exercising them.
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Inheritance Structures
Single Multiple Multiple Levels

BASE

BASE

BASE

BASE

SUBCLASS SUBCLASS

SUBCLASS

SUBCLASS

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

15

CmpSci 5/621 Fall06

10/17/06

Inheritance
B A C
Result Class

A + M1 C B + M2

Parent Class

A + M1
Modifier

+ M2

Adapted from Mary Jean Harrold , John D. McGregor , Kevin J. Fitzpatrick UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Implications of Inheritance
Initialization problems test whether a subclass specific constructor is correctly invoking the constructor of the parent class Semantic mismatch in subtyping inheritance the subclasses may have a different semantics and require different test suites Opportunity for test reduction can we trust features of classes we inherit from, or do we need to re-test derived classes optimistic view = no test is needed for classes derived from thoroughly tested classes more realistic view = methods of derived classes need to be re-tested in the new context
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

16

CmpSci 5/621 Fall06

10/17/06

Which fns must be tested in a subclass?


Base class contains: inherited(int x) redefined() - returns a number in range 1 to 10 inclusive

derived::redefined has to be tested afresh does derived::inherited() have to be retested?


Derived class contains: redefined() - returns a number in range 0 to20 inclusive //inherited() is inherited

derived::inherited() may not have to be completely tested


inherited contains the code: if (x<0) x = x/redefined() return x

if code in inherited() doesnt depend on redefined(), doesnt call it nor call any code that indirectly calls it
have to test when x<0, could divide by 0

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Implications of Inheritance
Re-use of test cases is it possible to use the same test cases generated for the base class during the testing of the derived classif not possible, find a way of partially reusing such test cases

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

17

CmpSci 5/621 Fall06

10/17/06

Test Reuse
base::redefined() and derived::redefined() are two different functions with different specifications and implementations

tests are derived from the different specification and implementation but the functions are likely to be similar, so the better the OO design, the greater the overlap
new tests are those for derived::redefined requirements that are not satisfied by the base::redefined tests

the simpler a test, the more likely it is to be reusable in subclasses but simple tests tend to find only the faults you target; typically not other faults by sheer luck

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Example
Base::describedSelf() is this code if (val < 0) message(Less) else if(val==0) message(Equal) else message(More)

Tests: input, expected output OK -1 Less Change 0 -----Equal Zero Equal OK 1 More Add 42 Jackpot

Tests: input, expected output -1 Less 0 Equal 1 More

Derived::describedSelf() is this code if (val < 0) message(Less) else if(val==0) message(Zero Equal) else { message(More) if(val==42) message(Jackpot) }

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

18

CmpSci 5/621 Fall06

10/17/06

Inheritance Testing
flattening inheritance each subclass is tested as if all inherited features were newly defined tests used in the super-classes can be reused many tests are redundant opportunity for test reduction -- incremental testing reduce tests only to new/modified features determining what needs to be tested requires automated support re-use of test cases -- ASTOOT Come back

to these

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Implications of Inheritance
Inheritance correctness test if it is truly expressing an is-a relationship or we are just in the presence of code reuse Testing of abstract classes abstract classes can not be instantiated and thus can not be thoroughly tested. Only classes derived from abstract classes actually can be tested, but errors can be present in the super (abstract) class Multiple inheritance increases the number of contexts to test specialization relationships
implementation specialization should correspond to problem domain specialization reusability of superclass test cases depends on this

programming convenience
unlikely they can be excused from testing by testing their superclass
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

19

CmpSci 5/621 Fall06

10/17/06

generic (parameterized class)


class Table (elemType) int numberElements; create( ); insert (elemType entry); delete (elemType entry); isempty() returns boolean; isentered(elemType entry) returns boolean; endclass;

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Testing generics
basically a change in the underlying structure need to apply white box testing techniques that exercise this change a generic class can not be tested without being instantiated -- specifying its parameters once instantiated, classes can be tested as with nongeneric classes parametric classes must be instantiated to be tested, but no assumptions can be made on the parameter that will be used for instantiation "trusted classes" are needed to be used as parameters - such classes can be seen as a kind of "stubs"

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

20

CmpSci 5/621 Fall06

10/17/06

Testing Generics
Parameterization may or may not affect the functionality of the access methods In Tableclass, elemType may have little impact on the implementations of the access methods of Table But, UniqueTable class would need to evaluate the equivalence of elements and this could vary depending on the representation on elemType
class Table (elemType) int numberElements; create( ); insert (elemType entry); delete (elemType entry); isempty() returns boolean; isentered(elemType entry) returns boolean; endclass; class UniqueTable extends Table insert(elemType entry); endclass;
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Unit Testing Object-Oriented systems


procedural programming
basic component: subroutine testing method: subroutine input/ output based

object-oriented programming
basic component: class = data structure + set of operations objects are instances of classes data structure defines the state of the object, thus correctness is not based only on output, but also on the state data structure is not directly accessible, but can only be accessed using the class operations (encapsulation)

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

21

CmpSci 5/621 Fall06

10/17/06

Basic Unit for Testing


the class is the natural unit for test case design
and aggregations of classes: class clusters and small subsystems

methods are meaningless apart from their class

Is this true?

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Class as the Unit to be Tested


OBJECT CLASS A
Method 1 Method 2

F
Method

E
Method

B
Adapted from Maj Nicko Petchiny Method

C
Method

D
Method

advantage dependency diagrams required for testing are less complex and association is easier to manage. disadvantage level of granularity could result in a lot of unnecessary regression testing at the integration and system levels.
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

22

CmpSci 5/621 Fall06

10/17/06

Method as the Unit to be Tested


OBJECT CLASS A
Method 1 Method 2

F
Method

E
Method

B
Adapted from Maj Nicko Petchiny Method

C
Method

D
Method

advantage
approach could result in significant savings because of the

reduction in the amount of regression testing required.

disadvantage

dependency associations between methods can quickly become complex and unmanageable.
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Clusters & Threads


A OUTPUT PORT EVENT

Class 1
meth1
1

Class 3
meth2 meth1

meth2
B

meth3
B OUTPUT PORT EVENT 2

Class 2
meth1 meth3
3

MM-Path

Message

meth2

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

23

CmpSci 5/621 Fall06

10/17/06

Method as Basic Unit for Testing


Jorgensen And Ericksen 5 Levels A Method - Unit Testing Message Quiescence - Integration Event Quiescence - Integration Thread Testing - System Thread Interaction - System Well come back to this
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Class as Basic Unit for Testing


intended use of a class implies different test requirements application-specific versus general-purpose classes abstract classes parameterized (template) classes testing a class instance (an object) can verify a class in isolation when individually verified classes are used to create objects in an application system, the entire system must be tested as a whole before it can be considered to be verified

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

24

CmpSci 5/621 Fall06

10/17/06

Testing Hierarchy
Class testing testing of individual classes inter-class testing testing of any set of cooperating classes, aimed at verifyingthat involved classes correctly interact. There are no constraint on how these classes are selected. intra-cluster testing (or cluster testing) testing of the interactions betweenthe different classes belonging to a subset of the system having some speci c properties (a cluster) a cluster is composed of cooperating classes providing particular functionalities
all the classes which can be used to access the file-system the classes composing a Java package
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Testing Hierarchy
intra-cluster testing (or cluster testing) -- more clusters should provide a well-defined interface, i.e., their interfaces should be well understood and they should mutually interact only by means of their interfaces. inter-cluster testing testing of the interactions between already tested clusters.The result of the integration of all clusters is the whole system integration testing a general way of indicating any of the above.

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

25

CmpSci 5/621 Fall06

10/17/06

class-level testing
techniques involved can be classified as black-box technique = program testing based on software specifications
State-based techniques, ASTOOT

white-box = testing based on the source code of the developed systems


adaptations of various coverage techniques -Harrold/McGregor & Buy/Orso/Pezze

white-box vs. black-box testing of O-O distance between object-oriented specification and implementation is typically small gap (and therefore usefulness) of the white-box/blackbox distinction is decreasing
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Incremental testing
Mary Jean Harrold, John D. McGregor and Kevin J. Fitzpatrick

Exploits the inheritance hierarchy to minimize the amount of testing that must be done First test each base class (no parents) Test each method Test the interactions among methods Then consider all classes that use only previously tested classes Child inherents its parents test suite Used as the basis for test planning Only need to develop new test cases for those entities that are directly or indirectly changed
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

26

CmpSci 5/621 Fall06

10/17/06

BaseClassTesting
use traditional unit testing techniques use a test suite that contains both specification-based and program-based test cases handle calls to other member functions (procedures) by providing stubs representing called member functions and drivers representing calling member functions testing history for a class contains triples: {mi, (TSi, test?), TPi, test?)) where mi is the member function TSi is the specification-based test suite TPi is the program-based test suite test? indicates whether the test suite is to be (re)run.
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

BaseClassTesting
Intra-class testing guided by a classgraph where each node represents either a member function in the class or a primitive data member, and each edge represents a message. class graph may be dkconnected where each connected subgraph represents those attributes in the class that interact with each other. use both specication-based and program-based test suites the history consists of triples {mi, (TSi, test?), TPi, test?)) where mi is the root node of the class graph TSi is the specification-based test suite

TPi is the program-based test suite test? indicates whether the test suite is to be (re)run.

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

27

CmpSci 5/621 Fall06

10/17/06

BaseClassTesting

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

ChildClass Testing
transforms the testinghistory for the parent class P to the testing history for the subclass R inputs Ps history, HISTORY(P), Ps class graph, G(P), and moditler M and outputs an updated HISTORY(R), for the subclass NEW or VIRTUAL-NEW member function attribute A must also be completely, individually testedsince it was not defined in P RECURSIVE or VIRTUAL-RECURSIVE member function attribute A requires very limited retesting since it was previously individually tested in P and the specification and implementation remain unchanged REDEFINED or VIRTUAL-REDEFINED attribute A in M requires extensive retesting but many existing specificationbased test cases may be reused since only the implementation has changed.
UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

28

CmpSci 5/621 Fall06

10/17/06

ChildClass Testing

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Incremental Inheritance based testing


Saves time Reduces number of new test cases Reduces execution time since there are fewer test cases Reduces number of test results that need to be evaluated May increase the cost of selecting new test cases Easily offset by reduction in human labor Actually a form of regression testing Minimizes the number of test cases needed to exercise a modified class

UNIVERSITY OF MASSACHUSETTS AMHERST DEPARTMENT OF COMPUTER S CIENCE CMPS CI 521/621 FALL 2006

Copyright W.Richards Adrion 2006 except were noted/cited

29