Class Testing

ECEN5033 University of Colorado

 Overview
 Class Testing
 Testing Interactions between objects
 Testing Class Hierarchies

2 January 27, 2002 ECEN5033 University of Colorado -- C

 Basics – How to test a single class
 Class Testing
 How to Test a Class

 Aspects of Class Testing

 How to Construct Test Cases

 When is a class test suite adequate?

 How to Construct a Test Driver

 Testing Interactions between objects

 Testing Class Hierarchies

3 January 27, 2002 ECEN5033 University of Colorado -- C

 Definition of class testing
 Verifying implementation of a class = verifying the
specification for that class

 If so, each of the instances should behave

properly.
 Assumption:
 The class in question has a complete and

correct specification that has been tested earlier

in the context of models
 Spec is expressed in natural language or as a

state transition diagram

4 January 27, 2002 ECEN5033 University of Colorado -- C
 Ways to test a class
 Code can be tested effectively by
 inspection (preferable when construction of a

test driver is too difficult)

 execution of test cases (lends itself to easy

regression testing later on)

 Remember: When you test a class, you are really:
 creating instances of that class and

 testing the behavior of those instances.

5 January 27, 2002 ECEN5033 University of Colorado -- C

 Aspects of Class Testing
 Decide
 test independently

 test as a component of a larger part of the

system
 How decide – combination of the following:
 Role of the class in the system – degree of risk

 Complexity of the class

 Amount of effort associated with developing a

test driver
 Sometimes, needs so many collaborators,
makes more sense to test in a cluster

6 January 27, 2002 ECEN5033 University of Colorado -- C

Who Tests? How much?
 Who – class usually tested by its developer
 Understands class’ spec

 Familiar with the code

 Test driver can be used by the developer to

debug code while writing it

 Perpetuates misunderstanding – needs

inspection at model stage to head that off

 What – ensure that the code for a class exactly
meets the requirements – no more, no less

7 January 27, 2002 ECEN5033 University of Colorado -- C

When are tests written?
 When – A test plan that identifies test cases
should be developed soon after a class is fully
specified and ready for coding.
 Especially if developer is the class tester

 Why?

 (What’s the danger in developer alone writing and

reviewing test cases for the class?)
 When again?
 Iterative development – the driver and the test

cases will be available to supplement or change

as the class is enhanced or modified
8 January 27, 2002 ECEN5033 University of Colorado -- C
How can you test “just a class”?
 How –
 Create a test driver that

 creates instances of the class

 sets up suitable environment around those

instances to run a test case
 sends one or more messages to an instance
as specified by a test case
 checks the outcome based on a reply value,
changes to the instance, or parameters to the
message
 deletes any instances it creates if responsible
for storage allocation
9 January 27, 2002 ECEN5033 University of Colorado -- C
How do we test static data in a class?
 How – continued:
 Static data members or operations
 testing required

 they belong to the class itself rather than to

each instance of the class

 the class can be treated as an object

 If the behavior of the instances of a class is

based on the values of class-level attributes
 test cases for testing the class-level attributes

must be considered as an extension of the

state of the instances
10 January 27, 2002 ECEN5033 University of Colorado -- C
How MUCH testing is done at this level?
 How much – Adequacy of testing measured in
terms of
 how much of the spec has been tested

 how much of the implementation has been

tested
 Want to test operations and state transitions in
many combinations
 Objects maintain state. “State” affects the
meaning of operations.

11 January 27, 2002 ECEN5033 University of Colorado -- C

 Constructing test cases - identification
 Identification of test cases
 Should be made from the spec that has been

reviewed rather than from the implementation

which may embody developer’s
misunderstandings
 Best: develop from spec and augment to test

boundaries introduced by the implementation

 If no spec exists: create one from the code and

verify it with the developer

12 January 27, 2002 ECEN5033 University of Colorado -- C

 Constructing test cases – identification2
 Identify requirements for test cases for all possible
combinations of situations in which
 a precondition can hold

 post conditions can be achieved

 Create test cases for those requirements

 specific input values – typical and boundary

 determine correct outputs

 eliminate conditions that are not meaningful

 Add test cases to show what happens when a

precondition is violated

13 January 27, 2002 ECEN5033 University of Colorado -- C

 Constructing Test Cases – from STD’s
 State Transition Diagrams
 They show behavior associated with instances

of a class
 Each transition represents a requirement for one
or more test cases

14 January 27, 2002 ECEN5033 University of Colorado -- C

 Constructing test cases from STD’s 2
 Suppose 6 transitions between states
 Plus 1 constructor and 2 destructors
 That makes 9 requirements
 Select representative values

 Select boundary values on each side of a

transition
 If the transition is guarded, select boundary

values for the guard condition

 Boundary values are based on the range of
attribute values associated with a state

15 January 27, 2002 ECEN5033 University of Colorado -- C

Top-level Statechart for Elevator Control

16 January 27, 2002 ECEN5033 University of Colorado -- C

 Adequacy of a Class Test Suite
 Ideally – exhaustively test each class
 Practically – impossible or too hard
 Worth it to exhaustively test some classes with
high risk
 Measures of adequacy to increase confidence that
we have tested enough
 state-based coverage

 constraint-based coverage

 code-based coverage

17 January 27, 2002 ECEN5033 University of Colorado -- C

 State-based coverage
 How many of the transitions in a state transition diagram
are covered by the test suite?
 “Covered” = touched at least once

 May reveal each transition was covered test values do

not adequately cover value ranges

 If test cases were generated correctly from a state
transition diagram with typical values and good boundary
values, the test cases will achieve adequate state-based
coverage
 If test cases generated from pre- and post conditions, then
it is useful to check them against the state transition
diagram to ensure each transition is covered.

18 January 27, 2002 ECEN5033 University of Colorado -- C

 State-based coverage: object interaction
 Note how operations interact w.r.t. transitions

Current Input or Action Output Next Test cases for the

State event State transition from
Sc to Sd may
Sa E1 O1 Sc
work if Sc was
reached from Sa
Sb E2 O2 Sc but not if Sc was
reached from Sb.
“State” is a
Sc E3 Sd function of
history.

19 January 27, 2002 ECEN5033 University of Colorado -- C

 State-based coverage: transition pairs
 Concerning problem on previous page:
 Check that the test cases cover all pairs of

transitions in the state transition diagram.

 In previous table, create test cases to test:

 SaSc and ScSd

 SbSc and ScSd

20 January 27, 2002 ECEN5033 University of Colorado -- C

 Statechart for
Elevator Control

21 January 27, 2002 ECEN5033 University of Colorado -- C

 Hierarchical statechart for
Elevator Control

22 January 27, 2002 ECEN5033 University of Colorado -- C

 Portion
enlarged

23 January 27, 2002 ECEN5033 University of Colorado -- C

 Constraint-based coverage
 How many pairs of pre- and post conditions have
been covered?
 Using the technique described earlier for
generating test cases from pre and post
conditions, if one test case is generated to satisfy
each requirement, then the test suite meets this
measure

24 January 27, 2002 ECEN5033 University of Colorado -- C

Constraint-Based coverage: object interaction
 For each operation opn that is not an accessor
operation,
 identify the operations op1, op2, etc. for which

their preconditions are met when the post

conditions for opn hold.
 That is, post condition(opn) satisfies

precondition(op1), etc.
 Then execute test cases for operation

sequences opn-op1, opn-op2, etc.

25 January 27, 2002 ECEN5033 University of Colorado -- C

 Code-based coverage
 How much of the code that implements the class
is executed across all test cases in the suite?
 Minimum: Ensure that every line was executed at
least once by the time all test cases (for that class)
have completed execution.
 Commercial tools
 Add test cases to the suite if some lines have not
been reached.
 Might still be inadequate – interaction between
methods may fail to exercise interactions between
methods
 Measure coverage for operation sequences

26 January 27, 2002 ECEN5033 University of Colorado -- C

 Test Driver Construction

27 January 27, 2002 ECEN5033 University of Colorado -- C

 Testing Interactions
 Class Testing
 Testing Interactions between objects
 Identifying & specifying object interactions

 Testing object interactions

 Collection classes

 Collaborator classes

 Testing and Design Approach

 Sampling

 COTS testing

 Patterns

 Testing exceptions

 Testing Class Hierarchies

28 January 27, 2002 ECEN5033 University of Colorado -- C
 Testing Interactions with other classes
 An OO program is made of a collection of objects
that collaborate to solve a problem.
 Correct collaboration is necessary for program
correctness.
 In this section, we assume interactions are
sequential, not concurrent (that’s a subject for next
semester)
 Focus: Ensure that messaging occurs correctly
between objects whose classes have already
been tested separately.
 Can happen embedded in application or in special
environment using a test harness

29 January 27, 2002 ECEN5033 University of Colorado -- C

 What is an object interaction?
 Object interaction is:
 request by a sender object to a receiver object

 to perform one of the receiver’s operations

 all of the processing performed by the receiver

to complete the request

 Includes messages between
 an object and its components

 an object and other objects with which it is

associated

30 January 27, 2002 ECEN5033 University of Colorado -- C

 Object interaction impact
 During the processing of any single method-invocation on
a receiving object
 Multiple object interactions can occur

 Want to consider impact of these on

 the internal state of the receiving object

 those objects with which it has an association

 Impact may be

 “no change”

 changes in attribute values in 1 or more objects

involved
 state changes in 1 or more objects

 including state changes of creation or deletion of

objects
31 January 27, 2002 ECEN5033 University of Colorado -- C
 Primitive and Non-Primitive Classes
 Interactions implied in a class spec where there
are references to other objects
 Primitive class can be instantiated and used
without needing to create any other instance
 Relatively few primitive classes in a program
 Since an OO program should model the objects in
a problem and all the relationships between them,
non-primitive classes are common & essential
 Non-primitive classes require the use of other
objects in some or all of their operations
 Identify the classes of those objects
32 January 27, 2002 ECEN5033 University of Colorado -- C
 Identifying Interactions
 Identify interactions by association relationships in the
class diagram
 regular associations

 aggregation

 composition

 Associations translate to class interfaces by:

 public operation names 1 or more classes as the
type of a formal parameter
 public operation names classes as the type of a
return value
 A class method creates an instance of another
class or refers to a global instance of some class
33 January 27, 2002 ECEN5033 University of Colorado -- C
 Recognizing collaborators
 Collaborators may be addressed
 directly, e.g. using a variable name

 by a pointer or a reference

 dynamic type of the object may be different

from the static type associated with the pointer

 pointers and references are polymorphic, bound

to an instance of any number of classes

 Pre- and post conditions for operations in the public
interface typically refer to states and/or specific
attribute values of any collaborating objects

34 January 27, 2002 ECEN5033 University of Colorado -- C

 Collection and collaboration
 Collection class
 maintains associations with instances of other

classes but never actually interacts with those

instances
 that is, never requests a service

 Collaborating class
 Class with more extensive interactions

 Collection classes are less common

35 January 27, 2002 ECEN5033 University of Colorado -- C

 Testing Collection Classes
 What collection classes do
 store references to these objects, representing

one-to-many relationships between objects in a

program
 create instances of these objects

 delete instances of these objects

 Specification
 refers to other objects

 does not refer to values computed based on

those objects

36 January 27, 2002 ECEN5033 University of Colorado -- C

 Identifying Collaborating Classes
 Non-primitive classes that are not collection classes
 Use other objects in 1 or more operations as part of
their implementation
 When a post condition of an operation in a class’
interface
 refers to the state of an instance of an object

 and/or specifies that some attribute of that object

is used or modified,
that class is a collaborating class.

37 January 27, 2002 ECEN5033 University of Colorado -- C

 Choosing testing “chunk” size
 Number of potential collaborations can grow too
large very quickly
 During class test: test composing object’s
interactions with its composed attributes
 As successive layers of aggregation are integrated,
 test the interaction between an object and its
associated objects
 Defect visibility issue – If too large a chunk is
chosen, intermediate results may be incorrect but
not seen at the level of test-result verification
 The more complex the objects, the fewer one
should integrate prior to a round of testing

38 January 27, 2002 ECEN5033 University of Colorado -- C

 What factors increase object complexity?
 number of parameters for each method
 number of methods

 number of state attributes in each object

Trying to test a chunk that is too complex can result

in defects that are hidden from the tests.

39 January 27, 2002 ECEN5033 University of Colorado -- C

  What are the implications of defensive and
contract design approaches when testing
interactions?

40 January 27, 2002 ECEN5033 University of Colorado -- C

 Specifying interactions
 Remainder of slides assume
 Operations defined by a class are specified by

preconditions, post conditions, and class

invariants
 contract design approach

41 January 27, 2002 ECEN5033 University of Colorado -- C

 Testing Collection Classes
 Collection classes are testing in ways similar to
primitive classes
 Test driver creates instances passed as parameters in
messages to a collection
 Test cases ensure that
 those instances are incorporated into and

 removed from the collection

 address limitations placed on collection capacity

 Test sequences of operations – the way modifier

operations on a single object interact with one another
 What additional test cases might be needed in a
defensive approach?
42 January 27, 2002 ECEN5033 University of Colorado -- C
 Testing Collaborator Classes
 More complex than testing collection or primitive
 For example, if certain objects must be passed as
parameters to a constructor, we can not test the
constructor or the instance that requests the
services of the constructor without ensuring there
are instances of the objects to be passed as
parameters

43 January 27, 2002 ECEN5033 University of Colorado -- C

 Contract vs. defensive approach &
testing
 Contract
 More responsibility on
human designer
 Less class-level testing
due to fewer paths due to
less error-checking code
 MORE interaction testing
required to ensure human
designer complied with
client side of contract
using precondition
constraints – are
preconditions in receiver
met by sender
 “illegal” for test cases to
violate preconditions
44 January 27, 2002
 Defensive
 More responsibility on
error-checking code
 More class-level testing
due to increased paths
due to more code
 Once the class-level error-
checking code is tested,
the interaction testing is
simplified since there is no
need to test if
preconditions are met
 Appropriate to violate
preconditions to see if
receiver catches that
ECEN5033 University of Colorado -- C
 Sampling
 Exhaustive testing: every possible test case
covering every combination of values
 Not reasonable much of the time
 Want to select the ones that will find the faults in
which we are the most interested
 Without prior information, random selection is

as good as we can do
 A sample is a subset of a population (e.g. all
possible test cases) that has been selected based
on some probability distribution

45 January 27, 2002 ECEN5033 University of Colorado -- C

 Use profile
 Gave a way to associate each use of the system
with a frequency
 These can be ranked by frequency
 Ranks can be turned into probabilities
 The higher the frequency of use, the larger the
probability of selection

46 January 27, 2002 ECEN5033 University of Colorado -- C

 Operational profile – stratified sample
 Stratified sample is a set of samples in which each
sample represents a specific subpopulation
 Example
 Select test cases that exercise each component

of the architecture
 Divide a population of tests into subsets so that

a subset contains all of the tests that exercise a

specific component.
 Sample on each subset independent of the others
 Need a basis for stratifying

47 January 27, 2002 ECEN5033 University of Colorado -- C

 A basis for stratification of test cases
 Use the actors from the use cases
 Select a sample of test cases from the uses of each actor
in the use cases
 Each actor uses some subset of the possible uses with
some frequency; rank by frequency
 Stratifying the test case samples by each actors provides
a way to increase the reliability of system
 Running selected tests
 uses the system the way it will be used in typical
situations;
 finds defects likely to be found in typical use

 gives largest possible increase in reliability with least

effort

48 January 27, 2002 ECEN5033 University of Colorado -- C

Test data generation for a range of values
 Generate test data first using the specification as basis
 Select values based on boundary values
 just within and just on the boundaries if contract

 just outside the boundaries also if defensive

 Select values within intervals by sampling

 Consider a random function over range 0.. N such as
int (random( ) * N) which generates a pseudo random
value between 0 and 1 according to a uniform distribution.
 +: many values will be tested over many iterations
 Need to log actual values used in case of failure so that
the failed test case can be re-created.
 A randomly chosen value causing failure is explicitly
added to test suite and used to test the repaired software
49 January 27, 2002 ECEN5033 University of Colorado -- C
 Systematic sampling
 Want to be able to increase level of coverage
systematically.
 We use boundary values coupled with sampling
over ranges
 We never omit boundary value test data

50 January 27, 2002 ECEN5033 University of Colorado -- C

Issues about sampling for interaction testing
 Sampling issues
 Objects passed as parameters

 Different states that can cause two objects from

the same class to behave differently

 A class family as a subset of a larger family

 Possibility of combinatorial explosion in the

number of test configurations

51 January 27, 2002 ECEN5033 University of Colorado -- C

 Example from McGregor & Sykes
Movable Sprite collideWith(self) Sprite

Puck Paddle Stationary Sprite Movable Sprite

Brick Wall Ceiling Floor Puck Paddle

LeftWall RightWall

Explosion of test cases ...

52 January 27, 2002 ECEN5033 University of Colorado -- C

 Parameter sampling to test interaction
 Consider two abstract classes that interact where class B
receives an instance of class A as a parameter
 Suppose B-method’s precondition places no restriction on
the parameter
 How do we determine the population from which to sample
objects to pass as this parameter?
 Suppose A is a base (abstract) class in a very large class
family (a set of classes related by inheritance)
 an object from any one of the classes in the family can

be substituted for its parameter

 This is the set from which to sample for possible

parameters
53 January 27, 2002 ECEN5033 University of Colorado -- C
 Impact of inheritance hierarchy on s.t.d.’s
 Each member of the family may have different states that can
cause two objects from the same class to behave differently.
 In the example, Puck and Wall have interesting differences in
their states.
 In families of classes, the state transition tables are related
along the lines of the inheritance hierarchy.
 as we look down the inheritance hierarchy, there will be

the same number of states or more states in the derived

class as in the base class
 should cover the states defined for each class,

emphasizing the new states added at that level in

inheritance hierarchy.

54 January 27, 2002 ECEN5033 University of Colorado -- C

 Class families as subsets of others
 If a class family is a subset of a larger family, after
the tests are designed, they can be applied to any
of the classes in the family, assuming the
substitution principle has been followed during
design*

55 January 27, 2002 ECEN5033 University of Colorado -- C

 Orthogonal array testing applied to OO
 Orthogonal array testing is a sampling technique to
limit the explosion of test cases
 Define pair-wise combinations of a set of interacting
objects because most faults result from interactions
due to two-way interactions
 An orthogonal array is an array of values in which
each column represents a factor.
 A factor is a variable in an experiment; in our case, a
specific class family in the software sys
 Each variable takes on a certain set of values called
levels; in our case, a specific class
 Each cell has an instance of the class, that is, a
specific state of an object

56 January 27, 2002 ECEN5033 University of Colorado -- C

 OATS – orthogonal array testing system
 The factors (columns) are combined pair-wise
rather than representing all possible combinations
of the levels for the factors.
 Suppose we have three factors – A, B, C – and
each has three levels – 1, 2, and 3
 How many possible combinations of these values
are there?
 How many pair-wise combinations? That is, how
many combinations where a given level appears
exactly twice?
factor=class family; level=class; cell=instance

57 January 27, 2002 ECEN5033 University of Colorado -- C

 array factor: factor:
class family class family
level: an instance of an instance of the
specific the specific class specific class a
class specific
level: an instance of an instance of the state
specific the specific class specific class a
class specific
state
level: an instance of an instance of the
specific the specific class specific class a
class specific
state

58 January 27, 2002 ECEN5033 University of Colorado -- C

 OATS – uses a balanced design
 Every level of a factor appears exactly the same
number of times as every other level of that factor
 Think of the rows of a table as test cases (In
example, 18 of the possible 27 are not being
conducted.
 This is a systematic way of reducing the number
of test cases.
 If we decide to add more later, we know which
ones have not been run.
 Also logical – most errors are between pairs of
objects rather than among several objects
 See handout for example

59 January 27, 2002 ECEN5033 University of Colorado -- C

 OATS adequacy criteria
 Ability to vary how completely the software under
test is covered
 Exhaustive – all combinations of all factors
 Minimal – only interactions between the base
classes from each hierarchy are tested
 Random – tester haphazardly selects cases from
several of the classes; not statistically random
 Representative – uniform sample ensures every
class is tested to some level
 Weighted representative – adds cases to the
representative approach based on relative
importance or risk associated with the class

60 January 27, 2002 ECEN5033 University of Colorado -- C

 Other testing – see McGregor & Sykes
 Testing off-the-shelf components – treat as a class
interacting with rest of your product
 Protocol testing – investigates whether the implementation
of a class satisfies its specification. Various protocols the
object participates in can be inferred from the pre- and post
conditions for individual operations.
 Test Patterns – design patterns for test software.
 explains the context of consideration, provides a set of

forces that guide a trade-off analysis, explains how to

construct the objects.
 When developer uses a certain design pattern, tester

can use a certain test pattern to structure the test code.

 Testing Exceptions

61 January 27, 2002 ECEN5033 University of Colorado -- C

 Testing Exceptions
 Class level testing: verify that each of the methods
does throw the exceptions it claims to throw in the
appropriate circumstance.
 Coverage criteria – class throws every exception
contained in the specifications of the methods
 At least one test case per exception

 Test driver sets up conditions to result in exception

event.
 Exceptions are objects and belong to classes so catch
statements can verify that it is of the correct type
 Integration level: being thrown at correct time in
correct place
 Many important points covered in book
62 January 27, 2002 ECEN5033 University of Colorado -- C
 Testing interactions at system level
 Components can be so complex it’s easier to test
them in the context of the application itself
 Which interactions are tested at system level?
 Those that can also be verified at the system

level
 That is, can see the direct results of the tests

 There is a direct relationship between user

interface and the ability to view test results

63 January 27, 2002 ECEN5033 University of Colorado -- C

 Testing Class Hierarchies
 McGregor & Sykes have 20 pages on this
 Inheritance provides powerful analysis & design tool
 Also provides powerful testing tool
 If inheritance applied during design in accordance
with the substitution principle
 There is an inheritance relationship that holds for
test suites.
 Test suites for subclasses can be derived from the
test suite for their parent classes
 Analyzing changes between subclass & parent,
can decide what test cases need to be added,
rerun, modified, and not run
64 January 27, 2002 ECEN5033 University of Colorado -- C