5/10/2019

Software Quality Engineering

Lecture – 10

Agenda
 Wrap-up
 Conclusion
– Structural Testing
– Unit Testing
 From Unit to Integration Testing
– OO Protocol Testing
– Finite State Machines for Testing

1
 5/10/2019

Unit Testing (Wrap up)

 While developing unit test strategy, we
consider:
– Test Requirement
– Documentation requirement
– Tools / Test utilities?
 Determine extent of unit testing (i.e., in
advance).
– do not just “test until time expires”
– prioritize, so that important tests definitely
performed
 Decide how and where to get the test input
 Estimate the resources required
– use historical data if available
 Arrange to track time, defect count, type &
source

Unit Test Cases Guidelines

 Considering our black-box testing knowledge, we write
test cases to test operations considering:
– normal parameter values
– limit parameter values
– outside parameter values
– (black box)
 Considering our white-box testing knowledge, we
write test cases to test operations ensuring:
– all instructions execute (statement coverage)
– all paths, including both sides of all branches execute(decision
coverage)
– use of all called objects
– handling of all data structures
– handling of all files

2
 5/10/2019

OO Protocol
 Procedural software
– unit = single program, function, or procedure
 Object oriented software
– unit = class
– unit testing = intra-class testing
– integration testing = inter-class testing (cluster of
classes)
 dealing with single methods separately is usually
– too expensive (complex scaffolding)
 Therefore, methods are usually tested in the
context of the class they belong to

Class in terms of State-Machine

 Natural representation with finite state Machines
– States correspond to certain values of the Attributes
– Transitions correspond to methods
 FSM can be used as basis for testing
– e.g. “drive” the class through all transitions, and verify
the response and the resulting state
 Test cases are sequences of method calls that
traverse the state machine
 State machine model can be derived from:
– Specification
– Code
 also using reverse engineering techniques
 or both …

3
 5/10/2019

Assume we had code…

public class stack
{
public final intmax = 10;
public Boolean isFull { … }
public add{ … }
public delete { … }
public destroy { … }
public { … }
public create{ … }
}

Extracting States and transitions

 States:
– Initial: before creation
– Empty: number of elements = 0
– Holding: number of elements >0, but less than
the max Capacity
– Full: number elements = max
– Final: after destruction
 Transitions:
– create, destroy
– actions that triggers the transition
 ex. Add, delete

4
 5/10/2019

Arranging transitions
 Initial -> Empty: action = “create”
– e.g. “s = new Stack()” in Java
 Empty -> Holding: action = “add”
 Empty -> Full: action = “add”
– if MAXcapacity=1
 Empty -> Final: action = “destroy”
– e.g. destructor call in C++, garbage collection in
Java
 Holding -> Empty: action = “delete”
– if s.size() = 1

State machine – stack

5
 5/10/2019

State based testing

The tests are derived from a state model of the
system. We can derive the state model in several
way, e.g. from
• Expected system behavior
• State part of a UML design or requirements
specification.
• Other state diagrams

Most system will, however, have a large number of

states

Fault types
• Missing or incorrect
– Transitions – new state is legal but incorrect
– Events – valid message ignored
• Extra, missing or corrupt state
• Sneak path
• Illegal message failure
• Trap door

6
 5/10/2019

Coverage Criteria
We can choose one or more of the following test
selection criteria:
 All states – testing passes through all states
 All events – testing forces all events to occur
at least once
 All actions – testing forces all actions to be
produced at least once

Test Strategies
 Possible strategies
– All round-trip paths
– All transition sequences beginning and ending in
the same state
– All simple paths from initial to final state
 This strategy will help you to find
– All invalid or missing states
– Some extra states
– All event an action faults

7
 5/10/2019

Question?
 What if Account
– we have concept level status: enum
balanace: int
documents to consider isActive(): boolean
– Could be isBlocked(): boolean
isClosed(): boolean
 UML documents gotBalance(): int
 Specifications block()
unblock()
 etc. close()
deposit(amount: int)
withdraw(amount: int)

Answer!
 We develop state-machines considering these
artifacts…
 But how…
Account
status: enum
balanace: int
isActive(): boolean
isBlocked(): boolean
isClosed(): boolean
gotBalance(): int
block()
unblock()
close()
deposit(amount: int)
withdraw(amount: int)

8
 5/10/2019

MODEL-BASED TESTING

Models
• A model is a description of a system’s behavior.
• Models are simpler than the systems they
describe.
• Models help us understand and predict the
system’s behavior.
• For Testing:
• State-machines as models
• Specifications as models e.g., UML diagrams
• Formal Specifications as models e.g., Object Z
• …

9
 5/10/2019

Against what are testing?

Specifications:

 Users needs (Requirements)

System Under Test ?  Objectives (specific)
 Informal specification
 Formal specification

The answer will help test team to establish a clear relationship between
the system under test, the specification and the objective to satisfy.

Models of Specification and

Implementation

 Conformance testing

conformance relation
abstract abstract
model of S model of I

assumptions/ assumptions/
test hypothesis test hypothesis
conformance relation
precise implementation I
specification S

10
 5/10/2019

Testing Systems Specified as State-Machine

 To confirm if an implementation conforms to its standard
– External tester applies a sequence of inputs to IUT and verifies its
behavior
– Issue1: preparation of conformance tests in coverage of IUTs all
aspects
– Issue2: time required to run test should not be unacceptably long
 Two main limitations
– Controllability: the IUT cannot be directly put into a desired state,
usually requiring several additional state transitions
– Observability: prevents the external tester from directly observing the
state of the IUT, which is critical for a test to detect errors
 Formal conformance testing techniques based on FSM
– Generate a set of input sequences that will force the FSM
implementation to undergo all specified transitions
– Black box approach: only the outputs generated by the IUT (upon
receipt of inputs) are observable to the external tester
 Requirement of a Fault Model

Fault Models
 A fault model is a hypothetical model of what types of faults
may occur in an implementation
– Most fault models are "structural", i.e. the model is a refinement of
the specification formalism (or of an implementation model)
 E.g. mutations of the specification or of a correct implementation
– It may be used to construct the fault domain used for defining what
"complete test coverage" means
 E.g. single fault hypothesis (or multiple faults)
 A fault model is useful for the following problems:
– Test suite development for given coverage objective
– Formalization of "test purpose"
– For existing test suite: coverage evaluation and optimization
– Diagnostics

11
 5/10/2019

Fault Model for FSM

 Output fault the machine provides an output different from
the one specified by the output function
 Transfer fault the machine enters a different state than that
specified by the transfer function
 Transfer faults with additional states: number of states of the
system is increased by the presence of faults, additional states
is used to model certain types of errors
 Additional or missing transitions: one basic assumption is that
the FSM is deterministic and completely defined (fully
specified). So the faults occur when it turns out to be non-
deterministic and/or incompletely (partially) specified

Round-trip paths and Sequencing Constraints

• A round-trip path tree
– Is built form a state transition diagram
– Includes all round-trip paths
• Transition sequences beginning and ending in the same state
• Simple paths for initial to final state. If a loop is present, we use
only one iteration
• Is used to
– Check conformance to explicit behavioral models
– Find sneak paths
A test strategy based on round-trip path trees will reveal:
• All state control faults
• All sneak paths – messages are accepted when they
should not
• Many corrupt states - unpredictable behavior
• Sequencing Constraints
– Guard Conditions
– Requirement for FSM test cases development

12
 5/10/2019

Transitions with Constraints

Each transition in a state diagram has the form
trigger-signature [guard] / activity. All parts are
optional
 trigger-signature: usually a single event that
triggers a potential change of state.
 guard: a Boolean condition that must be true
for the transition to take place.
 activity: an action that is performed during
the transition.

Sneak path test cases

A sneak path – message accepted when it
should not be accepted – can occur if
 There is an unspecified transition
 The transition occur even if the guard
predicate is false

13
 5/10/2019

Summary
 Unit Testing – Conclusion
 OO Protocol Testing
 State-Machine based testing
 Model-based testing
 Next Lectures
– Specification-based Testing
– Grey-box testing: An Introduction
– Testing Classes Clusters
 Method-Message Path Testing
– Testing Object-Oriented Systems

14