Innovations Syst Softw Eng (2008) 4:71–85
DOI 10.1007/s11334-007-0042-z
ORIGINAL PAPER
Patterns: from system design to software testing
Neelam Soundarajan · Jason O. Hallstrom ·
Guoqiang Shu · Adem Delibas
Received: 16 December 2007 / Accepted: 19 December 2007 / Published online: 12 January 2008
© Springer-Verlag London Limited 2008
Abstract Design patterns are used extensively in the design
of software systems. Patterns codify effective solutions for
recurring design problems and allow software engineers to
reuse these solutions, tailoring them appropriately to their
particular applications, rather than reinventing them from
scratch. In this paper, we consider the following question:
How can system designers and implementers test whether
their systems, as implemented, are faithful to the require-
ments of the patterns used in their design? A key considera-
tion underlying our work is that the testing approach should
enable us, in testing whether a particular pattern P has been
correctly implemented in different systems designed using
P, to reuse the common parts of this effort rather than hav-
ing to do it from scratch for each system. Thus in the approach
we present, corresponding to each pattern P, there is a set
of pattern test case templates (PTCTs). A PTCT codifies a
reusable test case structure designed to identify defects asso-
ciated with applications of P in all systems designed using
P. Next we present a process using which, given a system
designed using P, the system tester can generate a test suite
from the PTCTs for P that can be used to test the particular
system for bugs in the implementation of P in that system.
This allows the tester to tailor the PTCTs for P to the needs
N. Soundarajan (B) · G. Shu · A. Delibas
Computer Science and Engineering, Ohio State University,
Columbus, OH 43210, USA
e-mail: neelam@cse.ohio-state.edu
G. Shu
e-mail: shug@cse.ohio-state.edu
A. Delibas
e-mail: delibas@cse.ohio-state.edu
J. O. Hallstrom
School of Computing, Clemson University,
Clemson, SC 29634, USA
e-mail: jasonoh@cs.clemson.edu
of the particular system by specifying a set of specializa-
tion rules that are designed to reflect the scenarios in which
the defects codified in this set of PTCTs are likely to man-
ifest themselves in the particular system. We illustrate the
approach using the Observer pattern.
Keywords Design patterns · Contracts · Testing
1 Introduction
Design patterns [4,11,24] have had a profound impact on
how software systems are built. This is not surprising since
patterns capture the distilled wisdom of the software com-
munity, and provide effective solutions to recurring design
problems. They allow software engineers to reuse these solu-
tions, tailoring them to the needs of their particular systems,
rather than reinventing them from scratch. Patterns are used
primarily during the design phase of software systems. In
this paper, we focus on later stages of the software life-cycle,
the testing and maintenance phases. Specifically, the ques-
tion we are interested in is, how can system designers and
implementers test whether their systems, as implemented, are
faithful to the requirements of the underlying patterns? This
question is especially important for large systems since such
systems tend to have correspondingly large software teams
and hence a correspondingly greater likelihood that differ-
ent members of the team may have subtly different inter-
pretations of the patterns underlying the system’s design.
Hence the team members may implement their respective
parts of the system in ways such that when these parts are put
together, the resulting overall system is not compatible with
the patterns’ intent. Design-related bugs of this kind may not
manifest themselves in test cases that are developed using
standard approaches such as testing against the functional
requirements or even code-based criteria such as statement,
123
72
or branch, or path coverage [3]. The testing techniques and
supporting tools discussed in this paper will be of value in
reducing the likelihood of such bugs going undetected, and
will aid in their localization.
The problem of ensuring that a system is faithful to the
patterns underlying its design is even more serious during
the maintenance phase since system maintenance may be
the responsibility of team members who were not involved
with the original design or implementation of the system—
and consequently have only a superficial understanding of
its design. As a result, over time, the design integrity of the
system is likely to erode. The approach we develop in this
paper can help address this aspect of the problem as well.
This is because the test suite used to test whether the system
is faithful to its underlying patterns can be included as a
key part of the system’s design documentation in the same
manner as standard unit tests or other functionality tests. As
the system evolves during maintenance, the design test suite
will help system maintainers identify and localize changes
that may conflict with the original design. If one of these tests
were to fail, that doesn’t necessarily mean that the changes
in question are flawed. It may be that the design needs to
be modified; but this decision should be made consciously
with the design documentation being appropriately updated,
rather than simply modifying the code. We will return to this
point in the final section of the paper.
In standard software testing, we test software against its
specifications. These specifications must be formal since,
otherwise, there is no way to conclude whether a test was suc-
cessful or not1 . Similarly, to test whether a system is faithful
to the intent of the patterns underlying its design, we need
formal specifications of the patterns.
In previous work [12,26,31], we presented an approach to
providing precise specifications for patterns and their appli-
cations in the form of pattern contracts and subcontracts. It
is against these contracts and subcontracts that we wish to
test individual systems to validate their design correctness.
An important aspect of this approach to pattern speci-
fication is that the requirements and behavioral guarantees
specified in a pattern contract apply to all uses of the pattern;
a subcontract characterizes how the pattern is specialized in
a particular application. Thus the wisdom of the community
captured in a design pattern is reflected in the corresponding
pattern contract; it can be reused by every team that uses the
pattern in the design of its system. The individual team need
1 When is a test successful? One definition in the testing literature [3]
says that a test is successful when it shows that the software does not
meet its specification, since the purpose of the test is to find flaws in
the software. Others [20] use the convention that a test succeeds, or that
the software “passes” the test if, during the test execution, the software
behaves according to its specification. We will use this latter convention
in this paper.
123
N. Soundarajan et al.
only create the subcontract that specifies how the pattern is
specialized for use in a particular application.
A similar consideration is a primary motivation for the
approach we present in this paper. That is, a key goal of our
work is that the testing approach should enable us, in test-
ing whether a particular pattern P has been correctly imple-
mented in different systems designed using P, to reuse the
common parts of this effort rather than having to implement it
from scratch for each of these systems. Thus in the approach
we develop, corresponding to each pattern P, there is a set
of what we will call pattern test case templates (PTCTs). A
PTCT codifies a reusable test case structure designed to iden-
tify defects associated with applications of P in all systems
designed using P. Next we present a process using which,
given any system designed using P, the system tester can
generate a test suite from the PTCTs for P that can be used
to test the particular system for bugs in the implementation
of P. The process allows the system tester to tailor the set
of PTCTs corresponding to P to the needs of the particular
system by specifying a set of specialization rules that are
designed to reflect the structure and the scenarios in which
the defects codified in the set of PTCTs for P are likely to
manifest themselves.
An example will help illustrate our approach. Consider
the classic Observer pattern which we will use as our pri-
mary case study throughout the paper. According to the stan-
dard description [4,11] of the pattern, participating objects
play one of two roles: Subject or Observer. The intent of
the pattern is to keep multiple observers consistent with
the state of a single subject. When an object wishes to
become an observer of a subject, it invokes the subject’s
Attach() method. When it is no longer interested, it invokes
the Detach() method. In addition, Subject includes a
Notify() method which, according to the intended usage of
the pattern, is required to be called whenever the state of the
subject is modified. Notify() is required to invoke Update()
on each attached observer which must, in turn, update the
state of that observer to make it consistent2 with the current
subject state.
The standard UML diagram for the pattern appears in
Fig. 1. The subject maintains a set of references to the
currently attached observers in the variable _observers.
According to the standard description, when Attach() is
invoked, it adds a reference to the attaching observer to
_observers. But if this was all Attach() did, the intent of the
pattern would be violated; this is because the newly attached
observer’s state may not be consistent with the current state
of the subject – it will remain inconsistent until the next
2 What precisely terms such as “modified” and “consistent” mean is not
clear from such informal descriptions. The formal contract for Observer
which we will see in the next section will resolve this and other ambi-
guities in the informal description.
Patterns: from system design to software testing
Subject Observer
_observers
+Attach(in Observer) +Update()
1 *
+Notify()
+Detach(in Observer)
for all o in _observers
o.Update()
ConcreteSubject
ConcreteObserver
−subjectState *
1 −observerState
+Update()
Fig. 1 Observer pattern
notification cycle. To address this problem, Attach() must
invoke Update() on the newly attached observer. Failure to
do this is a common bug that appears in systems designed
using the Observer pattern. It is also a bug that can be missed
during standard (functional) testing since the inconsistency
of the newly attached observer may last only for a short
time, depending on when exactly the next modification in
the subject state takes place, resulting in a notification cycle
which will remove the inconsistency.
In our approach, given that this problem is common to sys-
tems designed using the Observer pattern, one of the PTCTs
for this pattern, as we will see, will correspond to this poten-
tial bug. When testing a system designed using Observer,
the system tester will specialize this PTCT appropriately to
generate a set of test cases that will help test the particular
system for the presence of this bug. These tests will confirm
that when a new observer in this system attaches to the sub-
ject, it is appropriately updated. In other words, these tests
will help ensure that the system is faithful to (this particu-
lar aspect of) the intent of the Observer pattern; and if the
system were to be modified during system evolution, the test
cases and the associated documentation will help the system
maintainers test for this bug and to identify any violations
of the intent of the underlying pattern that may be inserted
during this phase. Thus, the approach will help preserve the
design integrity of the system as it evolves.
Although some approaches to formalizing patterns have
been proposed [8,10,19,23] (in addition to our work cited
above), we are not aware of any other work focused on
the question of testing a system to see whether it correctly
implements the patterns underlying its design. Given the
widely recognized importance of design patterns and their
use in designing and implementing large software systems,
we believe that this is an important concern and believe that
the approach we present will prove valuable for this task.
73
Paper organization. Section 2 summarizes our approach to
pattern contracts and their relation to PTCTs. Section 3 devel-
ops the essential ideas of PTCTs and how they are defined
based on pattern contracts. A number of PTCTs for Observer
are presented based on the contract for the pattern. Section
4 considers how test cases, for a particular system designed
using a given pattern, may be generated from the PTCTs for
the pattern; a simple system, the simulation of a Hospital, is
used for illustration. Issues of coverage (and the associated
metrics) in the context of pattern-centric testing are also dis-
cussed. Section 5 briefly surveys elements of related work.
Section 6 concludes with a summary of the approach, and its
relation to test-driven design (TDD).
2 Pattern contracts
The contract for a design pattern P consists of a set of role-
contracts, one corresponding to each role of P, and a portion
that corresponds to the pattern as a whole. The role-contract
for a role R lists the state components of R, and specifies, in
standard pre-/post-condition format, requirements that must
be satisfied by the various methods of R. In a system designed
using P, a class C playing the role R will typically pro-
vide other methods in addition to those “named” in R. If
these “other” methods were to behave arbitrarily, then the
intent of the pattern would be violated even if the methods
corresponding to the named methods behaved according to
their specifications in the role-contract. In order to elimi-
nate this possibility, R’s role-contract will also include an
others specification that must be satisfied by all methods
of C, except those corresponding to the ones named in the
role-contract. The portion of the contract that corresponds
to the pattern as a whole consists of an invariant over the
(role) states of the various objects enrolled, at runtime, in an
instance of the pattern. The invariant will be satisfied when-
ever control is not inside any of the methods of any of the
participating objects.
One potential problem with formalizing a design pattern
is that its flexibility might be reduced or even eliminated.
A number of features of our formalism help guard against
this, the most important of these being the notion of an aux-
iliary concept. An auxiliary concept is a relation over one
or more states of one or more objects interacting with each
other according to the pattern. Auxiliary concepts are used
in the role-contracts and the pattern invariant, but their defin-
ition are not part of the pattern contract. Instead, definitions
tailored to particular systems, are provided in the subcon-
tracts corresponding to systems. The subcontract for a
particular system also includes a set of role-maps, one
corresponding to each class C playing a role R in the pattern,
as applied in this system. The C-R-role-map specifies how
the state, i.e., variables, of C map to the state of R (listed in its
123
74 N. Soundarajan et al.

role-contract), which methods of C correspond to each of the

named methods of R, etc.
As noted in Sect. 1, the pattern contract applies to all
systems designed using the pattern; the subcontract speci-
fies how the pattern is specialized for use in a given system.
Correspondingly, the pattern test case templates that we will
develop in the next section and will apply to all systems
designed using the pattern, will be based on the pattern’s con-
tract. The test cases for a given system will be obtained by
specializing the templates using the information in the sub-
contract and accounting for additional application-dependent
factors, as we will see later.
Consider the Observer pattern. The intent of the pattern, as
noted earlier, is to ensure that when the state of the subject is
modified, the observers “attached” to the subject are appro-
priately updated so that their states become consistent with
the current state of the subject. “Modified” does not, how-
ever, mean a change in any arbitrary bit or byte of the sub-
ject state; rather, some modifications are important enough to
require updates of the attached observers and others are not.
At the same time, which modifications are important enough
in this sense and which are not varies from system to system.
Hence, we use an auxiliary concept, Modified(), a relation
over two states of the subject, to distinguish “important”
changes in the subject state from “unimportant” ones. Sim-
ilarly, what it means for an observer state to be “consistent”
with the subject state varies from system to system; and, Fig. 2 Observer pattern contract (partial)
indeed, from one kind of observer that may be attached to
the subject to another that may be attached to the same sub- interactions are then expressed in the form of appropriate
ject. Hence we use a second auxiliary concept, Consistent(), conditions on the trace variables. As we will see in the next
a relation over an observer state and a subject state, to rep- section, PTCTs will be designed to test whether the sequence
resent this notion. The contract for Observer, a portion of of interactions that occur in a system designed using the pat-
which appears in Fig. 2, is written using these concepts and tern satisfy these conditions.
specifies the requirements that apply to all systems designed The contract for Observer starts by listing the auxiliary
using this pattern. concepts. Next (lines 6–8), we have a constraint that must
Many patterns are concerned with the sequences of inter- be satisfied by the definitions of these concepts in any sub-
actions between various objects, i.e., the sequences of method contract of Observer; we will consider this after consider-
calls invoked on the various objects involved in the pattern. ing the rest of the contract. Next (lines 10–11), we have
Thus, as we noted in Sect. 1, the Notify() method of the the pattern invariant, the reason (or the “reward” [26]) for
Subject role is required to call Update() on each of the using the pattern, that is guaranteed provided we meet all the
observers currently attached to the subject. Such require- requirements specified in the pattern contract. players[] is
ments are expressed in the pattern contract using the trace τ , the array of objects enrolled in the pattern instance in ques-
a ghost variable [15] provided by the formalism. In effect, tion and is another ghost variable of the contract formalism.
when a method m() starts execution, a corresponding trace In the case of an instance of Observer, the first player to
τ , initialized to the empty sequence, is automatically cre- enroll3 in any instance of the pattern is the subject; hence
ated. Each call that m() makes to a named method during
its execution is recorded as an element of the sequence, and 3 The complete pattern contract will also include clauses that specify
includes information about the object on which the method how an object enrolls to play a given role in a pattern instance, how
was invoked, the name of the method, and information about a pattern instance is created, etc. We omit the details of these here;
the associated arguments and return values. A number of see [12,26]. When testing a system designed using the pattern, it is,
of course, necessary that the requirements specified in these clauses
mathematical functions, some of which we will introduce
of the pattern contract (specialized for the particular system) are satis-
as needed, simplify trace manipulation, access to individual fied. Hence appropriate PTCTs must be defined, or these tests must be
trace elements, etc. Contract requirements concerning object included as part of other PTCTs.

123
Patterns: from system design to software testing 75

(a reference to) this object will be in players[0]. The rest in a particular application, that s1, s2 are two states of the
of the objects in players[] will be the objects that enroll to subject, and o1 is the state of an observer. Suppose that the
play the Observer role (by invoking the Attach() method, definitions of Modified() and Consistent() for this application
this being the action required for this purpose as specified are such that each of the following is true:
in the (elided from Fig. 2) enrollment clause for this role). Consistent (s1,o1), ¬Modified (s1,s2), and
Thus the invariant (lines 10–11) states that the state of the ¬Consistent (s2,o1)
first object in players[], i.e., the state of the subject in any
Suppose, at some point in the execution of this system, the
given instance of this pattern, is Consistent() with the states
subject is in the state s1 and the observer in the state o1.
of each of the observers (currently) enrolled in this instance.
Suppose, finally, that the subject state changes from s1 to
Next we have (part of) the Subject role contract. First we
s2. Then, according to the others specification in the Sub-
specify the state of the subject as consisting of obs, used to
ject role, the Notify() method will not be invoked and hence
store references to the currently attached observers. Next we
Update() will not be invoked on this observer and its state
have the specification of Attach(), one of the named methods
would remain as o1. At this point, the current state, o1, of the
of this role. The pre-condition requires that the attaching
observer will be inconsistent with the current state, s2, of
observer not already be attached, i.e., not have a reference
the subject, and the invariant that the pattern was intended
to it in obs. The post-condition states4 that a reference to the
to guarantee will be violated although the individual meth-
object is added to obs; that the subject itself is not modified,
ods in the individual roles, as actually implemented in the
in the sense of the auxiliary concept Modified(), and that one
particular application, satisfy their respective specifications.
call to a named method of the pattern has been made during
The problem is that we have conflicting notions of what a
the execution of Attach(), this being to the Update() method
sufficiently serious modification in the subject state is that
invoked on the attaching observer.
requires the observers to be updated on the one hand, versus
The others specification (lines 20–23) states that any
what it means for the subject state to be consistent with the
other method of the class playing the Subject role should
observer state, on the other. The constraint specified (lines
make no change in obs, and should either not modify (again
6–8) in the contract ensures that this problem doesn’t arise.
in the sense of Modified()) the subject state, or must invoke
Thus while the designers of a particular system are free to
the Notify() method (on the current subject). The (elided)
tailor the definitions of the auxiliary concepts to their partic-
specification of Notify() states that this method invokes
ular needs, they must ensure that these definitions satisfy the
Update() on each observer in obs.
constraint specified in the contract.
The Observer role contract states that the state of this
How realistic is this problem? That is, how likely is it
role consists of the variable sub (that holds a reference to
that the software team responsible for an actual application
the subject to which this observer is attached); that the
designed using the Observer pattern would base their sys-
Update() method does not change the value of sub, and
tem on such mutually conflicting notions of Modified() and
makes the state of the observer to be consistent (in the sense
Consistent()? It depends on the size of the team in question;
of the auxiliary concept Consistent()) with that of the sub-
as the team grows larger, the likelihood of such problems
ject. The others specification also states that it not modify
seem to also grow. But even for relatively small systems, the
sub, and leave the observer in a state that is consistent with
problem can creep in during system evolution; we will see
that of the subject5 .
an example of this later in the paper.
Let us now consider the constraint specified in the con-
Unfortunately, however, specific PTCTs (or, more pre-
tract (lines 6–8). Although the definitions of the auxiliary
cisely, test cases obtained by specializing them for a partic-
concepts, tailored to the needs of a particular application, will
ular application) cannot test these constraints because they
be provided by the corresponding subcontract, these defini-
are general conditions typically involving universal quanti-
tions cannot be completely arbitrary. For example, suppose,
fiers over the states of the various objects enrolled in the
pattern instance. Instead, violations of these constraints may
4 We use the “#” notation in the post-condition to refer to the pre- show up as violations of, for example, the pattern invariant—
condition value of the variable, i.e., its value when the method started although the individual methods invoked as part of the test
execution. case may satisfy their own specifications. We will see this in
5 Standard informal descriptions [11,25] of the pattern seem to suggest
the discussion of the example.
that the other methods of this role should not make any changes in
the state of the observer. But this is unnecessarily restrictive. As the
pattern contract states, all we need is that any changes in the observer 3 Pattern test case templates (PTCTs)
be such that they leave the observer state consistent with the sub-
ject state. This is an example [12,26] of how our pattern contracts can
identify dimensions of flexibility that may be missing in the standard Given the requirements captured in a pattern contract, how
informal descriptions. do we develop appropriate PTCTs that can be used to test

123
76
Fig. 3 Pattern-centric testing
whether systems implemented using the pattern satisfy these
requirements? It is important to stress that, given a pattern
contract, construction of the corresponding PTCTs is not a
mechanical activity. The purpose of the PTCTs is to help test
for common mistakes in implementing the particular pattern.
Clearly, which mistakes are common and which are not is a
question that can be answered only on the basis of experience
with systems designed using the pattern. In this section, we
will present three PTCTs, the first two corresponding to the
Observer pattern and the last one corresponding to the Com-
posite pattern. These are based on our assessment, based on
our own experience with implementing systems and based
on the reports in the literature, of the most common mis-
takes in implementing this pattern. PTCTs corresponding to
other patterns will similarly be based on an assessment of
commonly reported mistakes made in implementing them.
But these are, of course, not cast in stone. Indeed, with addi-
tional information about such mistakes, these PTCTs will be
refined and additional PTCTs introduced. The main contribu-
tion of this paper, then, is the approach presented for testing
systems to ensure the correct implementation of their under-
lying patterns—not the particular PTCTs presented. We will
return to this point in the final section.
Figure 3 schematically illustrates our approach to test-
ing systems against the patterns underlying their designs.
The pattern contract formally captures the requirements of
the design pattern, the subcontract specifies how the pat-
tern is specialized in the particular system. The PTCTs are
designed based on the most common mistakes that are made
in applying the pattern and are designed to check whether or
123
N. Soundarajan et al.
not the contract requirements are satisfied in those common
situations. These templates must then be specialized to obtain
the actual test cases corresponding to a particular system. The
system tester does this specialization, as we will see in the
next section, with help from JDUnit6 , a testing framework/-
tool that we are in the process of building, by combining
the information in the contract, in the subcontract and the
PTCTs, and taking account of any relevant details about the
particular system.
The language for defining PTCTs is intended to mirror
standard Java programming syntax to support rapid practi-
tioner adoption. Before looking at specific PTCTs, we sum-
marize the syntax of the structure of a PTCT. A PTCT is
represented as a segment of stylized Java code with inter-
spersed asserts that, based on the pattern contract that the
PTCT corresponds to, are expected to be satisfied at those
points. The asserts will be based on the pre- and post-
conditions of the named methods as well as the other spec-
ifications in the various role contracts of the pattern contract.
In addition, some of the asserts may also be based on
the pattern invariant. Since the PTCT is intended to apply to
all applications of the corresponding pattern, the construc-
tor and method calls used in its definition must conform to
the signatures specified in the relevant role contracts. When
the actual test cases corresponding to a given application are
generated from the PTCT, information in the corresponding
subcontract about the mapping of named methods of the
roles to particular methods of the corresponding classes and
the mappings of the role-method signatures to the class-level
method signatures will be used to translate the statements in
the PTCT to appropriate code in the test cases.
A PTCT begins with an initialization portion that creates
instances of one or more role types. named methods may
be invoked to bring the new player objects into states appro-
priate for the test case family being developed. The PTCT
developer may optionally include calls to other methods,
which serve as placeholders for class methods, which may
be introduced at the point of test case generation. Since the
precise number of other method calls required to bring an
object into the state appropriate for testing a particular system
varies, the PTCT syntax allows developers to specify that a
single other method may be instantiated by multiple class
methods, supplied at the point of test case generation. The
initialization section typically concludes with a PTCT pre-
condition. This assertion, expressed as a standard boolean
expression, captures properties assumed by the PTCT body.
The check ensures that the class-level constructor and method
6 The name of the framework, currently under construction, is intended
to draw a parallel to JUnit, the popular unit testing framework for Java;
the additional D emphasizes that the framework is used to test units of
design.
Patterns: from system design to software testing
calls introduced at the point of PTCT instantiation satisfy this
pre-condition.
Next, a PTCT defines a body. The body portion gener-
ally follows a standard testing idiom: First, local variables
are used to store the pre-conditional states of the players
participating in the PTCT. Next, invocations are placed to
the methods of interest, i.e., those relevant to the design
pattern requirements being tested. Finally, the PTCT con-
cludes with one or more assertions used to check that the
relevant pattern requirements are satisfied. These assertions,
as noted above, are based on the method and invariant spec-
ifications in the pattern contract and generally involve the
post-conditional states of the participating players, as well
as the pre-conditional states saved at the start of the body.
There is, however, one important deviation from the stan-
dard testing idiom: Since key parts of the pattern contract
involve the trace τ of method calls, the assertions contained
in a PTCT will correspondingly include conditions on a spe-
cial variable tau, used to record the trace of invocations orig-
inating from the PTCT. Note that the PTCT is not allowed
to contain any code that directly updates this variable; the
management of tau and updating it to reflect the method
invocations as they happen is the responsibility of JDUnit.
The one exception to this rule is that a PTCT may clear tau
when the earlier invocations recorded in it are no longer of
interest. In addition, JDUnit provides a number of helper
functions that we may use in a PTCT; these functions allow
us to obtain information about the current contents of tau.
We will see several of these in the PTCTs that we discuss
next.
There are some important differences in detail between
the trace τ used in the specifications in the pattern contract
and the trace tau maintained by JDUnit. The primary reason
for these differences is to enable us to construct more useful
PTCTs. In particular, suppose we have a PTCT that includes
a call to a method m1() followed by a call to method m2().
Thus the invocation of m1() completes before the invocation
of m2() begins; i.e., the execution of neither is nested inside
the other. Suppose the trace of method calls that takes place
during the execution of m1() is t1 and that during m2() is
t2. The trace tau maintained by JDUnit allows us to write an
assert following the completion of m2() that, for example,
imposes conditions on t2 based on the value of t1 or vice-
versa. We will see these details shortly.
The main portion of a PTCT intended to test the behav-
ior of the Attach() operation of the Subject role of the
Observer pattern appears in Fig. 4. As discussed earlier, a
common problem in using this pattern is not updating an
observer when it initially attaches to a subject. In the PTCT,
we begin by creating a subject s (line 1), and an observer o
(line 2). Next, we use a “getter” method provided by JDUnit,
to retrieve the pre-conditional value of s.obs before the call to
Attach() (line 7), and to store this value in pre_obs. Note
77
Fig. 4 PTCT for subject.attach()
that JDUnit provides appropriate getter methods for each
role field, including private fields.) Next, we add the attach-
ing observer to pre_obs (line 4) to simplify the expression
of the assertion check corresponding to the post-condition of
Attach() (lines 9–10). We additionally store the current state
of s (line 5), since this is required to check that Attach() not
modify s according to the applicable definition of Modified().
A couple of points should be noted here. The getter meth-
ods are more involved than simply returning the current value
of the field in question. In an actual system designed using the
pattern, the class C playing a particular role R may provide
a different set of fields from those specified in R’s role con-
tract. As we will see in the next section, the subcontract for
the system will specify the mapping from C’s fields to those
of R. The getter methods provided by JDUnit will make use
of this mapping to translate the values of C’s fields to the
corresponding values of R’s fields. Similarly, the clone()
operation applied to s (line 5) will create an object of what-
ever class plays the Subject role in the particular application.
Finally, JDUnit will use the auxiliary concept definitions
provided in the subcontract to evaluate assertions involving
concept references, such as the one involving Modified()
(line 9). We will return to these points in the next section.
Let us now consider contract requirements involving
traces. As we saw, in the contract formalism, each method
m() has an associated trace τ , initialized to the empty
sequence when the method starts execution. Information
about calls made by m() to named methods are recorded in
τ , with m()’s post-condition imposing necessary conditions
on τ . In those cases where a PTCT includes calls to methods
with trace requirements, the PTCT will include assert state-
ments that check relevant conditions on the associated trace
variables. Indeed, a PTCT may include assert statements
that require particular relations to hold across multiple trace
variables, each associated with methods preceding the asser-
tion check. To enable easy expression of such asserts, we use
the following approach. When the instantiated PTCT begins
execution, the testing framework creates a trace object tau,
initialized to the empty sequence. Each named method
123
78
invoked during the execution of the PTCT is recorded as
an element of tau, and includes information about the tar-
get object, the identity of the method invoked, etc. In addi-
tion, and this is the key difference from the use of τ in the
contract formalism, each element of tau itself includes the
trace of methods executed during the associated call’s exe-
cution. Thus, while there is only a single tau object, it main-
tains a branching structure corresponding to the computation
tree rooted at the PTCT. The traces contained in the individ-
ual elements of τ may be extracted using a simple accessor
(“tr()”) function provide by JDUnit.
Immediately prior to the call to Attach(), we clear tau
(line 6). This removes the trace entries associated with the
preceding constructor calls (lines 1–2), as well as any calls to
named methods introduced during the specialization process
in going from the PTCT to actual test cases; in the next
section we will see how the specialization may introduce
such calls. Hence, when control returns following the call to
Attach(), tau contains a single entry, corresponding to
this call. The trace of method calls that occurred during the
execution of this Attach() is extracted and stored in t1
(line 8). The assert that follows imposes appropriate condi-
tions on this trace, based on the requirements specified in the
pattern contract. The first two clauses require that s._obs be
appropriately updated (line 9), and that s not be Modified()
(line 10). The last three clauses (lines 11–13) address trace
conditions imposed on Attach(). Together, these clauses
require that Attach() invoke exactly one named method,
and that this call be to the Update() method of the attach-
ing observer7 .
The assert statement is, of course, based on the specifi-
cation of Attach() included in the Subject role contract
shown in Fig. 2. The effort involved in writing the PTCTs
corresponding to a pattern is primarily in identifying the
common mistakes that are made in applying the pattern and
coming up with appropriate test case templates based on this.
Once that has been done, the appropriate asserts to include
are determined by the pattern contract. But even here, the
PTCT designer may choose not to include all of the clauses
specified in the role contract as the post-condition of the
particular method. Even more importantly, at what points
to include checks of the pattern invariant and assertions that
require particular relations between the traces associated with
two or more method calls appearing earlier in the PTCT, are
all decisions the PTCT designer must make. And these deci-
7 We obtain the identify of the method invoked and the target object
by using the helper functions mt() and ob() provided by the JDUnit
framework. These functions may be applied to individual elements of a
trace in which case they return the identify of the method/target object
involved in that element. Or they may be applied to a trace, in which case
they return the sequence of method identities/target objects involved in
the various elements of the trace. We such usage in the pattern contract
in the last section.
123
N. Soundarajan et al.
Fig. 5 PTCT for subject.other()
sions will be based primarily on the common mistakes that
practitioners make in using the given pattern.
Note also the use of appropriate temporary objects to
capture pre-conditional values referenced in various post-
conditions; this is common in specification-based testing
given that post-conditions refer to both the state at the end of
the method and at its start. The pattern invariant (lines 6–7,
Fig. 2) could also have been checked as part of the assert
statement. This would amount to checking that Attach() not
only invokes Update() on the attaching observer, but also
that the latter method appropriately updates the observer’s
state to be Consistent() with the subject.
Next consider the PTCT shown in Fig. 5, intended to test
the behavior of Subject’s other methods. We begin by cre-
ating a subject (s) and two observers (o1, o2) (lines 1–3),
and attach both observers to the subject (line 4). We then
save the pre-conditional value of s.obs (line 5) and the value
of s as a whole (line 6). tau is then cleared, and some other
method is invoked on s (line 7). Finally, the trace associ-
ated with the other call is saved (line 8), and the behavior
of the method is checked against the requirements specified
in the pattern contract (lines 9–10). The assert statement
requires that s.obs be unchanged, and that both o1 and o2
be Consistent() with s.
In generating test cases for a particular system from this
PTCT, the s.other() call will have to be replaced by calls to
appropriate methods of the class playing the Subject role.
Some of these methods may require additional arguments. In
the next section, we will consider the problem of generating
suitable values for these arguments. At this point, however,
the more important issue is the mismatch between the assert
statement and the requirements specified in the Observer con-
tract. According to the contract (Fig. 2), when s.other()
terminates, either the state of s must not be Modified(), or
the Notify() method must have been invoked. As we have
already seen, this method will in turn invoke Update() on
each attached observer, bringing the objects into states con-
sistent with the new state of the subject. Hence, the assert
included in the PTCT is a test of the expected net behavior
of the participating objects, i.e., it is a more “global” test of
the system. There is, however, a risk in using such a PTCT,
especially if these were the only ones used. If, for instance,
Patterns: from system design to software testing
an other method were to modify the state of the subject,
neglect a call to Notify(), but by chance leave s in a state
Consistent() with the states of o1 and o2, the design defect
would go undetected.
In [27], we present a scenario in which this problem mani-
fests itself during system evolution; we summarize the exam-
ple here. A class S1 plays the Subject role, and provides two
fields, f1 and f2. A change in either field is considered to be
a modification of the subject according to the definition of
Modified() supplied in the system’s subcontract. A class O1
plays the Observer role. In an initial version of the system,
O1 is interested only in the value of f1; changes in f2 are
ignored. The Update() method of O1 uses an appropriate
getter method to retrieve the value of f1, and then updates
the observer’s state to become Consistent() with the state of
the subject. The Consistent() concept is defined suitably in
the subcontract. The S1 class includes a bug that omits a call
to Notify() when a change in f2 occurs. A PTCT similar to
the one above will not detect this defect since each observer
will remain Consistent() with the subject when the other
method terminates — even if f2 has changed without a cor-
responding call to Notify().
During system evolution, O1 is modified so that the value
of f2 becomes significant. Instances of O1 record the cur-
rent value of f2 associated with their corresponding subject.
Hence, the definition of Consistent() is suitably modified, as
is the implementation of O1’s Update() method. The new
implementation retrieves the values of both f1 and f2, and
updates the state of the observer appropriately. When the test
case derived from the PTCT shown in Fig. 5 is executed, the
assert statement will generate an error; the clause involv-
ing the Consistent() concept will be violated. The natural
assumption is that the fault lies in an area affected by sys-
tem evolution. In fact the defect lies in the original S1 class
— in particular, the failure of its other() method to invoke
Notify() when the value of f2 changes.
The solution is to revise the PTCT to fully test the interac-
tion requirements specified in the pattern contract, rather than
testing for net effects. More precisely, the last two clauses of
the assert in Fig. 5 (line 10) should be replaced with the
following conditions:
8 a s s e r t ( … clauses from Fig. 5 …
9 && (! M o d i f i e d ( pre_s , s ) ||
10 (( t1 . l e n g t h ( ) = = 1 ) & & ( t1 . m ( ) = = " N o t i f y "))
Before concluding this section, we will consider another
PTCT, this one for the Composite pattern. Composite is a
structural pattern [11] intended to compose objects into a
tree structure to represent part-whole hierarchies and to allow
clients to treat individual objects and composite objects in a
uniform manner. The pattern has three roles, Component,
Leaf, and Composite. Component is an abstract role with
Leaf and Composite being the two kinds of components.
79
Fig. 6 Composite role contract (partial)
Patterns such as Composite, although classified as structural,
also have behavioral aspects to them. Indeed, the main pur-
pose of the Composite pattern is to ensure that each operation
of a composite object is implemented by invoking the cor-
responding operation on each of its components and com-
bining the results returned by these invocations appropriately
to obtain the result of the original operation invocation.
A small portion of the Composite pattern contract, in par-
ticular a part of the Composite role contract, appears in
Fig. 6. The specification of any operation() of this role
states that when this method finishes, the set of children
is unchanged; that during the execution of this method, there
should have been a call to the corresponding method on each
child c; and that the result returned by this method should be
equal to the value obtained by appropriately combining the
results of these method calls invoked on the children. Since
the details of precisely how these results are combined to
obtain the result of the Operation() applied to the com-
posite will vary from one application designed using this
pattern to another, we use an auxiliary concept, Glue() to
represent this. Thus the final clause of the post-condition
of Operation() requires that the result returned by
the original call is equal to the value of Glue() applied to
the sequence of results in the method calls recorded in τ ,
this being obtained by means of the helper function,
resultSeq().
Figure 7 presents a PTCT to test that the requirements
specified in Fig. 6 are satisfied. We create two composites, c1
and c2, a leaf l, and create a suitable composite structure. We
then invoke operation() on the top level composite struc-
ture and, assert, after the call returns that the trace created
during the execution of this method is equal to the number of
children of this structure, and that the result returned matches
that obtained by applying the auxiliary concept Glue() on
the results returned by the calls recorded in that trace. The
definition of Glue() will, of course, be given in the sub-
contract corresponding to a particular system, for which we
will specialize this PTCT to produce actual test cases. It may
be worth noting that this PTCT does not completely test the
specification in Fig. 6; in particular, while it does test that the
123
80
Fig. 7 PTCT for composite.operation())
number of method calls recorded in the trace t1 is what it
should be according to the specification, it does not test which
methods were invoked in these calls, nor on what objects. It is
straightforward to modify the PTCT to include these checks
and we omit the details.
4 Generating test cases
Given PTCTs such as those in the last section, how do we
generate actual test cases corresponding to a given system S
designed using a particular pattern? Consider, for example,
line 7 of Fig. 4. The method Attach() that is being invoked
in this line may have an entirely different name in S. Indeed,
even the classes playing the Subject and Observer roles
will have their own names appropriate for the application.
To illustrate these and other issues involved in generating
actual test cases, we will use a simple Hospital Simulation
system, parts of which appear in Fig. 8, designed using the
Observer pattern.
The Hospital system consists of three main classes,
Patient, Nurse and Doctor. Instances of the Patient class
play the Subject role; instances of the other two play the
Observer role. Zero or one doctor object and zero or more
nurse objects are assigned to observe patient. The variables
_nurses and _doctor in the Patient class are used to keep
track of the assigned doctor/nurses. A nurse object tracks
the temperature (_temp) of the patient it is observing; a
doctor object tracks the heart-rate (_hrtRt). The checkVi-
tals() method updates _temp and _hrtRt as needed, and
invokes notify() if needed. The Nurse and Doctor classes
include an update() method to help keep track of the data
about the patients being observed. They also provide a get-
Status() method that returns information about the patients.
Before considering how test cases corresponding to the
Hospital system may be generated from the PTCTs of the
last section, we have to consider the subcontract that specifies
how the Observer pattern is specialized in this system; the
main part of this subcontract appears in Fig. 9.
123
N. Soundarajan et al.
Fig. 8 Hospital simulation code
The role map for Patient as Subject specifies how the
_obs of this patient viewed as as subject may be obtained
from the state of the patient. Next, the method maps specify
that addNurse() and setDoctor play the role of Attach()
and that notify() plays the role of Notify(). The role map
for Nurse as Observer illustrates a technical problem: since
a nurse may be observing multiple patients, each of which
is the subject in a distinct instance of Observer, how do
we identify the particular patient involved with the current
pattern instance? The contract formalism [12,26] provides
the notion of lead object to address this. The lead object in
a pattern instance is the first object that enrolls in a pattern
instance, i.e., the one that is in players[0]. The method map
for Nurse maps its update() (applied to the lead object) to
Update() of the Observer role. The role map for Doctor
as Observer is analogous.
Next we have the definitions of the auxiliary concepts.
According to the definition of Modified() (lines 25–27), the
state of a patient is considered to be modified if either _temp
value or the _hrtRt value has changed. The definition in lines
28–29 state that a nurse n’s state is consistent with that of the
patient p if n has the correct information about p._temp.
The definition (lines 30–31) of consistency of a doctor d
and patient p is similar.
Patterns: from system design to software testing
Fig. 9 Hospital-observer subcontract
Let us now turn to the main task of this section, generat-
ing, from the PTCTs for a pattern, test cases for a particu-
lar system designed using the pattern. For any such PTCT,
a (large) number of test cases may be generated. Each test
case would be obtained by replacing each role object in the
PTCT, such as s, o1, and o2 in the case of the PTCT in
Fig. 5, by corresponding application objects, as determined
by the subcontract, such as a patient, a nurse, and a doc-
tor, respectively, in the case of the Hospital system; replacing
the calls to role methods in the PTCT, such as s.Attach(o1)
and s.Attach(o2), by calls to the appropriate application
methods, again as determined by the subcontract, such as
s.addNurse(o1) and s.setDoctor(o2), respectively; etc.
While that much is straightforward, there are some impor-
tant additional issues to consider. For example, it may be that
the constructor functions (or other methods, for that matter)
of some of the application classes require additional argu-
ments. What values do we supply for these? The PTCT, of
course, imposes no requirements on these. As far as the PTCT
and the pattern are concerned, any values may be used for
these additional arguments. This means that the number of
possible test cases we can generate will be very large. Which
of these test cases will actually be useful will depend on
the system, in particular its implementation details. Hence
the human tester using our approach will be required to
81
suitably specialize the PTCT by defining appropriate test case
specializers.
A key purpose of a test case specializer (TCS) is to limit
the set of generated test cases, i.e., to specialize the PTCT
in such a manner that the test cases that are generated are
the ones most appropriate to the system under test. There
are a number of dimensions in which this specialization has
to be performed. First, the objects constructed in setting up
the scenario for a PTCT are role objects. If R is a role of
this pattern and, in the PTCT, we construct an object ob of
type R, and if, according to the subcontract for the system
in question the class C1 and C2 play the role R, then in an
actual test case, ob may be either a C1 object or a C2 object;
the TCS has to specify which it is.
Second, calls in the PTCT to named role methods (such as
Attach()) have to be replaced by calls to the corresponding
class methods, as specified in the appropriate role maps of
the subcontract. This process, by itself, does not require any
additional information from the TCS – unless more than one
class method plays the role method in question or, as is more
frequently the case, the class method in question requires
additional arguments beyond the ones expected by the role
method. These values will, of course, have to be provided by
the TCS. There are two important considerations to account
for here. First, the application class may specify, as part of
the pre-condition of the corresponding class method, some
conditions that must be satisfied by these argument values. If
some set of argument values generated according to the spe-
cialization in the TCS do not satisfy these conditions, that
test case cannot be used. For example, in the Hospital sys-
tem, we are not allowed to attach two doctor objects at the
same time to a given patient object. In other words, an
attempt to call addDoctor() on a patient object that
already has a doctor will violate the pre-condition of this
method and hence the method may behave in whatever way
it chooses and this is not a bug in the system, despite the fact
that, because of such behavior, an assert in the test case
may fail. At the same time, we cannot always eliminate this
type of problem when the test case is generated because some
of these assertions may be checkable only during execution
of the test case. In order to address this type of problem,
JDUnit inserts additional asserts in the test cases it gen-
erates immediately prior to calls to any methods. Each of
these asserts checks that the pre-condition of the method
being called is satisfied. If, when a test case is executed,
such an assertion is not satisfied, that is recorded in the log
maintained by the system; but the assertion failure does not
indicate a failure of the test case8 . The second issue that the
8 This is a distinction from the JUnit framework where every assertion
violation is reported as a failure. In order to allow a different treat-
ment for pre-condition violations, in our design of JDUnit, we use an
additional method, preAssert, to specify such assertions.
123
82
tester designing the TCS has to be concerned about in decid-
ing on the argument values is producing a large enough set of
test cases with different values for these arguments to ensure
adequate coverage of the system under test.
Next, each other() call that appears in the PTCT must
be converted to a call to one of the methods of the appro-
priate application class, other than the ones mapped to the
various named role methods. In addition, as in the case con-
sidered above, if the method in question requires additional
arguments, the TCS must provide these as well. And, as in
the case considered above, we must account for any pre-
conditions that may be imposed by the application class in
question.
The final dimension of specialization of the PTCT that the
TCS must account for is concerned with a construct that may
appear in a PTCT but, as it turns out, was not not used in
any of the PTCTs in the last section. This is the “other*();”
construct, which denotes a sequence of zero or more calls
to other methods. This construct may appear in a PTCT
at points where the PTCT designer wants to allow for the
possibility that a number of additional calls to other methods
may be needed in order to get the various objects into suitable
states for continuing the test. The TCS will have to specify
the sequence of method calls that must be made in place of
the other*() construct.
Let us now see how these considerations play out when
applied to some of the PTCTs (for the Observer pattern) when
we specialize them to test the Hospital system. Consider the
PTCT in Fig. 4. Line 2 of this PTCT requires an observer
object to be created. Suppose the TCS designer decides that
this object should be a doctor. In the notation for TCS that
we are designing concomitantly with the design of JDUnit,
this may be expressed as:
8 O b s e r v e r () { @2 } = D o c t o r ( /* args */ );
On the other hand, given that both Doctor and Nurse play
the Observer role in this application, the TCS designer may
want to create two test cases, one corresponding to each of
these possibilities. This is expressed in our notation as:
8 O b s e r v e r () { @2 } = D o c t o r ( /* args */ )
9 || N u r s e ( /* args */ );
Let us consider another statement that appears in the same
PTCT, the call to Attach() in line 7 of the PTCT in Fig. 4.
The following specializes that call so that it is replaced by a
call to the setDoctor() method:
8 A t t a c h ( o ) { @7 } =
9 { s e t D o c t o r (o , /* a d d i t i o n a l args */ )}
If, as we considered above, the observer that was con-
structed earlier (line 2 of the PTCT) could be either a doctor
or a nurse, our notation for TCS allows the tester to spec-
ify which method must be used when replacing the call to
Attach() in line 7:
123
N. Soundarajan et al.
8 A t t a c h ( o ) { @7 } =
9 { D o c t o r ( o ): s e t D o c t o r ( o , /* addnl args */ )
10 || N u r s e ( o ): a d d N u r s e ( o , /* addnl args */ )};
Strictly speaking, it should not be necessary for the tester to
provide this information since it is available from the sub-
contract (Fig. 9). But in those cases where multiple methods
of the same class may play the role of a named method, the
TCS will have to specify which of those must be used. This
is, of course, very much needed in dealing with other() calls
that appear in the PTCT since, in general, there will be many
class methods that qualify as other methods. In the case of
other*() calls, we must provide the sequence of method
calls that should replace it and the arguments for each call
in the sequence. For example, suppose the other() call in
line 5 of the PTCT in Fig. 5 was an other*() call. Then we
may have the following:
8 o t h e r *() { @7 } =
9 { c h e c k V i t a l s (); c h e c k V i t a l s ();}
Here the other*() call is replaced by a sequence of two calls
to checkVitals(). We can also use the “||” notation to allow
for alternative sequences of calls.
The detailed syntax for TCS is very much a work in
progress. For example, in the situation we just considered,
if the tester wanted to test a range of different sequences
of other calls, listing them all explicitly would be unde-
sirable. It would be more convenient if we could use regu-
lar expression-like notations for such situations. We intend
to apply our approach to a number of patterns and systems
designed using the patterns to determine the exact notational
facilities that TCS must include. In addition, practitioner-
friendliness will be a key consideration since it is the testers
who are part of individual software teams who will be respon-
sible for defining the TCSs for their particular systems.
Before we conclude this section, one final point about the
Hospital system is worth briefly discussing. Suppose, dur-
ing system evolution, it is decided that the nurse objects
observing a given patient object should keep track also of
the medicine-level, i.e., the value of the _medLvl variable
of the Patient class. It would be easy enough to modify the
Nurse class to do so, perhaps by introducing a new variable,
_medLevels, similar to the Nurse._vitals; and to revise
Nurse.update() so that the value of _medLevels as well
as _vitals is appropriately updated.
While this seems simple and reasonable, if this is all we
do, there will be a bug in this system! The problem is that, as
it is implemented, the Patient class methods do not invoke
notify() if only the _medLvl value changes. So in those sit-
uations where only this variable changes, the attached nurse
objects will have incorrect information about medLvl (until
the next time the patient’s _temp or _hrtRt changes, trig-
gering a call to notify()).
Patterns: from system design to software testing
Unfortunately, however, the test cases generated using the
PTCTs we have seen will not catch this error. The problem
is that, according to the definition of the auxiliary concepts
in Fig. 6, if only the value of _medLvl changes, the state
of the patient is not considered to be modified; moreover,
the state of nurse is considered to be consistent with that
of the patient as long as the former has the correct infor-
mation about the value of the _temp of the latter. Thus any
test cases generated from the PTCT in Fig. 5 will succeed
when executed. If we were to change the definition of one
of these concepts but not the other, the constraint specified
in the original pattern contract (lines 6–8 of Fig. 2) will be
violated. Since such a problem could easily arise during sys-
tem evolution, it is important to ensure that our PTCTs check
that constraints on the auxiliary concepts are satisfied. At the
same time, the fact that these constraints are typically uni-
versally quantified over the states of various roles makes this
a challenge. This is one of the questions we intend to address
in future work.
5 Related work
The PTCT testing approach builds upon our prior work in
pattern formalization [12,26,31]. The key conceptual ele-
ments of this work were summarized in Section 2. Several
other groups have also considered improving the precision of
pattern descriptions. Some have described techniques based
on diagrammatic notations that extend standard UML(-like)
syntax [7,21,32]. Closer to our specification approach is that
of Eden et al. [9,10]; the authors describe an approach to cap-
turing the structural properties of patterns using a logic nota-
tion. Patterns are expressed as logic formulae that specify
participating classes, methods, inheritance hierarchies, and
the structural relations among them. Mikkonen [19] describes
a behavioral specification approach based on DisCo [14], a
UNITY-like [5] action system notation. Data tuples represent
objects, and guarded actions model their interactions. Helm
et al. [13] describe a contract formalism that bears similarity
to ours. In particular, the formalism has constructs analogous
to our role contracts, auxiliary concepts, and a limited form of
trace conditions. The formalism additionally provides some
support for contract specialization and system-level reason-
ing. However, despite the effort invested in documenting pat-
terns precisely, there appears to be relatively little prior work
in testing the correctness of systems constructed according
to particular pattern descriptions, either formal or informal.
By contrast, this has been the focus of our work.
Some readers may take note of the fact that the phrase
“test(ing) patterns” appears throughout the literature.
Indeed, there is at least one workshop series devoted to testing
patterns [28–30], a corresponding online repository [16], and
widely referenced books that detail the subject [3,18]. So how
83
can the work described here be new? The standard meaning of
testing pattern refers to a pattern used to address a recurring
problem in the context of software testing. Indeed, a test(ing)
pattern is simply a special type of design pattern [17], one
intended to guide the design of testing code. Our work, how-
ever, is focused on validating the correct usage of design
patterns through software testing. We note that other authors
have also considered applying design patterns as part of the
system design process to improve testability [1,6]. Again,
however, our focus has been on ensuring the correct applica-
tion of patterns.
It is important to note that McGregor [17,18] extends the
traditional notion of a test(ing) pattern to patterns intended
for use in validating design pattern implementations. The
concept is similar to our use of PTCTs in that each test
pattern specifies a particular scenario of application objects
that interact in ways intended to check pattern requirements.
However, the scenarios are not based on precise pattern spec-
ifications. Equally important, McGregor does not consider
test suite specialization, nor automated generation. Still, the
similarities are important.
Finally, it may be useful to mention the SABER system
described by Reimer et al. [22]. The system is designed to
detect defects in large Java applications using static analy-
sis; SABER operates on systems designed using particular
frameworks. Correct use of these frameworks requires that
certain methods be invoked in certain orders, that others not
be invoked at certain points, etc. SABER attempts to iden-
tify violations of such requirements based on call sequence
rules specified using an XML dialect. Given the importance
of call sequence requirements in pattern-based systems,
elements of SABER may be useful in detecting pattern
violations statically, prior to pattern testing. We intend to
explore this possibility as part of future work.
6 Conclusion
We began with the observation that design patterns play
a central role in software practice, influencing the design
of most major software systems and class libraries. Conse-
quently, techniques for ensuring the correctness of pattern
implementations are likely to have a profound impact on the
reliability of real-world systems. To this end, we focused
on an approach to testing pattern implementations based
on precise pattern specifications. Whereas our prior work
provides the specification foundation, the work reported
here provides the foundation for specification-based testing
of pattern implementations.
In developing the testing approach, we observed that pat-
tern implementations follow a similar structure. This is not
surprising. After all, for a designer to claim that a particu-
lar group of classes have been designed according to a given
123
84
pattern, she must have satisfied certain requirements—
namely, those common to all applications of the pattern in
question. It then follows that the test cases used to vali-
date the pattern’s implementation share a common structure
across systems designed using the pattern. The central moti-
vation of our testing approach is that this common structure,
applicable to all systems designed using the pattern, should
be reusable. In defining the test cases for a particular system,
testers should only be required to invest the incremental effort
necessary to specialize the generalized structure as appropri-
ate to the system under test. Moreover, the ability to define
reusable test case structures enables the software engineer-
ing community to encode test case families that are likely to
reveal the most common pattern implementation errors.
We described a testing approach that satisfies these goals,
and the design of a software tool used to assist in test case gen-
eration. In our approach, pattern test case templates (PTCTs)
are used to define generalized test case structures reusable
across applications of particular patterns. Each template is
expressed using a Java-like syntax amenable to practitioner
adoption; the code is written in terms of the role types and role
methods specified by the corresponding pattern specification.
To test the behavior of a system implemented using a partic-
ular pattern, the corresponding PTCTs must be instantiated
to generate executable test cases. The instantiation process
is guided by a specification of the mappings between pat-
tern roles and system classes, as well as rules that govern
system-specific specializations. JDUnit assists in the instan-
tiation process by guiding developers in selecting the appro-
priate PTCTs and pattern specifications, and in introducing
application-specific objects, method calls, and arguments.
The output of the tool is a set of JUnit test cases used to
validate the correctness of the relevant patterns as applied in
the system under test9 .
We conclude with a short comment on the relationship
between our approach to pattern-centric testing (PCT), and
test-driven-development (TDD) [2]. In TDD, test cases that
capture the expected behavior of a component are written
before the component is implemented. Hence, TDD dictates
when test cases must be designed. On the other hand, PCT is
concerned with what must be tested. One could, as dictated by
TDD, develop our test cases before a system is implemented.
(Of course, relevant portions of the design must be complete
since otherwise we would not know which PTCTs to instan-
tiate.) Alternatively, following a more conventional process,
we may design and implement the system first, and then,
9 A key guideline for using the JUnit framework is that the tests should
be designed to test behavior rather than individual methods [20]. This
also applies to our approach, except that the behaviors in question are
those corresponding to the use of a particular pattern. Hence, the PTCTs
typically involve multiple roles. Consequently, the instantiated test cases
involve multiple system classes, by contrast to more traditional unit
tests.
123
N. Soundarajan et al.
during testing, use PCT to generate appropriate test cases.
Hence, PCT and TDD are fully compatible, but orthogonal
testing principles.
References
1. Baudry B, Le Sunyé Y, Jézéquel J (2001) Towards a ‘safe’ use of
design patterns to improve OO software testability. In: The 12th
international symposium on software reliability engineering, IEEE
Computer Society, Washington, DC, pp 324–329
2. Beck K, Gamma E (1998) Test infected: programmers love writing
tests. Java Rep 3(7): 37–50
3. Binder R (1999) Testing object-oriented systems. Addison-
Wesley, Menlo Park
4. Buschmann F, Meunier R, Rohnert H, Sommerlad P, Stal M
(1996) Pattern-oriented software architecture: a system of patterns.
Wiley, New York
5. Chandy K, Misra J (1988) Parallel program design. Addison-
Wesley, Menlo Park
6. Dasiewicz P (2005) Design patterns and object-oriented software
testing. In: The 2005 Canadian conference on electrical and com-
puter engineering, IEEE Canada, Dundas
7. Dong J (2002) UML extenstions for design pattern compositions.
In: Mingins C (ed) Proceedings of TOOLS, in special issue of
journal of object technology, vol 1, issue 3, pp 149–161
8. Dong J, Alencar P, Cowan D (2001) A behavioral analysis
approach to pattern-based composition. In: Proceedings of the 7th
international conference on object-oriented information systems,
Springer, pp 540–549
9. Eden A (2001) Formal specification of object-oriented design. In:
Proceedings of the international conference on multidisciplinary
design in engineering
10. Eden A (2002) LePUS: a visual formalism for object-oriented
architectures. In: Proceedings of the 6th world conference on inte-
grated design and process technology, IEEE Computer Society,
pp 149–159
11. Gamma E, Helm R, Johnson R, Vlissides J (1995) Design patterns:
elements of Reusable OO Software. Addison-Wesley, Menlo Park
12. Hallstrom J, Soundarajan N, Tyler B (2006) Amplifying the ben-
efits of design patterns. In: Aagedal J, Baresi L (eds) Proceedings
of the 9th international conference on fundamental approaches to
software engineering (FASE), Springer, pp 214–229
13. Helm R, Holland I, Gangopadhyay D (1990) Contracts: specifying
behavioral compositions in object-oriented systems. In: OOPSLA-
ECOOP, pp 169–180
14. Järvinen H, Kurki-Suonio R (1991) Disco specification language:
marriage of actions and objects. In: Proceedings of the 11th inter-
national conference on distributed computing systems, IEEE Com-
puter Society, Los Alamitos, pp 142–151
15. Jones C (1990) Systematic software development using VDM.
Prentice-Hall, Englewood Cliffs, New York
16. Marick B (2007 (date of last access)) Test patterns repository/
software testing patterns. http://www.testing.com/test-patterns/
patterns/
17. McGregor J (1999) Testpatterns: please stand by. J Object-oriented
Program 12:14–19
18. McGregor J, Sykes D (2001) A practical guide to testing object-
oriented software. Addison-Wesley, Menlo Park
19. Mikkonen T (1998) Formalizing design patterns. In: Proceedings
of 20th ICSE, IEEE Computer Society Press, pp 115–124
20. Rainsberger J (2005) JUnit recipes. Manning
21. Reenskaug T (1996) Working with objects. Prentice-Hall, Engle-
wood Cliffs
Patterns: from system design to software testing
22. Reimer D, Schonberg E, Srinivas K, Srinivasan H, Alpern B,
Johnson R, Kershenbaum A, Koved L (2004) “saber”: smart analy-
sis based error reduction. In: Proceedings of “ISSTA ’04”, ACM
Press, pp 243–251
23. Riehle D (1997) Composite design patterns. In: Proceedings of
OOPSLA, ACM, pp 218–228
24. Schmidt D, Stal M, Rohnert H, Buschmann F (1996) Pattern-
oriented software architecture: patterns for concurrent and net-
worked objects. Wiley, New York
25. Shalloway A, Trott J (2002) Design patterns explained. Addison-
Wesley, Menlo Park
26. Soundarajan N, Hallstrom J (2004) Responsibilities and rewards:
specifying design patterns. In: Finkelstein A, Estublier J,
Rosenblum D (eds) Proceedings of 26th international conference
on software engineering (ICSE), IEEE Computer Society, pp 666–
675
85
27. Soundarajan N, Hallstrom J (2006) Pattern-based system evolu-
tion: a case-study. In: Zhang K, Spanoudakis G, Visaggio G (eds)
Proceedings of 18th international conference on software engineer-
ing and knowledge engineering (SEKE 2006), Knowledge Systems
Institute, pp 321–326
28. Testing (2001) Patterns of software testing 1
29. Testing (2001) Patterns of software testing 2
30. Testing (2001) Patterns of software testing 3
31. Tyler B, Hallstrom J, Soundarajan N (2006) A comparative study of
monitoring tools for pattern-centric behavior. In: Hinchey M (ed)
Procedings of 30th IEEE/NASA sofware engineering workshop
(SEW-30), IEEE-Computer Society
32. Vlissides J (1998) Notation, notation, notation. C++ Report
123