Testing Visibly - Providing

a line of sight into test

operations
David
Kapfhammer
Global
Director
david.kapfhammer@keane.com
 Topics/Outline
The
Interrogatives
• What
is
“Testing
Visibly”?
• Why
is
it
important?
• When
should
the
techniques
for
Testing
Visibly
be
employed?
• Who
is
responsible
for
ensuring
that
we
Test
Visibly?
• Where
(organizationally)
should
this
technique
be
applied
and

where
are
the
relationships
with
other
Software/Systems

Engineering
Teams?
• How
does
it
work?

How
do
you
do
it?
• Where
do
we
go
from
here?
 What is Testing Visibly?
Determine
the
success
criteria,
metrics,
and
result

data
to
be
provided
by
testing
at
the
start
of
the

process,
deliver
these
results
constantly
and
review

frequently
 What is Testing Visibly?


Make
an
early
appearance
in
the
Software
Development
Life
Cycle
(SDLC)
• Test
Early
and
Often
• Reduce
overall
cost
of
defects



Share
Testing
Operations
Process
and
Policy
with
the
Organization
• Empowers
others
to
understand
how
we
work
and
to
set
expectations
• Provides
a
baseline
for
change
early
on;
if
needed




Keeping
an
open‐door
policy
• Do
not
remain
cloaked
in
secrecy
• Invite
other
teams
to
status
and
reporting
meetings
while
in
test
 Why is it important to test visibly?
Testing
is
the
microscope
into
 Value
Added
Contribution
viewing
an
organization’s
ability
to
 • Test
Early
and
Often
build
quality
software.

 • Reduce
overall
cost
of
defects
Commoditization
of
IT
and
the
relevance
to

It
tells
us
more
than
whether
or
not
 Business
Goals
software
satisfies
a
specified
 • Remaining
relevant
and
a
valued
contributor
re‐
enforces
requirement
for
test
requirement,
it
is
the
catalyst
for

illustrating
how
effective
the
 Visibility
makes
the
components
of
the
Triangle
of

organization
works. Quality
unambiguous
and
easy
to
understand
the

impact
of
each
component
When should the techniques of Testing Visibly be employed?

• Early
and
often
–
Sound
familiar?

• Plan
to
deliver
results • Seek
out
their
participation
• As
early
as
possible
and
frequently;
keep
 • Often
times
other
teams
have
“moved

your
audience
engaged
and
informed on”
to
the
next
release

• Know
when
to
report
and
to
whom
 Who is responsible for Testing Visibly?

Everyone
• All
layers
of
the
testing
organization
have
some
contribution
to
testing

visibly.

Test
Leadership
•Define
and
drive
the
vision

•Take
ownership
of
the
state
of
the
relationships

Test
Engineers
•Collect
data
for
reporting
purposes
•Communicate
and
collaborate
with
counterparts

Allowing
a
project
team
insight
into
testing
operations
does
not
discount
the

credibility
of
the
test,
nor
does
it
disintegrate
the
nature
of
being
unbiased.

 Relationships and Touch Points
!!0&.),12'
!"#$%&'()*"+,-".,/+' !!(3%,*".,/+'./'!456'
Quality !!7&8&1.'7"."'
Sponsors Offshore
Manager !!921"%".,/+'

Engineering Test Managers

Core Teams
DEV TM 1 …
Functional Unit 1 Functional Unit 2 Functional Unit N

Test Lead Test Lead Test Lead

PM TM 2 Test Analysts Test Analysts Test Analysts

Testers Testers Testers

Environment TM 3

UAT Team

Arch TM 4
 How does it work?

We
need
to
share
with
the
Organization
how
we
work
and
how
we
expect
to
work

with
others.
This
is
most
effectively
conveyed
via
a
User
Manual

• The
Users
Manual
is
a
guide
for
teams
to
use
in
its
interaction
with
the
testing
team.


It
describes
the
following:

✴ The
environment
and
systems
portfolio
that
the
testing
team
will
support

✴ The
organization
of
the
testing
team

✴ How
to
request
and
receive
services
from
the
testing
team

✴ Supporting
defining
documents
that
prescribe
how
the
testing
team
will
work
 The User Manual
The
underlying
principles
supporting
testing
services
include:

• 
A
consistent,
repeatable,
flexible,
and
enterprise‐oriented
framework

• 
Inserting
testing
into
all
phases
of
the
software
development
life
cycle

• 
Establishing
quality
as
part
of
the
task
from
its
inception


• 
Identification
of
risks
early,
and
applies
task
and
quality
resources
as
needed
to
mitigate

those
risks


• 
Aggressive
use
of
metrics,
trend
analysis,
and
lessons
learned
to
evaluate
performance

• Change
Control • Quality
Assurance
Request
Process
• Acceptance
Management
• Request
Prioritization • Risk
Management
• Status
Measuring
and

• Issue
Management
• How
to
check
request
status Reporting

• Performance
Measurement
and
 • Staff
Rotation/Replacement
• How
to
escalate
requests Reporting
 '"%02$/&
!"#"$%& '()(*"+")%&
'()(*"+")%& ,-()&
,-()& 7<++/&,-()&

Orchestrating
the Defining
3"/%&4%0(%"*5&
62/9&
'()(*"+")%&
./"0& ,-()&
'()1(-&
Documents
,=>& :);20<)+")%&
'()(*"+")%&&
,-()&

6"-"(/"& 78()*"&
'()(*"+")%& '()(*"+")%&
,-()&

The
User
Manual
is
established
as
a
hub
or
map
into
the
other
defining
documents
that
drive

how
the
testing
teams
operate.
 Where we go from here
We
must
maintain
relevance
in
the
SDLC
despite

organizational
and
budget
cut
backs

• Align
with
the
Business

• Enterprise
Wide
Testing
Strategy
• AS
IS
modeling
• Future
State
Definition
• Roadmap

.
• Roadmap
is
a
codified
artifact
that
illustrates

how
Testing
will
continue
to
grow
and
remain

relevant
to
the
Business
goals.

• A
roadmap
is
built
on
principles;
Testing
Visibly

is
one
of
them
Thank You
David
Kapfhammer
Global
Director
david.kapfhammer@keane.com