Test Reporting

Domain 15

Four Categories of Data

Test Transactions, Test Suites, Test Events
Defects
Efficiency
Interim

Interesting Points
Effective reporting cannot be accomplished
unless a complete test plan is ready by QA.
Development is about breaking down of
high level requirements to lower levels,
while QA goes in the opposite direction.

Functional Matrix
The purpose of creating a functional matrix
is to show the test plan and order of testing
to perform complete testing.
This can also be used to show the progress
of testing as complete.

Eight Interim Reports:

Functional Testing Status
Functions Working Timeline
Expected verses Actual Defects Uncovered
Timeline
Defects Uncovered verses Corrected Gap Timeline
Average Age of Uncorrected Defects by Type
Defect Distribution
Relative Defect Distribution
Testing Action

Functional Testing Status Report

This report will show percentages of the
functions which have been:
Fully Tested
Tested With Open Defects
Not Tested

Functions Working Timeline

This report will show the actual plan to
have all functions working verses the
current status of functions working.
An ideal format could be a line graph.

Expected verses Actual Defects

Uncovered
This report will provide an analysis between
the number of defects being generated
against the expected number of defects
expected from the planning stage.

Defects Uncovered verses

Corrected Gap
This report, ideally in a line graph format,
will show the number of defects uncovered
verses the number of defects being
corrected and accepted by the testing group.
If the gap grows too large, the project may
not be ready when originally planned.

Average Age Uncorrected

Defects by Type
This report will show the average days of
outstanding defects by type (Sev 1, Sev 2,
etc.).
In the planning stage, it is beneficial to
determine the acceptable open days by
defect type.

Defect Distribution
This report will show the defect distribution
by function or module. It can also show
items such as numbers of tests completed.

Relative Defect Distribution

This report will take the previous report (Defect
Distribution) and normalize the level of defects.
An example would be one application might be
more in depth than another, and would probably
have a higher level of defects.
However, when normalized over the number of
functions or lines of code, would show a more
accurate level of defects.

Testing Action
This report can show many different things,
including possible shortfalls in testing.
Examples of data to show might be number
of Sev 1 defects, tests that are behind
schedule, and other information that would
present an accurate testing picture.

Final Test Report

Three Objectives:
Define Scope of Testing
Present Test Results
Draw Conclusions/Recommendations
Two Goals:

Provide Data for Determination of

Readiness by Customer

Provide Data for Long Term Detection

of Problem Functions