Practices for Testing Code

By Ahmed Abdelhamid from Ronald Kirk's book

Testing cannot show that software is free of defects, but it can identify their existence. Thus, the purpose of
testing is not to show that a program works, but to find its defects. Following are several practices for
testing software.

1- Generate functional test cases from defined scenarios. These scenarios represent common
transactions that reflect the tasks that the users will carry out. These test cases should be
included in a regression test suite.

2- Use a code coverage tool. The reason for using a code coverage tool is that it provides
quantitative evidence on how thorough a test effort is. Weigers illustrates the value of a code
coverage tool when he said:
“Good programmers testing without benefit of a coverage tool are only achieving about 80 percent code
coverage, while average programmers are getting 50 to 60 percent, and poor programmers are hitting
less than 30 percent. Test strategies that emphasize only the functional behaviors of the software may
cover the requirements adequately, but this is no guarantee that the internal structure of the program has
been thoroughly exercised.”

3- Perform basis path testing. Basis path testing independently tests each conditional predicate
of a module. That is, the outcome of each condition that can affect control flow is independently
tested. Thus, basis path testing localizes a defect to a single subpath of a module because
there are no interactions between decision outcomes.
A further benefit of this approach is that the size of the testing effort is predictable before
testing begins, whereas other techniques are predicated on examining the ongoing progress of
a test effort. In basis path testing, the required number of test cases for a module is exactly its
cyclomatic complexity, which is computed by counting the number of conditional predicates of a
module plus one (each separate case of a case statement counts as one). In sum, basis path
testing is an effective, efficient, and quantifiable software testing technique that catches about
two thirds of all defects.
Consider the following function, which computes the factorial of a number, and its associated
flow graph, shown in the following figure.
 The cyclomatic complexity of this function is 2, indicating that two edges exist in the flow graph:
one test case is required for the condition 2 <= n and another one when 2 > n. Hence, n
equals 1 and n equals 2 are two valid inputs that will transverse every edge of the control flow
graph of the Fibonacci function, as shown in the previous table.

Note that many people have suggested breaking up complex modules into smaller modules, as
measured by cyclomatic complexity. This is not necessarily a good thing for two reasons:
First, decomposing a module into multiple modules may not make sense because it
may be difficult to decompose the larger module into smaller ones, each having specific
responsibilities. Thus, decomposition may turn out to be arbitrary and not very
meaningful.
Second, decomposing a module into more modules actually increases the testing effort.
Consider, for example, how one might decompose the greatest common denominator
function

4- Examine the boundary conditions affecting the control flow of a program. Test predicate
variables in a condition by testing the two values one unit of measurement from the specified
value. For example, if a predicate is x == 1, where x is an integer value, test cases should be
written so that x is bound to 0 and 2 when the condition is executed. For the factorial function in
the previous example checking the boundary conditions would yield the two test cases
corresponding to n equals 1 and n equals 3. By choosing the input values wisely for basis path
and boundary value testing, one can reduce the total testing effort. For this example, there are
three unique test cases — n equals 1, 2, and 3 — that satisfy the structural testing procedures
identified by Practices 3 and 4, which are shown in Table of factorial function. For the greatest
common denominator function of the previous factorial algorithm, three boundary value
conditions occur, which are satisfied by the wise selection of the inputs for basis path testing.
 Furthermore, examine loop statements, which are a special case of boundary value testing,
using the following heuristics.
- Execute each loop statement zero, one, and two times. This is more-or-less performing
boundary value testing of the initial step of the iteration variable. In addition, it has been
shown that executing a loop twice can detect some unique initialization problems.
- When using nested iteration statements, select a constant value for all iteration
variables except the one being tested, vary it as previously described, and then repeat
this process for the remaining nested iteration variables. This heuristic reduces the
number of required test cases.

5- Verify data and file usage patterns of a program. That is, check that a program opens files
before it operates on them and that it closes files before it terminates. Similarly, verify that a
program declares and initializes data before it operates on the data, and that it reclaims all data
before it terminates.

6- Verify that invalid and unexpected inputs are handled, as well as valid and expected ones.
People often detect numerous defects when they use a program in proper but unexpected
ways. Consequently, test cases containing unexpected and invalid inputs often find greater
numbers of defects than do test cases containing valid input. In addition, an important class of
inputs — command languages and data formatting protocols — should be rigorously tested.
Useful heuristics include:
(1) Executing repeated rewrite rules to find limitations in buffer sizes;
(2) Violating syntactic rules;
(3) Reversing syntactic tokens and other forms;
(4) Creating the null program;
(5) Creating null expressions (e.g., {});
(6) Examining code for missing, wrong, and too many delimiters; and
(7) Checking code for field value errors.
One might consider writing a syntax checker that mutates valid input strings in defined ways,
which requires a formal definition of the syntax.

7- Verify all critical timing modes and time-out conditions. A critical timing mode occurs
whenever execution timing is mission critical, consisting of:
(1) A fixed number of tasks, threads, or event-handlers running concurrently on a critical
processor;
(2) A fixed allocation of functions to tasks, threads, or event-handlers; and
(3) fixed priorities and frequencies for each task, thread, or event-handler.

8- Verify that systems work in a variety of configurations — as many as practical. Use the same
hardware and software environments that the deployment sites will use during the integration
test. The benefit of this is that an organization will detect defects unique to host environments of
the field sites before it deploys software to them.

9- Verify the accuracy of the documentation. This includes verifying the correctness of the
software installation procedures. Following this practice is important because documentation
defects represent 10 to 25 percent of all delivered software defects.