SOFTWARE ENGINEERING TECHNIQUES

LESSON 23:

Topics Covered
Importance of testing, Hurdles of testing and the testing Design
fundamentals, Test case design
27%
Objectives
Upon completion of this Lesson, you should be able to: Rqmts.
Design
• Know what is basically the importance of testing Code
Code Other
• Know what are the various hurdles in testing Rqmts. 7%
• Know what are the testing fundamentals 56%
Now let us see the importance of testing. Other
10%
Importance of Testing
Testing activity cannot be eliminated in the life cycle as the end
product must be bug free and reliable one. Testing is important Figure: Software defect Distribution
because: Testing Fundamentals
• Testing is a critical element of software Quality Assurance Before understanding the process of testing software, it is
necessary to learn the basic principles of testing.
• Post-release removal of defects is the most expensive
• Significant portion of life cycle effort expended on testing Testing Objectives
In a typical service oriented project, about 20-40% of project • Testing is a process of executing a program with the intent
effort spent on testing. It is much more in the case of “human- of finding an error.
rated” software. • A good test is one that has a high probability of finding an
For example, at Microsoft, tester to developer ratio is 1:1 as yet undiscovered error.
whereas at NASA shuttle development center (SEI Level 5), the • A successful test is one that uncovers an as yet
ratio is 7:1. This shows that how testing is an integral part of undiscovered error.
Quality assurance. The objective is to design tests that systematically uncover
OK, now we have understood that testing is important. But as different classes of errors and do so with a minimum amount
it is known that important tasks are not easy to be carried on, of time and effort.
same with testing. There are many hurdles in testing. We’ll see Secondary Benefits Include
what it is one by one.
• Demonstrate that software functions appear to be working
Hurdles in Testing according to specification.
As in many other development projects, testing is not free
• Those performance requirements appear to have been met.
from hurdles. Some of the hurdles normally encounters are:
• Data collected during testing provides a good indication of
• Usually late activity in the project life cycle
software reliability and some indication of software quality.
• No “concrete” output and therefore difficult to measure
Testing cannot show the absence of defects, it can only show
the value addition
that software defects are present.
• Lack of historical data
Test Information Flow
• Recognition of importance is relatively less A typical test information flow is shown in the following figure.
• Politically damaging as you are challenging the developer
• Delivery commitments
• Too much optimistic that the software always works
correctly

Defect Distribution
In a typical project life cycle, testing is the late activity. When the
product is tested, the defects may be due to many reasons. It
may either programming error or may be defects in design or
defects at any stages in the life cycle. The overall defect distribu- Figure: Test information flow in a typical software test life
tion is shown in the following figure. cycle

© Copy Right: Rai University

3E.253 83
 In the above fig,
SOFTWARE ENGINEERING TECHNIQUES
• Software Configuration includes a Software Requirements
Specification, a Design Specification, and source code.
• A test configuration includes a Test Plan and Procedures,
test cases, and testing tools.
• It is difficult to predict the time to debug the code, hence it
is difficult to schedule.
Test Case Design
Some of the points to be noted during the test case design are:
• Can be as difficult as the initial design.
• Can test if a component conforms to specification - Black
Box Testing.
• Can test if a component conforms to design - White box
testing.
• Testing cannot prove correctness as not all execution paths
can be tested.
Consider the following example shown in fig
A program with a structure as illustrated above (with less than
100 lines of Pascal code) has about 100,000,000,000,000
possible paths. If attempted to test these at rate of 1000 tests
per second, would take 3170 years to test all paths. This shows
that exhaustive testing of software is not possible.
Test Case Design
Introduction
Software can be tested either by running the programs and
verifying each step of its execution against expected results or by
statically examining the code or the document against its stated
requirement or objective. In general, software testing can be
divided into two categories, viz. Static and dynamic testing.
Static testing is a non-execution-based testing and carried
through by mostly human effort. In static testing, we test,
design, code or any document through inspection,
walkthroughs and reviews. Many studies show that the single
© Copy Right: Rai University
84
most cost-effective defect reduction process is the classic
structural test; the code inspection or walk-through. Code
inspection is like proof reading and developers will be benefited
in identifying the typographical errors, logic errors and devia-
tions in styles and standards normally followed.
Dynamic testing is an execution based testing technique.
Program must be executed to find the possible errors. Here, the
program, module or the entire system is executed (run) and the
output is verified against the expected result. Dynamic execution
of tests is based on specifications of the program, code and
methodology.
To test computer software, we spiral out along streamlines that
broaden the scope of testing with each turn.
Considering the process from a procedural point of view
testing within the context of software engineering is a series of
four steps that are implemented sequentially.
The steps are shown In Figure given below initially tests focus
on each module individually, assuring that it functions as a unit
hence the name unit testing. Unit testing makes heavy use of
white-box testing techniques, exercising specific paths in a
module’s control structure to ensure complete coverage and
maximum error detection. Next, modules must be assembled
or integrated to form the complete software package. Integration
testing addresses the issues associated with the dual problems
of verification and program construction. Black-box test case
design techniques are most prevalent during integration,
although a limited amount of white -box testing may be used
to ensure coverage of major control paths. After the software
has been integrated (constructed), sets of high-order test are
conducted. Validation criteria (established during requirements
analysis) must be tested. Validation testing provides final
assurance that software needs all functional, behavioral and
performance requirements. Black-box testing techniques are
used exclusively during validation.
The last high-order testing step falls outside the boundary of
software engineering and into the broader context of computer
system engineering. Software once validated must be combined
with other system elements (e.g., hardware, people, and
databases). System testing verifies the tall elements mesh properly
and that overall system function/performance is achieved.
High order tests
Requirement
Integration test
Design
Code
Unit test
Test Direction
Figure 5.3: Software Testing Steps
3E.253
Summary
The primary objective for test case design is to derive a set of
tests that have the highest likelihood for uncovering errors in
the software. The purpose of testing is to show that the
software works. Here most of the time, the software is
demonstrated in a normal sequence/flow. All the branches may
not be tested. This approach is mainly to satisfy the customer
and no value added to the program. Testing is an activity in
which a system or component is executed under specified
conditions; the results are observed and recorded and an
evaluation is made of some aspect of the system or compo-
nent. Many approaches have been defined in literatures. The
importance of any approaches depends on the type of the
system in which you are testing. Defects can exist in the
software, as it is developed by human who can make mistakes
during the development of software.
Exercises
1. Explain what are the various hurdles in testing?
2. Briefly explain what are the various testing fundamentals?
Case Study
A tiny software “bug” - three faulty computer instructions
lurking among several million that modern telephone switches
use to route calls -appears to have sparked the sudden, massive
failures of local telephone systems in recent weeks, the manufac-
turer of the equipment said yesterday. The flaws were hidden in
minor software changes that the Plano, Tex.-based company,
DSC Communications Corp., provided to its telephone
company customers in an effort to improve the equipment’s
performance. The software was sent out without major testing
because DSC judged that the changes were too small to require
it, DSC Vice President Frank Perpiglia said yesterday.
It was the first detailed explanation of what caused the
unprecedented string of failures. In rapid sequence in June and
earlier this month, local phone service in the Washington
region, Pittsburgh, San Francisco and Los Angeles mysteriously
crashed, for as long as eight hours.
Perpiglia said the software flaw appeared to be the “root cause”
of failures of DSC-built computers called signal transfer points
that have been at the heart of the investigation. But he said that
in the telephone network as a whole, in which equipment made
by many different companies is linked together, other causes
might be found as well.
Despite DSC’s statement, major telephone companies and
equipment manufacturers yesterday continued a high-energy
investigation of the incidents, which rank among the most
disruptive software-related failures in U.S. history. In the
meantime, telephone companies have installed in their switches
special software “patches” that DSC provided, and they said
that the fixes should prevent any recurrence of the failures.
Software consists of electronic instructions that in complex
programs can number in the millions. Computers move
through the instructions sequentially to perform tasks as diverse
as routing telephone calls, running a word processor, guiding
missiles and keeping track of company payrolls.
© Copy Right: Rai University
3E.253
A single misplaced command - one that tells a machine to look
SOFTWARE ENGINEERING TECHNIQUES
in the wrong data base when a particular piece of information is
needed, for instance - can throw a computer into confusion.
Usually such bugs are detected during laboratory testing, but
many survive this process.
DSC’s explanation underlines fears that as computer software
grows more complex and takes over
greater functions in companies and government, society will be
increasingly vulnerable to massive failures.
Software engineers are developing new ways to test software
and mathematically prove that it is error-free. Other theories say
software should be designed with the assumption that bugs
will occur but that any damage they cause will be containable.
Software developers continually come up with refinements in
existing programs. Perpiglia said DSC had made changes at the
request of one customer to basic software that runs the signal
transfer points. Between December last year and April, the
change was distributed to five of the seven regional telephone
companies.
“DSC did not go through the normal process” of four months
of testing before sending the change out, Perpiglia told
members of the House subcommittee on telecommunications
and finance yesterday. “Because the change was small ... we felt
that the change itself did not require such testing.”
Analysis of the computer instructions led DSC to pinpoint the
problem in a specific section of the change, Perpiglia said. The
flaws, he said, consisted of only three “binary digits” of
information - three flawed digits among millions of correct
ones.
DSC computers are designed to route calls around minor
failures that routinely occur in a modern telephone network,
such as the failure of a single circuit board. But due to the errors
in the program, the computers responded by pumping out
floods of erroneous internal messages that crowded out
routine messages that serve to route phone calls, causing the
systems to shut down.
Phone company executives said they had no way to explain why
the failures on both coasts had occurred so close in time to one
another. In a closed session yesterday, the Federal Communica-
tions Commission decided to step up its own research into
network reliability, to establish a special staff for that job and to
establish formal reporting requirements for network outages.
“The commission continues to have full faith in the fundamen-
tal strength of our public telephone network,” FCC Chairman
Alfred Sikes said. “And we are persuaded that the recent events
are - in all probability - inadvertent side effects of continuing
progress, rather than evidence of any fundamental flaws.”
Questions
Select two of the questions below, write the answers in the
space provided, and designate a member of the group to read
the answer(s) on behalf of the group.
1. Describe the incident in your own words. What went
wrong and what was the impact on the human lives?
2. State the root cause of the problem. Why did this problem
shut down the phone system?
85
 Further Readings and Information
SOFTWARE ENGINEERING TECHNIQUES

Sources
Software engineering presents both technical and mechanical
challenges. Books by Black(Managing the testing process,
Microsoft Press, 1999); Dustin, Rashka, and Paul (Test Process
Improvement: Step-by Step Guide to Structured Testing,
Addison-Wesley, 1999); Perry (Surviving the Top Ten Chal-
lenges of Software Testing: A People oriented Approach,
Dorset House, 1997).
A wide variety of information sources on software testing and
related subjects is available on the internet. An up-to date list of
World wide web references that are relevant to testing concepts,
methods and testing strategies can be found at the SEPA Web
site:http://www.mhhe.com/engcs/compsci/sepa/resources/
test-techniques.mhtml
Notes

© Copy Right: Rai University

86 3E.253