Вы находитесь на странице: 1из 149

SSD9: Software Specification, Testing, and

Maintenance
Unit 6. Build and Test the Solution
 6.1 Implementation
 6.2 Module Testing
 6.3 Integration Testing
 6.4 Product and Acceptance Testing
 6.5 CASE Technology

 Assessments
 Exercise 7
 Multiple-Choice Quiz 6
6.1 Implementation
 6.1.1 Reuse
 6.1.2 Choice of Programming Language
 6.1.3 Good Programming Practices and Coding
Standards
6.1.1 Reuse
 Module Reuse
 Costs and Constraints in Reuse
 What Can Be Reused and When?
 Reuse and Maintenance
Module Reuse
 The first issue one faces in implementation is
 whether to code everything from scratch 只靠自己的双手
从头做起?
 whether to reuse existing components.
 Although components from all phases of the
software process can be reused
 requirements,
 specifications,
 design,
 documentation, and
 testing suites,
 module reuse is by far the most common type of
reuse.
Reuse
 Reuse refers to taking the components of one product to
facilitate the development of a different product with a
different functionality.
 Reuse can be accidental 机会重用
 if developers realize that a previously developed component can be
reused
 reuse can be deliberate. 计划重用
 components are constructed with the idea of possible future reutilization
in mind.
 Deliberate reuse is better, insofar as such components are likely to be
better documented, better tested, and designed in a uniform style that
makes for easier maintenance.
 However, even reused components will have to be retested
during integration testing, because 重用组件要重新测试

 the range of input they get in a new product might go beyond that of the
Code reuse
 In the early days of system building, there was no
reuse: everything was constructed from scratch.
 The first common source of code reuse was
subroutine or class libraries, of which there are now
many in the public domain for different languages.
 many scientific computing subroutines have been developed
and refined for optimal performance, so in the case of each of
these subroutines there is no need to recode one.
 Application Programming Interfaces (APIs) are usually
implemented as operating system calls, but they can be
viewed as subroutine libraries.
 Code reuse is also a feature of the many varieties of toolkits
for building graphical user interfaces (GUIs).
Costs and Constraints in Reuse
 In reality, only a relatively small percentage of any
software product serves a novel purpose or provides
novel functionality ( 全新的功能 )
 approximately 15 %.
 That means that, in theory, about 85% could be standardized
and reused.
 In practice, on average, only 40 percent is reused.
 Smaller modules are frequently more reusable
because they are likely 软件组 件的粒度 问题
 to be documented better and
 less application specific.
Why only 40% reuse rate
 There are several good and some not so good reasons
that tend to limit the extent to reuse.
 One common reason is ego( 自负了点 ): the fact that many
software professionals would prefer to rewrite a piece of code
from scratch rather than use someone else’s. 不愿用别人的
 A related reason has to do with quality. Developers would reuse
code developed by others if they could be assured that it would
not introduce faults in the new product. 怕出问题 While this
concern is sometimes justified,
 both reasons reflect the attitude that no piece of code
is as good as one’s own! 真的 是这 样吗?
practical problems associated with reuse
 Retrieval 大海捞针
 unless code is deliberately developed and organized for easy
reuse, it may be difficult to locate it in a database of hundreds
and thousands of software modules.
 planning for reuse has a price
 an increase of 60 percent or higher in development cost — plus
the cost of reusing the component and the cost of implementing
a reuse process.
 there may be legal issues to consider. 避免官司 !
 In contract software development, the software belongs to the
client, unless it is just licensed.
 Internal software reuse is free from this constraint.
Object-oriented systems
 OO systems seem to be particularly suited to
code reuse. 貌似 很适合 reuse
 Objects are self-contained, which can make both
development and maintenance easier because it allows a
large product to be constructed as a collection of smaller
products.
 Maintenance is generally easier and cheaper, and some
classes of objects can be reused.
 但尺有所短,寸 有所长……
reuse in OO systems presents its own problems
 specifically due to inheritance !
 the fragile base class problem, occurs when a base class
changes.
 All classes depending on the base class have to be recompiled at least,
and possibly changed.
 indiscriminate use of inheritance: 滥用继承
 because dependent classes not only inherit all the attributes of their
superclasses but also have their own attributes, they can get big very
quickly!
 the use of polymorphism and dynamic binding. 多态与动态
绑定
 Polymorphism and dynamic binding represent the ability to invoke the
same method with arguments of different kinds and have the run-time
system determine the appropriate method to use based on the type of
argument.
 As powerful as it is, this mechanism makes maintenance and reuse of
code more difficult to understand and troubleshoot.
What Can Be Reused and When?
 Reuse is commonly perceived
 as being something you do with individual code modules,
 as in the case of subroutine libraries,
 but code is not the only thing that can be worth
reusing.
 if one can browse the inheritance tree of an object-oriented
system, its design can become reusable as well. 继承树本身
就可以重用的!
two approaches to incorporating reuse
 There are 2 ways to incorporating reuse into the
code design and implementation phases:
 One can develop the control logic for a product from scratch
and reuse pieces of code that incorporate specific operations
of the product.
 Alternatively, one can reuse an application framework that
 already provides the desired control logic and
 develop application specific operations.
 the set of classes used for the design of a compiler.
 One will need to code from scratch all classes specific to the language
and desired target machine.
Application framework reuse results in higher productivity
 Application framework reuse results in higher
productivity than toolkit or library reuse because
 more design is reused (recall that design typically has a
higher financial cost than implementation);
 logic is harder to design and code than are the specific
operations; and because the control logic will have been
tested.
 Code frameworks, those for performing basic
window manager operations are similar to
application frameworks in reusability. 所有 操作系
统平台 的 GUI 界面的窗口管 理都有类似 的!
Design patterns are another kind of design reuse.
 A design pattern is a solution to a general design problem in
the form of a set of closely interacting classes that have to be
customized to create a specific design.
 The idea of design pattern is drawn from the field of
building architecture.
 A GUI generator is an example of a design pattern
 a tool that assists the user in constructing a graphical user interface,.
 provides a set of abstract widget classes that the developer can combine
to create a custom user interface.
 A widget is a graphical object such as a window, a menu, a radio button,

 The abstract widget classes have concrete subclasses whose


instantiations are widgets with a specific "look-and-feel“ — such as
Motif, SunView, and others.
Software architecture
 Software architecture is another design concept that, in the
future, may lend itself to reuse.
 The term "software architecture" refers to design issues
concerning
 the organization of a product in terms of its components,
 product-level control structures,
 issues of communication and synchronization,
 databases and data access,
 physical distribution of the components,
 performance, and
 choice of design alternatives.
 Object-oriented architectures
 UNIX pipes and filters
 client-server architectures
Reuse and Maintenance
 Software reuse makes economic sense, particularly in an
organization that produces software for a specific
application domain.
 The higher up-front costs of designing and implementing for
reuse are more likely to pay off during development of
future products, resulting in both significant savings of cost
and time. 考虑越早 ,省 时省钱效 果越 好
 However, in order for reuse to bear fruit, there must be a
commitment from management: programmer initiative is
not enough.
potential beneficial effects of reuse
 Although the traditional reason for reuse has been to
shorten development, the potential beneficial effects of reuse
also extend to maintenance.
 reuse has more impact on maintenance than on development
回忆以前提过的各阶段时间成本比例
 because the cost of maintenance over a product’s lifetime is roughly
twice that of development (from requirements analysis through
integration testing and delivery).
 It is worth repeating that the crucial advantage of reuse is
the availability of well-documented and well-tested code.
 Reusing code that does not have these characteristics has little benefit
and potentially can cause significant harm.
6.1.2 Choice of Programming Language
 Programming Language Types
 Rapid-Prototyping Language
 Final Product Language
Programming Language Types
 Software practitioners often talk about fourth-
generation languages.
First-generation languages
 First-generation languages were binary machine
code, sequences of 0s and 1s, encoding information
at the level at which machines process it directly.
 First-generation languages were quite tedious to
write in and very hard to debug.
; 汇编语言程序结构例二(子程结构)
Second-generation
Second-generation languages
用二进制显示中断向量表中数据
;key EQU 0D0H D0H 的
个数
 Things improved a bit when
( 关 ; assembly languages
内容子程 ,
Things improved a bit whendisplay
; 用符号表示常量 assembly
用二 进 制 显 示 BX
PROC
languages,
键字 )
codesecond-generation
SEGMENT languages, wereMOV
; 代码段开始 developed
CX , 16 in the
late 1940s
ASSUME CS:code
and
begin: MOV AX , 0000H
early 1950s. 机器
rotate: 指令换成助
ROL BX , 1
MOV DL , BL
记符
 Although
MOV DS , AX
more understandable, assembly
AND DL , 01H languages
MOV SI , 0000H ADD DL , 30H
wereMOVstill
CX very
, 0400H processor dependent . AH , 2H
MOV
MOV BX , 0 INT 21H
 TheMOV instructions
AL , key reflected the architecture LOOP rotate of the
RET ; 子程返回
processor
next:
(how many registers
CMP [ SI ] , AL
JNZ point
there
display ENDPwere).
INC BX
 Individual
point: INC SI
instructions had to
code
be very
ENDS
specific (copy
; 代码段结束
END begin ; 指示程序结束
the contents of register 1 to register
LOOP next 和 3) and complex
CALL display ; 调用显示子程 ; 程序入口
control structures
MOV AH, 4CH ; 返回 DOS such as loops had to be spelled
outINT 21H
in excruciating detail.( 需要详 细分解, 跳转…
Third-generation languages
 third-generation languages: processor independence
and significantly more abstract control structures.
 These include familiar high-level languages developed
in the 60s and 70s, such as FORTRAN, Algol, Pascal,
COBOL, C, and Lisp.
 Third-generation languages are still procedural
languages, but each instruction in a high-level
language corresponds to five to ten machine code
instructions.
Third-generation languages
 Object-oriented languages can also be considered
third-generation languages
 though they organize data and computation in a very different
way.
 The first object-oriented language was Smalltalk, followed soon
thereafter by C++ and more recently Java.
 Object-oriented programming has become more and more
popular.
 Even artificial intelligence languages such as Lisp, designed for
symbolic computation, now have support for object-oriented
programming.
Fourth-generation languages
 4GLs were intended to abstract out even more
procedural detail by encoding in a single statement even
more lines of machine code (30–50 lines).
 Several of them are non-procedural —
 they specify what should happen

 but not how it will happen.


several special purpose 4GLs have been developed.
 A well-known 4GL is SQL, a language for accessing
information in relational databases.
 A probably less known example of a 4GL is Prolog,
commonly used in
 artificial intelligence applications
 especially in natural language processing.
 A Prolog statement can be used to specify that a sentence is
composed by a noun, followed by a verb, followed by another
noun.
 This statement can be used to analyze a text sentence and
determine whether it fits the noun-verb-noun pattern.
 Exactly how that happens is hidden from the user in the search
mechanism that is deep inside the Prolog interpreter.
4GL would increase programmer productivity significantly?

 Because of the high ratio of number of 4GL code lines


to machine code lines( 1:30-50), one would think that
use of a 4GL would increase programmer
productivity significantly.
 In fact, the results have been inconsistent, with
reports ranging from
 a tenfold increase in productivity, to
 a negligible increase, to
 a significant decrease.
It is difficult to evaluate!
 It is difficult to evaluate these reports and make a
general statement because
 much depends on the particular 4GL and its suitability for the
application.
 Other factors that influence the impact of introducing a 4GL
into an organization include
 the availability of CASE tools and
 the maturity of the organization. 组织的能力成熟度, CMM
There are also serious risks in using 4GLs.
 Many were intended for end-user programming ,
for allowing end-users 如二 次开 发等,
 to bypass the programmers in the organization and
 to write the code required by their specific information access
needs on their own. 用户自行编码
 The danger is that end users who are not experienced
programmers may
 place too much trust in the correctness of the output from their
4GL programs
 make harmful decisions based on erroneous output.
 write an incorrect program modifying a database and corrupting
shared data. 最糟糕的情形了
Rapid-Prototyping Language
 For implementing the final product, there may be
substantially less freedom in the choice of a
programming language:
 the client or the client’s platform may determine it.
 On the other hand, the client may simply specify
"the most suitable" programming language,
 giving the developer complete freedom and the responsibility
for making the right choice.
What is the right choice?
 There is probably no single right choice, but here
are some valuable criteria:
 A language that the client has experience and tools for
 A language that suits the application type
 A language that maximizes informational cohesion
 The language that has the best cost-benefit ratio
 The language with the lowest risk (risk analysis)
The criteria are not independent of each other.
 E.g., choosing a language that the developer does
not have much experience in or tools for will
 almost certainly result in higher costs for training the
programming staff
 and in higher risk both with respect to being able to 还带来
这些问题
 meet deadlines
 produce high quality code.
special comment for the second criterion
 The criterion: a language that suits the application
type.
 There is a tendency, currently, to use OO languages
for every type of application — C++.
 C++ compiles into C code.  C++ will work on any
platform that supports C.
 C programmers must be specially trained in the
object-oriented paradigm, which may be costly.
special comment for the second criterion
 Failure to invest the time and money to train them
properly  poorly designed and implemented code
 hard to maintain  even costlier 花的钱 更多
 If the client wants a scientific computing application,
chances are that the developer should opt for an old-
fashioned language like FORTRAN, for which a number of
public subroutine libraries are already available.
informational cohesion
 Another consideration in choosing a programming
language is
 that of maximizing informational cohesion
 choosing a language that allows co-location of independent
operations performed on the same data structure.
 Object-oriented languages are particularly good in
this regard,
 Other languages also support the creation of
abstract data types as well.
6.1.3 Good Programming Practices and Coding Standards
 Good Programming Practices
 Coding Standards
Good Programming Practices
 Choosing the programming language
 implementing the product design.
 A few general guidelines should be mentioned for
building code that is
 easier to understand
 easier to test
 easier to maintain
Coding style guidelines
 Coding style guidelines (indent the program lines)
are language specific.
 For a particular programming language have a
particular guide on good programming practice
 Coding style should keep in mind that 替维护 者着

 future program maintainers have to understand the code.
 This including using
 identifiers for procedures, variables, constants, object classes,
etc.,
 that are not meaningful only to the author of the code.
 The variable name xCoordRobotArm is significantly clearer
than just xCoord or xCdRbtArm.
self-documenting code
 Although self-documenting code exists, it is very seldom
found.
 The following is an example of a code fragment that is largely
self-documenting.
 // Read in the server response.
 buf.setLength(0);
while (bytesToRead > 0) {
c = in.read();
buf.append((char)c);
bytesToRead--;
}
c = in.read(); // what’s this?
 No line-by-line explanation is needed for the while loop, but
the last line needs a comment.
 the last read statement consume an extra control character,what for?
Coding Standards
 Coding standards may be formulated with fixed rules.
 a module will have between 35 and 50 lines of executable code.
 While statements such as these may be well motivated, they fail
to convey their real justification 硬性规定行数不恰当!
 the modules desire to build are logically and functionally
cohesive. 尽量做到这样 !
 A module has logical cohesion when it performs a
series of related actions, one of which is selected by
the calling module.
 performs all input and output operations in an application.
 A module has functional cohesion when it performs
exactly one action or achieves a single goal
 calculates sale commission.
Coding Standards
 A seemingly arbitrary coding standard such as the 35
– 55 lines of code per module example given above is
likely either to be ignored or to give rise to modules
with coincidental cohesion: 35 – 50 lines of
 unrelated code lumped together from smaller modules that do
not belong together 无关代码杂凑在一起
 or modules unnecessarily split up 或硬性分开到不同模块
 Modules so constructed are
 hard to understand
 hard to maintain
 hard to reuse
What a Coding standards do?
 Coding standards should
 be justified by providing the real motivation behind the standard and
 should be phrased in terms that leave some room for programmers
 to make the correct decisions,
 checking with their managers if those decisions result in code that goes
outside the guideline boundaries.
 In general, an organization should
 try to balance
 the usefulness of the restrictions with
 the burden they impose on programmers
 keeping in mind that
 poorly justified coding standards are a waste of time
 especially if they cannot be checked by machine
6.2 Module Testing
 the overview of the life-cycle phases that different kinds of
testing must be performed on code at different times:
 testing of individual modules happens during the implementation phase
 testing of the product as a whole happens during the integration phase.
 In fact
 because there are some complications involved in testing modules
completely independently of each other and
 because there are schedule constraints,
  more temporal overlap in the two types of testing than the above
description suggests.
 It is more likely
 testing  individual modules pass SQA  be integrated in accordance
with the chosen integration strategy
verification techniques 验证技术
 Verification is correctness checking
 the process of determining whether a phase of the software
process has been carried out correctly.
 Collectively termed validation that must be
performed before the product is delivered to the
client.
contents
 6.2.1 Execution-Based Testing
 6.2.2 Non-execution-Based Testing
 6.2.3 Other Testing Approaches
 6.2.4 A Comparison of Module-Testing Techniques
6.2.1 Execution-Based Testing
 General Remarks
 Black-Box Module-Testing Techniques
 Glass-Box Module-Testing Techniques
General Remarks
 Programmers are expected to test the correctness of
the modules before implementation of a module is
considered complete 程序员 测试以保证 其代码正

 but it is the responsibility of the developer’s SQA
group to perform methodical testing of each module
when it leaves the programmer’s hands.
 There are two basic approaches to testing modules,
each with its own weaknesses.
 Testing to Specifications
 Testing to Code
Testing to Specifications
 seeks to verify that the module conforms to the specified
input and output while ignoring the actual code.
 also known as
 black-box
 Structured (? 与教材有冲突,教材将其归为代码测试 )
 data-driven
 functional
 input/output-driven testing,
 It is rarely possible to test all modules for all possible input
cases, which may be a huge number of modules.
 Usually time constraints permit only about 1000 test cases
per module, so tests should be chosen carefully.
 Exhaustive testing( 穷尽测 试 ) is feasible only when input
combinations are small. 绝大多数时候不可行!
Testing to Code
 is quite the opposite of Testing to Specifications.
 It ignores the specification but tests each path
through the code. also called
 glass-box
 white-box
 behavioral (? 与教材有冲突,教材将其归为规格说明测试
)
 logic-driven
 path-oriented testing
Testing to Code
 Unfortunately, testing each path through the code is
generally not feasible, even for simple flowcharts.
 一个小程序的可能路径 10 N 数量级的
 it is possible to test every path without finding existing
faults, 有时即使 穷尽也 测不出 错误, why ?
 the fault lies in the decision criterion for selecting between paths.
 it is possible to exercise successfully all paths with data that
would not show the fault.
 E.g., taking the square root of a negative number.
 If code preceding the operation yielded only a nonnegative result, when
run with the test data, the fault would not be uncovered. 前边的代码不
产生负数结果,就没有机会进入这个操作,产生错误
所以 exhaustive testing 也不能保 证
 exhaustive testing to either code or specifications is
neither feasible nor, even if it were feasible, is
guaranteed to uncover all possible implementation
faults.
 A compromise is needed 寻求 折中的方法
 one that will highlight as many faults as possible, 只能尽量
多地找错,而无法… ..
 while accepting that not all faults will be detected. 接受残酷
的现实
Black-Box Module-Testing Techniques
 The objective in black-box testing is to
 select test cases that achieve the broadest possible testing
coverage
 maximizing the chances of detecting a fault
 while minimizing the possibility of wasting test cases by
having the same fault tested redundantly by more than one
test case. 最好不要多个用例检出一个错误
Equivalence testing 等价测试
 Equivalence testing is a technique based on the idea
that
 the input specifications give ranges of values for which the
software product should work the same way.
 These ranges of values are termed "equivalence classes" of
input.
 Boundary value analysis seeks to
 test the product with input values that lie on
 to the side of boundaries between equivalence classes.
 经验:选择处于等价类边界的测试用例时,检出错误的
可能性增大
a novel implementation of the division operation
 there are 7 equivalence classes:
 the positive numbers greater than 1
 1
 the positive numbers between 0 and 1
 0
 the negative numbers between 0 and -1
 -1
 the negative numbers less than -1
 Boundary value analysis would test the code with
the values 0, 1, -1, and a value for each of the
remaining four equivalence classes.
Output specifications
 Output specifications may be used similarly to
 establish equivalence classes and boundary values
 determine the input test cases required.
 These two techniques, used in combination, make
up a powerful approach to discovering faults.
functional testing
 In functional testing
 the tester identifies each item of functionality or each
function implemented in the module
 uses data to test each function.
 Functional testing is not simplistic since functions
are usually intertwined. 底层功能纠结 在一起
 A more realistic process for testing functionality called
functional analysis.
 However, functional testing is problematic in
general because functionality often spans across
module boundaries. 多模块实现一 个功能!
 As a result, integration testing and module testing become
blurred. 不能明显区分界限,有时要同时进行!
Glass-Box Module-Testing Techniques
 One technique for testing to code is structural
testing.
 In its simplest way,
 structural testing is called statement coverage and
 amounts to running tests in which every statement in the
code is executed at least once,
 using a CASE tool to keep track of statements that are yet to
be executed. 通常需要工具帮忙
 A better version of this technique, branch coverage,
 makes sure that each branching point is tested at least once.
path coverage 路径覆盖
 A further improvement is path coverage. 功能最 强
大的功 能测试方法
 Researchers are working on techniques for reducing
the potentially very large number of paths through
constructs such as loops.
 路径数 量极大,需设 法减小!
 An advantage of structural testing is
 when done thoroughly, it can help detect so-called "dead-
paths" through the code
 code fragments that can never be reached with the given
code control structure.
 Path 1 test case:
 value(k) = valid input, where k < i for 2 ≤ i ≤ 100
 value(i) = 999 where 2 ≤ i ≤ 100
 Expected results: Correct average based on k values and proper totals.
 Note: Path 1 cannot be tested stand-alone but must be tested as part of
path 4, 5, and
 6 tests.
 Path 2 test case:
 value(1) = 999
 Expected results: Average = 999; other totals at initial values.
 Path 3 test case:
 Attempt to process 101 or more values.
 First 100 values should be valid.
 Expected results: Same as test case 1.
 Path 4 test case:
 value(i) = valid input where i < 100
 value(k) < minimum where k < i
 Expected results: Correct average based on k values and proper totals.
 Path 5 test case:
 value(i) = valid input where i < 100
 value(k) > maximum where k <= i
 Expected results: Correct average based on n values and
proper totals.
 Path 6 test case:
 value(i) = valid input where i < 100
 Expected results: Correct average based on n values and
proper totals.
6.2.2 Non-execution-Based Testing
 Code Walkthroughs
 Code Inspections
 Remarks on Non-execution-Based Testing
Techniques
When view the code?
 Black-box and glass-box techniques test the code by
executing it.
 Non-execution-based techniques rely on review of
code either before or after it has been tested through
execution.
about reviews
 Software Life Cycle, reviews are used not just with
code but also with products of other stages of the
software life cycle, such as
 Requirements documents
 Specification documents
 Review tasks should be
 assigned to someone other than the original author of the
document and are
 probably best performed by more than one reviewer
Two basic types of reviews
 Two basic types of reviews are
 code walkthroughs 代码走查
 code inspections 代码审查
 They differ primarily in
 code walkthroughs have fewer steps and
 code walkthroughs are more informal.
Code Walkthroughs
 The walkthrough team should consist of 4 to 6
individuals, including at least
 a representative and the manager of the phase being tested
(implementation),
 a representative of the team that will perform the next phase
in the product’s life cycle (integration testing), and
 a client representative.
Code Walkthroughs
 The team should be chaired by a member of the
SQA group,
 because SQA is the group in the developer’s organization
that has the greatest stake 职责所在 in assuring the
correctness of the code.
 Participants should
 receive material before the walkthrough and
 prepare lists of items
 they do not understand
 they believe to be incorrect.
The goal of the walkthrough team is to detect faults
 The goal of the walkthrough team is to detect faults, not to
correct them.
 The person leading the walkthrough guides the other
members of the team through the code.
 The walkthrough can be driven by …… with team members
raising their concerns at the appropriate time. 两种方 式
 the lists of issues compiled by team members
 the code itself
 In both cases, each issue will be discussed as it comes up and
resolved into
 either a fault that needs to be addressed or 需要指出的错误
 a point of confusion that will be cleared up in the discussion 需要澄清
的问题
Code Inspections
 An inspection is a far more formal activity than a
code walkthrough.
 It is conducted by a team of 3 to 6 people that
includes
 representatives from the group responsible for the current
phase (implementation, testing) and
 representatives from the next phase or phases (integration
testing).
 One member of the team plays the role of moderator, leading
and managing the team.
 Another team member is the recorder, who writes down the
faults found during the inspection.
The review process consists of 5 formal steps:
 the overview step
 the preparation step
 the inspection step
 the rework step
 the follow-up step
The review process consists of 5 formal steps:
 In the overview step
 the author of the module gives a presentation to the team.
 In the preparation step
 the participants try to
 understand the code in detail and compile lists of issues
 ranked in order of severity.

 use a checklist of potential faults which to be found to help in


this process
The review process consists of 5 formal steps:
 In the inspection step
 perform a thorough walkthrough of the code
 aiming for fault detection through complete coverage of the code.
 Within a day of the inspection, the moderator produces a meticulous
written report 详细的书面报告 .
 In the rework step
 the individual responsible for the code resolves
 all faults
 issues noted in the written report.
 In the follow-up step
 the moderator must make sure that each issue has been resolved by
 either fixing the code or
 clarifying confusing points.
How to do with the fault
 An important product of an inspection is the
number and kinds of faults found rated by severity.
统计错 误数量及类型 ,给错误分 级
 In an inspection, if a module exhibiting a
significantly larger number of faults than other
modules in the system 远远超过平 均水平
 it is a good candidate for rewriting. 就要重写
 If the inspection of two or three modules reveals a
large number of errors of specific types, 存在大 量
特定类 型的错误
 this may warrant (re)checking other modules for similar
errors. 其他模块也会存在!
Something helpful
 Something help in conducting a second inspection of
the module later, such as A detailed breakdown 分
类 of
 types
 severity
 number of errors found
 If more than 5 percent of the material inspected
must be reworked, the team must reconvene for a
full re-inspection. 如此高的出错 率,要找原 因了

Remarks on Non-execution-Based Testing Techniques
 Walkthroughs and inspections have been shown to be a very
powerful method for finding faults in code and other
documents. 注意不仅是代码,其他阶段产品也使用该
方法
 Some studies have shown that inspections are capable of
finding the vast majority of faults in a product in the design
and implementation phase.
 The additional time spent on code reviews and design
document reviews more than pays for itself by leading to
early fault detection. 检查越早开始越好,时间成本越小
 it is less time consuming and more cost effective to conduct
reviews than
 to rely on expensive and not altogether reliable execution-based testing
techniques.
some impediments (阻碍) to effective non-
execution-based testing techniques
 First
 unless the product is modularly decomposable, it will be very
difficult to conduct an effective code review.
 Second
 during the review process, it is sometimes necessary to refer
to documents produced by earlier phases, so these must be
completed and up to date for the current phase of the project.
some impediments to effective non-execution-based
testing techniques
 Third
 unless they are carefully managed, both walkthroughs and
inspections can easily degenerate into performance evaluation
sessions for team members 对小组成员的表现评估,??
 where points are scored for finding the maximum number of faults
or 给发现最多错误的人计分
 where team members are evaluated negatively for not finding
faults. 反之……
 Finally
 it is important that a review session be kept to a maximum of two
hours, since performance of the team is likely to deteriorate if the
time for the review extends beyond that. 时间太长,人易疲劳,
效率下降
6.2.3 Other Testing Approaches
 Correctness Proofs
 Complexity Metrics
 Fault Statistics and Reliability Analysis
 The Cleanroom Technique
Correctness Proofs
 Correctness proofs are formal, mathematical proofs
that a code fragment satisfies its input and output
specifications.
 Ideally, they should be developed along with the
code, not as an afterthought. 而不是事后 再证明!
 Textbook section 6.5.1 ( P119 ) provides a
 simple example of a correctness proof.
Do we need correctness proofs every time?
 Many software engineering practitioners claim that
correctness proofs should not be viewed as a standard
software engineering technique for a variety of
reasons.
 The technique is
 not suited to every software product and
 best left for critical portions ( 最重要的部分 ) of an application
 for situations when no other technique is able to provide the
desired certainty that the code is correct. 其他方法无能为力时
 Why no longer or not always valid. next page……
Many of these reasons are no longer or not always valid.-1
 First, it is claimed that correctness proofs require a
mathematical preparation that most software
practitioners do not have. 需要很 强的 数学背 景
 However,YOU , computer science graduates currently receive
sufficient training in mathematics and logic to be able to
construct such proofs, or to learn how to, on the job. 你们学习
过哪些相关课程?
 A second claim is that correctness proving is too
expensive.
 Whether this is in fact the case should be determined by a cost-
benefit analysis on a project-by-project basis.
Many of these reasons are no longer or not always valid.-2
 Another claim has been that proving the correctness
of code is too hard,
 but several systems have been proved correct and nowadays the
process is facilitated by the use of tools such as theorem
provers.
 Still, there are some difficulties in formally proving
the correctness of coding
 including the need to rely on the correctness of the theorem-
proving program itself. 所依据的定理本身正确吗?
Complexity Metrics 复杂性 度量
 Although complexity metrics might be considered a
glass-box technique because they look at the code,
 they are not used for testing modules
 but rather for determining which modules should be
emphasized in testing.
 They are based on the assumption that the more complex a
module is, the more likely it is to contain faults. 越复杂越容
易出错
Different complexity metrics have been developed
 the simplest of which is
 based just on the number of lines of code.
 More sophisticated predictors have incorporated
information such as
 the number of binary decisions, 二元决策( if-then-else )
 the total number of operators and operands, and
 the number of distinct operators and operands.
complexity metrics can be a useful tool
 There is evidence that
 complexity metrics can be a useful tool for determining
which modules should be given special attention,
 although it’s not clear that
 more complex metrics provide significant improvement as a
criterion for measuring complexity 具有明显的提升作用
 Than over just lines of code.
Fault Statistics and Reliability Analysis
 Fault statistics provide a useful metric for determining
 whether to continue testing a module or product or
 whether to recode it.
 The number of faults detected via execution- and non-
execution-based techniques must be recorded.
 Data on different types of faults found via code inspections
 misunderstanding the design
 initializing improperly
 using variables inconsistently
 These data can be incorporated into checklists 数据还可 以
用来……
 for use during later reviews of the same product and future products.
Reliability analysis
 Reliability analysis uses statistical-based techniques
to provide estimates of
 how many faults are remaining and
 how much longer it is desirable to keep testing.
 It can also be applied in
 both implementation
 and integration phases.
 It is based on the assumption that 基于一个 假定
 the chance of a failure occurring decreases exponentially( 成
指数降低 ) as testing proceeds 
 the longer a product runs without failure, the greater the
likelihood that the product is fault free.
An example of a statistical-based testing technique
 the zero-failure technique. 零故障技术
 It uses
 the number of faults stipulatedin the product’s specifications,
在规格说明中规定的故障数量
 the number of faults detected so far during testing, and
 the total number of hours of testing up to the last failure,
 determine how long to test in order to be confident
that the product’s fault rate satisfies specifications.
The Cleanroom Technique 净室技 术
 The Cleanroom technique is a combination of several
different software development techniques.
 Under this technique, a module isn’t compiled until
it has passed an inspection, or another kind of non-
execution-based review,
 a code walkthrough or inspection.
 The relevant metric is testing fault rate, 故障率
 the total number of faults detected per KLOC (thousand lines
of code)
 a commonly used measure of code quality in the software
industry.
Comparison of the metric
 However, in other techniques this metric
 is applied by the SQA group
 after the programmer has informally tested a module, and
 does not include faults found and corrected by the
programmer during development and desk checking. 开发调
试时发现的故障
 In contrast, in the Cleanroom technique, the metric
 includes the faults found from the time of compilation and
through execution, 编译执行时的故障
 but not the ones found by previous inspections and other
non-execution-based techniques. 不含走查时发现的故障的
数目
Cleanroom technique has considerable success
 The Cleanroom technique has had considerable success in
 finding and weeding out faults before execution-based testing.
 In one case,
 all faults were found via correctness-proving techniques, using largely
informal proofs, but with a few formal proofs as well.
 The resulting code was found to be free of errors both at compilation
and at execution time.
 In a study of 17 other Cleanroom software products, the
products did not perform quite as faultlessly, but they still
achieved remarkably low fault rates, an average of 2.3 faults
per KLOC.
6.2.4 A Comparison of Module-Testing Techniques
 we have surveyed a number of module-testing
techniques,
 execution-based
 non-execution-based
 Different studies have tried to draw conclusions
about the relative effectiveness of these techniques,
but the results have not always been consistent or
conclusive( 确定 的 ).
The most general conclusion
 The most general conclusion that can be drawn is that all
techniques are roughly equally effective from
 the perspective of finding faults, and
 each presents some advantages and disadvantages.
 One study concluded that
 professional programmers were able to detect more errors more quickly
with code reading,
 while advanced students did equally well whether it was with a black-
box or code-reading technique. ( why ?)
 Both groups performed better with black-box and code reading than
with glass-box techniques, which also tend to be quite expensive on a
per fault basis.
 Overall
 code reading found more interface faults,
 black box found more control faults.
special difficulties when to test object-oriented code
 Methods associated with object classes may change
the internal state of object instances, but
 the correctness of the resulting internal state cannot be tested
without developing methods and sending messages that give
the value of state variables. 需要开发方法
 In addition, even though a method may have been
tested for a superclass, it will still need to be retested
for instances of subclasses,
 due to the fact that inheritance permits superclasses and
subclasses of objects to have different methods. 子类经过了
扩充,加入了许多新的方法
What the management must do?
 Because module-testing practices, except correctness
proofs, are still largely an imperfect science,
 management needs to judge testing process, in
particular with respect to the kind of testing and its
exhaustiveness. 详尽程度 , 到什么 程度为止
 Ultimately, the management needs to decide
 whether the cost of proving correctness exceeds the benefit
of checking that the product satisfies its specifications.
 when to stop testing.
 A set of techniques from reliability analysis can be
used to provide statistical estimates of the remaining
number of faults.
What the management must do?
 Another decision that management may need to make is
 whether to keep testing a module that is found to contain many faults or
 whether to have it recoded from scratch.
 We mentioned earlier that
 finding a significantly larger than average number of faults in a module
argues for recoding that module.
 the probability that further faults will remain increases with the number
of problems found during development. 继续测,会有更多错误出来
 Management must
 decide the maximum acceptable number of faults, and
 request that a module be recoded if that number is exceeded.
6.3 Integration Testing
 Top-Down Implementation and Integration
 Bottom-Up Implementation and Integration
 Sandwich Implementation and Integration
 General Remarks on Integration Testing
Integration testing
 Integration testing tests the modules' ability to work
together correctly.
 it is difficult to test modules in isolation, particularly
modules that call each other.
module calls other modules
 it is necessary to create stubs — minimal versions of the
called modules.
 such a stub for module X should at the very least give a
message saying: "Module X was called."
 Better yet, a stub should return values corresponding to
preplanned test cases.
module called by other modules
 the module being tested is itself called by other modules,
 a driver must be coded to pass the appropriate arguments.

 The effort put into creating drivers and stubs may be


minimal, but it still consumes resources for work that
will be thrown away. 貌似没有 什么 工作量 ,但 还是
要消耗 资源的
combine module and integration testing
  For this reason, if possible, it is best to combine
module and integration testing. 结合 单元测 试与 集成
测试, 安排好 模块 实现的 次序 !
 First, modules should be tested individually.
 After integrating two modules, the tester should check that the
partial product continues to behave as it did before adding the new
module. 加入新模块后,是否一样正常
three approaches for performing integration testing
 There are at least 3 approaches for performing
integration testing, each with its own advantages
and disadvantages.
 Top-Down Implementation and Integration
 Bottom-Up Implementation and Integration
 Sandwich Implementation and Integration
Top-Down Implementation and Integration
 Suppose that a system has the interconnection
pattern shown in the graph below
top-down integration testing sequence
 A top-down integration strategy begins by testing
the integration of module a with modules b, c, and d.
It then goes on to testing the integration of modules
b and e, c and f, d and f, d and g, and so on, until the
leaf modules are reached.
advantages to top-down integration
 Top-down integration helps isolate faults to lower
modules as they are being added
 if the higher modules have been tested, faults appear when
new lower modules are added
  faults can be attributed to those lower modules. 或存在
与与之的接口中
 Design faults also show up early in the testing
process. Because……
 Software products include two kinds of modules:
 Logic modules, which incorporate the decision-making, flow-
of-control aspects of the product; usually found at upper levels.
 Operations modules, which perform the actual operations of
the product; usually found at the lower levels.
It is wiser to code and test logic modules before
operations modules.
 Logic modules are usually more complex,
 design faults are likely to show up in them.
 Operation modules are likely to be reusable, even if
they are coded first. 一般是 原子操作, 文件读写

 if top-level modules are re-implemented to correct
logic faults
 the interconnections between the logic and operational
modules will still need to be retested,
 resulting in unnecessary work. 需要做另外的测试工作
 设计错误发现越晚,修改的成本越大
some disadvantages to top-down integration
 The lower and potentially reusable operational modules may
be inadequately tested
 as the testing process runs up against time constraints in the
development schedule. 时间紧迫时首先牺牲的是对底层模块的详细
测试
 操作模块的可重用性,也会使程序员忽略对之的详细测试
 软件被设计的“太好”,模块也可能测试不彻底
 logic tests may prevent the lower modules from ever being tested
 it promotes defensive programming. (保护程序的存在)
 If (X>=0) then y = SquqreRoot(x, errorflag);
 the caller module include safety checks.
 input (X = -2) should not occur , 不能通过条件检查
 A better practice is responsibility-driven design, in which safety checks
(possibly assertions) are built into the called modules. 如将 X 的取值
范围测试放到 SquareRoot 内
Bottom-Up Implementation and Integration
 In bottom-up integration,
 lower-level modules are coded and tested first,
 higher modules coded as drivers.
 coding and testing sequence
 the leaf modules (h, i, l, m, k) would be coded and tested
first,
 then their callers (modules e, f, j, g) would be coded and
tested (possibly at the same time),
 integration of each caller-called module pair would be tested.
Advantage and disadvantage
 Bottom-up integration
 isolates faults to the upper modules
 results in thorough testing of the lower-level operations
modules. 专门开发的测试驱动程序,没有保护程序屏蔽
,使底层模块充分地被测试
 The drawback is that major design faults will be
detected late in the process. 因为体 现 设计 的 逻辑
模块 集成后 期再测试 的,设计错 误发现很晚
Sandwich Implementation and Integration
 The sandwich integration strategy
 combines the top-down and bottom-up approach
 yielding the advantages
 relieving the disadvantages of both approaches.
 扬长避短!
Sandwich Implementation and Integration
The sandwich integration strategy
 sequence
 Logic modules are implemented and integrated top down
 operations modules are implemented and integrated bottom
up.
 The interfaces between the two groups of modules are tested
one by one.
 Note that there is some flexibility in determining
which modules belong to which group. 灵活划分
 module j could be treated as an operational module while g
could be treated as a logic module.
 module j could be included among the logic modules or g
among the operational modules.
General Remarks on Integration Testing
 Access to inconsistent copies of the design document is a
likely cause of type of problem as following. 实现依据不一
致造成的
 Inconsistent assumptions held by different programmers — may
produce faults that show up during the integration phase.
 such as the range of input
 the number of arguments that a module must accept
 所以应该由第三人来决定谁对谁错!
 it is better to let the integration test plan and the test phase
itself be run by the SQA group.
 Because none of the individuals who participated in coding the
inconsistent modules may be willing to admit that they were responsible
for the mistake,
 This group also has the most to lose if testing is not performed properly.
质保小组的职责所在!
项目实 践统计表 明
 The three approaches to integration testing apply
equally well to object-oriented and non-object-
oriented systems.
 top-down

 bottom-up

 sandwich
Integration testing of products with GUI, however,
poses particular problems.
 In non-GUI testing, it is possible to use a rudimentary
CASE tool ( 一般 的 CASE 工具 ) to set up input test
cases and expected outcomes, allowing tests to be
rerun when new modules get integrated. ( 输入与 期
望输 出数 据放在 一个文 件中 与实际 输出 进行 比较
,得 到测 试结果 )
 This will not work for products that incorporate a
GUI because user-input events ( mouse clicks )
cannot be stored in a file as other test data can.
 The solution is to use a special CASE tool, a GUI testing
environment such as QAPartner or Xrunner.
 this type of tool create a machine-readable version of a manual
GUI test case, called a script that can be run as a normal test
case.
6.4 Product and Acceptance Testing
 Product Testing
 Acceptance Testing
 Module testing and integration testing are
verification activities. they check to see
 the individual modules are correct
 the modules communicate correctly with each other.
  integration testing is completed  ……  the
product is delivered 
 test the whole product of other behavioral properties.
 Who do this?
 it is the responsibility of the SQA group in the developer’s
organization
 Ensure to find and remove any residual product faults are
Product Testing
 The goal of product testing is validation of the
product
 checking that the software meets behavioral specifications
including, but not limited to, correctness.
For COTS software
 the SQA team’s primary concern is that the product
be free of faults, since it will go to as many
customers as possible.
 the financial and public image cost of selling faulty
COTS products can be prohibitive 经费及公众形
象方面 ,代价巨大
  after internal product testing is satisfactorily
completed
  the product is shipped to alpha and beta test -
sites (prospective buyers) 
 receive feedback on any remaining faults that the
SQA team may have overlooked.
For COTS software
 Another important test to be performed both in-
house and at test sites concerns the utility of the
product,( 产品 效用 ) , the extent to which a
product meets customer needs
 when it is correct
 when it is used under the conditions permitted by its
specifications.
 The concept of utility includes
 whether the product does something useful,
 whether it is easy to use, and
 whether it is cost effective relative to competing products.
 如金山 wps, 与微软 ms office
For contract software
 most of the remaining testing before delivery will
take place in-house.
 The SQA’s primary responsibility is to ensure
 the product will pass the acceptance test.
 Failure to do so 否则 ……
 reflects badly on the management capabilities of the
developer organization 管理无能!
 may create a major public relations problem.
 So , The SQA group must perform a number of
tests.
What we should do?
 The product as a whole must be subjected to
correctness testing
 using black-box techniques
 similar to the testing at the module level.
 The reliability of the product must be tested. include
 estimates of mean time between failures
 the severity of the effects of a failure.
 Severity results must be broken down to include
 mean time to repair the product when a fault occurs
 the time required to repair the consequences of that fault (the
corruption of client data as a result of the fault occurring).
What we should do?
 The robustness of a product under a range of
operating conditions must be tested.
 when it receives input that falls outside the range of
legal input given in the specifications. 非法输入 时
 The product should not crash
 it should minimally respond with an error message saying
that the input could not be processed
 preferably giving the reason why the input could not work.
What we should do?
 The product should be subjected to stress testing to
check its behavior 强度测 试
 under very high user loads (many users logged in at once)
 volume testing (very large data files, or a large number of
transactions).
 The product should also be tested against any other
specified constraints, such as
 performance (response times)
 storage, and
 security.
 All documentation should be checked against
standards set out by the software project
management plan (SPMP).
What we should do?
 Finally, the product should be checked for
compatibility with other software used by the client
(installation testing).
 When the SQA group determines that the product
has passed all these tests,
 the product is ready to be delivered to the client for
acceptance testing.
Acceptance Testing
 Acceptance testing is driven by the client.
 It can be performed
 directly by the client,
 by the developer’s SQA in the presence of a client representative
 by an SQA group hired by client.
 Acceptance testing may include all of the types of testing
that were performed by the developer during product
testing, but the 4 primary components are testing
 correctness,
 robustness,
 performance, and
 documentation.
 Unlike testing at the developer’s organization,
acceptance testing is performed using the
customer's hardware and actual data.
 If test data should be a reflection of actual data,
 the specifications may be incorrect
 the SQA may have misunderstood them.
 If the product replaces an existing product 软件更

 the specifications should say that the new product would be
run in parallel with the old one until the client is satisfied that
it works satisfactorily.
 At that time, the old product can be retired.
6.5 CASE Technology
 CASE Technology in General
 Coding Tools
 Version Management Tools
 Build Tools
 Integrated Environments
 When to Use CASE Technology
CASE Technology in General
 When discussing CASE tools (computer-assisted
software engineering tools), it is important to keep
in mind the following distinctions:
 Programming-in-the-small:
 coding at the level of the code of a single module
 Programming-in-the-large:
 software development at the module level,
 including aspects such as architectural design and integration

 Programming-in-the-many:
 software production by a team
 either at the module level or at the code level
CASE tool scan aid in all aspects of software development
 CASE tools, workbenches, and environments can
 aid in all aspects of software development.
 are also helpful in keeping online a single consistent version
of documentation and
 Aid in communication among developers.
 store email exchanges over product development issues
 provide a record of design decisions.
 A particular CASE tool assists in only one aspect of
software production. Code editor
 CASE tools are typically classified as front-end or
upperCASE, 不同阶 段起作用
 if they assist with the requirements, specification, and design
phases.
 Similarly, back-end or lowerCASE tools
 assist with implementation, integration, and maintenance.
 programs for constructing graphical representations of
software such as data flow diagrams
 tools for constructing data dictionaries.
CASE workbenches and environments
 In addition to CASE tools, there are also CASE
workbenches and environments.
 Workbenches are collections of tools that support a
small number of related activities — like
 editing, compiling, linking, testing, and debugging.
Debugging , IDE
 is relevant throughout the entire software process,
or at least a large portion of it.
Coding Tools
 Coding tools assist the programmer with many aspects of
programming.
 Commonly available coding tools include:
 Structure editors: tools for writing code that know about the programming
language in which the code is written and that can catch syntax errors;
 Pretty printers: tools that produce nicely laid out programs;
 Online interface checkers: tools that check that procedures, functions,
and/or methods exist and are being called with the right number and type of
arguments;
 Operating system front ends: tools that facilitate compiling, linking, and
loading;
 Informative error messages: tools capable of displaying run-time errors at a
level appropriate for the programming language (for example, "Line 24:
division by 0," instead of "Segmentation fault" for a FORTRAN program);
and
 Source-level debuggers: run-time tools that produce trace output or can be
used to debug the code interactively by setting breakpoints.
structure editor
 a structure editor
 incorporates some or all of the other coding tools,
 support for online documentation,
 the resulting collection of tools constitutes a programming
workbench.
Version Management Tools
 During development and maintenance of a software
product, several revisions of each module may arise
in response to requests
 for fault removals
 enhancements
 These changes produce multiple versions of
modules, which can become a nightmare unless
properly managed. 程序员 的恶梦!
 The CASE tools help manage software versions.
 Version control should begin when the module has passed
SQA inspection.
 Before that, there will be too many versions for each module.
Software-versioning tools
 Software versioning tools help organizations
manage
 multiple versions
 Software-versioning tools help developers keep track of
different versions of the software product or of components of
the product, such as
 following an incremental life-cycle model 增量开发
 making revisions during maintenance. 维护时修改
 variations. 模块的异构版
 Matters are further complicated by the coexistence of module
variations.
 different implementations of the same module for different hardware
platforms.
 Software-versioning tools help developers keep track of
different versions of the software product or of components
of the product, such as
 might arise when following an incremental life-cycle model or
 when making revisions of a product during maintenance.
 Although there will be only a single version of each
component (module) in a specific product version, multiple
versions of components will need to be accessible and
distinguished from each other as the new version of the
product is developed.
 Matters are further complicated by the coexistence of
module variations.
 different implementations of the same module for different hardware
platforms.
 Software versioning tools help organizations manage multiple versions
and variations.
Revision and variations
 Configuration-control tools are used to specify
which versions and which variations of each
component are required for a particular version of
the complete product — that is, its configuration.
不同的 配置,形成不 同的最终完 整产品
 Baselines and access control systems prevent
programmers from working on the same module
simultaneously.
How the Baselines and access control systems do?
 The maintenance manager creates a baseline
configuration for the product.
 A programmer needing to work on a module
 copies that configuration into his or her private workspace
 freezes or locks the module that needs to be changed. 做标
记,锁定! Microsoft sourcesafe
 No other programmer is allowed to work on a
frozen module.
 the changes have been implemented and tested
 the new version of the module is installed and
unfrozen
 changing the baseline.
Build Tools
 Build tools assist in selecting and assembling all the
correct versions of the code.
 A build tool needs to be used in conjunction with a
configuration-control tool, ideally to be integrated
with it.
 the UNIX make command, which uses a makefile to specify
all components of the system that must be
 compiled
 linked

 in which order.
Integrated Environments
 In the context of CASE, integrated environments
usually in reference to user interface integration —
 an environment where tools
 all have the same visual appearance and
 provide access to similar functionality in similar ways.

 Macintosh and the Microsoft Windows environments.


 Process integration refers to an environment that
supports a specific software process. 与模型有 关
 In tool integration, all tools communicate via the
same data format.
 Team integration takes place when a CASE
environment promotes effective team coordination
and communication. 调度与 通信
 management integration refers to a CASE
environment that supports management of the
process.
 Until now, by and large only tool integration has
been achieved.
When to Use CASE Technology
 CASE technology can
 improve programmer productivity,
 shorten development time, and
 increase product user satisfaction.
 However, CASE technology requires training and one
should use it with care.
 CASE environments, like programming languages, need to
be matched with the situation of use.
 Low maturity organizations can use CASE tools and
workbenches, but not environments.
 Environments automate a process, so it is important that it be the right
process for the organization.
 When there is no process in place, they automate chaos. 使用不当,适
得其反!