Software Testing Unit - I: Learning Objectives

MCA 317, Software Testing
Software Testing Unit -I
Bharati Vidyapeeths Institute of Computer Applications and Management, New Delhi-63, by Shalini Singh Jaspal
U1. 1
Learning Objectives
What is software testing and why it is so hard? Error, Fault, Failure, Incident Test Cases Testing Process Limitations of Testing No absolute proof of correctness Overview of Graph Theory
U1. 2
Reasons for Emphasis on S/W Testing

Software an integral part of Daily-usage devices Life-support systems Wider audience thus implying Uncertainty in usage patterns and environments Probably untrained users Higher Risk Factor Requirement for round-the clock availability
U1. 3
Bharati Vidyapeeths Institute of Computer Applications & Management, by Shalini Singh Jaspal
Objective Behind Testing

Improve Quality of the Software Find Bugs Enhance usability (end users perspective) Reduce costs!!!
U1. 4
Definition
Process that Establishes the S/W is working satisfactorily as per the users requirements. Proves the product is free of errors. Certifies that the product is ready to be used.
U1. 5
In Practice
Testing is a task of locating errors. Includes running / examining the S/W product with the intent of making it fail. By Glen Myers Testing is the process of executing a program with the intent of finding errors Objective The basic idea is to make sure that the errors are tracked and submitted for rectification before the product is finally delivered.
Bharati Vidyapeeths Institute of Computer Applications and Management, New Delhi-63, by Shalini Singh Jaspal U1. 6
Basic Questions
When to test? Cost and time factors What to test? What is Software How to test? Automated or manual Who should test? Does the developer qualify
U1. 7
When to Test
Phased Process => Errors at one stage would propagate to the next unless checked in time. Later an error is found, the more it costs to fix. Thus, it is important to: Prevent occurrence of errors. Detect errors in the same phase as they are introduced and prevent their migration across phases of development. Thus, in order to minimize costs we need to test right from the stage one where requirements are being gathered .
How to Test
Laying down processes Considerations behind Automation Analysis of tools available in terms of Suitability Economic feasibility In-time procurement User Training Portability Interoperability Available support
U1. 9
Who should test

Problems with developers assigned the job of testing Overconfidence Biased view Already tested this syndrome Work on basis of Requirement Specifications A tester should be someone who Has an unbiased perspective Understands users requirements Has a knowledge of the entity being tested and the domain it applies to. Knows the feature being tested. Knows the purpose behind the testing exercise.
S/W Failure
Error, Fault, Failure, Incident Errors due to: Commission Omission Could lead to deviation from user requirements or requirement specifications.
U1. 11
S/W Failure
A common cause behind failures is that we can pay attention to the wrong conditions. But we can't pay attention to all the conditions The phenomenon of blindness due to inattention Humans (often) don't see what they don't pay attention to. Programs (always) don't see what they haven't been told to pay attention to. (Programs are more precise and less flexible.)
U1. 12
S/W Failure: Various Ways

Program state System state Program state, including uninspected outputs System state
Intended inputs
Configuration and system resources From other cooperating processes, clients or servers
System under test
Monitored outputs
Impacts on connected devices / system resources To other cooperating processes, clients or servers
U1. 13
Testing
A technical investigation done to expose quality-related information about the product under test
U1. 14
Explanation
A technical
We use technical means, including experimentation, logic, mathematics, models, tools (testing-support programs), and tools (measuring instruments, event generators, etc.)
investigation
An organized and thorough search for information. This is an active process of inquiry. We ask hard questions (aka run hard test cases) and look carefully at the results.
done to expose quality-related information
Discussed next
about the product under test & the related processes.
U1. 15
Quality Related Information

Find important bugs, to get them fixed Check interoperability with other products Help managers make ship / no-ship decisions Block premature product releases Minimize technical support costs Assess conformance to specification Minimize safety-related legal risk
U1. 16
Challenges
What are various possible conditions in which it is expected to work. Does the S/W function as it is expected to under those conditions. How to generate those conditions. Convey the findings in a positive manner. Testing time and resources are generally limited. Deciding when to stop testing
U1. 17
Pre-requisites of Good Testing

Well designed test case (having a high probability of detecting an uncovered fault) Addresses both valid and invalid features Addresses both functional and non-functional requirements Addresses users perspective Based upon what, why and how of testing Is well documented and reusable Lists expected outcomes and success criteria Is reliable
U1. 18
Limitations of testing
The objective of testing is to show presence of errors. What it definitely cant do is prove absence of errors. One can never find the last bug in an application, no matter how hard he tries as: Domain of possible inputs is too large. The program has a number of possible paths. The challenge is to achieve maximum coverage through minimum test cases. It is rightly said that testing process is only as efficient as the test data.
Are Well Designed Test Cases Sufficient

Apart from well designed test cases it is important to: Analyze the results properly Establish relation between faults and incidents Report the detected fault effectively Association with a module Steps for regenerating the problem encountered Optionally a probable impact
U1. 20
Qualities of a Good Tester

Soft Skills Communication: within and across group Positive attitude Understanding of users perspective Ability to prioritize Ability to set goals and chase them effectively Reporting Skills Knowledge Domain expertise Programming Languages Automation Tools Capability of planning Capability of Analyzing Risks
Testing Principles: Myers

A necessary part of a test case is a definition of the expected output or result A programmer should avoid testing his or her own program A programming organization should not test its own programs Thoroughly test the results of each test Test cases must be written for invalid and unexpected, as well as valid and expected input conditions.
U1. 22
Testing Principles: Myers

Examining a program to see if it does not do what it is supposed to do is only half of the battle. The other half is seeing whether the program does what it is not supposed to do. Avoid throw-away test cases unless the program is truly a throwaway program. Do not plan testing effort under the tacit assumption that no errors will be found. The probability of existence of more errors in a section of a program is proportional to the number of errors already found in the section
Few Directions
Use testing for Quality Assurance rather than Quality Control Always remember the customers perspective Keep redesigning the tests to address untested areas Remember that the principle proposed by Myers that the probability of existence of an error in a part of code is proportional to the number of errors already found Should know when to stop testing
Testing Cost Curve
U1. 25
Impracticality of Testing All Paths

For most programs, it is impractical to attempt to test all execution paths through the product, due to a combinatorial explosion [1]. It is also not possible to develop an algorithm for generating test data for paths in an arbitrary product, due to the inability to determine path feasibility [2]
U1. 26
No Absolute Proof of Correctness

Howden claims that there is no such thing as an absolute proof of correctness[3]. Instead, he suggests that there are proofs of equivalency,i.e., proofs that one description of a product is equivalent to another description. Hence, unless a formal specification can be shown to be correct and, indeed, reflects exactly the users expectations, no claim of product correctness can be made [4]
U1. 27
Conclusion
What is software testing and why it is so hard? Error, Fault, Failure, Incident Test Cases Testing Process Limitations of Testing No absolute proof of correctness Overview of Graph Theory: Students Presentation
U1. 28
Objectives
Life cycle testing Verification Validation V Model W Model 11 Steps of Testing: W. Perry
U1. 29
Life-Cycle Testing
What it is: Implies continuous testing of solution even after plans are complete and tested system is implemented. Objective: Identify defects at earliest possible time Requirements: Formally developed processes A well defined Test Plan Structured Methodologies. Features: Development and Testing processes start at the same time (with the project) with the same information
10
Life-Cycle Testing
Responsibilities Development team defines and documents the requirements, carries out development related activities Test team formulates test plans and runs compliance tests to uncover defects
U1. 31
Verification & Validation

Traditionally testing used to be an End-of-the-developmentcycle activity With Life-Cycle testing model there is a requirement to test deliverables of each phase. The associated activities can be classified as Verification & Validation. Thus, software testing is essentially the verification and validation processes put together. Both the processes are complementary in nature and provide their own set of error filters
Verification
Verification: Are we building the product right? Static and proactive in nature (QA). Checks if product (result of a particular phase of SDLC) conforms with its specifications. Is static in nature. Involves manual checking, doesnt include execution of code. The target documents are Req. Specs., HLD, DB Design The methods used are Inspection, Walkthrough etc.
11
Validation
Validation: Are we building the right product? Dynamic and reactive in nature (QC). Checks the software to insure it meets customers requirements. Is dynamic in nature. Involves execution of code. The target could be a component, module, a set of integrated modules or system as a whole. The methods used are Black box, white box testing etc.
U1. 34
Verification & Validation

Req Specs HLD Detailed Design Code Integration Verify Verify Verify Verify & Validate Verify & Validate System Testing UAT
Verification and Validation: IEEE

Verification: The process of evaluating a system or component to determine whether the productssatisfy the conditions imposed Validation: The process of evaluating a system or componentto determine whether it satisfies specified requirements.
U1. 36
12
Verification vs. Validation

Verification ensures that the system complies with an organizations standards and processes, relying on review or non-executable methods Did we build the right system? Validation physically ensures that the system operates according to plan by executing the system functions through a series of tests that can be observed and evaluated Did we build the system right?
U1. 37
Drivers of World-Class Testing

Planning Training Management Support User Satisfaction Use of processes Efficiency Tools Quality Control
U1. 38
V Model
The V-model is a software development process which can be presumed to be the extension of the waterfall model. Instead of moving down in a linear way, the process steps are bent upwards after the coding phase, to form the typical V shape. The V-Model demonstrates the relationships between each phase of the development life cycle and its associated phase of testing.
U1. 39
13
V Model
The V-model deploys a well-structured method where each phase can be implemented by the detailed documentation of the previous phase. Testing activities like test designing start at the beginning of the project well before coding and therefore Saves a huge amount of the project time and thus helps in cost reduction Assures better validation Assures involvement of people with appropriate skills.
U1. 40
V Model
U1. 41
V Model
U1. 42
14
V Model
The V-Model further promotes the notion of early test preparation. finds faults in baselines and is an effective way of detecting faults early The V-model promotes the idea that the dynamic test stages (on the right hand side of the model) use the documentation identified on the left hand side as baselines for testing.
U1. 43
V Model: Problems
Derivative of Waterfall model => no provision for increments V Model focuses on specific dynamic test stages And has no mention of the effectiveness of static tests such as reviews, inspections, static code analysis and so on. This is a major omission and the V-Model does not support the broader view of testing as a constantly prominent activity throughout the development lifecycle.
U1. 44
V Model: Problems
No mention of important activities like: Planning Post implementation maintenance Testing Quality Control
U1. 45
15
Modified V
Addresses the parallelism in S/W development and accommodates incremental development. Introduces concept where development products go through phases of testing Each unit/component/module is given an entry / exit criteria to go through a testing stage The components that satisfy the entry criteria are moved into the corresponding testing phase The components that satisfy the exit criteria are moved out of the current testing phase
W Model
W-Model focuses on the development products themselves and utilizes static testing techniques to verify intermediate deliverables. Essentially, every development activity that produces a work product is shadowed by a test activity.
U1. 47
W Model
The purpose of the test activity specifically is to determine whether the objectives of a development activity have been met and the deliverable meets its requirements. The W-Model of testing focuses specifically on the product risks of concern at the point where testing can be most effective.
U1. 48
16
W Model
U1. 49
11-Step of Testing: W. Perry

Address the shortcomings of W-Model by incorporating phases for planning, post-implementation maintenance and Quality Control
U1. 50
11-Step of Testing: W. Perry

The steps are: 1. Access development plan and status 2. Develop the test plan 3. Test software requirements 4. Test software design 5. Program phase testing 6. Execute and record results 7. Acceptance Test 8. Report test results 9. Test Software installation 10.Test Software Changes 11.Evaluate test effectiveness
17
Define Software requirements
Step1 Assess Development P lan and Status Step2 Develop the test P lan Step3 Test Software requirements
Build Software
Step4 Test Software design Step5 Program Phase Design Step6 Execute and record result Step7 Acceptance Test Step8 Report Test Results Step9 Test Software Installation Step10 Test Software Changes Step11 Evaluate Test effectiveness
Install Software
Operate and Maintain Software
U1. 52
Detailed Analysis of 11-Step Model
U1. 53
Assess Development Plan and Status

Problems in Project Plan can have a direct effect on Software Quality. Thus, testing the Project Development prerequisite for building the test plan Plan is a
Testers can challenge the completeness and correctness of the development plan Potential problem areas could be: An inaccurate estimate Choice of inadequate development tools / procedures
18
Develop the Test Plan

Test Plan acts as a contract between development team / users testers and It describes Tests to be conducted Test objectives and risks Resources and schedule necessary for completion Reports (Interim and Final) to be generated
ensuring
Potential problems areas in ensuring proper execution of Test Plan: Lack of training / test tools / management support / time / user involvement
U1. 55
Test Software Requirements

Important because if a defect is identified at this point, it will not be incorporated in design and code Most cost effective phase for detecting a system flaw Objectives Ensure requirements clearly represent users needs Verify a cost-benefit analysis has been performed and a reasonable alternative is chosen Ensure business constraints have been addressed These objectives are addressed by identifying and evaluating various risk factors by means of a risk matrix
U1. 56
Test Software Design

Involves testing of design phase deliverables like: I/O Specifications System Flowcharts H/W & S/W requirements Success factors are defined on the basis of attributes found by analyzing existing systems Design reviews and inspections are conducted to evaluate the deliverables against listed success factors
U1. 57
19

Design phase helps in developing a solution to the business problem Testing of Software Design addresses design phase objectives, e.g.: Develop specifications for manual and automated system Comply with policies, procedures, standards Complete project within budgetary, staffing and scheduling constraints
U1. 58

Some important concerns to be analyzed during this phase are: Conformance to requirements Compliance with methodology Design of Data Integrity Controls, Authorization Rules, Audit Trails, Contingency Plans Maintainability Portability
U1. 59
Program Phase Testing

Primary objective is to ensure design specifications have been correctly implemented. Technical and requires involvement of someone with programming experience Important to include static testing techniques as well Ensure programs are not released for Integration, System or Acceptance testing before being adequately tested.
U1. 60
20
Execute and Record Results

Aims at uncovering misunderstandings in the translations between user, designers and programmers. Involves execution in test environment similar to operational environment and recording deviations from expected results. Appropriate action is taken based upon the severity level of the detected problem.
U1. 61
Acceptance Test
Describes procedures for identifying acceptance criteria for interim life cycle products Final acceptance acknowledges: Software product meets users requirements Adequacy of development process Responsibilities of user: Ensure involvement in developing system requirements and acceptance criteria Identify interim and final products for acceptance Plan, schedule and delegate responsibilities for acceptance activities Act according to the plan and acceptance decision for each product
Report Test Results

Testing and reporting activities stretch throughout the software lifecycle. Test results / reports must be capable of answering various questions like: Project status Success status, reliability of interim deliverables In order to ensure software projects are accurate, complete and meet users needs, test reports must: Be available when needed Contain adequate test information
21
Test Software Installation

Tests the process that places system in production status Not concerned with verification of functioning of the application Aims to validate: Proper programs are placed in production status Needed data is prepared and available Operating and user instructions are prepared and used
U1. 64
Test Software Changes

Changes could be required: During the development process is on After the software has been placed in to production Require Impact analysis Reiteration of some of the previous steps and Usage / extension of previously used test data. Test process aims at ensuring that the changed application will function properly in the operating environment. Corrective changes can be minimized by ensuring a well planned test process that is conducted by well trained testers.
Evaluate Test Effectiveness

Aims at: Evaluating and improving test process Evaluating and enhancing individual / team performance Identifying areas that need improvement The assessment process includes: Establishing assessment objective (identifying weaknesses, need for tools, good practices etc) Identify what characteristics are to be measured Assigning measurement responsibility Identify needed facts Collect the data Assess the effectiveness of testing by using metrics calculated on basis of data collected
22
Conclusion
Life cycle testing Verification Validation V Model W Model 11 Steps of Testing: W. Perry Detailed Discussion
U1. 67
Verification Techniques
U1. 68
Objectives
Verification Techniques Inspection Walkthrough Pass-around Desk Checks Pair Programming
U1. 69
23
Recall
There are two basic types of testing Execution-based testing Non-execution-based testing V & V Verification Determine if the workflow was completed correctly Validation Determine if the product as a whole satisfies its requirements Remember The terms verify / review are also used for all non-executionbased testing
U1. 70
Verification and Validation
U1. 71
Reviews
A review is any activity in which a work product is distributed / presented to reviewers who examine it and give feedback. In execution-based testing, one defect may mask another so several executions might be required. On the other hand, many different defects may be discovered in a single review. Because of the reuse of domain and programming knowledge, reviewers are likely to have seen the types of error that commonly arise In a peer review, co-workers of a person who created a software work product examine that product to identify defects and correct shortcomings
24
What Can be Reviewed

All interim and final development work products, including: Requirements specifications User interface specifications and designs HLD, LLD Source code Test plans, designs, cases, and procedures Software development plans, including project management plan, configuration management plan, and quality assurance plan
U1. 73
Objective of Review
Verify whether the work product Correctly satisfies the specifications found in any predecessor work product. Adheres to standards. Suggest improvement opportunities to the author. Gain consensus amongst the project team members. Secure approval from stakeholders. Promote the exchange of techniques and education of the participants.
Review: Categories
Technical Ensure conformance to specifications. Check if development is in accordance to plans, standards and guidelines. Ensure changes to software elements are implemented properly. Managerial Check if project making progress as per plan. Identify and evaluate alternatives. Maintain global control through adequate resource allocation.
U1. 75
25
Identifying High Risk Components

Components that use new technology, techniques, or tools Key architectural components Complex logic or algorithms that are difficult to understand but must be accurate and optimized Mission-critical / security-critical components with dangerous failure modes. Components having many exception conditions or failure modes. Components that are intended to be reused
U1. 76
Identifying High Risk Components

Components that will serve as models or templates for other components Components that affect multiple portions of the product Complex user interfaces Components created by less experienced developers Code modules having high complexity Modules having a history of many defects or changes
Which Review Method to Use

Work products that fit in one or more of categories discussed, are considered high risk. A product is considered low risk if an undetected error will not significant affect the projects ability to meet its schedule, quality, cost, and feature objectives. Use inspections for high-risk work products, or the high-risk portions of large products, and for major work products that are about to be baselined. Less formal reviews are acceptable for other work products.
U1. 78
26
Inspections
U1. 79
Participants
In general, a work product should be reviewed by: The author of any predecessor document or specification Someone who must base their subsequent work on the work product Peers of the author Anyone responsible for a component to which the work product interfaces Attendance by anyone with supervisory authority over the author is by invitation of the author only.
U1. 80
Roles - Responsibilities
Author: Creator or maintainer of the work product to be inspected. Initiates the inspection process by asking the peer review coordinator to assign a moderator. States his objectives for the inspection. Delivers work product and its specification or predecessor document to moderator. Works with moderator to select inspectors and assign roles. Addresses items on the Issue Log. Reports rework time and defect counts to moderator.
U1. 81
27
Moderator Plans, schedules, and leads the inspection events. Works with author to select inspectors and assign roles. Assembles inspection package and delivers it to inspectors at least 3 days prior to the inspection meeting. Determines whether preparation is sufficient to hold the meeting. If not, reschedules the meeting
U1. 82
Moderator (Contd.) Facilitates inspection meeting. Corrects any inappropriate behavior. Solicits input from inspectors as reader presents each section of the work product. Records any action items or side issues that arise during the inspection. Leads inspection team in determining the work product appraisal. Serves as verifier or delegates this responsibility to someone else. Delivers completed Inspection Summary Report to the organizations peer review coordinator.
U1. 83
Reader Presents portions of the work product to the inspection team to elicit comments, issues, or questions from inspectors. Recorder Records and classifies issues raised during inspection meeting.
U1. 84
28
Inspector Examines work product prior to the inspection meeting to find defects and prepare for contributing to the meeting. Records preparation time. Participates during the meeting to identify defects, raise issues, and suggest improvements. Verifier Performs follow-up to determine whether rework has been performed appropriately and correctly.
U1. 85
Peer Review Coordinator Custodian of the organizations inspection metrics database. Maintains records of inspections conducted and data from the Inspection Summary Report for each inspection. Generates reports on inspection data for management, process improvement team, and peer review process owner.
U1. 86
Entry Criteria
The author selected inspection as the approach for the product being reviewed. All necessary supporting documentation is available The author has stated his objectives for this inspection. Reviewers are trained in the peer review process. Documents to be inspected are identified with a version number. All pages are numbered and line numbers are displayed. The documents have been spell-checked.
U1. 87
29
Entry Criteria
Source code to be inspected is identified with a version number. Listings have line numbers and page numbers. Code compiles with no errors or warning messages using the projects standard compiler switches. Errors found using code analyzer tools have been corrected. For a re-inspection, all issues from the previous inspection were resolved. Any additional entry criteria defined for the specific type of work product are also satisfied.
U1. 88
Planning: Tasks & Responsibilities

Give moderator the work product to be inspected and supporting documents, such as specifications, predecessor documents, or pertinent test documentation: Author Determine whether work product satisfies inspection entry criteria: Moderator Based on the size and complexity of the work product, determine how many inspection meetings will be required. Moderator and Author
U1. 89
Planning: Tasks & Responsibilities

Select inspectors and assign roles to individuals. Gain agreement from the other inspectors to participate: Moderator and Author Determine whether an overview meeting is required: Author Schedule the inspection, and possibly overview, meetings and distribute a meeting notice: Moderator Distribute the inspection package to the participants at least 3 working days prior to the inspection meeting: Moderator or Author
U1. 90
30
Overview: Tasks & Responsibilities

Describe the important features of the work product to the rest of the inspection team. State inspection objectives: Author Evaluate the assumptions, history, and context of the work product: Inspectors
U1. 91
Preparation
Ask individual inspectors to prepare with specific objectives in mind, such as: checking for interface errors; checking traceability to, and consistency with, predecessor specifications; or checking conformance to standards. Moderator and Author Examine the work product, to understand it, find defects, and raise questions about it. Use the appropriate defect checklist to focus attention on defects commonly found in the type of product being inspected. Use other analysis methods to look for defects as appropriate. Inspectors Log minor defects found, such as typographical errors or style inconsistencies, on the Typo List. Deliver this to the author at or prior to the inspection meeting. Inspectors
Inspection Meeting: Process

Open the Meeting: Introduce the participants (if necessary) and state their roles, state the purpose of the inspection, and direct inspectors to focus their efforts toward finding defects, not solutions. Remind participants to address their comments to the work product under review, not to the author. Moderator Establish Preparedness: Ask each inspector for his preparation time and record the times on the Inspection Summary Report. If preparation is insufficient, reschedule the meeting. Moderator Present Work Product: Describe portions of the work product to the inspection team. Reader
31

Raise Defects and Issues: Point out concerns, potential defects, questions, or improvement opportunities after the reader presents each section. Inspectors Record Issues: Capture the information on the Issue Log for each issue raised. State aloud what was recorded to make sure it was recorded accurately. Recorder Answer Questions: Respond briefly to any specific questions raised, and contribute to defect detection based on special understanding of the work product. Author

Make Product Appraisal: After all meetings scheduled for a given inspection are complete, decide on the work product appraisal. If the inspectors disagree, assign the most conservative appraisal offered by any of the inspectors. Inspectors Sign Inspection Summary Report: All participants sign the Inspection Summary Report to indicate their agreement with the inspection outcome. Inspectors Collect Inspection Feedback. Ask the inspectors to evaluate the inspection and suggest improvements, using the Inspection Lessons Learned Questionnaire. Moderator
U1. 95
Information to Record for Each Defect Found

Origin Type Development phase in which the defect was introduced Missing (something needs to be there but is not) Wrong (something is erroneous or conflicts with something else) Extra (something unnecessary is present) Usability Performance Non-defect issue (question, point of style, suggestion, clarification needed) Major (could cause product failure or cost significantly more to correct in the future) Minor (non-fatal error, cosmetic problem, annoyance, or a workaround is available) Page and line or section number where the defect is located
Severity
Location
Description Concise description of the issue or possible defect

32
Inspected Work Products: Possible Appraisals

Appraisal Accepted As Is Meaning Modifications may be required in the work product, but verification of the modification is not necessary.
Accept Defects must be corrected, and the changes must be Conditionally verified by the individual named on the Inspection Summary Report. Re-inspect Following Rework Inspection Not Completed A substantial portion of the product must be modified, or there are many changes to make. A second inspection is required after the author has completed rework. A significant fraction of the planned material was not inspected, or the inspection was terminated for some reason.
U1. 97
Rework
Task Responsib le Correct defects and typos found, resolve issues raised, and Author modify work product accordingly. Mark issues list to indicate action taken. Correct any other project documents based on defects identified in the inspected work product. Record any uncorrected defects in the projects defect tracking system. If rework verification is not needed, report the number of major and minor defects found and corrected and the actual rework effort to the moderator. Record the actual rework effort on the Inspection Summary Report. Author Author Author
Moderator
U1. 98
Follow-Up
Task Responsible Confirm that the author has addressed every item on the Verifier Issue Log. Determine whether the author made appropriate decisions as to which defects not to correct and which improvement suggestions not to implement. Examine the modified work product to judge whether the Verifier rework has been performed correctly. Report any findings to the author, so rework can be declared complete, incorrect rework can be redone, or items that were not originally pursued can be addressed. Report the number of major and minor defects found and corrected and the actual rework effort to the moderator. Author
U1. 99
33
Follow-Up
Task Responsible
Check whether the exit criteria for the inspection and for the Moderator peer review process have been satisfied. If so, the inspection is complete. Check the baselined work product into the projects configuration management system. Author
U1. 100
Rules to be followed
Inspection should be carried out by 3-6 participants All participants can act as an inspector The author shall not act as inspection leader The author should not act as reader or recorder Roles may be shared among the team members Individual participant may act in more than one role
U1. 101
Deliverables
1. 2. 3. 4. 5. Baselined work product Completed Inspection Summary Report Completed Issue Log Completed Typo Lists Counts of defects found and defects corrected
U1. 102
34
Exit-Criteria
All of the authors inspection objectives are satisfied. Issues raised during the inspection are tracked to closure. All major defects are corrected. Uncorrected defects are logged in the projects defect tracking system. The modified work product is checked into the projects configuration management system. Moderator has collected and recorded the inspection data.
U1. 103
Exit-Criteria
If changes were required in earlier project deliverables, those deliverables have been correctly modified, checked into the projects configuration management system, and any necessary regression tests were passed. Moderator has delivered the completed Inspection Summary Report and defect counts to the peer review coordinator
U1. 104
Measurements
The moderator shall collect the data items from each inspection. These data items are used to calculate the process metrics and to monitor and improve the inspection process. The moderator shall record the data items in the appropriate spaces on the Inspection Summary Report and Issue Log and report them to the organizations peer review coordinator. The peer review coordinator shall maintain these metrics in a repository and produce periodic reports of summary data for practitioners and managers
U1. 105
35
Data to be Collected
Data Item Effort: Planning Definition total labor hours spent by the moderator and author in planning, scheduling meetings, assembling, duplicating, and distributing materials, and any other related tasks total labor hours spent by the participants in an overview meeting, if one was held total labor hours spent by the inspectors and author preparing for the inspection total labor hours the author spent correcting defects in the initial deliverable and making other improvements; include verification time from the follow-up stage duration of the inspection meeting in hours
U1. 106
Effort: Overview Effort: Preparation Effort: Rework
Time: Meeting
Data to be Collected
Data Item Defects Found: Major, Minor Definition total number of major and minor defects found by the inspection team; do not include non-defect issues raised, such as questions, requests for clarification, points of style, or items from the Typo Lists
Defects Corrected: total number of major and minor defects corrected Major, Minor during rework Size: Planned, Actual total physical lines of code (not including comments and blank lines) or number of document pages that were planned for inspection and that were actually inspected number of active participants in the inspection meeting inspection teams decision about disposition of the inspected work product (accepted as is, accepted conditionally, re-inspect following rework)
U1. 107
No. of Inspectors Inspection Appraisal
Process Metrics to be Calculated

Metric Percent.Inspected Percent.Majors Rate.Inspection Rate.Preparation Formula 100 * Size.Actual / Size.Planned 100 * Defects.Found.Major / Defects.Found.Total Size.Actual / Time.Meeting Size.Planned / (Effort.Preparation / Number.of.Inspectors)
Rework.per.Defect
Effort.Rework / Defects.Corrected.Total
U1. 108
36
Process Metrics to be Calculated

Metric Defect Density Defects Found: Total Formula Defects.Found.Total / Size.Actual Defects.Found.Major + Defects.Found.Minor
Defects Corrected: Defects. Corrected.Major + Defects. Total Corrected.Minor Effort: Inspection Effort.Planning + Effort.Overview + Effort.Preparation + Effort.Meeting + Effort.Rework Number.of.Inspectors * Time.Meeting Effort.Inspection / Defects.Found.Total
Effort: Meeting Effort per Defect
Effort per Unit Size Effort.Inspection / Size.Actual

U1. 109
A Sample Form
Inspection Log Example
U1. 111
37
Review Techniques
Walkthrough
U1. 112
Walkthroughs
A walkthrough is an informal way of presenting a technical document in a meeting. The author runs the walkthrough: calling the meeting, inviting the reviewers, soliciting comments and ensuring that everyone present understands the work product. After the meeting, the author should follow up with individual attendees who may have had additional information or insights. The walkthrough is preceded by preparation by designers/developers Lists of items Items not understood Items that appear to be incorrect
U1. 113
Managing walkthroughs
Document driven Person responsible for document walks the reviewers through the document Reviewers interrupt with comments triggered by the presentation Interactive process Not to be used for the evaluation of participants
U1. 114
38
Participants & Entry Criteria

Participants The author selects the participants in a walkthrough. No specific roles are assigned. Entry Criteria The author selects a walkthrough review approach for the product being reviewed. The author has stated his objectives for the review.
U1. 115
Tasks & Responsibilities

Task Select review participants, obtain their agreement to participate, and schedule a walkthrough meeting. Distribute work product to reviewers prior to the meeting. Describe the work product to the reviewers during the meeting in any appropriate way. Lead discussion on the topics of interest or concerns about the work product. Present comments, possible defects, and improvement suggestions to the author. Based on reviewer comments, perform any necessary rework of the work product. Author Responsibl e
Author Author
Reviewers Author
U1. 116
Deliverables, Verification, Exit-Criteria

Deliverables Modified work product Verification No verification of rework is required. Changes are made at the authors discretion. Exit-Criteria The author has made any appropriate changes in the work product.
U1. 117
39
Inspection Vs. Walkthroughs

Walkthrough
Adv. Circulation of Documents Adv. Preparation by reviewers Presentation in the meeting by Rigor Purpose Effectiveness Not Required No Authors Informal to formal Judge quality, find defects, training Low to medium Yes Yes Readers Formal Measure/improve quality of product and process Low to very high, depending on training and commitment
U1. 118
Inspection
Passaround
U1. 119
Participants & Entry-Criteria

Participants The author selects the participants in a passaround. No specific roles are assigned. However, it can be beneficial to ask individual reviewers to examine the work product from specific perspectives like: checking internal cross-references, confirming interface definitions, checking conformance to standards, or identifying grammatical and typographical errors. Entry Criteria The author selected a passaround review approach. The author has stated his objectives for the review. The document has been spell-checked.
40

Task Either distribute a physical or electronic copy of the work product to each reviewer, or create a file to which all reviewers have read/write access. Responsible Author
Notify reviewers that work product is available and indicate Author the date by which review comments are to be provided. Either hand-write comments directly on the work product, or enter comments into the work product file, using a revision marks convention or an annotation feature of the tool being used. If working with an individual copy of the work product, deliver the work product with comments to the author after completing the review. Reviewers
U1. 121

Task Responsibl e After the final date of the review period, remove work Author product from the shared file location and examine added comments. Based on reviewer comments, perform any necessary Author rework of the work product. If a shared file was used to accumulate comments, delete Author each comment from the work product as it is resolved. Leave any comments that require discussion in the work product until agreement is reached on actions to be taken.
U1. 122
Deliverables, Verification, Exit-Criteria

Deliverables Modified work product Verification No verification of rework is required. The author is responsible for making appropriate decisions on issues and for correctly performing any rework. Exit-Criteria The author has addressed all review issues raised..
U1. 123
41
Code Review
U1. 124
Code Review
A code review is a special kind of inspection in which the team examines a sample of code and fixes any defects in it. In a code review, a defect is a block of code which does not properly implement its requirements, or does not function as the programmer intended, or is not incorrect but could be improved For example, it could be made more readable or its performance could be improved
U1. 125
Good candidates for code review

A portion of the software that only one person has the expertise to maintain Code that implements a highly abstract or tricky algorithm An object, library or API that is particularly difficult to work with Code written by someone who is inexperienced or has not written that kind of code before, or written in an unfamiliar language Code which employs a new programming technique An area of the code that will be especially catastrophic if there are defects
42
Checklist for Code Review: Example

Control flow Data declaration
All variables declared? Default values understood? Arrays and strings initialized? Variables with similar names? Correct initialization? Each loop terminates? DO/END statements match?
Input/output
OPEN statements correct? Format specification correct? End-of-file case handled?
U1. 127
Deskchecks
U1. 128
Deskchecks
A deskcheck is a simple review in which the author of a work product distributes it to one or more reviewers. The author sends a copy of the work product to selected project team members. The team members read it, and then write up defects and comments to send back to the author.
U1. 129
43
Deskchecks
Unlike an inspection, a deskcheck does not produce written logs which can be archived with the document for later reference. Deskchecks can be used as predecessors to inspections. In many cases, having an author of a work product pass his work to a peer for an informal review will significantly reduce the amount of effort involved in the inspection.
U1. 130
Pair Programming
U1. 131
Pair Programming
Pair programming is a technique in which two programmers work simultaneously at a single computer and continuously review each others work. Although many programmers were introduced to pair programming as a part of Extreme Programming, it is a practice that can be valuable in any development environment. Pair programming improves the organization by ensuring that at least two programmers are able to maintain any piece of the software.
U1. 132
44
Pair Programming
In pair programming, two programmers sit at one computer to write code. Generally, one programmer will take control and write code, while the other watches and advises. Some teams have found that pair programming works best for them if the pairs are constantly rotated; this helps diffuse the shared knowledge throughout the organization. Others prefer to pair a more junior person with a more senior for knowledge sharing.
U1. 133
Pair Programming
The project manager should not try to force pair programming on the team; it helps to introduce the change slowly, and where it will meet the least resistance. It is difficult to implement pair programming in an organization where the programmers do not share the same nine-to-five (or ten-to-six) work schedule. Some people do not work well in pairs, and some pairs do not work well together.
U1. 134
Conclusion
Inspection Walkthrough Pass-around Desk Checks Pair Programming
U1. 135
45
References
1. Beizer, B., Software Testing Techniques. New York: Van Nostrand, 1983 2. Howden, W. E, A Functional Approach to Program Testing & Analysis, New York: McGraw-Hill, 1987 3. Howden, W. E, Functional Program Testing and Analysis, IEEE Trans. Software Eng SE-12, 10 (Oct. 1986), 997-1005 4. Paul C. Jorgensen, Software Testing, A Craftsmans Approach, CRC Press, 2002 5. William Perry, Effective Methods for Software Testing, John Wiley & Sons, New York, 1995.
U1. 136
Review Questions: Short Type

Define Software Testing Difference between Verification & Validation Difference between Error, Fault & Failure In which phase of SDLC, the fixing of bugs is least expensive an in which phase is it most expensive? What are shortcomings of V-model? 5. What is modified V Model 6. What are the shortcomings of V Model that are addressed in W Model 7. Shortcomings of W Model addressed in 11-Steps by W Perry 8. Discuss importance of testing Software Design 9. Discuss importance of Test Plan 10. Explain importance of Life Cycle testing 11. What is the difference between Walkthrough & Inspection
1. 2. 3. 4.
Review Questions: Long Type

1. How would you decide when to stop testing? 2. Why is testing difficult? What are the challenges facing Software Testing? 3. What are the limitations of testing? 4. Discuss Testing Principles by Myers. 5. Discuss importance of processes in Software Development & Testing. 6. What are the different factors contributing to System Failure? 7. What is the importance of Code Review?
U1. 138
46
This document was created with Win2PDF available at http://www.win2pdf.com. The unregistered version of Win2PDF is for evaluation or non-commercial use only. This page will not be added after purchasing Win2PDF.

Software Testing Unit - I: Learning Objectives

Загружено:

Сведения о документе

Исходное описание:

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Software Testing Unit - I: Learning Objectives

Загружено:

Авторское право:

Доступные форматы

MCA 317, Software Testing