Вы находитесь на странице: 1из 52

Software Quality & Testing

IT SCENARIO

US Govt. Study on Software Development Projects reveals 60 % of Projects are behind Schedule

50%

were over cost

45 %

of delivered projects were unusable

Success/Failure Rate of Projects


PROJECT SUCCESS HISTORY
60 50 40

53 46 40 31 16

30 20 10 0 1994

33 28 27 26

Success Failed Challenge

1996

1998

SDLC
Requirement Analysis

Architectural DESIGN (HLD) Detailed Design (LLD)

Maintenance

Implementation

Coding and Testing

Distribution of Bugs

Others Coding 10% 7%

Design 27%

Requirement 56%

Source: Writing Testable Requirements by Dick Bender

DISTRIBUTION OF EFFORTS
Coding 1% Design 13%

Others 4%

Requirement 82%
S ou rc e: W r itin g T es ta b le R eq u ir e m en ts b y D ic k B en d e r

Quality ?

Problem with Quality?

The problem with quality is not what people dont know about it?

The problem is what they think they do know.

Quality has much in common with Sex Bill Crossby

Everybody is for it. (Under certain conditions, of course) Everyone feels they understand it (Even though they would not want to explain it)

Everyone thinks execution is only a matter of following natural inclinations. (After all we do get along somehow)
Most people feel that problems in these areas are caused by others. (if only they would take the time to do things right)

What Is Quality?

As defined by a producer of a product


Meeting specified requirements of an acceptable

PRODUCT
As defined by a customer

FITNESS FOR USE


Should be available at the RIGHT

TIME, PLACE, FORM, PRICE

What Is Quality?

As defined by ISO 9000 :


The totality of characteristics of an entity (product or service) that bear on its ability to satisfy

STATED and/or IMPLIED needs.

Software Quality :McCalls Quality Factors

Factors that can be directly measured. Factors that can be measured only indirectly. Compare the software to some datum and arrive at an indication of quality.

Software Quality :McCalls Quality Factors

PRODUCT REVISION

PRODUCT TRANSITION

PRODUCT OPERATION

Software Quality : Product Operation

Correctness The degree to which the software satisfies stated needs as indicated by:

Suitability Accuracy Interoperability Compliance Security

Software Quality : Product Operation

Reliability: The amount of time that the software is available for use as indicated by:

Maturity

Fault Tolerance
Recoverability

Software Quality : Product Operation

Usability: The degree to which the software is easy to use as indicated by:

Understandability

Learnability
Operability

Software Quality : Product Operation

Integrity:

The extend to which access to Software or Data by unauthorized persons can be controlled

Software Quality : Product Operation

Efficiency: The degree to which the software makes optimal use of system resources as indicated by:

Time Behavior Resource Behavior

Software Quality : Product Revision

Maintainability: The ease with which repair may be made to the software is easy to use as indicated by:

Analyzability Changeability Stability Testability

Software Quality : Product Revision

Flexibility

The effort required to modify the programs

Software Quality : Product Revision

Testability

The effort required to test the programs to ensure that it performs its intended functions

Software Quality : Product Transition

Portability: The ease with which software can be transposed from one environment to another as indicated by:

Adaptability Installability Conformance Replaceability

Software Quality : Product Transition

Reusability The extend to which a program [or parts of programs] can be reused in other applications - related to the packaging and scope of the functions that the program perform.

Software Quality : Product Transition

Interoperability

The effort required to couple the system to another

Software Cost of Errors at Each Stage of the SDLC

Software Economics

Relative Cost of Software Defects


140 120 100 80 60 40 20 0 Req Design Code Test Production Line 1

Testing Economics
Stage Err. Int. Traditional Continuous & Cost Left Cost Left Cost -----------------------------------------------------------------Req. - 1 20 20 000 10 010 Des. - 3 20 40 000 15 055 Code - 5 20 60 000 18 140 Test - 10 12 480 04 280 Prdn -100 00 1680 00 680

Definitions

Errors Error is an oversight or a wrong decision Defects Non fulfillment of an intended requirement or reasonable expectation.

usage,

A deviation from specification or standard Anything that causes customer dissatisfaction on conformance to standards. Anything less/extra requirement from specification

Definitions
Software Verification & Validation Software V & V processes determine whether development products of a given activity conform to the requirements of that activity & whether software satisfies its intended use and user needs

Definitions
Verification IEEE defines Verification as the process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase e.g. Do we have clear and complete requirements Is this a good design

Definitions
Validation IEEE defines Validation as the process of evaluating a system or component during or at the end of the development whether it satisfies specified requirements.

e.g. It involves executing the actual software or a simulated mock type.

V & V Objectives

Facilitate early detection and correction of errors


Enhance management insight into process and product risk Support software life cycle processes to ensure compliance with program performance, schedule and budget requirements.

What is Quality Assurance (QA)?

All those planned and systematic activities implemented within the quality system and demonstrated as needed to provide adequate confidence that an entity will fulfil requirements for quality

What is Quality Control (QC)?

The operational techniques and activities that are used to fulfil requirements for quality

Quality Assurance Vs Quality control

Process Based
Proactive

Product based Reactive

Has to be designed, evolved as a culture Requires Quality Specialists Selection of QA tools.


Processes

S/W Test cases


Require testers

QC can be automated.
Inspection/Audit

Quality Assurance Vs Quality control (contd..)

Satisfies the Customer

Satisfies the Process

Continuous Improvement

Time/Project Bound

Training on quality (Awareness, Processes, Costing).

Dynamic Software Testing, Test Reviews etc.

Quality Models

Kaizen
Total Quality Management (TQM) Tick-IT

Spice
SEI CMM

SEI PCMM

ISO9000

SEI CMMI

CMM - Capability Maturity Model

5 levels of software process maturity


Continuously improving process

|Optimizing (5) Managed (4)

Improvement institutionalized

Predictable process Standard consistent process


Disciplined

Quantitative control

Defined (3)

Technical practices are integrated with management practices and institutionalized

process

Repeatable (2)

Project Management practices are institutionalized

Initial (1)

Process is informal & adhoc

Optimizing (5)
Process change management Technology change mgmt. Defect prevention

Managed (4)
Quality management Quantitative Process management

Defined (3)
Peer reviews Inter group coordination Software product engineering Integrated software management Training program Organization process definition Organization process focus

Repeatable (2)
Software configuration management Software quality assurance Software subcontract management Software project tracking and oversight Software project planning Requirements management

Intial(1)

The Maturity Levels


5 Optimizing
Focus on continuous process improvement

Process measured and controlled

Quantitatively Managed

Defined
Process characterized for the organization and is proactive

Managed
Process characterized for projects and is often reactive

Initial 1
Process unpredictable, poorly controlled, and reactive

CMMI-SE/SW (Staged Representation)


Maturity Level 5 Optimizing Focus Continuous Process Improvement Quantitative Management Process Areas Organizational Innovation and Deployment Causal Analysis and Resolution Organizational Process Performance Quantitative Project Management Requirements Development Technical Solution Product Integration Verification Validation Organizational Process Focus Organizational Process Definition Organizational Training Integrated Project Management Risk Management Decision Analysis and Resolution Requirements Management Project Planning Project Monitoring and Control Supplier Agreement Management Measurement and Analysis Process and Product Quality Assurance Configuration Management
Risk Rework Quality Productivity

4 Quantitatively Managed

3 Defined

Process Standardization

2 Managed

Basic Project Management

1 Initial

Software Quality Assurance

Software Quality Assurance includes:

Scope Management Time Management

Cost Management
Quality Management Change Management

Software Quality Assurance includes:

Human resources Management Communication Management

Risk Management
Procurement Management Integration Management

Software Engineering Solutions

Joint Application Development (JAD) Structured Methodology Visual Diagramming/Prototyping Model Iterative Model Spiral Model

Software Engineering Solutions

Joint Application Development (JAD)


Implemented by IBM User and Systems Team Work together thru the entire Cycle Creates an Ownership Reduces Communication Gap Detection of Errors on Time User gets trained during the development

Software Engineering Solutions

Structured Methodology

Based on 4 Principles
1. 2. 3.

4.

Abstraction Formality Divide and Rule Hierarchical Ordering

Software Engineering Solutions

Prototyping/Visual Modeling
1.

2.
3.

Make Prototype for User Communication Use Diagrammatic Techniques Visual Modeling helps better understanding

Software Engineering Solutions

Spiral Model

Consists of a Cyclic approach


1. 2. 3.

4.

Planning Risk Assessment Engineering Review/Evaluation

Project Planning Process Design and Construction Process Change Control Process

Software Requirements

User Documentation Process

System Testing Process

Relationship of Requirements to Project Development Processes

Requirements Engineering
Requirements Definition

Requirements Elicitation Requirements Analysis Requirements Documentation Requirements Review

Software Requirements Specifications

Requirements Management
Requirements Change Management Requirements Traceability