SW Quality Standards

Learning Objectives
By the end of the lecture you should be able to:
Describe Quality standards mandated by
ISO 9001: 1994
SEI CMM
Quality Models
•ISO 9001 : 1994 standard
•SEI CMM

The quality of a software system is governed by the quality of the process used to develop
and evolve it - Watts S. Humphrey

ISO 9001 : 1994 Standard - 20 Clauses

ISO Clause 4.1 - Management Responsibility
•The Management defines and documents its policy for quality and commitment to
quality.
•The Management Representative (MR) ensures that the Quality Management
System (QMS) is established, implemented and maintained in accordance with ISO.
• Management Review meetings are held at periodic intervals to discuss the
achievements and critical management issues (including project and Support Group
issues) with the resolution or corrective actions

ISO Clause 4.2 - Quality System

•This requires the organization to establish, document and maintain a QMS as
per ISO requirements and the organization's quality policy

ISO Clause 4.3 - Contract Review

This requires the organization to establish and maintain documented procedures for
contract and proposal review and for the coordination of these activities.

•All the contracts pertaining to software development are reviewed by Quality and
Risk Management (QRM) Group member/Directors /Support Group Managers as
appropriate, using a checklist.

•All the proposals are reviewed by QRM Group member Directors/Support Group
Managers as appropriate, before submitting them to the client. The review records for
proposal and contract are maintained.

2
 ISO Clause 4.4 - Design Control
This requires the organization to prepare and implement plans for each design and
development activity. Also necessary reviews need to be carried out at appropriate
stages.
Correspondingly:
• All projects carry out different activities in a project according to the Project’s Software
Project Plan.
• All software projects would identify and follow a definitive lifecycle (SDLC)
• Reviews (Peer and formal reviews, Checkpoint reviews)of the software items, project
documents are carried out as per SQA Plan
• Also reviews are carried out before introducing any changes in the existing system
ISO Clause 4.5 - Document & Data Control
The Document Control System ensures that
•Pertinent documents are available, as and when required, to persons
authorized to use them
•The documents are reviewed, approved and authorized prior to use
•A master index of documents is maintained
•The QMS is maintained online

3
 ISO Clause 4.6 - Purchasing
• Purchasing procedures are applicable to procurement of any software (such as tools),
hardware, consumables and sub-contractor services.
• Purchasing activities are carried out as per the procedures defined in Purchase
Procedures Manual.

ISO Clause 4.7 - Control of customer supplied product

This requires the organization to establish and maintain documented procedures

for the control of verification, storage and maintenance of customer supplied
product.
Correspondingly:
•Any product (especially a software product) supplied by client for inclusion in
the deliverable, is properly reviewed on receipt and maintained.
•Any nonconformity or defect detected is promptly recorded, brought to the
notice of the client and issues arising thereof are resolved.
•Included product acquired from a third party is treated in a similar manner as
client supplied product.

4
 ISO Clause 4.8 - Product identification & traceability
This requires the organization to establish and maintain documented procedures for the
identification of the product during its entire lifecycle.
Correspondingly:
• Configuration control for the projects is planned, managed and controlled as per the
Software Configuration Management Plan
• Changes to the baselined software items are carried out as per the approved change
control strategy.
• Projects also maintain a traceability matrix which traces the analyzed requirements
components to Design components to construction program units to test cases and
conditions. This is used for validation of the product prior to delivery to client.
• Traceability of deliverables is further ensured through document and data control
ISO Clause 4.9 - Process Control
This requires the organization to use appropriate standards, guidelines and process
control procedures during the SDLC

Correspondingly:
•Every project uses appropriate standards and guidelines defined in the QMS
•If the client provides specific standards and guidelines, then these are reviewed and
approved by a reviewer designated by the SEPG/SQAG.

5
 ISO Clause 4.10 - Inspection and Testing
This requires the organization to establish and maintain documented procedures for
inspection and testing activities. Correspondingly:
• Receiving Inspection and Testing
– Incoming products/ tools / specifications etc. (client supplied or purchased) are
used in projects after appropriate verification activities are carried out.
• Test Planning and Execution
– The project customizes inspection and testing methods and documents the same
in the Software Project Plan and Test Plans for a project.
• Final Inspection (Formal Review)
– Final Inspection / Formal Review of every deliverable is done by Project Quality
Advisor and identified SME / PM before it is delivered to the client.
• Unit Testing, System Testing, Acceptance testing are carried out

ISO Clause 4.11- Control of Inspection,measuring and test equipment

This requires the organisation to establish and maintain documented

procedures to control, calibrate and maintain inspection, measuring and test
equipment

6
 ISO Clause 4.12- Inspection & test status
The inspection and test status of the product shall be identified by suitable means ,
which indicate the conformance or nonconformance of the product with respect to the
tests performed. The test records shall be maintained.
Correspondingly:
• The External Quality Advisor during an audit checks that appropriate test records are
maintained
• Test Problem Reports (TPRs) have been raised during unit / system / acceptance
testing.
• All the TPRs are tracked to closure

ISO Clause 4.13- Control of non-conforming product

This requires that the products that do not conform to specified requirements are
prevented from inadvertent use or installation.

Correspondingly:
•Standard code used from a previous project or code supplied by the client are
treated as non-conforming until tested or peer reviewed.
•The non-conforming product is handled by the Project Leader / PM and verified
and validated by PQA / PM prior to delivery to client or its usage internally.

7
 ISO Clause 4.14 - Corrective & Preventive Action
This requires the organization to establish and maintain documented procedures for implementing corrective
and preventive action.
Correspondingly,
– The PL/PM initiates preventive action at checkpoints or as appropriate during entire SDLC
– Applies controls to ensure that corrective actions are taken and these are effective
– The effectiveness of the preventive actions taken by projects is evaluated in the Management Review
Meeting and checkpoint review meetings/team discussion forums etc. and continuous improvement is
ensured.
– SEPG works according to a plan ensuring continuous process improvements
ISO Clause 4.15- Handling, storage, packaging, preservation & delivery
This requires the organization to establish and maintain documented procedures for handling,
storage, packaging, preservation and delivery of product
Correspondingly, Packaging and Delivery
•All the deliverables are verified by the project team to ensure that the correct number of copies
and the correct version of software items are delivered.
Installation
•The project team assists the client in installing the software as stated in the contract/client
requirement specifications.
Handling, Storage and Preservation
•The PM identifies the person responsible for backup and security of computer medium of the
project.
•Regular backups of the system software and project libraries are kept with the Operations Group.

8
 ISO Clause 4.16 - Control of Quality Records
This requires the organization to establish and maintain documented
procedures for identification, collection, indexing, access, filing, storage,
maintenance and disposition of quality records.
Correspondingly,
– The Project Leader / PM identifies the project quality records in the SPP for the project
– The retention period is one year after formal project closure unless specified otherwise in
the contract / client requirement specifications.
– A filing scheme for maintenance of project quality records is mentioned in the SPP

ISO Clause 4.17- Internal Quality Audits

This requires the organization to establish and maintain documented
procedures for planning and carrying out audits
Correspondingly,
•Internal quality audits are conducted periodically across the projects and
support groups in a planned manner to evaluate the compliance
•The Lead Auditor / Quality Manager/SQAG Lead are responsible for
scheduling and managing audit activities.
•Non-conformances encountered during audits are reported to persons
responsible and non-conformances / deviations are tracked to closure.

9
 ISO Clause 4.18- Training
This requires the organization to establish and maintain documented procedures for
identifying training needs and provide training for all people in the organization
Correspondingly,
– Training plans are based on inputs from the projects, support groups and
individuals
– Training needs of the staff members are identified
– Comprehensive and structured training is provided to the staff members
– Effectiveness and need fulfillment of all training programs are evaluated and if
required, retraining is organized
– Records of all training programs, waivers etc are maintained

ISO Clause 4.19- Servicing

This requires the organization to establish and maintain documented
procedures for performing servicing e.g. warranty and post implementation
support after installation of software

10
 ISO Clause 4.20 - Statistical Techniques
This requires the organization to establish and maintain statistical techniques for verifying
process capability.
Correspondingly,
– Projects collect product and process metrics periodically during the entire SDLC at
defined intervals as per SPP. This helps the Project Manager to formulate plans, take
appropriate corrective and preventive measures, enhance product and process
capabilities, and implement a continuous product/process improvement program

CMM Framework
Capability Maturity Model
• A framework that describes the key elements of an effective software process
• Evolutionary path from adhoc to mature, disciplined process
• Key Practices which when followed improve the ability of organizations to meet goals
for cost, schedule, functionality and quality
• A yardstick against which it is possible to judge a software process and compare it with
industry

11
 Components of CMM
• Maturity levels
• Process capability
• Key Process Areas (KPAs)
– Common features Carrying out the Key Practices results in meeting
– Key Practices Goals of the KPA

Components of CMM

12
 Process Capability
• Describes the range of expected results that can be achieved by following a software process
• Is one of the means by which we can predict the most likely outcome to be expected from
the next software project the organization undertakes
Key process areas (KPAs)
•Each maturity level is composed of KPAs
•Each KPA identifies
A cluster of related activities that
When performed collectively, achieve a set of goals for establishing process
capability at the maturity level
•KPAs have been defined to reside at a single maturity level
Level 1 - Initial
• Environment not stable for developing and maintaining software
• Inadequate management and software engineering practices
• Ineffective planning
• Reaction-driven commitment systems
• Emphasis on coding and testing during crisis
• Success depends on having exceptional people
• Unpredictable software process capability
• Unpredictable schedules, budgets, functionality, and quality
• Few stable software processes 13
 The KPAs by Maturity Levels
Continuously Optimizing
improving (5)
process Optimizing (5)
Process change management
Technology change management
Defect prevention
Predictable Managed
process (4)
Managed (4)
Software
Quality management
quality management
Process measurement
Quantitative and analysis
process management
Standard, Defined
consistent (3)
process
Defined (3)
Peer reviews
Intergroup coordination

Disciplined Repeatable Software product engineering

Integrated software management
process (2) Training program
Organization process definition
Organization process focus

Initial Repeatable (2)

(1) Software configuration management
Software quality assurance
Software subcontract management
Software project tracking and oversight
Software project planning
Requirements management

Initial (1)

14
 Level 2 - Repeatable
• Policies for managing software projects
• Planning and managing based on experience
• Allows repeatability of successful practices
• Specific processes implemented by the projects may differ
• Realistic project commitments
• Costs, schedules and functionality tracked
• Software requirements and work products are baselined
• Standards defined and conformed to
• Strong customer-supplier relationship with subcontractors
Level 3 - Defined
• Organization-wide standard software processes
• Effective software engineering practices
• Integration of software engineering and management processes
• Software managers and technical staff perform more effectively
• Software Engineering Process Group (SEPG)
• Organization-wide training program
• Project’s “Defined Software Process”
• Good management insight into the technical progress on all projects

15
 Level 4 - Managed
• Quantitative goals for software products and processes
• Organizational measurement program
• Organization-wide software process database
• Well-defined and consistent measurements
• Variation in process performance narrowed
• Meaningful variations can be distinguished from random variation
• Risks known and carefully managed
• Software products are of high quality

Level 5 - Optimising
• Organization focused on process improvement
• incremental advances in existing processes
• innovations using new technologies and methods
• Proactive identification of weaknesses to strengthen processes
• Goal of preventing occurrence of defects
• Cost-Benefit analyses of introducing new technologies and proposed process changes
• Reuse of organizational learning
• Error-Cause removal

16
 Goals
• Summarizes
– The key practices of a KPA
– Used to determine whether an organization / project has effectively implemented the
KPA
• Signifies
– The scope, boundaries and intent of each KPA

Common features
• Consists of the following five features
– Commitment to Perform
– Ability to Perform
– Activities Performed
– Measurement and Analysis
– Verifying Implementation
• Indicates whether the implementation and institutionalization of KPA is
effective, repeatable and lasting

17