# Reliability growth model:

A reliability growth model is a model of how the system reliability changes over time during the
testing process.As system failures are discovered, the underlying faults causing these failures
are repaired so that the reliability of the system should improve during system testing and
debugging. To predict reliability, the conceptual reliability growth model must then be translated
into a mathematical model.
Reliability growth modeling involves comparing measured reliability at a number of points of time
with known functions that show possible changes in reliability

# What is quality?
It is the standard of something as measured against other things of a similar kind; the degree of
excellence of something.

SOFTWARE QUALITY​ is the degree of conformance to explicit or implicit requirements and

expectations.

● The degree to which a system, component, or process meets specified requirements.

● The degree to which a system, component, or process meets customer or user needs or
expectations
● software quality: The totality of functionality and features of a software product that bear
on its ability to satisfy stated or implied needs.

Importance of Software Quality:

The main reasons for the importance of software quality are as follows:
● Quality software saves time and money.
● When you have fewer defects, it makes testing and maintenance easy.
● Greater reliability in software increases customer satisfaction and lowers the
maintenance cost.
● As the maintenance cost decreases, it helps in reducing the overall cost of the software
project.

Examples:
Let’s look at an example to understand how important software quality is.

● On June 4, 1996, Ariane 5 crashed on its maiden journey just 40 seconds after the take off.
At an altitude of about 3700 m, the launcher veered off and broke and exploded. After
detailed investigation, it was found that the failure in software caused the crash that resulted
in the loss of half a billion dollars.
● Similarly, on September 23​rd,​ in the year 1999, Mars Climate Orbiter disappeared after it
began to orbit Mars. The cost of this mission was about 125 US million dollars. On
investigating, it was found that the error was due to the transfer of information between the
 team in Colorado and the team in California. One team used English units whereas the
other one used metric unit for key space craft operation.

If the quality of the software was taken care of, these incidents could have been avoided easily.
There are 2 aspects of software quality – functional aspect and structural aspect.
The structural aspect includes all the non functional aspects like reliability, consistency,
maintenance and so on.

# Key Aspects of Software Quality

From a customer point of view, the key aspects of quality include the below points.
● Professional looking good design : Look and feel of the software is very important from a
customer point of view.
● Functionality : Software does what is expected of it. It should meet all the requirements
that are mentioned at the beginning of the software development phase. Nobody will use
software that does not function properly.Example – Assume that you are testing the
functionality to withdraw a certain amount from your ATM software. Some of the basic
test cases would be to check whether the system turns on, connects to the bank, reads
your card, accepts your PIN, asks to choose an account, lets you enter the amount,
checks if the cash is available, allows correct transaction for the amount specified if it is
within the limits and so on. If these basic functions do not work properly, then the
software quality is poor.
● Reliability : Reliability indicates the acceptable level of failures or breakdown. This
means the software needs to be reliable if it has to satisfy end user.One example of
testing reliability is to verify that it works on every operating system correctly. It should
also work on the desktop, laptop, iPad and mobile.
● Consistency : To ensure quality, there should be consistency in the behavior of the
software throughout.
● Value for Money : It is very important to deliver cost effective software to the customer.
The final product should meet all the requirements of the customer, should be reliable
and should behave the way it is expected to.
● Excellent Maintenance service : Maintenance is an important aspect of software quality.
Maintenance comes into the picture after the product is shipped to the customer. To
keep the customer happy, it is important to provide good after sales service.For
example, if the customer after using the software for 6 months needs to make a change,
the software team should be ready to make the necessary changes as a part of
maintenance service.

# DEFECT​: defined as a variance between expected and actual.

Defect is an error found AFTER the application goes into production.
It commonly refers to several troubles with the software products, with its external behavior or
with its internal features. It is the deviation of the customer requirement.
ERROR​: An error is a mistake, misconception, or misunderstanding on the part of a software
developer. In the category of developer we include software engineers, programmers, analysts,
and testers. For example, a developer may misunderstand a de-sign notation, or a programmer
might type a variable name incorrectly – leads to an Error. It is the one which is generated
because of wrong login, loop or due to syntax. Error normally arises in software; it leads to
change the functionality of the program.
It is a person act that generates an erroneous result.

FAILURE​: A failure is the inability of a software system or component to perform its required
functions within specified performance requirements. When a defect reaches the end customer
it is called a Failure. During development Failures are usually observed by testers.

FAULT:​ An incorrect step, process or data definition in a computer program which causes the
program to perform in an unintended or unanticipated manner. A fault is introduced into the
software as the result of an error. It is an anomaly in the software that may cause it to behave
incorrectly, and not according to its specification. It is the result of the error.

# Defect prevention​:
Making sure the defects do not happen in the first place
Defect Prevention Methods and Techniques:
Some traditional and common methods that have been in use since a long time for defect
prevention are listed below;
1) ​Review and Inspection​: This method includes the review by an individual team member
(self-checking), peer reviews and inspection of all work products.

Use of formal methods like formal specification and formal verification.Formal specification is
concerned with producing consistent requirements specification, constraints and designs so that
it reduces the chances of accidental fault injections. With formal verifications, correctness of
software system is proved.

2) ​Walkthrough​: This is more or less like a review but it’s mostly related to comparing the
system to the prototype which will give a better idea regarding the correctness and/or the
look-and-feel of the system.

3) ​Defect Logging and Documentation​: This method provides some key information,
arguments/parameters that can be used to support analyzing defects.

4) ​Root Cause Analysis​: Root cause analysis includes two major approaches:
I) Pareto Analysis
2) Fishbone Analysis:

# LOC (Lines of code)​: A line of code is any line of program text that is not a comment or blank
line, regardless of the number of statements or fragments of statements on the line.This
specifically includes all lines containing program headers, declarations, and executable and
non-executable statements. Thus their method is to count physical lines including prologues and
data definitions (declarations) but not comments

# Function point:
A function can be defined as a collection of executable statements that performs a certain task,
together with declarations of the formal parameters and local variables manipulated by those
statements (Conte et al., 1986).
A function point is a unit of measurement to express the amount of business functionality an
information system (as a product) provides to a user.
The functional user requirements of the software are identified and each one is categorized into
one of five types:
inputs, outputs, inquiries, internal and external interfaces

Importance of function point?

1)The outcome of a function point count provides the metric ‘unit of software delivered’ and can
be used to assist in the management and control of software development, customisation or
major enhancements from early project planning phases, through to the ongoing support of the
application.

2)Functionally sizing the requirements for the application quantifies the different types of
functionality delivered by an application. The function point count assigns function points to
each of the function types: External Inputs, Outputs, Enquiries, and Internal and External Files.

3)Knowing the software size facilitates the creation of more accurate estimates of project
resources and delivery dates

4) If the software to be developed is planned to replace existing production applications, it is

useful to assess if the business is going to be delivered more, less or the same functionality

5)Assessing Replacement Cost

6) Once the scope of the project is agreed, the estimates for effort, staff resources, costs and
schedules need to be developed

# Software Quality Metrics

Software metrics can be classified into three categories: product metrics, process metrics, and
project metrics.
Product metrics describe the characteristics of the product such as size, complexity, design
features, performance, and quality level.

Process metrics can be used to improve development and maintenance activities of the
software. Examples include the effectiveness of defect removal during development, the pattern
of testing defect arrival, and the response time of the fix process.

Project metrics describe the project characteristics and execution. Examples include the number
of software developers, the staffing pattern over the life cycle of the software, cost, schedule,
and productivity.

Some metrics belong to multiple categories. For example, the in- process quality metrics of a
project are both process metrics and project metrics.

Software quality metrics are a subset of software metrics that focus on the quality aspects of the
product, process, and project. In general, software quality metrics are more closely associated
with process and product metrics than with project metrics.

Software quality metrics can be further divided into three categories −

- Product quality metrics

- In-process quality metrics
- Maintenance quality metrics

PRODUCT QUALITY METRIC:

This metrics include the following −
● Mean Time to Failure
● Defect Density
● Customer Problems
● Customer Satisfaction

Mean Time to Failure

It is the time between failures. This metric is mostly used with safety critical systems such as the
airline traffic control systems, avionics, and weapons.
Mean time to failure (MTTF) is the length of time a device or other product is expected to last in
operation.
MTTF is used for non-repairable products. When MTTF is used as a measure, repair is not an
option.

Defect Density
It measures the defects relative to the software size expressed as lines of code or function
point, etc. i.e., it measures code quality per unit. This metric is used in many commercial
software systems.
number of defects over the opportunities for error (OFE) during a specific time frame.

Defect Density is the number of defects confirmed in software/module during a specific period of
operation or development divided by the size of the software/module.

It enables one to decide if a piece of software is ready to be released.

Defect density is counted per thousand lines of code also known as KLOC.
Formula to measure Defect Density:
● Defect Density =​ Defect ​count/ size of the release
Size of release can be measured in terms of line of code (LoC).

Customer Problems
It measures the problems that customers encounter when using the product. It contains the
customer’s perspective towards the problem space of the software, which includes the
non-defect oriented problems together with the defect problems.
rom the customers' standpoint, all problems they encounter while using the software product,
not just the valid defects, are problems with the software. Problems that are not valid defects
may be usability problems, unclear documentation or information, duplicates of valid defects
(defects that were reported by other customers and fixes were available but the current
customers did not know of them), or even user errors. These so-called non-defect-oriented
problems, together with the defect problems, constitute the total problem space of the software
from the customers' perspective.

Customer Satisfaction

Customer satisfaction is often measured by customer survey data through the five-point scale −
 ● Very satisfied
● Satisfied
● Neutral
● Dissatisfied
● Very dissatisfied
Satisfaction with the overall quality of the product and its specific dimensions is usually obtained
through various methods of customer surveys. Based on the five-point-scale data, several
metrics with slight variations can be constructed and used, depending on the purpose of
analysis. For example −
● Percent of completely satisfied customers
● Percent of satisfied customers
● Percent of dis-satisfied customers
● Percent of non-satisfied customers
Usually, this percent satisfaction is used.

Similarities between MTTF and Defect density:

- Both are product quality metrics
- Both measure Intrinsic product quality

Differences​ ​between MTTF and Defect density:

- The MTTF metric is most often used with safety critical systems such as the airline traffic
control systems, avionics, and weapons. The defect density metric, in contrast, is used
in many commercial software systems.

- one measures the time between failures, the other measures the defects relative to the
software size (lines of code, function points, etc.).

- although it is difficult to separate defects and failures in actual measurements and data
tracking, failures and defects (or faults) have different meanings.

- For special-purpose software systems such as the air traffic control systems or the
space shuttle control systems, the operations profile and scenarios are better defined
and, therefore, the time to failure metric is appropriate. For general-purpose computer
systems or commercial-use software, for which there is no typical user profile of the
software, the MTTF metric is more difficult to implement and may not be representative
of all customers.

- gathering data about time between failures is very expensive. It requires recording the
occurrence time of each software failure. It is sometimes quite difficult to record the time
for all the failures observed during testing or operation. To be useful, time between
failures data also requires a high degree of accuracy. This is perhaps the reason the
MTTF metric is not widely used by commercial developers.
 - Finally, the defect rate metric (or the volume of defects) has another appeal to
commercial software development organizations. The defect rate of a productor the
expected number of defects over a certain time period is important for cost and resource
estimates of the maintenance phase of the software life cycle.

IN PROCESS QUALITY METRIC:

In-process quality metrics deals with the tracking of defect arrival during formal machine testing
for some organizations. This metric includes −
● Defect density during machine testing
● Defect arrival pattern during machine testing
● Phase-based defect removal pattern
● Defect removal effectiveness

Defect density during machine testing

Defect rate during formal machine testing (testing after code is integrated into the system
library) is correlated with the defect rate in the field. Higher defect rates found during testing is
an indicator that the software has experienced higher error injection during its development
process, unless the higher testing defect rate is due to an extraordinary testing effort.
This simple metric of defects per KLOC or function point is a good indicator of quality, while the
software is still being tested. It is especially useful to monitor subsequent releases of a product
in the same development organization.

Defect arrival pattern during machine testing

The overall defect density during testing will provide only the summary of the defects. The
pattern of defect arrivals gives more information about different quality levels in the field. It
includes the following −
● The defect arrivals or defects reported during the testing phase by time interval (e.g.,
week). Here all of which will not be valid defects.
● The pattern of valid defect arrivals when problem determination is done on the reported
problems. This is the true defect pattern.
● The pattern of defect backlog overtime. This metric is needed because development
organizations cannot investigate and fix all the reported problems immediately. This is a
workload statement as well as a quality statement. If the defect backlog is large at the
end of the development cycle and a lot of fixes have yet to be integrated into the system,
the stability of the system (hence its quality) will be affected. Retesting (regression test)
is needed to ensure that targeted product quality levels are reached.
The objective is to look for defect arrivals that stabilize at a very low level or time btw failures
that are far apart b4 ending the testing effort and releasing the s/w
Phase-based defect removal pattern
This is an extension of the defect density metric during testing. In addition to testing, it tracks the
defects at all phases of the development cycle, including the design reviews, code inspections,
and formal verifications before testing.
Because a large percentage of programming defects is related to design problems, conducting
formal reviews, or functional verifications to enhance the defect removal capability of the
process at the front-end reduces error in the software. The pattern of phase-based defect
removal reflects the overall defect removal ability of the development process.
With regard to the metrics for the design and coding phases, in addition to defect rates, many
development organizations use metrics such as inspection coverage and inspection effort for
in-process quality management.

Defect removal effectiveness

It can be defined as follows −
DRE=(Defect removed during a development phase /
Defects latent in the product)×100%
Because the total number of latent defects in the product at any given phase is not known, the
denominator of the metric can only be approximated. It is usually estimated by:
Defects removed during the phase + defects found later
This metric can be calculated for the entire development process, for the front-end (before code
integration) and for each phase. It is called early defect removal when used for the front-end
and phase effectiveness for specific phases. The higher the value of the metric, the more
effective the development process and the fewer the defects passed to the next phase or to the
field. This metric is a key concept of the defect removal model for software development.

MAINTENANCE QUALITY METRICS:

Although much cannot be done to alter the quality of the product during this phase, following are
the fixes that can be carried out to eliminate the defects as soon as possible with excellent fix
quality.
● Fix backlog and backlog management index
● Fix response time and fix responsiveness
● Percent delinquent fixes
● Fix quality

Fix backlog and backlog management index

Fix backlog is a workload statement for software maintenance Fix backlog is related to the rate
of defect arrivals and the rate at which fixes for reported problems become available. It is a
simple count of reported problems that remain at the end of each month or each week. Using it
in the format of a trend chart, this metric can provide meaningful information for managing the
maintenance process.
Backlog Management Index (BMI) is used to manage the backlog of open and unresolved
problems.
BMI=Number of problems closed during the month/ Number of problems arrived during the
month×100%
If BMI is larger than 100, it means the backlog is reduced. If BMI is less than 100, then the
backlog increased.
With enough data points, the techniques of control charting can be used to calculate the backlog
management capability of the maintenance process. More investigation and analysis should be
triggered when the value of BMI exceeds the control limits. Of course, the goal is always to
strive for a BMI larger than 100. A BMI trend chart or control chart should be examined together
with trend charts of defect arrivals, defects fixed (closed), and the number of problems in the
backlog.

Fix response time and fix responsiveness

For many software development organizations, guidelines are established on the time limit
within which the fixes should be available for the reported defects. Usually the criteria are set in
accordance with the severity of the problems. For the critical situations in which the customers'
businesses are at risk due to defects in the software product, software developers or the
software change teams work around the clock to fix the problems. For less severe defects for
which circumventions are available, the required fix response time is more relaxed.

The fix response time metric is usually calculated as the mean time of all problems from open to
close. Short fix response time leads to customer satisfaction.
The important elements of fix responsiveness are customer expectations, the agreed-to fix time,
and the ability to meet one's commitment to the customer.
For example, John takes his car to the dealer for servicing in the early morning and needs it
back by noon. If the dealer promises noon but does not get the car ready until 2 o'clock, John
will not be a satisfied customer. On the other hand, Julia does not need her mini van back until
she gets off from work, around 6 P.M. As long as the dealer finishes servicing her van by then,
Julia is a satisfied customer. If the dealer leaves a timely phone message on her answering
machine at work saying that her van is ready to pick up, Julia will be even more satisfied. This
type of fix responsiveness process is indeed being practiced by automobile dealers who focus
on customer satisfaction.
Percent delinquent fixes

For each fix, if the turnaround time greatly exceeds the required response time, then it is
classified as delinquent:

Fix Quality
Fix quality or the number of defective fixes is another important quality metric for the
maintenance phase. A fix is defective if it did not fix the reported problem, or if it fixed the
original problem but injected a new defect. For mission-critical software, defective fixes are
detrimental to customer satisfaction. The metric of percent defective fixes is the percentage of
all fixes in a time interval that is defective.
A defective fix can be recorded in two ways: Record it in the month it was discovered or record it
in the month the fix was delivered. The first is a customer measure; the second is a process
measure. The difference between the two dates is the latent period of the defective fix.
Usually the longer the latency, the more will be the customers that get affected. If the number of
defects is large, then the small value of the percentage metric will show an optimistic picture.
The quality goal for the maintenance process, of course, is zero defective fixes without
delinquency.

# SQA:
Software Quality Assurance (SQA) consists of a means of monitoring the software engineering
processes and methods used to ensure quality. It does this by means of audits of the quality
management system under which the software system is created.It is distinct from software
quality control which includes reviewing requirements documents, and software testing. SQA
encompasses the entire software development process, which includes processes such as
software design, coding, source code control, code reviews, change management, configuration
management, and release management. Whereas software quality control is a control of
products, software quality assurance is a control of processes.
SQA is used to test the software, rather than checking the quality after completion. SQA
processes, test for quality in each phase of development until the software is complete. With the
SQA the software development moves to the next phase if the current phase of software is
compiled with required quality standards.
SQA encompasses:
1. analysis, design, coding and testing methods and tools
2. formal technical reviews that are applied during each software engineering step 3. a multi
tiered testing strategy
4. control of software documentation and the changes made to it
5. a procedure to assure compliance with software development standards
6. measurement and reporting mechanism
Below are the Quality assurance criteria against which the software would be evaluated against:
● correctness
● efficiency
● flexibility
● integrity
● interoperability
● maintainability
● portability
● reliability
● reusability
● testability
● usability
Different activities/role of SQA are as follows
● Maintaining the quality of the system as per the specification and business requirements.
● Defect prevention and formal methods for other defect prevention technique.
● Defect Reduction
● Direct fault detection and removal without executing the project scenario.
● Testing the project for Failure observation and bug removal.
● Risk identification
● Defect tracking techniques and methods. Software fault tolerance. Concluding remarks
and maintaining reports.
SQA ACTIVITIES
http://www.academia.edu/9760547/LECTURE_NOTES_2_Software_Quality_Assurance

Prepares an SQA plan for a project. The plan is developed during project planning and is
reviewed by all interested parties. Quality assurance activities performed by the software
engineering team and the SQA group are governed by the plan.
The plan identifies • evaluations to be performed • audits and reviews to be performed •
standards that are applicable to the project • procedures for error reporting and tracking •
documents to be produced by the SQA group • amount of feedback provided to the software
project team

Participates in the development of the project’s software process description. The software team
selects a process for the work to be performed. The SQA group reviews the process description
for compliance with organizational policy, internal software standards, externally imposed
standards (e.g., ISO-9001), and other parts of the software project plan.

Reviews software engineering activities to verify compliance with the defined software process.
The SQA group identifies, documents, and tracks deviations from the process and verifies that
corrections have been made.
Audits designated software work products to verify compliance with those defined as part of the
software process. The SQA group reviews selected work products; identifies, documents, and
tracks deviations; verifies that corrections have been made; and periodically reports the results
of its work to the project manager.

Ensures that deviations in software work and work products are documented and handled
according to a documented procedure. Deviations may be encountered in the project plan,
process description, applicable standards, or technical work products.

Records any noncompliance and reports to senior management. Noncompliance items are
tracked until they are resolved

# Statistical Quality Assurance (SQA)​. is used to identify the potential variations in the
manufacturing process and predict potential defects on a parts-per-million (PPM) basis. It
provides a statistical description of the final product and addresses quality and safety issues
that arise during manufacturing.

Statistical Quality Assurance

Statistical quality assurance reflects a growing trend throughout industry to become more
quantitative about quality. For software, statistical quality assurance implies the following steps:
1. Information about software defects is collected and categorized
2. Each defect is traced back to its cause
3. Using the Pareto principle (80% of the defects can be traced to 20% of the causes) isolate
the "vital few" defect causes
4. Move to correct the problems that caused the defects in the “vital few”

This relatively simple concept represents an important step towards the creation of an adaptive
software engineering process in which changes are made to improve those elements of the
process that introduce error. To illustrate this, assume that a software engineering organization
collects information on defects for a period of one year. Some of the defects are uncovered as
software is being developed. Others are encountered after the software has been released to its
end-users.
Once the vital few causes are determined, the software engineering organization can begin
corrective action. For example, to correct MCC, the software developer might implement
facilitated application specification techniques (Chapter 11) to improve the quality of customer
communication and specifications. To improve EDR, the developer might acquire CASE tools
for data modeling and perform more stringent data design reviews. It is important to note that
corrective action focuses primarily on the vital few. As the vital few causes are corrected, new
candidates pop to the top of the stack.

# standard of quality ​which is a criterion or statement that describes the acceptable level of
something
Quality assurance is the process of defining how software quality can be achieved and how the
development organization knows that the software has the required level of quality. The main
activity of the quality assurance process is the selection and definition of standards that are
applied to the software development process or software product.

There are two main types of standards.

1) The product standards are applied to the software product, i.e. output of the software
process. It documents the minimum requirements that a product must meet to be fit for
sale. If the outcome misses some of these specifications, the company must discard or
correct the product before selling it, states the International Organization for
Standardization, a global player in defining industrial quality management standards. To
ensure adequate throughput, a business owner may choose to set the requirements
slightly higher than the clients’ expectations, so that the product always meets or
exceeds the customers’ needs.

2) The process standards define the processes that should be followed during software
development.Process quality standards protect the business owner from unnecessary
costs in product repair or manufacturing rejects. These standards ensure that employees
building products or providing services follow a specific procedure so that the results
always meet the design quality standards. These process guidelines come in the form of
instructions that describe each step and explain how to differentiate a well-performed
action from an error.

3) The software standards are based on best practices and they provide a framework for
implementing the quality assurance process.

ISO 9000 is an international set of standards that can be used in the development of a quality
management system in all industries. ISO 9000 standards can be applied to a range of
organizations from manufacturing to service industries. ISO 9001 is the most general of these
standards. It can be applied to organizations that design, develop and maintain products and
develop their own quality processes.
. The ISO 9001 standard describes various aspects of the quality process and defines the
organizational standards and procedures that a company should define and follow during
product development. These standards and procedures are documented in an organizational
quality manual.

Documentation standards in a software project are important because documents can represent
the software and the software process. Standardized documents have a consistent appearance,
structure and quality, and should therefore be easier to read and understand. There are three
types of documentation standards:
1. Documentation process standards. These standards define the process that should be
followed for document production.
 2. Document standards. These standards describe the structure and presentation of
documents.
3. Documents interchange standards. These standards ensure that all electronic copies of
documents are compatible.

#​http://blog.pilgrimquality.com/iso90012015-quality-mgmt-principles/
The International Standard for Quality management (ISO 9001:2015) adopts a number of
management principles, that can be used by top management to guide their organizations
towards improved performance.

1. Customer focus[​edit​]
The primary focus of quality management is to meet ​customer requirements​ and to strive to
exceed customer expectations.
Rationale
Sustained success is achieved when an organization attracts and retains the confidence of
customers and other interested parties on whom it depends. Every aspect of customer
interaction provides an opportunity to create more value for the customer. Understanding
current and future needs of customers and other interested parties contributes to sustained
success of an organization [5]
​

2. Leadership[​edit​]
Leaders at all levels establish unity of purpose and direction and create conditions in which
people are engaged in achieving the organization’s quality objectives.
Rationale
Creation of unity of purpose and direction and engagement of people enable an organization to
align its strategies, policies, processes and resources to achieve its objectives [6]
​
3. Engagement of people[​edit​]
Competent, empowered and engaged people at all levels throughout the organization are
essential to enhance its capability to create and deliver value.
Rationale
To manage an organization effectively and efficiently, it is important to involve all people at all
levels and to respect them as individuals. Recognition, empowerment and enhancement of
competence facilitate the engagement of people in achieving the organization’s quality
objectives.​[7]

4. Process approach[​edit​]
Consistent and predictable results are achieved more effectively and efficiently when activities
are understood and managed as interrelated processes that function as a coherent system.
Rationale
The quality management system consists of interrelated processes. Understanding how results
are produced by this system enables an organization to optimize the system and its
performance.​[8]

5. Improvement[​edit​]
Successful organizations have an ongoing focus on improvement.
Rationale
Improvement is essential for an organization to maintain current levels of performance, to react
to changes in its internal and external conditions and to create new opportunities.​[9]

6. Evidence based decision making[​edit​]

Further information: ​decision making
Decisions based on the analysis and evaluation of data and information are more likely to
produce desired results.
Rationale
Decision making can be a complex process, and it always involves some ​uncertainty​. It often
involves multiple types and sources of inputs, as well as their interpretation, which can be
subjective. It is important to understand ​cause-and-effect​ relationships and potential ​unintended
consequences​. ​Facts​, ​evidence​ and ​data analysis​ lead to greater ​objectivity​ and confidence in
decision making.​[10]

7. Relationship management[​edit​]
Further information: ​Relationship management
For sustained success, an organization manages its relationships with interested parties, such
as ​suppliers​, retailers.
Rationale
Interested parties influence the performance of an organization. Sustained success is more
likely to be achieved when the organization manages relationships with all of its interested
parties to optimize their impact on its performance. Relationship management with its supplier
and partner networks is of particular importance
Custi focus, ppl engagement, leadership, improvement, evidence based decision making,
relationship mgmt
Celier

Quality costs are the costs associated with preventing, finding, and correcting defective work.
These costs are huge, running at 20% - 40% of sales. Many of these costs can be significantly
reduced or completely avoided. One of the key functions of a Quality Engineer is the reduction
of the total cost of quality associated with a product.

Cost of Quality (COQ) is a measure that quantifies the cost of control( cost of conformance) and
the cost of failure of control (Cost of Non-Conformance).
In other words, it sums up the costs related to prevention and detection of ​defects​ and the costs
due to occurrences of defects.
 ● Definition by ISTQB: cost of quality: The total costs incurred on quality activities and
issues and often split into prevention costs, appraisal costs, internal failure costs and
external failure costs.

● Cost of Control (Also known as Cost of Conformance)

○ Prevention Cost
■ The cost arises from efforts to prevent defects.
■ They are planned and incurred before actual operation,
■ Example: ​Quality Assurance​ costs, qlty planning, training
○ Appraisal Cost
■ The cost arises from efforts to detect defects.
■ Example: ​Quality Control​ costs, verification , audits
● Cost of Failure of Control (Also known as Cost of Non-Conformance)
○ Internal Failure Cost
■ The cost arises from defects identified internally and efforts to correct
them. (defects discovered before the product or service is delivered to
the customer)
■ Example: Cost of Rework (Fixing of internal defects and re-testing)
○ External Failure Cost
■ The cost arises from defects identified by the client or end-users and
efforts to correct them.
■ Example: Cost of Rework (Fixing of external defects and re-testing) and
any other costs due to external defects (Product service/liability/recall,
etc)
■ Repair, complaints, servicing, warranty
Cost of Quality (COQ) = Cost of Control + Cost of Failure of Control
where
Cost of Control = Prevention Cost + Appraisal Cost
and
Cost of Failure of Control = Internal Failure Cost + External Failure Cost

# Benchmarking

Benchmarking is a technique in which a company measures its performance against that of best
in class companies, determines how those companies achieved their performance levels and
uses the information to improve its own performance. Subjects that can be benchmarked
include strategies, operations and processes.

Benchmarking is the process of measuring products, services, and processes against those of
organizations known to be leaders in one or more aspects of their operations. Benchmarking
provides necessary insights to help you understand how your organization compares with
similar organizations, even if they are in a different business or have a different group of
customers.

Additionally, benchmarking can help you identify areas, systems, or processes for
improvements—either incremental (continuous) improvements or dramatic (business process
reengineering) improvements.

Benchmarking has been classified into two distinct categories:

Technical benchmarking — Performed by design staff to ascertain the capabilities of products or

services, especially in comparison to the products or services of leading competitors. For
example, on a scale of one to four, four being best, how do designers rank the properties of
your organization’s products or services? If you cannot obtain hard data, the design efforts may
be insufficient, and products or services may be inadequate to be competitive.

Competitive benchmarking — Compares how well (or poorly) an organization is doing with
respect to the leading competition, especially with respect to critically important attributes,
functions, or values associated with the organization’s products or services. For example, on a
scale of one to four, four being best, how do customers rank your organization’s products or
services compared to those of the leading competition? If you cannot obtain hard data,
marketing efforts may be misdirected and design efforts misguided.

http://www.qasigma.com/2008/11/benchmarking.html
PERFORMANCE.
Key Points:
● Benchmarking is an increasingly popular improvement tool.
● Benchmarking concerns processes and practices.
● Benchmarking is a respected means of identifying processes that require major change.
● Benchmarking is done between consenting companies that may or may not be
competitors.
● Benchmarking compares your process or practice with the target company’s
best-in-class process or practice.
● The goal of benchmarking is to find “secrets of success” and then adapt and improve for
your own application.

Benchmarking Approach and Process:

● Obtain management commitment.
● Baseline your own processes.
● Identify your strong and weak process and document them.
● Select processes to be benchmarked.
● Form benchmarking teams.
● Research best-in-class.
● Select candidate best-in-class benchmarking partners.
● Form agreements with benchmarking partners.
 ● Collect data.
● Analyze data and establish the gap
● Plan action to close the gap/surpass.
● Implement change.
● Monitor.
● Update benchmarks; continue the cycle

McCall’s quality model:

https://www.researchgate.net/figure/McCall-Software-Quality-Model_fig2_260391390

Boehm’s quality model:

https://www.researchgate.net/figure/Boehms-quality-model_fig2_228947488

Software quality metrics:

https://www.tutorialspoint.com/software_quality_management/software_quality_management_m
etrics.htm