4-4 Milestone Three: Repeatability and Reproducibility

Repeatability and Reproducibility

Repeatability is the variation observed when you have the same Quality Control
(QC) Inspector examine the same machined part repeatedly with the same device. While
Reproducibility is the variation observed when different Inspectors examine the same parts
using the same device (Pyzdek & Keller, 2014). The data will be from Ultrasonic Flaw
Detection in the course of the casting machining process, a variety of internal
discontinuities can occur in the metal. These include voids, porosity, inclusions, and
cracks. All of these conditions produce ultrasonic indications that can be identified by a
trained Inspector using an ultrasonic flaw detector with appropriate transducers.
In order to collect the data, have an expert Ultrasonic Inspector examines five machined
castings from the process that include both good and bad samples using flaw detection
device to establish the master parameters. Then have two UT Inspectors, not aware of the
expert inspectors judgments, examine the same parts twice (in random order), using their
Ultrasonic Inspection procedure and then compare against the mater as shown below
Results

Sample
1
2
3
4
5

Judgement
by Expert
Inspector
Defective
Good
Defective
Defective
Good

Inspector A
Inspector
A_1
Defective
Good
Good
Defective
Good

Inspector
A_2
Defective
Good
Defective
Defective
Good

Inspector B
Inspector
B_1

Inspector
B_2

Defective
Good
Defective
Defective
Good

Defective
Good
Defective
Defective
Good

4-4 Milestone Three: Repeatability and Reproducibility

In order to trust the data is accurate, find the individual effectiveness of each
Inspector as shown below

Inspector A
Inspector A_1
1
2
3
4
5

Defective
Good
Good
Defective
Good

Inspector A_2

Within Inspector A

Defective
Good
Defective
Defective
Good

1
1
0
1
1

Inspector A with Expert

Inspector
1
1
0
1
1

# Match
# Inspected
% Agree

4
5
4/5 x 100 = 80%

4
5
4/5 x 100 = 80%

An Inspector would be effective with respect to the sample test if the judgements given
by the same Inspector at different times matches and if the judgements match the one given
by the expert Inspector. From the analysis Inspector A is in agreement with himself and the
Expert Inspector 80% of the time. Ideally the percentage should be 100 percent, but
generally, the rule of thumb is 90 percent and above (Chien, n.d.). The data from Inspector A
is not accurate and not reliable.

While for Inspector B, the individual effectiveness is 100%, which means he is in

agreement with himself and the Expert Inspector 100% of the time. The data from Inspector
B is accurate and reliable as shown from the analysis shown in the table below

4-4 Milestone Three: Repeatability and Reproducibility

For Inspector B
Inspector B_1
1
2
3
4
5

Defective
Good
Defective
Defective
Good

Inspector B_2

Within Inspector B

Defective
Good
Defective
Defective
Good

1
1
1
1
1

# Match
# Inspected
% Agree

5
5
5/5 x 100 = 100%

Inspector B
with Expert Inspector
1
1
1
1
1
5
5
5/5 x 100 = 100%

In measurement system analysis, repeatability is the variation in values obtained with one
measuring instrument when used several times by an operator while measuring the identical
feature on the same part. Reproducibility is the variation in the average of values made by
different operators using the same measuring instrument when measuring the identical feature on
the same parts (Milivojevich, 2015). Measurement plays a significant role in six sigma
program. Usually, the gage repeatability and reproducibility (GR&R) study needs to be
conducted prior to the process capability analysis for verifying the accuracy of
measuring equipments.
The gage study consists of several parts that are repeatedly measured by multiple appraisers.
While the numbers can vary, most studies use 10 parts and three appraisers who measure the
parts at least three times each.

The gage study consist of seven parts selected at random from a process, the
parts represent the typical variation exhibited by the process. In the study the
parts were presented to two inspectors, each measure the impellers tip
diameter. Once the inspectors complete their measurements they repeat the
process again.
The data set used, for both inspectors, appears in the table below, the table
shows three values for the same part feature. To compute the average range we
must first calculate the range for each part by Inspector. Next, is to compute the
average range.

4-4 Milestone Three: Repeatability and Reproducibility

A gage R&R study will tell operators if the measurement system is acceptable for its
intended use. The gage study also shows which part of the measurement system is
contributing the most to the variation of the measurements and helps operators plan
improvements to the system.

Measurement systems contain variation from three main sources: the products themselves,
the appraiser taking the measurements and the equipment used to perform the measurement.
The gage study shows the contribution of each of these areas
The repeated measurements are called trials. Calculations are then made to determine the level of
variation between the appraisers, parts and across the trials

Operator A
Part
Trial 1
Trial 2
Trial 3
Average
Ranges

1
2.83
2.80
2.95
2.86
0.15

2
2.80
2.95
2.92
2.89
0.15

3
2.95
2.92
2.93
2.93
0.03

4
2.92
2.93
2.97
2.94
0.05

5
2.93
2.97
2.95
2.95
0.04

6
2.97
2.95
2.86
2.93
0.09

7
2.95
2.86
2.89
2.90
0.09

Operator B
Part
Trial 1
Trial 2
Trial 3
Average
Ranges

1
2.86
2.89
2.78
2.84
0.11

2
2.89
2.78
2.96
2.88
0.18

3
2.78
2.96
3.02
2.92
0.09

4
3.02
2.93
2.88
2.94
0.14

5
2.96
3.02
2.93
2.97
0.09

6
2.93
2.88
2.97
2.93
0.09

7
2.88
2.97
3.04
2.96
0.16

Part Average

2.85

2.89

2.93

2.94

2.96

2.93

2.93

Repeatability is the variation observed when you have the same Quality Control (QC) Inspector
examine the same machined part repeatedly with the same device. While Reproducibility is the
variation observed when different Inspectors examine the same parts using the same device

4-4 Milestone Three: Repeatability and Reproducibility

In Measurement Systems Analysis, repeatability is the variation between measurements that

occurs when one person measures the same item several times, using the same measuring
equipment.
(Pyzdek & Keller, 2014).
Reproducibility is the variation in the average measurements of different appraisers who measure
the same items using the same measuring equipment. It may also be used to compare different
measuring devices at different locations (Pyzdek & Keller, 2014).

Conducting the Study

The gage study consists of several parts that are repeatedly measured by multiple appraisers.
While the numbers can vary, most studies use 10 parts and three appraisers who measure the
parts at least three times each.

Using the same gauge, three operators will measure the same dimension on 10 different part
samples, three times each

The repeated measurements are called trials. Calculations are then made to determine the level of
variation between the appraisers, parts and across the trials

Measurement plays a significant role in Six sigma program. Usually, the gauge
repeatability and reproducibility (GR&R) study needs to be conducted prior to the
process capability analysis for verifying the accuracy of measuring equipments and
helping organizations improve their product and service quality. Therefore, how to
ensure the quality of measurement becomes an important task for quality practitioners

4-4 Milestone Three: Repeatability and Reproducibility

The Average and Range method requires multiple parts, appraisers and trials to quantify the
repeatability and reproducibility. The following is a typical Data sheet used in industries
There are three basic and widely used methods for determining the Gage R&R. They are:
Range method Average and Range method Analysis of Variance method (ANOVA) Let us see
the Average and Range method in detail. The Average and Range method is a statistical method
that provides an estimate of the following components. Part Variation Repeatability
Reproducibility R&R Total Variation This method computes the total measurement system
variability, which can be separated into components like repeatability, reproducibility and part
variation.

Average and Range Method(Example) Consider the following example: (Taken from
Measurement System Analysis Reference Manual)
No. of Appraisers = 2 No. of Trials = 3 No. of parts

=5

Reference
Chien, C., J. (n.d.) Making Sense of Attribute Gage R&R Calculations. Retrieved from

4-4 Milestone Three: Repeatability and Reproducibility

https://www.isixsigma.com/tools-templates/measurement-systems-analysis-msa-gagerr/making-sense-attribute-gage-rr-calculations/
Pyzdek, T., & Keller, P. (2014). Six Sigma Handbook, (4th ed.). New York, NY: McGraw-Hill.
QSO620 Module 4 Lecture Note (2016). Measurement System Analysis (PowerPoint). Retrieved
from https://bb.snhu.edu/bbcswebdav/pid-11631608-dt-content-ridMASTER_ImportedContent_20130514033243/Learning%20Modules/Module%20Four
%20Module%20Lecture/QSO620_Module4_Lecture.pdf