Attribute Gage R&R Studies - Part 2 - BPI Consulting

8/28/2018 Attribute Gage R&R Studies - Part 2 | BPI Consulting
Support (/support) Contact Us (/contact-us)

Revision History (/revision-history-for-spc-for-excel-version-5) View Cart (/cart)

My Account (/user)
(/)
Attribute Gage R&R Studies - Part 2
June 2010
This is the second newsletter on attribute Gage R&R studies. As stated last month, sometimes a
measurement system has a measurement value that comes from a nite number of categories. The easiest
one of these is a go/no go gage. This gage simply tells you if the part passes or it fails. There are only two
possible outcomes. In the last newsletter, we use a simple go/no go gage to demonstrate how to compare
the appraisers. We used cross tabulation tables and calculated kappa values to compare the appraisers.
In this newsletter, we will
Develop con dence intervals for % agreement for each appraiser

Calculate the system e ectiveness
Compare each appraiser's results to the reference value
Draw conclusions based on the appraiser's e ectiveness, miss rate and false alarm rate
In this issue:
Example
Con dence Interval for Appraisers
System % E ectiveness Score
The Reference Values
The Reference Values and Cross-Tabulation Tables
The Reference Values and Kappa Values
The Reference Values and Con dence Intervals
The Reference Values and the System E ectiveness Score
Conclusions
Quick Links
https://www.spcforexcel.com/knowledge/measurement-systems-analysis/attribute-gage-rr-studies-part-2 1/13
Some of us use the manual Measurement Systems Analysis to help us understand our Gage R&R studies.
Information on this manual can be found at this website: www.aiag.org
(http://www.aiag.org/scriptcontent/index.cfm). This newsletter is a summary of their information
contained in the third edition of the manual but provides more details of the calculations contained in the
manual. Of course, our SPC for Excel software performs all the calculations automatically for you. Note that
the fourth edition of Measurement Systems Analysis is now available, but the author has not yet reviewed it.
EXAMPLE
The example was introduced in last month's newsletter. Suppose you are in charge of a production process
that makes widgets. The process is not capable of meeting speci cations. You produce widgets that are out
of speci cation. The process is in control and, as of yet, your Black Belt group has not gured out how to
make it capable of meeting speci cations. Your only alternative, at this time, is to perform 100% inspection
of the parts and separate the parts that are within speci cations from those that are out of speci cations.
You have selected an attribute go/no go gage to use. This gage will simply tell if the part is within
speci cations. It does not tell you how "close" the result is to the nominal; only that it is within speci cations.
To determine the e ectiveness of the go/no gage, you decide to conduct an attribute gage R&R study. You
select three appraisers (Bob, Tom and Sally). You nd 30 parts to use in the trial. Each of these parts was
measured using a variable gage and rated as passing (within speci cations or conforming) or failing (out of
speci cation or nonconforming). There are 21 conforming parts and 9 nonconforming parts in the study.
Each appraiser measures each part three times using the go/no go gage and the results are recorded. The
parts must be run in random order without the appraiser knowing which parts he/she is measuring. In other
words, randomize the 30 parts and have an appraiser measure each part. Then randomize the order again
and repeat the measurement.
The results from the study are shown below. P indicates the part passed (conforming), while F indicates that
the part failed (nonconforming). The rst column is the reference value for the part. It represents the "true"
value of the part based on the variable gage measurements.
Table 1: Attribute Gage R&R Study Results
Appraiser BobBobBobTomTomTomSallySallySally
Reference Part/Trial 1 2 3 1 2 3 1 2 3
P 1 P P P P P P P P P
F 3 F F F F F F F F F
P 6 P P F P P F P F F
P 7 P P P P P P P F P
F 12 F F F F F F F P F
P 14 P P F P P P P F F
P 21 P P F P F P F P F
F 22 F F P F P F P P F
F 26 F P F F F F F F P
F 30 F F F F F P F F F
CONFIDENCE INTERVAL FOR APPRAISERS
Most software packages will calculate a con dence interval for each appraiser to show how well the
appraiser agrees with himself/herself on all trials. We will demonstrate the calculations using Bob's data. The
rst step is to determine how frequently Bob agrees with himself - that is, how often did Bob rate each part
the same for each of the trials?
Looking through Table 1 for Bob's data, you can see that there were 25 times Bob rated the part the same
each time. There were ve times when Bob did not rate the part the same each time (for parts 6, 14, 21, 22,
and 26). This represents a percent agreement of 25/30 = 83.3%. So, Bob rated each part the same each time
83.3%. If we repeated this study, would Bob have this same percentage agreement? We don't know but
probably not because there are common causes of variation always present. We can develop a con dence
interval around this average to give us an idea of the amount of possible variation in Bob's results.
First, let's de ne the following:
m = number of times the each part was rated the same

N = total number of parts
The calculations for the 95% con dence interval for the % agreement are given below. The F distribution is
used to determine the con dence interval.
The lower con dence interval is given by the equation:
where:
v1 = 2m
v2 = 2(N-m+1)
F = 2.5 percentile of the F distribution corresponding to v1 and v2 degrees of freedom.
From Bob's data:
v1 = 2m = 2(25) = 50
v2 = 2(N-m+1)= 2(30-25+1) = 12
F = 0.4512 (for alpha = 0.05 with 50 and 12 degrees of freedom)
You can nd the value for F in statistical tables. You can also use the FINV function in Excel. But be aware that
this function returns the inverse of the right-tailed F probability distribution. You want the left-tail so you
would need to enter FINV(0.975, 50, 12) to get the correct tail portion. Excel 2010 has a new function F.INV
which returns the inverse of the left-tailed F probability distribution. If you have Excel 2010, you can use
F.INV(0.025, 50, 12) to get the F value.
The lower con dence limit is then:
Lower con dence = 50(0.4512)/(12+50(0.4512)) = 0.653 = 65.3%
The upper con dence limit is given by the following equation:
where:
v1 = 2(m+1)
v2= 2(N-m)
F = 97.5 percentile of the F distribution corresponding to v1 and v2 degrees of freedom
From Bob's data:
v1 = 2(m+1) = 2(26) =52

v2 = 2(N-m) = 2(5) = 10
F = 3.216 (for alpha = 0.05 with 52 and 10 degrees of freedom)
The upper con dence is then:
Upper con dence = 52(3.216)/(10+52(3.216)) = 0.944 = 94.4%
Thus, the 95% con dence interval for Bob's percent agreement is 65.3% to 94.4%. The con dence intervals
for all three appraisers are given in the table below.
Table 2: Appraiser Con dence Intervals

Bob Tom Sally
Total Inspected (N) 30 30 30
Number Matched (m) 25 26 23
95% Upper 94.4% 96.2% 90.1%
% Agreement 83.3% 86.7% 76.&%
95% Lower 65.3% 69.3% 57.7%

Since the % agreement for each appraiser lies in the con dence interval for each other appraiser, we must
conclude that there is no statistically signi cant di erence between the three appraisers.
SYSTEM % EFFECTIVENESS SCORE
You can also determine an overall e ective score for all three appraisers. The calculations are the same as
for the con dence interval. The di erence is that you rst start by determining the number of parts where all
three operators agreed on all trials, i.e., all operators rated the part the same for each trial - either a pass or
fail. As shown in the data in rst table, this occurred for 22 parts. There were 8 parts where the appraisers
disagreed for some trial or trials. Thus, the overall % agreement is given by:
Overall % agreement = 22/30 = 0.733 = 73.3%
The 95% con dence interval for the overall agreement is calculated as shown above. The 95% con dence
interval for the overall agreement is 54.1% to 87.7%.
THE REFERENCE VALUES
The rst column in Table 1 contains the "Reference" value. This is the "true" value of the part as determined
by a variable measurement system. The reference value can be used in the calculations in this newsletter
(and the last one) by considering those results to be like another appraiser.
THE REFERENCE VALUES AND CROSS-TABULATION TABLES
Thus, we can construct cross-tabulation tables (as shown in the last newsletter) that compare the results for
each individual appraiser to the reference. The cross-tabulation table for Bob is shown below.
Table 3: Cross-Tabulation Table for Bob and the Reference
Reference
Fail
Pass Total

Count 25 3 28
Fail
Expected 8.4 19.6 28
Bob
Count 2 60 62
Pass
Expected 18.643.4 62
Count 27 63 90
Total
Expected 27 63 90

The cross-tabulation table simply organizes the results to make it easier to see if there are signi cant
di erences. Consider the rows labeled "Count." The following is true from the table:
Bob failed the part 25 times when the reference value was "Fail"
Bob failed the part 3 times when the reference value was "Pass" (this is known as false alarm)
Bob passed the part 2 times when the reference value was "Fail" (this is known as a miss)
Bob passed the part 60 times when the reference value was "Pass"
Thus, Bob made the correct decision 85 times out of 90. The row labeled "Expected" are the expected
accounts. This is the count you would expect if there was no di erence between the appraiser and the
reference.
The cross-tabulation tables for Tom and Sally are shown below. Tom made the correct decision 86 out of 90
times. Sally made the correct decision 79 out of 90 times.
Table 4: Cross-Tabulation Table for Tom and the Reference
Reference
Fail
Pass Total

Count 25 2 27
Fail
Tom
Count 2 61 63
Pass
Expected 18.944.1 63
Count 27 63 90
Total
Expected 27 63 90

Table 5: Cross-Tabulation Table for Sally and the Reference
Reference
Fail
Pass Total

Count 23 7 30
Fail
Sally
Count 4 56 60
Pass
Expected18.0 42.0 60
Count 27 63 90
Total
Expected 27 63 90
THE REFERENCE VALUES AND KAPPA VALUES
We introduced the kappa value in the last newsletter. It can be used to measure the agreement of the
appraiser with the reference value. Kappa can range from 1 to -1. A kappa value of 1 represents perfect
agreement between the appraiser and reference value. A kappa value of -1 represents perfect disagreement
between the appraiser and the reference value. A kappa value of 0 says that agreement represents that
expected by chance alone. So, kappa values close to 1 are desired.
Kappa is calculated using the following equation:
kappa = (po -pe)/(1 - pe)
where:
po = the sum of the actual counts in the diagonal cells/overall total

pe = the sum of the expected counts in the diagonal cells/over total
The sum of counts in the diagonal cells is the sum of the counts where the appraisers agreed (both either
passed or failed a part). The sum of expected counts is the same thing but you use the expected counts
instead of the counts.
Using the data for Bob and the reference, the value of kappa is calculated as shown below.
po = (25 +60)/90 =0 .944
pe = (8.4 + 43.9)/90 = 0.576
kappa = (po - pe)/(1 - pe) = (0.944 - 0.576)/(1 - 0.576) = 0.87
The kappa values against the reference for all three appraisers are shown below:
Bob: 0.87
Tom 0.89
Sally 0.72
The MSA manual reference above says:
"A general rule of thumb is that values of kappa greater than 0.75 indicate good to excellent
agreement (with a maximum kappa = 1); values les than 0.40 indicate poor agreement."
Based on this information, Bob and Tom have good agreement with the reference value. Sally appears to
have a somewhat poorer agreement with the standard but is close to the 0.75.
THE REFERENCE VALUES AND CONFIDENCE INTERVALS
We can construct con dence intervals around how often each appraiser agreed with the reference value
using the same procedure as above. For example, Bob's three trials agreed with the reference value 25
times. For ve parts, Bob had a di erent result on at least one trial for the part. These numbers are identical
to Bob's "within" agreement. In other words, there were no parts where Bob did not agree with the reference
value at least once. This will often be true in these types of studies. The con dence intervals for the
agreement for each appraiser versus the reference values are given below.
Table 6: Appraiser Con dence Intervals versus the Reference

Bob Tom Sally
Total
Inspected 30 30 30
(N)
Number
Matched 25 26 23
(m)
95%
94.4% 96.2% 90.1%
Upper
%
83.3% 86.7% 76.&%
Agreement
95%
65.3% 69.3% 57.7%
Lower
THE REFERENCE VALUES AND THE SYSTEM EFFECTIVENESS

SCORE
You can also calculate the system e ectiveness versus the reference values. Again, the numbers are identical
to the system e ective score given above.
CONCLUSIONS
So, after all these calculations, what have we learned? In their manual Measurement Systems Analysis (3rd
Edition), AIAG provides these guidelines.
Table 7: Acceptance Guidelines
False
Miss
Decision E ectiveness Alarm
Rate
Rate
Acceptable
for the > 90% < 2% < 5%
appraiser
Marginally
2%
acceptable 5%to
80% to 90% to
for the 10%
5%
appraiser
Unacceptable
for the < 80% > 5% > 10%
appraiser

If the result is marginally acceptable, the measurement system may need improvement. It depends on the
situation. If the result is unacceptable, then improvement is needed.
The e ectiveness number in the above table is simply the % agreement for each appraiser. There are two
new terms in the table: miss rate and false alarm rate. The manual does a poor job of de ning miss rate and
false alarm rate in the manual. The miss rate is applied to the nonconforming parts in the study. The miss
rate is the percentage of time that an appraiser did not reject a failing part. This is determined by the
following equation:
Miss Rate = 100( Number of misses/Number of opportunities for a miss)
There were 9 nonconforming parts in the study. With three trials for each part, each appraiser had 3*9 = 27
opportunities for a miss. Bob had two misses (with parts 22 and 26). So Bob's miss rate is 100(2/27) = 7.4%.
Tom also had two missed (with parts 22 and 30) so his miss rate is also 100(2/27) = 7.4%. Sally had 4 misses
(with parts 12, 22 (twice), and 26). Her miss rate is 100(4/27) = 14.8%. Note that the number of misses is
contained in the cross-tabulation tables above under the Fail column for the Reference and the Pass row for
the appraiser.
The false alarm rate is applied to the conforming parts in the study. The false alarm rate is the percentage of
time that an appraiser rejects a conforming part. It is de ned as:
False Alarm Rate = 100(Number of false alarms/number of opportunities for a false alarm)
There were 21 conforming parts in the study. With three trials each, each appraiser had 3*21 = 63
opportunities for a false alarm. Bob had three false alarms (with parts 6, 14 and 21) so his false alarm rate is
100(3/63) = 4.8%. Tom has two false alarms (with parts 6 and 21) so his false alarm rate is 100(2/63) = 3.2%.
Sally had 7 false alarms (with parts 6 (twice), 7, 14 (twice), and 21 (twice)) for a false alarm rate of 100(7/63) =
11.1%. Note that the number of false alarms is given in the cross-tabulation table under the Pass column for
the Reference and the Fail row for the appraiser.
The table below shows the ratings for each appraiser.
Table 8: Acceptance Results
Appraiser E ectiveness Miss Rate False Alarm Rate

Bob 83.3% 7.4% 4.8%
Tom 86.7% 7.4% 3.2%
Sally 76.7% 14.8% 11.1%

Based on the e ectiveness measure, Bob and Tom are marginal and Sally needs improvement. The same is
true for the miss rate. The false alarm rate is acceptable for Bob and Tom, but Sally needs improvement.
Based on this analysis, the overall measurement system needs to be improved.
QUICK LINKS
SPC for Excel Software (/spc-for-excel-software)
Visit our home page (/home)
SPC Training (/seminars-on-site-training)
SPC Consulting (/implementation)
Ordering Information (/ordering-information)
Thanks so much for reading our publication. We hope you nd it informative and useful. Happy charting and
may the data always support your position.
Sincerely,
Dr. Bill McNeese

BPI Consulting, LLC
(http://www.linkedin.com/in/billmcneese)
Connect with Us
 (https://www.facebook.com/spcforexcel) 
(https://plus.google.com/u/0/b/112490535975760615454/+Spcforexcel/posts) 
(https://www.linkedin.com/company/3569889) 
(https://www.youtube.com/channel/UCnogTilST4u1z_TQFops_LQ)
Measurement Systems Analysis
<< Return to Categories (/spc-for-excel-publications-category)

ANOVA Gage R&R - Part 1 (/knowledge/measurement-systems-analysis/anova-gage-rr-part-1)
Acceptance Criteria for Measurement Systems Analysis (MSA) (/knowledge/measurement-
systems-analysis/acceptance-criteria-for-MSA)
Attribute Gage R&R Studies - Part 2 (/knowledge/measurement-systems-analysis/attribute-
gage-rr-studies-part-2)
Attribute Gage R&R Studies: Comparing Appraisers (/knowledge/measurement-systems-
analysis/attribute-gage-rr-comparing-appraisers)
Destructive Gage R&R Analysis (/knowledge/measurement-systems-analysis/destructive-
gage-rr-analysis)
Evaluating the Measurement Process - Part 1 (/knowledge/measurement-systems-
analysis/newsletter/evaluating-measurement-process-part-1)
Evaluating the Measurement Process - Part 2 (/knowledge/measurement-systems-

analysis/evaluating-measurement-process-part-2)
Gage R&R for Non-Destructive and Destructive Test Methods (/knowledge/measurement-
systems-analysis/gage-rr-for-non-destructive-and-destructive-test-methods)
Levey-Jennings Charts (/knowledge/measurement-systems-analysis/levey-jennings-charts)
Measurement Systems - Is Yours Any Good? (/knowledge/measurement-systems-
analysis/measurement-systems-yours-any-good)
Monitoring Destructive Test Methods (/knowledge/measurement-systems-
analysis/monitoring-destructive-test-methods)
Monitoring Test Methods Using SPC (/knowledge/measurement-systems-analysis/monitoring-
test-methods-using-spc)
Operational De nition of a Consistent Measurement System (/knowledge/measurement-
systems-analysis/operational-de nition-consistent-measurement-system)
Operational De nitions/Measurement System Analysis (/knowledge/measurement-systems-
analysis/operational-de nitions-measurement-system-analysis)
Probable Error and Your Measurement System (/knowledge/measurement-systems-
analysis/probable-error-and-your-measurement-system)
SPC and the Lab (/knowledge/measurement-systems-analysis/spc-and-the-lab)
Speci cations and Measurement Error (/knowledge/measurement-systems-
analysis/speci cations-and-measurement-error)
The Impact of Measurement Error on Control Charts (/knowledge/measurement-systems-
analysis/measurement-error-impact-on-control-charts)
Three Methods to Analyze Gage R&R Studies (/knowledge/measurement-systems-
analysis/three-methods-analyze-gage-rr-studies)
Variable Measurement Systems - Part 1: Stability (/knowledge/measurement-systems-
analysis/variable-measurement-systems-part-1-stability)
Variable Measurement Systems - Part 2: Bias (/knowledge/measurement-systems-
analysis/variable-measurement-systems-part-2-bias)
Variable Measurement Systems - Part 3: Linearity (/knowledge/measurement-systems-
analysis/variable-measurement-systems-part-3-linearity)
Variable Measurement Systems - Part 4: Gage R&R (/knowledge/measurement-systems-
analysis/variable-measurement-systems-part-4-gage-rr)
Catalog (/catalog)
Software (4) (/catalog/software)
Complete Teaching Guides to SPC (11) (/catalog/complete-teaching-guides-spc)
Connect with Us
 (https://www.facebook.com/spcforexcel) 
(https://plus.google.com/u/0/b/112490535975760615454/+Spcforexcel/posts) 
(https://www.linkedin.com/company/3569889) 
(https://www.youtube.com/channel/UCnogTilST4u1z_TQFops_LQ)
SPC Knowledge Base Sign-up

Click here (/newsletter-signup) to sign up for our FREE monthly publication, featuring SPC and other
statistical topics, case studies and more!
SPC Around the World

SPC for Excel is used in over 60 countries internationally. Click here for a list of those countries.
(http://www.spcforexcel.com/spc-for-excel-used-internationally)
Copyright © 2018 BPI Consulting, LLC. All Rights Reserved.

Site developed and hosted by ELF Computer Consultants (http://www.elfcc.com).

Attribute Gage R&R Studies - Part 2 - BPI Consulting

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Attribute Gage R&R Studies - Part 2 - BPI Consulting

Загружено:

Авторское право:

Доступные форматы

8/28/2018 Attribute Gage R&R Studies - Part 2 | BPI Consulting

Support (/support) Contact Us (/contact-us)

In this newsletter, we will

Develop con dence intervals for % agreement for each appraiser

Table 1: Attribute Gage R&R Study Results

CONFIDENCE INTERVAL FOR APPRAISERS

First, let's de ne the following:

m = number of times the each part was rated the same

N = total number of parts

The lower con dence interval is given by the equation:

From Bob's data:

The lower con dence limit is then:

Lower con dence = 50(0.4512)/(12+50(0.4512)) = 0.653 = 65.3%

The upper con dence limit is given by the following equation:

From Bob's data:

v1 = 2(m+1) = 2(26) =52

The upper con dence is then:

Upper con dence = 52(3.216)/(10+52(3.216)) = 0.944 = 94.4%

Table 2: Appraiser Con dence Intervals

SYSTEM % EFFECTIVENESS SCORE

Overall % agreement = 22/30 = 0.733 = 73.3%

THE REFERENCE VALUES

THE REFERENCE VALUES AND CROSS-TABULATION TABLES

Table 3: Cross-Tabulation Table for Bob and the Reference

Table 4: Cross-Tabulation Table for Tom and the Reference

Table 5: Cross-Tabulation Table for Sally and the Reference

THE REFERENCE VALUES AND KAPPA VALUES

Kappa is calculated using the following equation:

kappa = (po -pe)/(1 - pe)

po = the sum of the actual counts in the diagonal cells/overall total

po = (25 +60)/90 =0 .944

pe = (8.4 + 43.9)/90 = 0.576

kappa = (po - pe)/(1 - pe) = (0.944 - 0.576)/(1 - 0.576) = 0.87

The MSA manual reference above says:

THE REFERENCE VALUES AND CONFIDENCE INTERVALS

Table 6: Appraiser Con dence Intervals versus the Reference

THE REFERENCE VALUES AND THE SYSTEM EFFECTIVENESS

Table 7: Acceptance Guidelines

Miss Rate = 100( Number of misses/Number of opportunities for a miss)

The table below shows the ratings for each appraiser.

Table 8: Acceptance Results

Appraiser E ectiveness Miss Rate False Alarm Rate

SPC for Excel Software (/spc-for-excel-software)

Visit our home page (/home)

SPC Training (/seminars-on-site-training)

SPC Consulting (/implementation)

Ordering Information (/ordering-information)

Dr. Bill McNeese

Measurement Systems Analysis

<< Return to Categories (/spc-for-excel-publications-category)

Evaluating the Measurement Process - Part 2 (/knowledge/measurement-systems-

SPC Knowledge Base Sign-up

SPC Around the World

Copyright © 2018 BPI Consulting, LLC. All Rights Reserved.

Вам также может понравиться