University of Michigan Online Six Sigma Black Belt Course

Measurement Systems Analysis

Measurement Systems Analysis (MSA) Reference Manual, AIAG, 1995.
(www.aiag.org)

Copyright, Pat Hammett, University of Michigan. All Rights Reserved.

Topics
I. II. III.

Components of Measurement System Sources of Measurement Error Types of Measurement System Studies
Gage Accuracy Study Gage R&R Study
A. B.

University of Michigan Online Six Sigma Black Belt Course

University of Michigan Online Six Sigma Black Belt Course

Decomposing Variation
Measurement variation is inherent to a process!
2total 2inherent
(random variation)

2special cause
(special causes - these may be 0)

2product 2repeatability 2static

2gage 2reproducibility 2dynamic

(load/unload between measurements)

2batch 2material

2?

I. Components of a Measurement System

Equipment or Gage
Type of Gage:
Attribute: go-no go, vision systems (part present or not present) Variable: calipers, probe, tape measure, coordinate measurement machines, checking fixture with inspection device

Discrimination of Measurement General Rules:

At least 1/10 of tolerance (tol = 1 mm, measure to at least 0.1) Or, at least 1/10 of 6*process standard deviation (6)

Operator & Operating Instructions Part locating or orientation scheme

gage must be able to consistently locate the part being measured.
4

University of Michigan Online Six Sigma Black Belt Course

University of Michigan Online Six Sigma Black Belt Course

II. Sources of Measurement Error

Systematic Variations
1. accuracy: improper calibration of equipment 2. reproducibility: different operators using same equipment with different techniques

Random Variations
3. repeatability: (for example, inconsistent part locating)

Periodic Variations
4. stability: wear, deterioration, environmental conditions
5

1. Accuracy
difference between the true average and the observed average
Accuracy or Mean Bias True Average

true average may be obtained using a more precise measuring tool

Observed Average

University of Michigan Online Six Sigma Black Belt Course

University of Michigan Online Six Sigma Black Belt Course

2. Reproducibility
variation in average of measurements made by different operators using the same gage measuring the same part.
Reproducibility Operator C

Operator B Operator A
7

3. Repeatability
random variation in measurements when one operator uses the same gage to measure the same part several times.
Observed Average

Repeatability
8

University of Michigan Online Six Sigma Black Belt Course

University of Michigan Online Six Sigma Black Belt Course

4. Stability
difference in the average of at least 2 sets of measurements obtained with a gage as a result of time.
Stability

Time 1

Time 2

III. Types of Measurement Systems Analysis Methods

A. Accuracy Gage Accuracy (Bias) Study B. Repeatability & Reproducibility Study
Gage R&R (Methods: ANOVA or X-bar/Range) - Note: repeatability may be further decomposed into Static & Dynamic Repeatability

10

University of Michigan Online Six Sigma Black Belt Course

University of Michigan Online Six Sigma Black Belt Course

A. Gage Accuracy (Bias) Study

Measure a master (reference) part using a more precise instrument. Repeatedly measure master part with gage.
Reference Part (mm) 62.93

Measurement System Current Gage Trial-1 63.07 Trial-2 63.05 Trial-3 63.05 Trial-4 63.03 Trial-5 63.08 Trial-6 63.03 Trial-7 63.01 Trial-8 63.04 Trial-9 63.06 Trial-10 63.07 Average 63.05 Difference 0.12

One-Sample T: Current Gage Test of mu = 62.93 vs mu not = 62.93 Variable Current Gage Current Gage p-value 0.000
11

N 10

Mean 63.0490

StDev 0.0218

( 63.0334, 63.0646)

B. Gage R&R Study

Gage R&R Study is used to identify if gage variation is sufficiently low relative to tolerance width and/or part variation. In this module, we will show how to conduct a Gage R&R study and interpret results using the Average and Range Gage R&R method.
Note: Gage R&R also may be done using a TwoFactor Analysis of Variance.
12

University of Michigan Online Six Sigma Black Belt Course

University of Michigan Online Six Sigma Black Belt Course

Trials and Operators

In conducting a Gage R&R study, we need to identify # parts, # trials per part, and # operators. We also need tolerance width for each feature.
Tolerance Width = USL LSL
USL ~ Upper Spec Limit and LSL ~ Lower Spec Limit.

Common Applications (parts x trials x operators):

5 or 10 parts 2 or 3 trials 2 or 3 operators

Example: 5x3x2 Two operators will measure each of 5 parts three times.
13

Data Sheet 10x3x3

Gage worksheet: gagerr.xls
Gage R & R Study Worksheet Date: Gage Number: Gage Cert. Level: Gage Cert. Date: Gage Build Source: Operator: No Operators: Number of Trials: Operator A Trial Number 1 2 3 Average Range 1 2 A Tolerance Width: Number of Parts: 3 Part 4 5 6 7 8 9 10 Average Part Number: Part Name: Characteristic: Engineering Level: B Study Variation: 5.15 # of standard deviation

X-bar R-bar 1 2 3 Part 4 5 6 7 8 9 10 Average

Trial Operator Number B 1 2 3 Average Range Trial Operator Number C 1 2 3 Average Range

X-bar R-bar 1 2 3 Part 4 5 6 7 8 9 10 Average

X-bar R-bar

14

University of Michigan Online Six Sigma Black Belt Course

University of Michigan Online Six Sigma Black Belt Course

Example: Car Door

Suppose you conduct a Gage R&R study for a Car Door.
13 Features, 5 parts, 3 trials, 2 Operators

P6 P5 P4 P3 P2

P7 P8

P9 P10 P11 P12 P13

Measurements deviation from nominal.

Spec 0 +/- 0.5 mm

P1

15

Gage R&R Data Feature P1

(see gagerr.xls for data set)
No Operators: Number of Trials: Trial Operator Number A 1 2 3 Average Range Operator B Trial Number 1 2 3 Average Range 2 3 1
-0.02 0.03 0.04

Tolerance: Number of Parts: 2

0.05 0.03 0.03

1 5 5
-0.03 -0.04 0.01

3
-0.04 -0.05 -0.01

Part 4
0.05 0.07 0.08

0.02 0.06 1
0.04 0.07 0.05

0.04 0.02 2
0.05 0.07 0.05

-0.03 0.04 3
0.02 -0.03 -0.01

0.07 0.03 Part 4

0.09 0.09 0.09

-0.02 0.05 5
-0.04 -0.05 -0.03

0.05 0.03

0.06 0.02

-0.01 0.05

0.09 0.00

-0.04 0.02
16

University of Michigan Online Six Sigma Black Belt Course

University of Michigan Online Six Sigma Black Belt Course

Testing Range Stability?

1st Step test if any range measurements are out-control (Ri > UCLR)

If individual range measurements for a set of trials is out-of-control, than we may over-estimate gage variation. Note: Range tests help identify potential outliers.

2nd Step, if all measurements are in-control (stable), we may then compute the % Repeatability, % Reproducibility, and % Gage R&R.
17

Feature 1: Range Stability Test (from worksheet)

R-Bar X-Dif UCLr LCLr Max Range Range Stable? 0.03 0.02 0.08 0.00
0.06 Yes

Max Range Between Trials = 0.06 UCLr ~ 0.08 (calculated from data) Thus, the trials are in statistical control.
18

University of Michigan Online Six Sigma Black Belt Course

University of Michigan Online Six Sigma Black Belt Course

Gage R&R Calculations

Most software will automatically compute:
% Repeatability (% Equipment Variation) % Reproducibility (% Appraiser Variation) % R&R (% Repeatability and Reproducibility)

We typically compute these value as:

% of the tolerance width, and/or % of total variation in the parts measured.

19

Repeatability Calculations (use software)

K = 5.15 or 6 (# of std deviations to use to estimate variation)* Repeatability Requires R-bar R-bar average range between trials

EV = R

d2

*K

Trials 2 3

d2 1.128 1.693

repeatability = R d

Note: K=6 (# of sigmas) - represents 99.73% of values if normal distribution K=5.15 (# of sigmas) - represents 99% of values if normal distribution

20

University of Michigan Online Six Sigma Black Belt Course

10

University of Michigan Online Six Sigma Black Belt Course

Reproducibility Calculations (use software)

Reproducibility Requires X-bar diff X-bar Diff Range of Operator Means

X diff = Max( X i ) Min( X i ) AV = X diff * K 2 ) 2 EV 2 / nr )

reproducibility = AV / K
Trials 2 3 d2 1.128 1.693 K2 (SV=5.15) 3.65 2.7

n number of parts r number of trials

K2 (SV=6) 4.25 3.15

Note: 6 - represents 99.73% of values if normal distribution 5.15 - represents 99% of values if normal distribution

21

Calculations - continued
R&R Calculations (see attached worksheet for sample) d K (SV=5.15) K (SV=6) # parts
2 3 3

R & R = EV + AV R&R = R & R / K

2

2 3 4 5 6 7 8 9 10

1.128 1.693 2.258 2.823 3.388 3.953 4.518 5.083 5.648

3.650 2.701 2.300 2.080 1.930 1.820 1.740 1.670 1.620

4.252 3.146 2.680 2.423 2.249 2.120 2.027 1.946 1.887

PV = Rp * K 3 TV = R & R 2 + PV 2

22

University of Michigan Online Six Sigma Black Belt Course

11

University of Michigan Online Six Sigma Black Belt Course

Gage R&R Ratio

To determine acceptability, a ratio is typically formed comparing gage variation to either: Tolerance Width - % R&R Tolerance K * R& R
% R & R - Tolerance = USL LSL

Total Process Variation (TV) - % R&R TV

% R & R - TV = K R & R K TV

Note: K=6 - represents 99.73% of values if normal distribution K= 5.15 - represents 99% of values if normal distribution 23

Repeatability and Reproducibility Ratios

Similar ratios may be computed for repeatability and reproducibility. % Equipment Variation - Tolerance:

% EV - Tolerance =

K * repeatabiity USL LSL

% Appraiser Variation Tolerance K * reproducibility

% AV - Tolerance = USL LSL

Note: may compute ratios vs. Total Variation Observed

% EV - TV =

repeatabiity TV

% AV - TV =

reproducibility TV

24

University of Michigan Online Six Sigma Black Belt Course

12

University of Michigan Online Six Sigma Black Belt Course

General Criteria for Gage R&R

Measures: R&R % (Tolerance) or R&R% - TV if R&R% is: < 10% ~ measurement system is acceptable 10-30% ~ measurement system may be acceptable based upon importance of application, gage cost, cost of repairs .. > 30% ~ measurement system needs improvement
25

Repeatability Requirement
Typically, manufacturers also have a requirement for EV (repeatability) % EV Tolerance < 18% OR, % EV Total Variation < 18%

26

University of Michigan Online Six Sigma Black Belt Course

13

University of Michigan Online Six Sigma Black Belt Course

Car Door Example: Feature 1

Note: feature 1 has tolerance +/- 0.5 mm.
No Operators: Number of Trials: Trial Operator Number A 1 2 3 Average Range Operator B Trial Number 1 2 3 Average Range 2 3 1 -0.02 0.03 0.04 0.02 0.06 1 0.04 0.07 0.05 0.05 0.03 2 0.05 0.03 0.03 0.04 0.02 2 0.05 0.07 0.05 0.06 0.02 Tolerance: Number of Parts: Part 3 4 -0.04 0.05 -0.05 0.07 -0.01 0.08 -0.03 0.04 3 0.02 -0.03 -0.01 -0.01 0.05 0.07 0.03 Part 4 0.09 0.09 0.09 0.09 0.00 5 -0.03 -0.04 0.01 -0.02 0.05 5 -0.04 -0.05 -0.03 -0.04 0.02 6 1 5 6 Average 0.00 0.01 0.03 0.01 0.04 Average 0.03 0.03 0.03 0.03 0.02

X-bar R-bar

X-bar R-bar

27

Results From Worksheet

Calculations from Excel File: gagerr-v3.xls
Are the repeatability data? Is this gage acceptable?

%EV-Tol %AV-Tol %RR-Tol Pass/Fail

9.7% 5.8% 11.3%

%EV-TV %AV-TV %RR-TV %PV-TV Pass/Fail

38.6% 23.0% 44.9% 89.3% Fail

R-Bar X-Dif UCLr LCLr Max Range Range Stable?

0.03 0.02 0.08 0.00

0.06 Yes

Pass

28

University of Michigan Online Six Sigma Black Belt Course

14

University of Michigan Online Six Sigma Black Belt Course

Interpretation: Feature 1
% R&R (tolerance) = 11%
Relative to the tolerance, this measurement system meets R&R criteria.

% R&R TV = 45%
Note: although fail 30% criteria, the variation in the parts for the study is quite small.
5.15TV = 0.26 which is approx tolerance width of 1 mm
29

% TV Versus % Tolerance
Industry Debate: % TV versus % Tolerance
Opinion 1: purpose of gage is to separate gage variation from product variation. Thus, use % TV requirement Opinion 2: purpose of gage is to measure relative to a tolerance. Thus, use % Tolerance

Instructor Opinion:

Measure Both. If fail % TV, make sure that you have taken parts with sufficient variation relative to the tolerance range.

30

University of Michigan Online Six Sigma Black Belt Course

15

University of Michigan Online Six Sigma Black Belt Course

Minitab
Gage R&R Analysis also may be done in minitab.
Under Stat >> Quality Tools

Types of Gage R&R Studies in Minitab

Crossed design (when each part is measured multiple times by each operator may use either ANOVA or Average/Range Method) Nested design (when each part is measured by only one operator, such as in destructive testing. In destructive testing, the measured characteristic is different after the measurement process than it was at the beginning. Crash testing is an example of destructive testing.

31

Minitab: Feature #1
STAT >> Quality Tools >> Gage R&R (Crossed)
Part Numbers PartNo Operator Operator Measurement Data Value Method Analysis X-bar/Range Options >> Process Tolerance = 1

32

University of Michigan Online Six Sigma Black Belt Course

16

University of Michigan Online Six Sigma Black Belt Course

Results Using Minitab

Minitab (if set K (study variation sigma) = 5.15 and X-bar / Range method option, results should be ~ same)
Quality Tools >> Gage R&R Study Crossed
StdDev Source (SD) Study Var (5.15*SD) %Study Var (%SV) %Tolerance (SV/Toler)

Total Gage R&R Repeatability Reproducibility Part-to-Part Total Variation

2.18E-02 1.86E-02 1.13E-02 4.37E-02 4.88E-02

0.112145 0.095814 0.058276 0.224966 0.251369

44.61 38.12 23.18 89.50 100.00

11.21 9.58 5.83 22.50 25.14

33

Results from Minitab

If use Minitab Gage tool, range chart is automatic output
Gage R&R (Xbar/R) for Value
Components of Variation
100 0.10 %Contribution %Study Var %Tolerance 0.05 0.00 -0.05 Gage R&R Repeat Reprod Part-to-Part

Gage name: Date of study: Reported by: Tolerance: Misc:

Response by PartNo

Percent

50

PartNo
0.10 UCL=0.08238 0.05

Sample Range

Range Chart

R Chart by Operator
0.10
A B

Response by Operator

0.05 R=0.032 0.00 LCL=0 0.00 -0.05

Operator

Xbar Chart by Operator

0.10
A B

Operator*PartNo Interaction
0.10

Operator
A B

Sample Mean

0.05

Average

UCL=0.05474 Mean=0.022

0.05

0.00 -0.05

LCL=-0.01074

0.00

PartNo

34

University of Michigan Online Six Sigma Black Belt Course

17

University of Michigan Online Six Sigma Black Belt Course

Is it the R or the R?
If we fail Gage R&R study, we often look at equipment and appraiser variation estimates individually.
So, if % EV is higher than % AV we consider repeatability a greater concern than reproducibility.

35

Improving Gage Capability

How do we improve reproducibility?
operator training, OR more clearly define measurement scales (easier to read).

How do we improve repeatability?

gage maintenance, OR may need to redesign the gage or the gage locating scheme which affects how a part is held during measuring.
36

University of Michigan Online Six Sigma Black Belt Course

18