Академический Документы
Профессиональный Документы
Культура Документы
uncertainty estimation
on measurement,
calibration and testing
Alexandre ALLARD
LNE, France
Outline
Introduction
Reference documents
GUM methodology
Supplement 1 to the GUM (Monte Carlo)
Case studies
Linear example
Conclusion
Bayesian methods
General conclusion
20/05/2015
20/05/2015
20/05/2015
Laboratory B
Laboratory A
Laboratory B
Laboratory A
Max. Tolerance
LImit
Min. Tolerance
Limit
Specification Area
20/05/2015
Measurement Result
and Uncertainty
Max. Tolerance
Limit
Min. Tolerance
Limit
Specification Area
20/05/2015
Max. Tolerance
Limit
Min. Tolerance
Limit
Specification Area
20/05/2015
10
Min. Tolerance
Limit
Guard band
Acceptation Area
Max. Tolerance
Limit
Specification Area
11
Guard band
Acceptation Area
Min. Tolerance
Limit
Specification Area
Max. Tolerance
Limit
12
shared risk
Acceptation Area
Min. Tolerance
Limit
Max. Tolerance
Limit
Specification Area
13
Documentation - JCGM
WG1
GUM +
Supplements
20/05/2015
WG2
International
Vocabulary of
Metrology (VIM)
14
Documentation - VIM
VIM (JCGM 200:2012) : International Vocabulary of
Metrology Basic and general concepts and associated
terms, 2012.
20/05/2015
15
Alternative method when the conditions for a valid application of the GUM
are not fulfilled
Requires a software
20/05/2015
16
20/05/2015
17
Terminology
Measurand (VIM 2012, 2.3) : quantity intended to be
measured
The measurand has a unique and unknowable true value
Uncertainty (VIM 2012, 2.26) : non-negative parameter
characterizing the dispersion of the quantity values being
attributed to a measurand, based on the information used
Standard uncertainty (VIM 2012, 2.30) : Measurement uncertainty
18
20/05/2015
19
,,
+ +
1
Workshop measurement uncertainty Jakarta
20
20/05/2015
21
q0.50 is the median : half possible values of X are below the median and
the other half are over the median.
2.5%
q0.025
20/05/2015
22
23
Population
,,
Data
Mean
Standard
deviation
20/05/2015
Sample
,,
1
1
24
g(x)
-2
x
+
g ( x ) dx = 1
25
Step 2:
Quantification
of the sources
of uncertainty
of the xi
Standard
Uncertainty
or
distribution
Mathematical
Model
of the Measurement
Process
Measurand
Y = f(Xi)
Final
Expression of
the Result
yU
f(X1,..,Xn)
Feedback
20/05/2015
26
,,
Workshop measurement uncertainty Jakarta
27
28
1)1) Distance
Distance between
centre and
upper side
entre the
le centre
dethe
la face
of the gauge and
joint plan,
20Csur
and at a
suprieure
de the
la cale
et leatplan
lequel
elle est adhre, 20 C et
vertical position.
en position verticale.
2)2) Distance
Distance between
centres
of the sides of
entre the
lestwo
deux
centres
des
facesatde
la cale,
20 is
C,
lahorizontal
the gauge,
20C.
The gauge
at a
cale
tant en position horizontale.
position.
3) Distance entre deux plans
3) parallles,
Distance between
plans,
at 20C.
20 two
C,parallel
la cale
tant
en
horizontale.
Theposition
gauge is at
a horizontal position.
20/05/2015
29
True value
(unknown)
systematic
random
error
Measurement error (VIM 2012, 2.16) : measured quantity value minus a
reference quantity value
Systematic measurement error (VIM 2012, 2.17) :
component of
measurement error that in replicate measurements remains constant or
varies in a predictable manner
Random mesurement error (VIM 2012, 2.19) : component of measurement
error that in replicate measurements varies in an unpredictable manner
20/05/2015
30
20/05/2015
31
20/05/2015
32
20/05/2015
33
34
,,
The input quantities X1, XN represent all the usable data for the
calculation of the result.
The f function does not merely express a physical law, but the
measurement process. More specifically, it must contain all the
quantities that contribute significantly to the final resulting uncertainty.
20/05/2015
35
0
= % 1 + ( ) 20 cos /
1 + 1 ) 20
corrective term
barometer verticality
corrective term
gravity acceleration
dilatation
density of mercury at
measurement temperature
20/05/2015
36
,,
20/05/2015
37
Outline
Introduction
Reference documents
GUM methodology
Supplement 1 to the GUM (Monte Carlo)
Case studies
Linear example
Conclusion
Bayesian methods
General conclusion
20/05/2015
38
Measurement Process
analysis
Reproducibility Tests
no
yes
Analytical Method
Reference
documents
Analytical
method
assumptions?
yes
ISO 5725
ISO GUM
ISO GUM
non
supplment 1
Variance Analysis
Law of Propagation
Result and
Uncertainty
"Validation" and update
20/05/2015
39
y, u(y)
y U (k=2)
x 3, u(x 3)
y~ ;u ( y~)
[ylow;yhigh]
20/05/2015
40
Step 2:
Quantification
of the sources
of uncertainty
of the xi
Standard
Uncertainty u(xi)
or
distribution
Mathematical
Model
of the Measurement
Process
Measurand
Y = f(Xi)
Final
Expression of
the Result
yU
f(X1,..,Xn)
Feedback
20/05/2015
41
Type A
No Experimental Data
A Priori Information
Type B
20/05/2015
42
Examples :
o
20/05/2015
43
Linear operator
=4
Variance
o
o
54
+ 2
<
5 56 789
20/05/2015
44
Temperature of the
laboratory (C)
22.3
22.7
23.5
23.1
23.3
22.8
23.0
22.9
22.4
10
23.2
20/05/2015
Mean value
====
= = 22.92 7
Standard deviation
2 = =
1
1
= =
= 0.38 7
45
Available
knowledge ?
Probability
distribution
Standard
deviation u(Xi)
46
Available
knowledge ?
Verification certificate
Expert knowledge
Physical limits
20/05/2015
47
Probability
distribution
Gaussian
Resolution q of a displaying
device
Rectangular
Rectangular
Oscillating phenomenon
between two values a and b
Arcsine (U-shaped)
Probability
distribution
Standard
deviation u(Xi)
20/05/2015
Standard deviation
D
=
E
12
F 4
3
G5
8
48
o Interval of values
o A distribution of probability
-a
or
-a
20/05/2015
u(xi)
(GUM)
or S (Stats)
...
49
1 H 1
789
=H
.2
.2
50
35
35
25
25
15
15
5
15
20
25
20/05/2015
15
20
25
51
22.3
23.2
22.7
15.0
23.5
23.6
23.1
24.6
23.3
25.3
22.8
15.8
23.0
19.2
22.9
20.6
22.4
20.3
10
23.2
22.2
Mean
22.92
20.98
Std dev
0.38
3.52
20/05/2015
= 0.41
52
X1= 2Q1-5Q3+Q4-3Q8
X2 =Q1+6Q2+Q8
u ( x 1 , x 2 ) = u ( F (Q 1 , Q 2 ,..., Q L ), G (Q 1 , Q 2 ,..., Q L )) =
i =1
F G
u 2 (Q i
Q i Q i
53
Step 2:
Quantification
of the sources
of uncertainty
of the xi
Standard
Uncertainty u(xi)
or
distribution
Mathematical
Model
of the Measurement
Process
Measurand
Y = f(Xi)
Final
Expression of
the Result
yU
f(X1,..,Xn)
Feedback
20/05/2015
54
,,
,,
20/05/2015
55
20/05/2015
56
N 1 N
f
f f
2
(
)
u c2 (y ) =
u
x
+
2
u xi, x
i
i =1 j = i +1 x i x j
i =1 x i
N
2 2
c
i u (x i ) + 2
i =1
ci
i =1 j = i + 1
N 1
u c2 (y ) = c i2u 2 ( x i ) + 2
i =1
u 2 (y ) =
c j u xi , x j
N 1 N
u c2 (y ) =
c c u (x ) u (x )r (x , x )
N
i =1 j = i +1
N 1
i =1
i =1 j = i +1
2 2
c
i u ( xi ) + 2
ci c j u (xi , x j )
Revised GUM
20/05/2015
57
I 2
58
Uncertain
Input
Quantities :
xi
Mathematical
Model
of the Measurement
Process
Measurand
Final
Expression of
the Result
yU
Y = f(Xi)
f(X1,..,Xn)
Feedback
20/05/2015
59
y U unit (value of k )
Expanded uncertainty
Coverage interval
20/05/2015
Coverage factor
P Q; P + Q
60
Quantity defining an
measurement
around
expressed
as
standard deviation
about
measurement
the
interval
result
that
may
of
a
be
u(y)
k : coverage factor
U = k u(y)
20/05/2015
61
U = k uc(y)
Expanded
uncertainty
Coverage Factor
Combined standard
uncertainty of y
(Step 3 Result)
62
k =2
So that p =
p 95%
compute
No
(1 )%
eff
(dof)
(1 )%
k=2
So that p =
p=?
Monte Carlo
Simulations
See GUM
20/05/2015
63
In simpler terms:
A phenomenon may be represented by a normal distribution if it results
from a great number of additional, small and independent effects
20/05/2015
64
y U unit (value of k )
In order to have a physical significance, a measurement result must be
rounded according to the uncertainty related to the result.
o U is rounded up to at most 2 significant digits
o The measurement result y is rounded accordingly
For instance, there is no need to announce a length measurement by the
micron if the uncertainty related to the measurement with a sliding caliper
corresponds to the tenth of a millimeter
The rounding must not be carried out in several stages but only once, when
reporting the final result
20/05/2015
65
U = k u(y)
u(y) obtained at Stage 3
Very often k = 2
U rounded to 2 significant digits
y U
unit
(k=2)
20/05/2015
66
Outline
Introduction
Reference documents
GUM methodology
Supplement 1 to the GUM (Monte Carlo)
Case studies
Linear example
Conclusion
Bayesian methods
General conclusion
20/05/2015
67
Reproducibility Tests
no
yes
Analytical Method
Reference
documents
yes
ISO 5725
ISO GUM
ISO GUM
non
supplment 1
Variance Analysis
Law of Propagation
Result and
Uncertainty
"Validation" and update
20/05/2015
68
69
N 1 N
(y ) = f u 2 (x i ) + 2 f f u x i , x j
i =1 x i
i =1 j =i +1 x i x j
N
y, u(y)
output quantity Y is Gaussian
The Central Limit Theorem is to be applied
independence of xi s
The variance associated to Y is greater than any ci (xi)
component for an xi whose distribution is not gaussian
20/05/2015
yU
70
y, u(y)
y U (k=2)
x 3, u(x 3)
y~ ;u ( y~)
[ylow;yhigh]
20/05/2015
71
Step 1
Y=f(x1,x2,x3)
Choice of distributions
for input variables
Step 2
Step nb.1
Step nb.2
Step 3
Step nb. M
X1
X2
x11
x12
x1M
x21
x22
x2M
X3
Calculation of M values
resulting from Y
Empirical distribution of
the measurand
Step 3
y1
x31
x32 f y 2
x3M
y M
Random generation of M
input variable samples
20/05/2015
Step 4
~
y ; u( ~
y)
[ylow;yhigh]
Extraction of the:
Mean
standard deviation
interval at p%
72
y, u(y)
y U
x 3, u(x 3)
20/05/2015
y~ ;u ( y~)
[ylow;yhigh]
73
o Interval of values
o A distribution of probability
-a
or
-a
20/05/2015
u(xi)
(GUM)
or S (Stats)
...
74
Triangular
20/05/2015
75
20/05/2015
76
20/05/2015
77
Distribution to be assigned
to X according to
maximum entropy
Lower limit: a
Higher limit: b
R (a,b)
Rectangular
N (x, u(x))
X varies sinusoidally
between limits a and b
(a<b)
U-shaped(a,b)
Best estimation of a
positive quantity
graph
Gaussian
Arcsine Derivative
1
e( )
Exponential
Normal distribution corresponds to maximum entropy taken from all the distributions which
have an identical mean x and a identical standard deviation u(x)
20/05/2015
78
113.6
113.2
115.4
113.1
113.6
113.5
114.4
112.7
113.6
113.5
Mean = 113.55
Scale (experimental std
deviation) = 0.68
Number of df = 14
Student Standard Deviation :
20/05/2015
79
Period
(G-1()= x1)
G( x ) =
g (t )dt
20/05/2015
80
R = 0.71
0.5534
20/05/2015
81
GUM-S1 Step 3
Step N1
Step N2
Step 3
Step NM
X1
X2
x11
x12
x1M
x21
x22
x2M
X3
Calculation of M values
resulting from the empirical
measurand distribution Y
Step 4
y1
x31
x32 f y 2
x3M
y M
20/05/2015
82
Software validation,
20/05/2015
83
Period T = 219937-1
20/05/2015
84
GUM-S1 Step 4
Propagation of Uncertainty (GUM)
Most frequently k = 2
x 1, u(x 1)
x 2, u(x 2)
y, u(y)
y U
x 3, u(x 3)
Interval at p%
Coverage interval
20/05/2015
[ylow;yhigh]
85
1
TU =
F
T TU
86
p = 0.95
1
(
2
0.975
0.025
[\ <
(
2
Symmetrical distribution
(
<
\
[
; [\ < 1
2
20/05/2015
[\ <
1.96
(
2
avec (
(
2
1.96
87
[\ <
is minimum
Iy = 8.13-1.74 = 6.36
20/05/2015
Iy = 7.48-1.14 = 6.34
Iy = 7.68-1.47 = 6.21
88
89
Reproducibility Tests
no
yes
Analytical Method
Reference
documents
yes
ISO 5725
ISO GUM
ISO GUM
non
supplment 1
Variance Analysis
Law of Propagation
Result and
Uncertainty
"Validation" and update
20/05/2015
90
91
yes
Step 2
quantify
Standardized
Method
+
Interlaboratory
tests
ISO 5725 : sr et sR
no
Estimate
One intermediate
uncertainty
yes
u(xi)
participation ?
no
Proof
Normalized
Method
yes
Step 3
uncertainty
propagation
uc(y)
uc(y) = sR
Step 4
Expanded uncertainty
U = kuc(y)
20/05/2015
92
Minimum
5725-2
Maximum
Reproducibility
conditions (R)
repeatability (r)
conditions
Intermediate Precision
5725-3
same method
identical individuals
same laboratory
same operator
same equipment
short time interval
20/05/2015
same method
identical individuals
different laboratories
different equipment
93
y11
Mean
T
y1n
Lab i
Descriptive
Statistics
s1
yi1
Tb
yin
Lab p
Standard
deviation
si
yi1
ypn
Grubbs
sp
Homogeneity Tests (filter p p*)
Cochran
Hypothesis : ni = n constant
Tc =
Tb
20/05/2015
1
1
repeatability
standard deviation
Tb Tc
reproducibility
standard deviation
94
Outline
Introduction
Reference documents
GUM methodology
Supplement 1 to the GUM (Monte Carlo)
Case studies
Linear example
Conclusion
Bayesian methods
General conclusion
20/05/2015
95
t = tu +
20/05/2015
+ t
Workshop measurement uncertainty Jakarta
96
20/05/2015
97
GUM-S1
Trueness
o
2 tu =
Trueness
v
w
= 0.05 mm
o Gaussian distribution
o Mean : 0, std deviation :
v
.
2 tu = w =
= 0.05 mm
Quantification
o
Rectangular distribution
Quantification
o Rectangular distribution
x
= 0.029 mm
3.1 3.0
Repeatability
3.1 2.9
n = 10 repetitions
3.2 3.2
t = 3.05 mm
3.0 3.1
2 t =
= 0.11 mm
20/05/2015
2.9 3.0
Repeatability
o Student distribution
o t = 3.05 mm
o I5t = = 0.11 mm
98
GUM-S1
2 t = 2 tu + 2
+ 2 t
2 t =
+ 2 t
2 t =
2 tu + 2
D = 2 2 t = 0.25 mm
Coverage interval :
T D; T + D = 2.80; 3.30
20/05/2015
t = 3.05
2 t = 0.14
Coverage interval :
tWKX ; tY ZY = 2.78; 3.32
Workshop measurement uncertainty Jakarta
99
GUM-S1
Input quantity
Coontribution
to the variance
(%)
Input quantity
Coontribution
to the variance
(%)
Trueness eT
16%
Trueness eT
14%
Repeatability t
79%
Repeatability t
81%
Quantification q
5%
Quantification q
5%
20/05/2015
100
Example 1 : Conclusion
20/05/2015
101
20/05/2015
102
GUM
o
o
a
o
o
W
o
o
r, = 100 000 mg
2 r, = 0.05 mg
mR,c
o
mR,c
R
o
o
r, = 1.234 mg
2 r, = 0.02 mg
0 = 1.20 kg/m
2 0 = 0.058 kg/m
0 = 8000 kg/m
2 0 = 577 kg/m
0r = 8000 kg/m
2 0r = 29 kg/m
20/05/2015
mR,c
o Gaussian distribution
o Mean : r, = 100 000 mg, Standard
deviation : 2 r, = 0.05 mg
mR,c
o Gaussian distribution
o Mean :r, = 1.234 mg, Standard
deviation : 2 r, = 0.02 mg
a
o Rectangular distribution
o Lower bound : 1.10 kg/m, Upper
bound : 1.30 kg/m
W
o Rectangular distribution
o Lower bound : 7000 kg/m, Upper
bound : 9000 kg/m
R
o Rectangular distribution
o Lower bound : 7950 kg/m, Upper
bound : 8050 kg/m
103
GUM
2 = 7r, 2 r, + 7r, 2 /r,
+7 2 0 +7 2 0 +7r 2 0r
2 =
2 r, + 2 /r,
2 = 0.05 + 0.02
2 = 5.4 10< kg
= 1.234 10< kg
= 7.5 10< kg
2
D = 2 2 = 11 10< kg
Coverage interval :
T D; T + D = 1.124 10< ; 1.343 10<
20/05/2015
Coverage interval :
WKX ; Y ZY =
1.083 10< ; 1.383 10<
104
GUM
Input quantity
Coontribution
to the variance
(%)
Input quantity
Coontribution
to the variance
(%)
Convent. Mass of
the reference
weight mR,c
86%
Convent. Mass of
the reference
weight mR,c
43%
Convent. Mass of
the added weight
mR,c
14%
Convent. Mass of
the added weight
mR,c
8%
Interaction
between the mass
densities a and
W
Not available
Interaction
between the mass
densities a and
W
49%
20/05/2015
105
Outline
Introduction
Reference documents
GUM methodology
Supplement 1 to the GUM (Monte Carlo)
Examples
Linear example
Conclusion
Bayesian methods
General conclusion
20/05/2015
106
GUM
Measurement model
GUM-S1
Linearity
Linear or
approximately linear
No assumptions
Distribution associated
to the measurand
Often Gaussian
Any distribution
Software
implementation
Excel sheet
Sensitivity analysis
Directly availabe
(partial derivatives)
Requires additional
calculations or simulations
Indirect measurements
20/05/2015
Not addressed
107
Bayes theorem
| =
Likelihood function
;
|
Posterior probability
distribution
108
109
,,
110
Computational codes
GUM-S1 steps :
,,
20/05/2015
111
Small deviations from the best estimates (the linear approximation of the
GUM is then often adequate).
Despite its wide use in the world, the GUM needs revisions
20/05/2015
112
Questions ?
Thank you !
Merci !
terima kasih !
20/05/2015
113