Instrument Calibration and

Uncertainty Analysis

Calibration of an Instrument

The static performance charactertistics of an instrument can be determined

by ANSI/ISA Standard 51.1 Process Instrumentation Terminology. This
standard gives the definition for the following important characteristics:
1. Accuracy the maximum difference between measurements and a standard (or reference) as determined by testing over the range of the instrument.
2. Repeatability the maximum difference between repeated measurements of the same standard in the same direction of measurement (i.e.
increasing or decreasing the input).
3. Resolution the smallest increment you can read on the scale. If you
can see a half increment (i.e. the space between the tick marks), then
the resolution will be one half of the unit spacing.
4. Range the maximum and minimum values you can measure (for example: 0 10 mm).
5. Span the algebraic difference between the upper and lower range
values.

Example 1
You wish to determine the accuracy and repeatability of your bathroom
scale in terms of kg. You measure calibration-standard masses (with an
accuracy rating of 0.0005 kg) over the range of the scale. The range of the
scale is 0 100 kg. Your scale has 1 kg increments but you can make out
half increments (i.e. the resolution is 0.5 kg). Following ANSI/ISA 51.1, you
find the results shown in Table 1.
Table 1: Weigh scale calibration data.
Scale Reading (kg)
Standard (kg) Up Down Up Down Up
0.000
0.0
0.0
20.000
19.5
21.0 22.0 19.0
19.5
40.000
43.0
39.0 39.5 40.0
41.0
60.000
60.0
59.5 61.0 58.0
59.5
80.000
80.0
82.0 79.5 79.0
80.0
100.000
100.0
99.5
100.0

Down
0.0
20.0
39.0
61.0
79.5

Solution
The easiest way to find the accuracy and repeatability is to make a deviation
table shown in Table 2.
Table 2: Weigh scale deviation data.
Standard (kg)
0.000
20.000
40.000
60.000
80.000
100.000

Deviation from standard (kg)

Down Up Down Up Down
0.0
0.0
0.0
-0.5
1.0
2.0
-1.0
-0.5
0.0
3.0
-1.0
-0.5
0.0
1.0
-1.0
0.0
-0.5
1.0
-2.0 -0.5
1.0
0.0
2.0
-0.5
-1.0
0.0
-0.5
0.0
-0.5
0.0
Up

Max. Dev.
of Ups (kg)
2.5
3.5
1.5
0.5
0.5

Max. Dev.
of Downs (kg)
0.0
2.0
1.0
3.0
3.0

According to ANSI/ISA 51.1 the measured accuracy of the scale is +3.0/-2.0 kg.
Notice that the accuracy is not symmetric. If the error was random and
normally distributed, and enough measurements were taken, it would be expected that the accuracy would be symmetric. In this course, we will report
2

the accuracy symmetrically using the most conservative estimate. This would
be similar to the accuracy rating given by a manufacturer. This gives an accuracy of 3.0 kg or 3% Full Scale (or FS) or Span. We will state the
accuracy this way because: 1) it makes further uncertainty analysis easier,
and 2) we rarely have time to repeat the experiment enough times to ensure
random occurrences do not skew our results (typically a minimum of five up
and five down traverses are used).
The repeatability is simply the maximum deviation between repeated
measurements of the same value approached from the same direction. In this
case the repeatability is 3.5 kg or 3.5% of span.
Note: According to ANSI/ISA 51.1, if the accuracy rating of the standard
is less than one tenth of the accuracy of the instrument tested, then the
accuracy rating (or inaccuracy) of the standard may be ignored. Since the
accuracy rating of the masses is 0.0005 kg and the accuracy of the scale is
3.0 kg, the uncertainty in the masses is simply ignored.

Uncertainty in a Measurement

Lets say you know the accuracy of an instrument and you use the instrument
to measure some property. What is the uncertainty in that measurement?
This section will explain how to determine the uncertainty in an experimental
measurement, but first we must cover a few important definitions.
The error is defined as the difference between the true value and the
measured value. Since we can never know the true value of a property we
can NEVER know the error in the measurement. What we can do is estimate
the uncertainty in our estimate of the true value. The uncertainity is simply
the estimate of the error with a given confidence interval.
The International Standards Organization (ISO) defines two types of uncertainty:
1. Type A Uncertainty (Px ) The uncertainty estimated from the data,
sometimes this is called precision or repeatability uncertainty. (However, as we will see later, this is not the same repeatability as defined
above in the ANSI/ISA standard. Therefore, I will try to stick to the
term precision when discussing Type A uncertainty.)
2. Type B Uncertainty (Bx ) Uncertainties that cant be analyzed from
3

the data and must be estimated from other sources. Sometimes this is
called systematic or bias uncertainty.
The total uncertainty (Ux ) in a measurement is:
qX
Bx2 + Px2
Ux =

2.1

(1)

Precision Uncertainty (Px )

By taking repeated measurements of the property we are trying to determine

we will obtain a distribution of measurements. An important result from
statistics is the central limit theorem (see Sec. 3.6 in Beckwith). Neglecting
any bias error and for a large sample size, the central limit theorem states:
Sx
Sx
x zc/2 < < x + zc/2
n
n

(2)

where is the true mean value of the property, x is the mean of the measurements, zc/2 is the z-score of the normal distribution with c% confidence,
Sx is the standard deviation of the measurements, and n is the number of
Sx
measurements taken. Therefore, the estimate of the true value is x zc/2
n
Sx
with c% confidence. The term zc/2 is simply the precision uncertainty
n
Sx
(i.e. Px = zc/2 ).
n
If the sample size is small (n <30), the t-distribution must be used,
therefore:
Sx
Px = t 2 ,
(3)
n
where = 1 c, and = n 1 is the degrees of freedom. Values for the
t-distribution are shown in Figure 1.
Note: A confidence interval of 95% is almost universally used in calculations.

2.2

Bias Uncertainty

Bias error will consistently introduce error in our measurement and since it
is systematic there is no way to determine it from making repeated measurements. Therefore we are left to making estimates of this uncertainty (see
4

Figure 1: Students t-Distribution (Beckwith et al., 2006)

5

Sec. 3.9 in Beckwith). A very common estimate of the bias uncertainty is

by using the manufacturers specification for accuracy or the accuracy determined by your own calibration of an instrument. Regardless of the source
of this estimate, it is important that the estimate of the bias uncertainty
has the same confidence interval as the precision uncertainty. If we use the
ANSI/ISA 51.1 definition for accuracy we have no idea what the confidence
interval is. However, most people assume that the accuracy rating has a
95% confidence. In fact this will be a conservative estimate considering that
accuracy is defined as the maximum deviation from the standard.
Other bias errors might not be covered in the accuracy of the instrument
and must be accounted for elsewhere, such as spatial variation of the property
or any effect the instrument might have on the system.
Example 2
You weigh yourself 5 times on the bathroom scale discussed in the previous example (accuracy of 3.0 kg). Your measurements are: 62.5 kg, 63.0 kg,
61.0 kg, 62.0 kg, and 62.0 kg. Estimate your mass and give the uncertainty
with 95% confidence.
Solution
Since the sample size is small the precision uncertainty can be found from
Sx
Px = t 2 , .
n

(4)

The desired confidence interval is 95%, so = 1 0.95 = 0.05 and

= 0.025. The number of degrees of freedom are = n 1 = 4. Using

2
these values, t0.025,4 = 2.776 is found from Figure 1.
The standard deviation, Sx , is found from
sP
(xi x)2
Sx =
.
(5)
n1
Using the data we get x = 62.1 kg and Sx =0.742 kg.
(0.742 kg)

Therefore, Px = 2.776
= 0.92 kg.
5
In this case we estimate the bias uncertainty as 3 kg. Therefore,
p
p
Ux = Bx2 + Px2 = (3 kg)2 + (0.92 kg)2 = 3.1 kg
6

(6)

Therefore, your weight is estimated to be 62.1 3.1 kg with a confidence

of 95%.
NOTE: In this case the precision uncertainty had little effect on the total
uncertainty. Therefore, less measurements could have been made without
much compromise in the total uncertainty in the measurement. To make a
drastic improvement in the measurement uncertainty a more accurate scale
would be required.

2.3

Single-Sample Precision Uncertainty

Ideally several measurements are taken of a property to minimize the precision uncertainty. However, it may occur that only one measurement is
possible. If this is the case Eq. 3 will no longer hold because Sx and t 2 , are
undefined. In this case it is common to estimate the precision uncertainty
as,
Px zc/2 e 2e ,
(7)
where e is the standard deviation of measurements that were taken at some
other time with the same instrument (e.g. when the instrument was calibrated), and where the zc/2 2 for a 95% confidence interval.
Example 3
You weigh yourself once with the bathroom scale described in Example 1
and obtain a measurement of 62.5 kg. Estimate your mass and give the
uncertainty with 95% confidence.
Solution
Since you have only made one measurement you must estimate the precision uncertainty using the single-sample assumption. In this case you can
use the calibration data in Example 1 to find e . The standard deviation of
all the data points in Table 2 is 1.02 kg. Therfore,
Px 2(1.02 kg) = 2.04 kg

(8)

and
Ux =

p
p
Bx2 + Px2 = (3 kg)2 + (2.04 kg)2 = 3.6 kg

(9)

Therefore, your weight is estimated to be 62.5 3.6 kg with a confidence

of 95%.

Propagation of Uncertainty

If we know the equations that describe a system, it is possible to estimate

the probable uncertainty in a variable from the known uncertainties in other
variables. This is sometimes called an external estimate of error, because it
can be made without doing any experiments.

3.1

Analysis of Equations for Small Errors

Consider an output y that is a function of inputs x1 , x2 , etc. y = y(x1 , x2 , x3 , . . . , xn )

Then, take the derivative
dy =

y
y
y
dx1 +
dx2 + +
dxn
x1
x2
xn

(10)

Variation dy of y is produced by the variations dx1 etc. Interpret these

variations as uncertainties, .
y
y
y
1 +
2 + +
n
(11)
x1
x2
xn
We cant evaluate y directly because we dont know if the errors are
positive or negative. To get around this square both sides, and note that on
average the cross product terms
y =

y y
1 5 + . . .
(12)
x1 x5
will disappear. It can be proved that these cross products will vanish if
the errors are symmetrically distributed random values that are statistically
independent of each other. Neglecting cross products leads to a simple
expression for the mean square uncertainty
+

2y

2
n 
X
y
xi

i=1

This result may be stated as:

8

2i

(13)

The mean square uncertainty in a quantity y is found by adding

the mean square uncertainties of all variables contributing to y,
as long as the errors are random and independent of each other.
Because 2y and 2i are just the variances of each component, this is the
same as the theorem that states: the variances of random components add
to produce the functions variance.

3.2

Fractional Uncertainty in Purely Multiplicative Equations

For equations that are purely multiplicative, the fractional or percentage

uncertainty can be found without first calculating the partial derivatives in
Eq.13. The short cut method is as follows:
First take the natural logarithm of both sides of the equation.
Then take the derivative of both sides of the equation, noting
that for any variable xi ,
d(ln xi ) =

dxi
xi

Interpret the differentials, dxi ,as small uncertainties so that

dxi i
Square both sides of the equation, discarding the cross products.
(This is equivalent to simply squaring each term separately and
summing).
For example, consider the purely multiplicative equation
Q = A1 F a Gb H c

(14)

where A1 , a, b and c are constants that can be positive or negative. To find

the effect of uncertainties in F, G and H on the function Q follow the above
four steps.
ln Q = ln(A1 F a Gb H c )
(15)
9

Differentiate, noting A1 =constant

dQ
dF
dG
dH
=a
+b
+c
Q
F
G
H

(16)

Interpret differentials as uncertainties and square each term to obtain

 2
 2
 2
 2
Q
F
G
H
= a2
+ b2
+ c2
(17)
Q
F
G
H
Here we see the square of the fractional (or percentage) uncertainty in each
variable is weighted by the square of its exponent.
This result is very helpful in planning an experiment. It tells us that
variables that appear to large exponents should be measured much more accurately than those with small exponents. For example, if we were measuring
F, G and H in an equation like
Q = A1 F 1/2 G3 H 1/3
The percentage uncertainty in G is six times more important than uncertainties in F because G has an exponent of 3 and F has an exponent of 12 .
Example 4
The power P dissipated by an electrical resistor R is related to voltage
E and current I by P = EI where E = IR. If we know that the voltmeter
has an uncertainty of 2% of reading and the resistance is known to 1.5%,
what is the percentage uncertainty in current I and in power P ?
Solution
P =

E2
R

2 
2
P
P
=
E +
R
E
R

2 
2
P
P
E2 2
E4 2
2
p =
E +
R = 4 2 E + 4 R
E
R
R
R
2p

Divide this equation by P 2 =

E4
to get fractional uncertainty
R2
10

 2  2
2P
R
E
+
=4
2
P
E
R
p
P
= 4(0.02)2 + (0.015)2 = 0.043
P
which is a 4.3% probable error for power. For current, first solve E = IR
for I to make I the dependant variable.
I=

E
R

2 
2
I
I
=
E +
R
E
R
2 
2

E
1
E +
R
=
R
R2

2I

E2
for fractional error
R2
 2  2  2
I
E
R
=
+
I
E
R
or
  p
I
= (0.02)2 + (0.015)2 = 0.025
I
which is a 2.5% estimate of the error for current.
Divide by I 2 =

Alternate Solution
Because the equations for power and current are purely multiplicative,
take ln value of both sides
E2
R
ln P = 2 ln E ln R
P =

differentiate

dP
dE dR
=2

P
E
R
Note that this equation shows that uncertainties in E are a factor of 2
more important than uncertainties in R. This is because the exponent of E
is 2 in the equation for P . Interpret differentials as s and square
11

 2
P

 
P

P
p

 2  2
E
R
= 2
+
E
R

= (2(0.02))2 + (0.015)2 = 0.043

P
Similarly, for the uncertainty in I, first solve the purely multiplicative
equation for I,
E
I=
R
Then take the logarithm of both sides,
ln I = ln E ln R
The derivation of this is
dE dR
dI
=

I
E
R
Interpret dI as I ect.
 2
I

 2
E

 2
R

Using the given numerical values,

p
I
= (0.02)2 + (0.015)2 = 0.025
I
Example 5
Two electrical signals are combined, each having a random uncorrelated
RMS (root-mean-square) noise level:
SIGNAL A
2.0 V with 3 mV RMS noise
SIGNAL B
6.0 V with 5 mV RMS noise
The basic equation is E = EA + EB . What will the magnitude of the
combined signal and RMS noise be?

12

Solution
The noise signals are random and uncorrelated with each other, so
their variances (i.e. mean square
p levels) add, and the RMS combined noise is
= (0.003 V)2 + (0.005 V)2 = 0.0058 V
The combined signal looks like this

Example 6
The volume of a block of gold 2 cm 5 cm 4 cm is measured with a
micrometer. Find the uncertainty in volume measurement if
a.
A random (careless) error 2% occurs in length L, width
W , and height H
b.
The micrometer reading is always 2% too small
this will illustrate the difference between random uncorrelated errors and
systematic dependant errors.

13

Solution
a.

Volume V = LW H

2 
2 
2
V
V
V
2
V =
L +
W +
H
L
W
H
= (W HL )2 + (LHW )2 + (LW H )2

Divide by V 2 = (LW H)2

r     
V
L
W
H
=
+
+
V
L
W
H
p
= (0.02)2 + (0.02)2 + (0.02)2
= 0.035
This is 3.5% in volume
b.
However if all three readings are too small by 2%, they combine
systematically
V
= 1.0 (0.98)3
V
= 1.0 0.941
= 0.059
This is 5.9% in volume
To minimize this systematic summing effect it is best to design experiments
and connect instruments so errors remain independent of each other.
Example 7
To illustrate how non-linear terms in an equation can alter the relative
importance of errors, consider the volume flow rate Q of water over a weir.
The water flows at depth H over a weir of width B.
Q = 3.33(BH 3/2 0.2H 5/2 )
The constant 3.33 has the units of m1/2 /s, the square root of gravitational
acceleration g.
14

If the absolute uncertainty in measuring B is twice as large as the absolute

uncertainty in H, which weir gives the smallest percentage uncertainty in Q.
Weir # 1 has: B1 =1.4 m, H1 =2.0 m
Weir # 2 has: B2 =3.0 m, H2 =1.0 m
Solution
For uncorrelated random uncertainties in B and H,

2 
2
Q
Q
2
B +
H
Q =
B
H
= (3.33H

3/2

 2


5 3/2
3
1/2
BH 0.2 H
H
B ) + 3.33
2
2
2

Set B = 2H and divide by Q2 to get

2Q
=
Q2

H 3/2
2H
BH 3/2 0.2H 5/2

2
+

3
BH 1/2 21 H 3/2
2
H
BH 3/2 0.2H 5/2

!2

For weir # 1
2Q
= 4.02H + 0.552H = 4.552H
2
Q
For weir # 2
2Q
= 0.5102H + 2.042H = 2.552H
2
Q
Thus, weir # 2 is the better because it produces the smallest flow
measurement error. So, a broad weir with shallow depth H is better than a
narrow weir operating at a larger flow depth.

15

Example 8
The moment of inertia of a cylinder of radius r, length L and density is
given by
mr2
Ix =
2
where the mass m = r2 L if the uncertainty in is 2% and the
uncertainty in L is 1%, how precisely must we measure r in order that the
uncertainty in Ix is less than 5%?
Solution
Combining the two equations, the dependant variable, Ix , is
4
r L
2
Taking ln values of both sides of this purely multiplicative equation
Ix =

ln(Ix ) = ln
Then

+ ln + 4 ln r + ln L
2

d
dr dL
dIx
=0+
+4 +
Ix

r
L

From which we can see that the uncertainty in radius r is four times as
important as uncertainties in of L.
Squaring
 2  2
 2  2

Ix
r
L
=
+ 16
+
Ix

r
L
 2
r
(0.05)2 = (0.02)2 + 16
+ (0.01)2
r
we find
r
= 0.011
r
so the radius r must be in error no more than 1.1%, and independent of
error in L (e.g. use a different ruler, and a different person to read it).

16