Lecture 9

Estimation Theory
 25/1/2016
• The theory of statistical inference may be defined to be
those methods by which one make inferences or
generalization about a population. Statistical inference may
be divided into two major areas: estimation and test of
hypotheses.
• Classical Methods of Estimation
• An estimate of a population parameter may be given as a
point estimate or as interval estimate.
• The set of all possible actions that can be taken in an
estimation problem is called the action space or decision
space.
• A statistic Θ is said to be an unbiased
estimator of the parameter θ if μΘ = E(Θ) =θ.
• If we consider all possible unbiased estimators
of some parameters θ, the one with the
smallest variance is called the most efficient
estimator of θ.
 Estimating The Mean

• A point estimator of the population mean μ is

given by the statistic XX . The sampling
distribution of XX is centered at μ and in most
application the variance is smaller than that of
any other estimator.
Confidence Interval For μ; σ known

A (1-α)100% confidence interval for μ is

where XX is the mean of a sample of size n from

a population with known variance σ2 and Zα/2
is the value the standard normal distribution
leaving an area α/2 to the right.
 Example

• The mean and standard deviation for the

quality point average of a random sample of
36 college seniors are calculated to be 2.6 and
0.3, respectively. Find the 95% and 99%
intervals for the entire senior class.
Solution
XX = 2.6, s =0.3,the z value leaving an area to
the right = 0.05/2 = 0.025 and therefore an
area of 0.975 to the left, is z0.025 = 1.96 from table.
Hence 95% confidence interval is

which reduces to 2.5˂ μ˂ 2.70

To find 99% confidence interval, we find z value
leaving an area of 0.005 to the right and 0.995 to the left,
using the table, z0.005 = 2.575 and the 99% confidence interval is

2.47 ˂ μ ˂ 2.73
• If XX is used as an estimate of μ, we can be (1-
α)100% confident that the error will be less
than Zα/2(σ/√n).
• If XX is used as an estimate of μ, we can be (1-
α)100% confident that the error will be less
than a specified amount e when the sample
size is
• n = (Zα/2 σ/e)2
 Example

• How large a sample is required in the above

example if we want to be 95% confident that
our estimate of μ is off by less than 0.05?
• Solution
• The sample standard deviation s = 0.3 will be
used for σ. Then
• n =(1.96x0.3/.05)2 = 138.3
• We use n = 139.
Confidence Interval For μ; σ Unknown And n˂30

A (1-α)100% confidence interval for μ is

where xx and s are the mean and standard deviation

, respectively, of a sample of size n˂30 from an approximate
normal population, and tα/2 is the value of the t distribution,
with ν = n-1 degrees of freedom, leaving an area of α/2 to the right.
 Example

• The contents of seven similar containers of

sulfuric acid are 9.8, 10.2, 10.4, 9.8, 10.0, 10.2,
and 9.6 liters. Find 95% confidence interval for
the mean off all such containers, assuming an
approximate normal distribution.
 Solution

• The sample mean and standard deviation for the

given data are
• xx = 10.0 and s = 0.283
• From t tables t0.025 = 2.447 for ν = 6 degrees of
freedom. Hence the 95% confidence interval for μ
is
• 10.0 – (2.447)(0.283/√7)˂μ˂ 10.0 + (2.447)
(0.283/√7)
• 9.74 ˂ μ ˂ 10.26
Estimating the difference between two means
Confidence Interval For μ1 – μ2; σ12 and σ22 Known

A (1-α)100% confidence interval for μ1 – μ2 is

 Example

• A standardized chemistry test was given to 50

girls and 75 boys. The girls made an average
grade of 76 with a standard deviation of 6,
while the boys made an average grade of 82
with standard deviation of 8. Find a 96%
confidence interval for the difference μ1 – μ2,
where μ1 is the mean score of all boys and μ2
is the mean score of all girls who might take
this test.
 Solution

• The point estimate of μ1 – μ2 is x1x - x2x = 82 -76 = 6.

• Since n1 and n2 are both large, we can substitute s1
= 8 for σ1 and s2 = 6 for σ2. Using α = 0.04, we find
z0.02 = 2.054 from table. Substituting these values
• 6 -2.054√(64/75 +36/50)˂ μ1 – μ2˂ 6 +
2.054√(64/75 +36/50)
• 3.42 ˂ μ1 – μ2˂ 8.58
Small Sample Confidence Interval μ1 – μ2, σ12 = σ22 But Unknown

A (1-α)100% confidence interval for μ1 – μ2 is

Where x1x and x2x are the means of small

independent samples of size n1 and n2,
respectively. Sp is the pooled standard
deviation and tα/2 is the value of the t
distribution with ν = n1 + n2 -2 degrees
of freedom leaving an area of α/2 to the right.
Sp2 = ((n1-1)S12 + (n2 -1)S22)/(n1 + n2 -2)
 Example
• In a batch chemical process, two catalysts are being
compared for their effect on the output of the process
reaction. A sample of 12 batches are prepared using
catalyst 1 and a sample of 10 batches were obtained using
catalyst 2. The 12 batches for which catalyst 1 was used
gave an average yield of 85 with a sample standard
deviation of 4, while the average for the second sample
gave an average of 81 and a sample standard deviation of
5. Find 90% confidence interval for the difference between
the population means assuming the population to be
approximately normally distributed with equal variances.
Solution

The point estimate of μ1 – μ2 = x1x - x2x = 85 – 81 = 4

Sp2 = ((n1-1)S12 + (n2 -1)S22)/(n1 + n2 -2)
= ((11)(16) + (9)(25))/(12 + 10 -2) = 20.05
Sp = 4.478 , α = 0.1, ν = 20 degrees of freedom, tα/2 = 1.725

Which simplified to 0.69 ˂ μ1 – μ2˂ 7.31

• If the population variances are considerably
different we still obtain good results when the
populations are normal, provided that n1 = n2.
Therefore, in a planned experiment, one
should make every effort to equalize the size
of the samples.
Small Sample Confidence Interval μ1 – μ2,
σ12 ≠ σ22 and Unknown
A (1-α)100% confidence interval for μ1 – μ2 is

Where x1x and s12,and x2x and s22, are the means
and variances of small independent samples of size n1
and n2, respectively, from approximate normal
distributions and tα/2 is the value of the t distribution with

degrees of freedom leaving an area of α/2 to the right.

 Example
• Records for the past 15 years have shown the
average rainfall in certain region of the country for
the month of May to be 4.93 cm with standard
deviation of 1.14. A second region of the country
has had an average rainfall in May of 2.64 cm of rain,
with a standard deviation of 0.66 cm during the past
10 years. Find a 95% confidence interval for the
difference of the true average rainfalls in these two
regions, assuming that the observations came from
normal population with different variances.
Solution

x1x = 4.92 and s1= 1.14 and n1 = 15 for the first region.
For the second region x2x = 2.64, s2 =0.66, and n2 = 10.

The point estimate of μ1 – μ2 = x1x - x2x = 4.93 – 2.64 = 2.29.

Using α = .05, we find t0.025 = 2.069 for ν =23 degrees of
freedom from tables.
Substituting the above values in the following equation

Which simplified to 2.02 ˂ μ1 – μ2˂ 2.56

Confidence Interval For μ1 – μ2 = μD For Paired Observation

A (1-α)100% confidence interval for μD is

Where dd and sd are the mean and standard deviation

of the difference of n pairs of measurements and tα/2
is the value of the t distribution with ν = n-1 degrees of
freedom, leaving an area of α/2 to the right.
 Example

• Twenty college freshmen were divided into 10 pairs,

each member of the pair having approximately the
same IQ. One of each pair was selected at random
and assigned to a mathematics section using
programmed materials only. The other member of
each pair was assigned to a section in which the
professor lectured. At the end of the semester each
group was given the same examination and the
following results were recorded:
Pair Programmed Lectures d
materials
1 76 81 -5
2 60 52 8
3 85 87 -2
4 58 70 -12
5 91 86 5
6 75 77 -2
7 82 90 -8
8 64 63 1
9 79 85 -6
10 88 83 5
Find a 98% confidence interval for the true difference
in the average grades of the two learning procedures.
Solution
Since the observations are paired, μ1 – μ2 = μD and
the point estimate of μD is given by dd = -1.6.
The variance sd2 of the sample difference
sd2 = (n∑di2 – (∑d)2)/(n(n-1)) = ((10)(392)-(-16)2)/(10)(9)=40.7
Taking the square root, we obtain sd = 6.38. Using α = 0.02,
we find t0.01 = 2.821 for ν =10-1=9 degrees of freedom.
Therefore substituting in the formula

-7.29 < μD < 4.09

Estimating The Variance

An unbiased estimate of the population variance σ2

is provided by the sample variance s2.
Hence the statistic S2 is called an estimator of σ2.

Confidence Interval For σ2

A (1-α)100% confidence interval for the variance σ2
of a normal population is

Where s2 is the variance of a random sample of size n,

and Χ and Χ are the value of a chi-square distribution
with ν = n-1 degrees of freedom, leaving area of α/2
and 1-α/2, respectively to the right.
 Example

• The following are the weights, in decigrams, of

10 packages of grass seed distributed by a
certain company:46.4, 46.1, 45.8, 47.0, 46.1,
45.9, 45.8, 46.9, 45.2, and 46.0. Find a 95%
confidence interval for the variance of all such
packages of grass seed distributed by the
company.
Solution

S2 = (n∑xi2 – (∑xi)2)/n(n-1)=((10)(21,273.12)-(461.2)2)/(10)(9) = 0.286

To obtain 95% confidence, α=0.05, from chi-square table with
ν =9 degrees of freedom, we find Χ0.025 =19.023 and Χ0.975 = 2.700,
substituting in the formula

0.135 < σ2 < 0.953

Estimating The Ratio Of Two Variances

A point estimate of the ratio of two variances σ12/σ22 is

given by the ratio of s12/s22 of the sample variances.
Hence the statistic S12/S22 is called an estimator of σ12/σ22.
Confidence Interval For σ12/σ22
A (1-α)100% confidence interval for the ratio σ12/σ22 is

Where s12 and s22 are the variances of independent

samples of size n1 and n2 respectively, from normal
population, fα/2 (ν1,ν2) is the value of F distribution with
ν1 = n1-1 and ν2 = n2 -1 degrees of freedom leaving an area
of to the right, and fα/2 (ν2,ν1) is a similar f value with
 Example
• A standardized placement test in mathematics
was given to 25 boys and 16 girls. The boys
made an average grade of 82 with standard
deviation of 8, while the girls made an average
grade of 78 with standard deviation of 7. Find a
98% confidence interval for σ/σ and σ/σ, where
σ and σ are the variances of the populations of
grades for all boys and girls, respectively, who
at some time have taken or will take this test.
Solution

We have n1 = 25, n2 =16, s1 =8, s2 = 7. For 98% confidence

interval α = .02. Using F table, f.01(24,15) = 3.29, and f0.01(15,24) = 2.89.
Substituting the values in the following formula

which simplified to
0.397 < σ12/σ22 < 3.75
Taking the square root of the confidence limit
0.630 < σ1/σ2 < 1.943
 Tolerance Limits

• For a normal population of measurements with

unknown mean μ and unknown standard deviation
σ, tolerance limits are given by xx ± ks, where k is
determined so that one can assert with 100γ%
confidence that the given limits contain at least
the proportion 1- α of the measurements.
• Values of k are given in tables for 1-α = 0.90, 0.95,
0.99; γ=0.95, 0.99; and for selected values of n
from 2 to 1000
 Example

• A machine is producing metal pieces that are

cylindrical in shape. A sample of these pieces
is taken and the diameters are found to be
1.01, 0.97, 1.03, 1.04, 0.99, 0.98, 0.99, 1.01,
and 1.03 centimeters. Find the 99% tolerance
limits that will contain 95% of the metal pieces
produced by this machine, assuming an
approximate normal distribution.
 Solution

• The sample mean and standard deviation for the data are
• xx = 1.0056 and s = 0.0245
• from table for n = 9, γ = 0.99, and 1 – α = 0.95, we find k=
4.550.
• Hence the 99% tolerance limits are given by
• 1.0056 ± (4.550)(0.0245)
• Or o.894 and 1.117
• That is we are 99% confident that the tolerance interval
from 0.894 to 1.117 will contain 95% of the metal pieces
produced by this machine.