Академический Документы
Профессиональный Документы
Культура Документы
• Point estimation
• Desirable properties of point estimations
• Interval estimations
• Confidence intervals
Estimator
Assume that we have a sample (x1,x2,,,xn) from a given population. All parameters of
the population are known except some parameter θ. We want to determine
from the given observations unknown parameter - θ. In other words we want to
determine a number or range of numbers from the observations that can be
taken as a value of θ.
Estimator – is a method of estimation.
Estimate – is a result of an estimator
Point estimation – as the name suggests is the estimation of the population
parameter with one number.
Problem of statistics is not to find estimates but to find estimators. Estimator is not
rejected because it gives one bad result for one sample. It is rejected when it
gives bad results in a long run. I.e. it gives bad result for many, many samples.
Estimator is accepted or rejected depending on its sampling properties.
Estimator is judged by the properties of the distribution of estimates it gives
rise.
Properties of estimator
Since estimator gives rise an estimate that depends on sample points (x1,x2,,,xn)
estimate is a function of sample points. Sample points are random variable
therefore estimate is random variable and has probability distribution. We want
that estimator to have several desirable properties like
2. Consistency
3. Unbiasedness
4. Minimum variance
Note that estimator is a sample statistic. I.e. it is a function of the sample elements.
Properties of estimator: Consistency
Here we used the fact that expectation and summation can change order (Remember
that expectation is integration for continuous random variables and summation
for discrete random variables.) and the expectation of each sample point is
equal to the population mean.
Knowledge of population distribution was not necessary for derivation of
unbiasedness of the sample mean. This fact is true for the samples taken from
population with any distribution for which the first moment exists..
Example of biased estimator: Sample variance.
Given sample of size n from the population with unknown mean (θ) and variance (σ
2
) we estimate mean as we already know and variance (intuitively) as:
1 n 1 n 2
tn = ∑ ( xi − x ) = ∑ xi − x
2 2
n i =1 n i =1
What is the bias of this estimator? We could derive distribution of tn and then use it
to find expectation value. If population has normal distribution then it would
give us multiple of χ2 distribution with n-1 degrees of freedom. Let us use a
direct approach:
1 n 1 n 1 n
1 n n
1
E ( tn ) = ∑
n i =1
E ( xi
2
) − E (( ∑
n i =1
xi ) 2
= E ( x 2
) −
n 2
E ( ∑ x
i =1, j =1
x
i j ) = E ( x 2
) −
n 2
( E ( ∑
i =1
xi
2
) −E ( ∑
i≠ j
xi x j )) = E ( x 2 ) − 2 ( nE ( x 2 ) − n( n − 1) E ( x ) 2 )
n
1 n −1 n −1 n −1 2
= E( x2 ) − E( x2 ) − E ( x )2 = ( E ( x 2 ) − E ( x )2 ) = σ
n n n n
Sample variance is not an unbiased estimator for the population variance. That is
why when mean and variance are unknown the following equation is used for
sample variance: s 2 = 1 n ( x − x )2
n − 1 i =1
∑ i
Property of estimator: mean square error and bias
Expectation value of the square of the differences between estimator and the
expectation of the estimator is called its variance:
Vθ = E (tn − E (tn )) 2
Exercise: What is the variance of the sample mean.
As we noted if estimator for θ is tn then difference between them is error of the
estimation. Expectation value of this error is bias. Expectation value of square
of this error is called mean square error (m.s.e.):
M θ = E ( tn − θ ) 2
One of the estimator is plug-in. It has only intuitive bases. If parameter we want to
estimate is expressed like θ=t(F) then estimator taken as θˆ = t ( Fˆ ) . Where F is
thepopulation distribution and F̂ is its sample equivalent.
Example: population mean is calculated as:
θ = ∫ xf ( x )dx
Since sample is from the population with the density of distribution f(x) sample mean
is plug-in estimator for the population mean.
Exercise: What is the plug-in estimator for population variance? What is the plug-in
estimator for covariance. Hint: Population variance and covariance are
calculated as: 2
σ = ∫ ( x − µ ) f ( x)dx and
2
cov( X , Y ) = ∫ ( x − µ x )( y − µ y ) f ( x, y )dxdy
Replace the integration with summation and divide by the number of elements in the
sample. Since sample was drawn from the population with a given distribution
it is not necessary to multiply by f(x)
Least-squares estimator
Another well known and popular estimator is the least-square estimator. If we have a
sample and we think that (because of some knowledge we had before) all
parameters of interest are inside the mean value of the population then least
squares methods estimates by minimising the square of the differences between
observations and mean value:
n
∑ w ( x − µ (θ ))
i =1
i i
2
→ min
Exercise: Verify that if only unknown parameter is the mean of the population and all
wi are equal to each other then the least-squares estimator will result in the
sample mean.
Interval estimation
Estimation of the parameter is not sufficient. It is necessary to analyse and see how
confident we can be about this particular estimation. One way of doing it is
defining confidence intervals. If we have estimated θ we want to know if the
“true” parameter is close to our estimate. In other words we want to find an
interval that satisfies following relation:
P (GL < θ < GU ) ≥ 1 − α
I.e. probability that “true” parameter θ is in the interval (GL,GU) is greater than 1-α.
Actual realisation of this interval - (gL,gU) is called a 100(1- α)% of confidence
interval, limits of the interval are called lower and upper confidence limits. 1-
α is called confidence level.
Example: If population variance is known (σ2) and we estimate population mean then
x−µ
Z= is normal N (0,1)
σ/ n
We can find from the table that probability of Z is more than 1 is equal to 0.1587.
Probability of Z is less than -1 is again 0.1587. These values comes from the
tables of the standard normal distribution.
Interval estimation: Cont.
Now we can find confidence interval for the sample mean. Since:
P ( −1 < Z < 1) = P ( Z < 1) − P ( Z < −1) = 1 − P ( Z > 1) − P( Z < −1) = 1 − 2 * 0.1587 = 0.6826