MATH 3080 Final 7.

4 The term on the left is the sample mean, and we can replace
Information and Efficiency the term on the right with k . Then, the Neyman Pearson
Fisher information I() is: Theorem tells us that the rejection the most powerful test for
6.4   our H0 and HA is of the form
Distributions based on a normal sample
I() = V ln {f (X; )} x k
If X1 2v1 , X2 2v2 , and independent, then
Alternatively, when we select k such that we have an = .05 So
X1 + X2 2v1 +v2
= .05 = P X k | H0 : = 10


2
 
If Z1 , Z2 , . . . , Zn are independent standard normal, then I() = E ln{f (X; )}
2 And now we solve for k
Z12 + Z22 + 2n
 

k 10

In () = V ln {f (X1 , X2 , . . . , Xn ; )} = nI() .05 = 1
If X1 , X2 , . . . , Xn are from a random sample of normal dist, 16/ 16
then The Cramer Rao inequality asserts the smallest possible
= k = 11.645
(n 1)S 2 / 2 2n1 variance (lower bound of variance) for an unbiased estimator T
= x 11.645
X 1
T = V (T )
S/ n In () so x 11.645 is the rejection region for the most powerful test
for testing H0 vs HA
7.2 9.4
Maximum Likelihood Estimation P-values 10.1
Maximize the liklihood function, f (x1 , x2 , . . . , xn ; 1 , . . . , n ) The P-value is the probability, assuming the null hypothesis, CI for two means
for your parameter of getting the test statistic or something even weirder. Testing two means when variance is known
Example: Suppose Xi s are from exponential distribution with s
parameter , then, 9.5 12 2
XY = + 2
Neyman - Pearson Theorem, UMP m n
f (x1 , . . . , xn ; ) = (ex1 ) (exn ) = n exi Neyman-Pearson Theorem: For testing a simple null
hypothesis against a simple alternative hypothesis, form the x y 0
Take ln: ln {f (x1 , . . . , xn ; )} = n ln() xi z= q 2
region such that the liklihood ratio is atleast k. Then the 1 2
Set m
+ n2
using the rejection region has the smallest type II error
probability for a given . If m > 40 and n > 40, then a 100(1 )% CI for a two sided
/[ln(likelihood)] = 0 = n/ xi = 0 test of 1 2 is :
= = n/xi = 1/x L(0 )
k s
L(A ) s21 s2
So the MLE is = 1/X x y z/2 + 2
Inside the critical region. m n
Invariance principle: If is the MLE for , then the MLE for Example:

any function h() is h() Suppose Xi s are a random sample from a normal population 10.2
with mean and variance 16. Then the most powerful test two sample t-tests
7.3 with the best critical region with a sample size n = 16 and For a t test,
Factorization Theorem, sufficiency significance level = .05 for testing H0 : = 10 vs x y 0
t= q 2
HA : = 15 is: s1 s2
if + n2
m
(32)16/2 exp (1/32)(xi 10)2
 
f (x1 , . . . xn ; ) = g(t(x1 , . . . , xn ); ) h(x1 , . . . , xn ) L( = 10) Is distributed by a t distribution with df of:
= k
L( = 15) (32)16/2 exp [(1/32)(xi 15)2 ]  2
Then t(x1 , . . . , xn ) is a sufficient statistic (doesnt depend on   s2 s2
1 1
+ 2
(xi 10)2 (xi 15)2 k


the parameter ) = exp m n
Example: Suppose f (xi ) = 1/. Then 32
(s1 /m)2 s2
2 /n

1

m1
+ n1
x2i 20xi + 1600 x2i + 30xi 3600


f (x1 , . . . , xn ; ) = 1/ 1/ = 1/n = exp
32 10.3
Introduce an indicator function where I(A) = 1 iff k Paired data
{x1 , . . . , xn } [0, ] (doesnt change the distribution)

1
 D = X Y , D = 1 2 = 0
= exp (10xi 2000) k

1
 32 d 0
f (x1 , . . . , xn ; ) = I(max{x 1 , . . . , xn } ) t=
n sD / n
= 10xi + 2000 32 ln(k)
I(0 min{x1 , . . . , xn }) xi 1 A two sided 100(1 )% CI for D is:
= (32 ln(k) 2000)
So max{x1 , . . . , xn } is a sufficient statistic 16 160 d t/2,n1 sD / n
10.5 From 6.4 again, Then (remember that AT B = B T A)
Inferences about Two Population Variances 21 /v1
Fv1 ,v2 T  = (Y X)T (Y X) (3)
22 /v2
S12 /12 T  = (Y T T X T )(Y X) (4)
F = Fdist , v1 = m 1, v2 = n 1
S22 /22 From (* c) and (** g), T  = Y T Y T X T Y Y T X + T X T X (5)
T T T T T T T T
H0 : 12 = 22 test statistic = f = s21 /s22 (c)/(I 1) 2I1 /(I 1)   = Y Y X Y X Y + X X (6)
= F(I1),I(J1)
HA : 12 > 22 f F,m1,n1 ( g)/I(J 1) 2
I(J1) /I(J 1) T  = Y T Y 2 T X T Y + T X T X (7)
HA : 12 < 22 f F,m1,n1 
(I1)s2
 Now take the partial w.r.t. , and set it equal to zero
X
/I 1 (minimizing T  = SSE)
HA : 12 6= 22 f F/2,m1,n1 2 /J
=  PI = F(I1),I(J1)
T 

2
or f F1/2,m1,n1 i=1 (J1)si
/I(J 1) = 0 2X T Y + 2X T X (8)
2

11.1 = 0 = 2X T Y + 2X T X (9)
ANOVA
Regression
T T
True regression equation: = 0 = X Y + X X (10)
Suppose we have 3 populations:
= X T Y = X T X (11)
I1 I2 I3 y = 0 + 1 x +
J1 2 3 2 = (X T X)1 X T Y = (X T X)1 X T X (12)
Estimated regression equation: T 1 T
J2 3 4 4 = (X X) X Y = (13)
J3 4 5 6 y = 0 + 1 x = (X T X)1 X T Y (14)
xi x1 = 3 x2 = 4 x3 = 4
s2i s21 = 1 s22 = 1 s23 = 4
(xi x )(yi y)2
If 1 =
H0 : 1 = 2 = 3 (xi x)2
is true, then we can estimate 2 using 0 = y 1 x
V (Xi ) 2
V (X i ) = =
J J
SST = (yi y)2 = (n 1)s2y
2
s2X = 2 J sX
2
y y)2 = (n 1)s2y
SSR = (
J i i

From 6.4, y yi ) 2
SSE = (
(n 1)s2 SST = SSR + SSE
2(n1)
2 SSR
r2 =
(I 1)s2 SST
= X
2(I1)
2 1
X t=  
S
(I 1)s2 XY
SSX
= X
2(I1) (*c)
2 /J
Derivation of OLS (matrix form): Our equation is:
If H0 is false, then we can estimate 2 using
PI Y = X +  (1)
2
i=1 si
2 Where Y is a column matrix of the response variable, X is the
I
matrix of a column of ones (for the intercept) and the columns
From 6.4, of predictor variables which correspond with Y , is the
(n 1)s2 column matrix of coefficients (starting with the intercept, 0 )
2(n1) Then, our error term is:
2
(J 1)s2i  = Y X (2)
= 2(J1)
2
PI 2 Our SSE (sum of squared errors) is:
i=1 (J 1)si
= 2I(J1) (**g)
2 SSE = T