Analysis of Variance and Design g of Experiments Experimentsp -I

MODULE II
LECTURE - 8

GENERAL LINEAR HYPOTHESIS AND ANALYSIS OF VARIANCE

Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur

Case 3: Test of H 0 : L =
Let us consider the test of hypothesis related to a linear parametric function. Assuming that the linear parameter function
L is estimable where L = ( 1 , 2 ,..., p ) is a p 1 vector of known constants and = ( 1 , 2 ,..., p ) . The null hypothesis

of interest is

H 0 : L =
where is some specified constant. Consider the set up of linear model Y = X + where Y = (Y1 , Y2 ,..., Yn ) follows N ( X , 2 I ). The maximum likelihood estimators ti t of f and d 2 are
= ( X X ) 1 X y

and

)( y X ), respectively 2 = (y X ) respectively.

1 n

with The maximum likelihood estimate of estimable L is L

) = L E(L ' ) = 2 L( X X )1 L Cov( L ~ N L , 2 L( X X )1 L L

and

2 n

~ 2 (n p)
2 n
are also independently distributed.

and assuming X to be the full column rank matrix. Further, L

Under H 0 : L = , the statistic

t=

) ( n p )( L 2 L( X X ) 1 L n

follows a t-distribution with (n p) degrees of freedom. So the test for H 0 : L = against H1 : L rejects H 0 whenever

t t

(n p )

where t1 (n1 ) denotes the upper 100 % points on t-distribution with n1 degrees of freedom.

Case 4: Test of H 0 : 1 = 1 , 2 = 2 ,..., k = k

Now we develop the test of hypothesis related to more than one linear parametric functions. Let the ith estimable linear parametric function is i = L'i and there are k such functions with Li and both being p 1 vectors as in the Case 3. Our interest is to test the hypothesis

H 0 : 1 = 1 , 2 = 2 ,..., k = k
where 1 , 2 ,..., k are the known constants. Let = (1 , 2 ,..., k ) and = (1 , 2 ,.., k ). Then H 0 is expressible as H 0 : = L = where L is a k p matrix of constants associated with L1 , L2 ,..., Lk .
= L' The maximum likelihood estimator of i is : i i

= ( , ,..., ) = L . Then 1 2 k
) = Also E ( ) = 2V Cov (

where V = (( L'i ( X X ) 1 L j ))

where ( L'i ( X X ) 1 L j ) is the (i, j )th element of V. Thus

)V 1 ( ) (

2
follows a 2 distribution with k degrees of freedom and

2 n

2 follows a distribution with (n - p) degrees of freedom where

1 )( y X ) is the maximum likelihood estimator of 2 . 2 = (y X n

5
1 2 Further ( )V ( ) and n are also independently distributed.

Thus under H 0 : =

)V 1 ( ) ( 2 k 2 n 2 (n p )
or

)V 1 ( ) n p ( k 2 n

follows FF distribution with k and (n p) degrees of freedom. freedom So the hypothesis H 0 : = is rejected against

H1 : At least one i for i = 1, 2,..., k whenever F F1 (k , n p) where F1 (k , n p) denotes the upper 100 % points
of F-distribution with k and (n p) degrees of freedom.

One-way classification with fixed effect linear models of full rank

The objective in the one way classification is to test the hypothesis about the equality of means on the basis of several samples which have been drawn from univariate normal populations with different means but the same variances. Let there be p univariate normal populations and samples of different sizes are drawn from each of the population. Let
2 yij ( j = 1, 2,..., ni ) be a random sample from the ith normal population with mean i and variance , i = 1, 2,..., p , i.e.,

Yij ~ N ( i , 2 ), j = 1, 2,..., ni ; i = 1, 2,..., p.

The random samples from different population are assumed to be independent of each other. These observations follow the set up of linear model
Y = X +

where

Y = (Y11 , Y12 ,..., Y1n1 , Y21 ,..., Y2 n2 ,..., Y p1 , Y p 2 ,..., Y pn p ) ' y = ( y11 , y12 ,..., y1n1 , y21 ,..., y2 n2 ,..., y p1 , y p 2 ,..., y pn p ) '

= ( 1 , 2 ,..., p ) = (11 , 12 ,..., 1n , 21 ,..., 2 n ,..., p1 , p 2 ,..., pn ) '

1 2 p

7
1 0...0 n1 values 1 0 0 0 1...0 n values 2 X = 0 1...0 0 0...1 n p values 0 0...1

1 if i occurs in the j th observation x j xij = or if effect i is present in x j 0 if effect i is absent in x j n = ni .

i =1 p

So X is a matrix of order n p, is fixed and

' first n1 rows of are 1 = (1, 0, 0,..., 0), ' next n2 rows of are 2 = (0,1, 0,..., 0) ' and similarly the last n p rows of are p = (0, 0,..., 0,1).

2 Obviously, rank ( X ) = p, E (Y ) = X and Cov(Y ) = I .

This completes the representation of a fixed effect linear model of full rank.

Th null The ll h hypothesis th i of fi interest t ti is H 0 : 1 = 2 = ... = p = (say) ( ) and H 1 : At least one i j (i j ) where and 2 are unknown. W would We ld d develop l h here the h lik likelihood lih d ratio i test. I It may b be noted d that h the h same test can also l b be d derived i d through h h the h l least squares method. This will be demonstrated in the next module. This way the readers will understand both the methods. We already have developed the likelihood ratio for the hypothesis H 0 : 1 = 2 = ... = p in the case 1.
) d The e whole o e pa parametric a et c space is s a ( p + 1) dimensional e s o a space .

= {( , 2 ) : < i < , 2 > 0, i = 1, 2,..., p} .

Note that there are ( p + 1) parameters 1 , 2 ,..., p and 2 . Under H 0 , reduces to two dimensional space as

= {( , 2 ); < < , 2 > 0} .

The likelihood function under is
1 1 2 L( y | , 2 ) = exp 2 2 2 2
2 n p ni

( y
i =1 j =1

ij p

i ) 2
ni ij

n 1 L = ln L( y | , ) = ln (2 2 ) 2 2 2 L = 1 =0 i i ni

( y
i =1 j =1

i )2

y
j =1

ni

ij

= yio

L 1 p ni 2 = = 0 ( yij yio ) 2 . 2 n i =1 j =1

9
The dot sign in yio indicates that the average has been taken over the second subscript j. The Hessian matrix of

(o)

2 which second order partial derivation of ln L with respect to i and 2 is negative definite at = y io and 2 =

ensures that the likelihood function is maximized at these values. Thus the maximum value of L( y | , 2 ) over is
1 1 2 Max L( y | , ) = exp 2 2 2 2
2 n

( y
i =1 j =1 n /2

ni

ij

)2 i

n = p ni 2 2 ( yij yio ) i =1 j =1

n p . exp 2

The likelihood function under

n 2

is

1 1 2 exp L( y | , ) = 2 2 2 2
and

( y
i =1 j =1 p ni

ni

ij

)2

1 n l L( y | , ) = ln(2 ln l (2 2 ) 2 2 2
2

( y
i =1 j =1

ij

)2 .

The normal equations and the least squares estimates are obtained as follows:
p ni ln L( y | , 2 ) =1 =0 yij = yoo n i =1 j =1

ln L( y | , 2 ) 1 p ni 2 = 0 = ( yij yoo ) 2 . 2 n i =1 j =1

10
The maximum value of the likelihood function over under H 0 is
1 1 2 Max L ( y | , 2 ) = exp 2 2 2 2
n

(y
i =1 j =1 n/2

ni

ij

)2

n = ni p 2 2 ( y ij y oo ) i =1 j =1

n exp . 2

The likelihood ratio test statistic is

Max L ( y | , 2 ) Max L ( y | , 2 )
n/2

p ni 2 ( yij yio ) =1 =1 = i p jni 2 ( yij yoo ) i =1 j =1

We have

(y
i =1 j =1

ni

ij

y oo ) = ( yij yio ) + ( yio y oo )

2 i =1 j =1 ni p p

ni

= ( yij yio ) 2 + ni ( yio y oo ) 2 .

i =1 j =1 i =1

11

Thus

( y
i =1 j =1

ni

ij

yi ) 2 + ni ( yio yoo ) 2 I =1 p ni ( yij yio ) 2 i =1 j =1

p

n 2

q = 1 + 1 q2

n 2

where
q1 = ni ( yio yoo ) 2
i =1 p

q2 = ( yij yio ) 2 .
i =1 j =1

ni

Note that if the least squares principal is used, then

q1 q2

: sum of squares due to deviations from H 0 or the between population sum of squares, : sum of squares due to error or the within population sum of squares,

q1+q2 : sum of squares due to H 0 or the total sum of squares.

Using the Theorems 6 and 7 7, let
Q1 = ni (Yio Yoo ) 2
i =1 p p

Q2 = Si2
i =1

where Si2 = (Yij Yio ) 2 ,

i =1 ni

Yoo =

1 p ni Yij , n i =1 j =1

Yio =

1 ni

Y
j =1

ni

ij

12
then under H 0

Q1
2

~ 2 ( p 1) ~ 2 (n p)

and

Q2
2

Q1 and Q2

are independently distributed.

Thus under H 0 Q1 2 p 1 ~ F ( p 1, n p). Q2 2 n p The likelihood ratio test reject H 0 whenever

q1 >C q2
where the constant C = F1 ( p 1, n p ).

13
The analysis of variance table for the one way classification in fixed effect model is

Source of variation
Between populations

Degrees of freedom
p -1

Sum of squares
q1

Mean squares
q1 p 1

F - value
n p q1 . p 1 q2

C = F1 ( p, n p )

Within populations

n-p

q2
H0 : = 0

q2 (n p)

Total

n-1

q1 + q2

Note that Q E 2 =2 n p Q E 1 =2 + p 1 1 p = i . p i =1

(
i =1

) ;

p 1