GLRTs and MLEs

Let o be the MLE of and let o be the MLE of 0.

Then the LRT statistic is

L(o | x)
(x)
L( | x)
Example: Normal GLRT

Let X1,...Xn be a random sample from a N( , 1) population .

Test : H0 : 0 versus H1 : 0.
Then the LRT statistic is :
n
( xi 0 ) 2 / 2
L(o | x) (2 ) -n/2 e i 1
(x) n
L( x | x) ( xi x ) 2 / 2
(2 ) -n/2 e i 1
n
2
n
2
( xi 0 ) ( xi x )
i 1 i 1
e 2

n n
Note that ( xi 0) ( xi x) 2 n( x 0) 2 .
2

i 1 i 1

n ( x 0 ) 2
(x) e 2
Example: Normal GLRT (contd)

We will reject H0 if (x) c. We have:

n ( x 0 ) 2
{x : ( x) c} {x : e 2
c}

{x : ( x 0) 2 (2 ln c) / n}

{x :| x 0 | (2 ln c) / n }
Therefore, the LRTs are those tests that reject H0 if
where 0 (2 ln c) / n .
the sample mean differs from the value 0 by
more than

(2 ln c) / n .
Example: Size of the Normal GLRT

Chose c such that sup 0 P ( (X) c) .

For the previous example, we have :
0 { 0}, and n ( X 0) ~ N (0,1) if 0.
The test :
z / 2
reject if | X 0 | ,
n

where z / 2 satisfies P(Z z / 2)
2
wi th Z ~ N(0,1), is a size test.
Sufficient Statistics and GLRTs

Theorem: If T(X) is a sufficient statistic for ,

and *(t) and (t) are the LRT statistics
based on T and X, respectively, then
*(T(x))=(x) for every x in the sample
space.
Example: Normal GLRT with unknown
variance

Let X1,...Xn be a random sample from a N( , 2 ) population .

Test : H0 : 0 versus H1 : 0. (Note : 2 is a nuisance parameter) .
Then the LRT statistic is :
max L( , 2
| x) max L( , 2
| x)
{ , 2 : 0 , 2 0} { , 2 : 0 , 2 0}
(x)
max L( , 2
| x) 2
{ , 2 : , 2 0} L( , | x)

1 if 0
2
L( 0, 0 | x)
if 0
2
L( , | x)
which is equivalent to a test based on the Student' s t statistic.
Example: Normal GLRT with unknown
variance (contd)

If 0, then
n
2 ( xi 0 ) 2 / 2 0
L( 0, 0 | x) (2 0) - n/2 i 1
e
(x)
2 n
2
2 ( xi ) / 2
L( , | x) (2 ) - n/2 i 1
e

n n 2
n
( xi 0 )
n
2 ( xi 0 ) 2 i 1

2
(2 0) - n/2
e i 1
( 0) - n/2


0
n n
2
( x )2
( ) -n/2
i

2 2
n
( xi ) 2 i 1
(2 ) - n/2
e i 1
Example: Normal GLRT with unknown
variance (contd)

Note that :
21 n n 1 2
( xi x ) 2 S
n i 1 n
and
2 1 n 1 n n 1 2
0 ( xi 0) ( xi x ) 2 n( x 0) 2
2
S ( x 0) 2
n i 1 n i 1 n
Therefore, (x) c when

2 n 1 2 n 1
S
n n
(x) c' and x 0;
0 n 1 S 2 ( x 0) 2 n 1 ( x 0) 2
n
n S2
X - 0
this is analogous to rejecting H0 when tn 1, .
2
S
n
 Asymptotic Distribution of the
GLRT Simple H0
Theorem : For testin g H0 : 0 versus Ha : 0, suppose X1,...Xn

are iid f ( x | ), is the MLE of , and f ( x | ) satisfies the
the following regularity conditions :
(1) The parameter is identifiab le; i.e., if ' , then f ( x | ) f ( x | ' ).
(2) The densities f ( x | ) have some common support, and f ( x | )
is differenti able in .
(3) The parameter space contains an open set of which the true
parameter value 0 is an interior point.
(4) x X , the density f ( x | ) is three times differenti able with respect
to , the third derivative is continuous in , and f ( x | )dx can
be differenti ated three times under the integral sign.
(5) , c 0 and a function M( x) (both depend on 0) such that :
3
log f ( x | ) M( x) x X , 0 - c 0 c,
3
wi th E 0[M(X)] .