1

Alternate Form for CRLB

2
) ; ( ln
1
)

var(

x p
E
See Appendix 3A
for Derivation
Sometimes it is easier to find the CRLB this way.
This also gives a new viewpoint of the CRLB:
From Gardners Paper (IEEE Trans. on Info Theory, July 1979)
Consider the Normalized version of this form of CRLB
Posted
on BB

2
2
2
) ; ( ln
1 )

var(

x p
E
Well derive
this in a way
that will re-
interpret the
CRLB
2
Consider the Incremental Sensitivity of p(x; ) to changes in :
If +, then it causes p(x; ) p(x; + )
How sensitive is p(x; ) to that change??

= =

) ; (
) ; (
in change %
) ; ( in change %
) ; (
) ; (
) (
~

x
x x
x
x
x
p
p p
p
p
S
p
Now let 0:

= =

) ; ( ln
) ; (
) ; (
) (
~
lim ) (
0
x
x
x
x x
p
p
p
S S
p p
Recall from Calculus:
x
x f
x f x
x f

) (
) (
1 ) ( ln
| |
)
`

2
2
2
2
2
) (
1
) ; ( ln
1 )

var(
x
x
p
S E
p
E

Interpretation
Norm. CRLB =
Inverse Mean
Square
Sensitivity
3
Definition of Fisher Information
The denominator in CRLB is called the Fisher Information I( )
It is a measure of the expected goodness of the data for the
purpose of making an estimate

=
2
2
) ; ( ln
) (

x p
E I
Has the needed properties for info (as does Shannon Info):
1. I( ) 0 (easy to see using the alternate form of CRLB)
2. I( ) is additive for independent observations
follows from:
| |

=

=
n n
n x p n x p p ) ]; [ ( ln ) ]; [ ( ln ) ; ( ln x
If each I
n
( ) is the same: I( ) = NI( )
4
3.5 CRLB for Signals in AWGN
When we have the case that our data is signal + AWGN then
we get a simple form for the CRLB:
Signal Model: x[n] = s[n; ] + w[n], n = 0, 1, 2, , N-1
White,
Gaussian,
Zero Mean
Q: What is the CRLB?
First write the likelihood function:
( )
( )

=
1
0
2
2 2 /
2
] ; [ ] [
2
1
exp
2
1
) ; (
N
n
N
n s n x p

x
Differentiate Log LF twice to get:
( )

=

)

1
0
2
2
2
2 2
2
] ; [ ] ; [
] ; [ ] [
1
) ; ( ln
N
n
n s n s
n s n x p

x
Depends on
random x[n]
so must take
E{}
5
{ }
2
1
0
2
1
0
2
2
2
0
] ; [
2 2
2
] ; [
] ; [ ] ; [
] ; [ ] [
1
) ; ( ln

=
=

|
|
|
.
|

\
|
=

N
n
N
n
n s
n s
n s n s
n s n x E p E
! ! ! " ! ! ! # $
" # $
x
Then using this we get the CRLB for Signal in AWGN:

1
0
2
2
] ; [
)

var(
N
n
n s

2
] ; [

n s
Note: tells how
sensitive signal is to parameter
If signal is very sensitive to parameter change then CRLB is small
can get very accurate estimate!
6
Ex. 3.5: CRLB of Frequency of Sinusoid
Signal Model:
1 , , 2 , 1 , 0 0 ] [ ) 2 cos( ] [
2
1
= < < + + = N n f n w n f A n x
o o

0 0.1 0.2 0.3 0.4 0.5
0
2
4
6
x 10
-4
f
o
(cycles/sample)
C
R
L
B


(
c
y
c
l
e
s
/
s
a
m
p
l
e
)
2
0 0.1 0.2 0.3 0.4 0.5
0.01
0.015
0.02
0.025
f
o
(cycles/sample)
C
R
L
B
1
/
2


(
c
y
c
l
e
s
/
s
a
m
p
l
e
)
Error in
Book
Bound on
Variance
Bound on
Std. Dev.
| |

=
+

1
0
2
) 2 sin( 2
1
)

var(
N
n
o
n f n SNR

Signal is less
sensitive if f
o
near 0 or
7
3.6 Transformation of Parameters
Say there is a parameter with known CRLB

But imagine that we instead are interested in estimating

some other parameter that is a function of :
= g( )
Q: What is CRLB

CRLB
g
CRLB
2
) (
) var(
|
.
|

\
|

=
Captures the
sensitivity of to
Proved in
Appendix 3B
Large g/ small error in gives larger error in
increases CRLB (i.e., worsens accuracy)
8
Example: Speed of Vehicle From Elapsed Time
Known Distance D
start
Laser
Sensor Sensor
Laser
stop
Measure Elapsed Time T
Possible Accuracy Set by CRLB
T
T
T
T V
CRLB
D
V
CRLB
T
D
CRLB
T
D
T
CRLB
=

|
.
|

\
|
=

|
.
|

\
|

=
2
4
2
2
But really want to measure speed V = d/T
Find the CRLB
V
:
) / (
2
s m CRLB
D
V
T V

Accuracy Bound
Less accurate at High Speeds (quadratic)
More accurate over large distances
9
Effect of Transformation on Efficiency
Suppose you have an efficient estimator of :

But you are really interested in estimating = g( )

Suppose you plan to use )

g =
Q: Is this an efficient estimator of ???
A: Theorem: If g( ) has form g( ) = a + b, then
is efficient.
)

g =
affine transform
Proof:
First: ( ) ( ) ( )

CRLB a a b a
2 2

var

var

var = = + =
= because efficient
Now, what is CRB

? Using transformation result:

( )

CRLB a CRLB
b a
CRLB
a
2
2
2
=

+
=
=
! !" ! !# $
( )

CRLB =

var
Efficient!
10
Asymptotic Efficiency Under Transformation
If the mapping = g( ) is not affinethis result does NOT hold
But if the number of data samples used is large, then the
estimator is approximately efficient (Asymptotically Efficient)

( ),

p g =

of pdf
Small N Case
PDF is widely spread
over nonlinear mapping

( ),

p g =

of pdf
Large N Case
PDF is concentrated
onto linearized section