Lecture 08 - Non Parametric SE-Nice

8/12/2008
Lecture 08: Spectrum estimation nonparametric methods

Instructor: Dr. Gleb V. Tcheslavski Contact: gleb@ee.lamar.edu Office Hours: Room 2030 Class web site: http://www.ee.lamar.edu/ gleb/adsp/Index.htm
Material comes from Hayes
ELEN 5301 Adv. DSP and Modeling
Summer II 2008
Introduction
As we discussed previously, the power spectrum of a wss process is the Fourier transform of its autocorrelation sequence. Therefore, estimating the spectrum is equivalent to estimating the autocorrelation For an autocorrelation ergodic process: autocorrelation.
N 1 lim N xn+k xn* = rx ( k ) N 2 N + 1 n =
(8.2.1)
Therefore, if xn is known for all n, estimating the power spectrum is (in theory) straightforward. However, there are two primer limitations making spectral estimation an extremely challenging problem. 1. The amount of data available for the analysis is never unlimited and, in many situations, might be very small (short observation time or quasi-stationarity limitations). 2. The data is often corrupted by noise or contaminated with an interfering signal.
Therefore, spectrum estimation is a problem involving estimating Px(ej) from a finite number of noisy data sample.
Summer II 2008
8/12/2008
Introduction
Spectral estimation is a problem that is of great importance in many applications including data analysis, Wiener filtering (to build an optimum filter, filt power spectra of desired signal and noise must be computed), signal t fd i d i l d i tb t d) i l detection, classification, and tracking, etc. There are two major classes of spectral estimators: 1. Nonparametric (classical) methods begin by estimating the autocorrelation sequence from a given data. The power spectrum then is estimated via Fourier transform of an estimated autocorrelation sequence; 2. Parametric (non-classical) methods the analyzed process is replaced by an appropriate model with known (for the specific parameters) spectrum.
Summer II 2008
The Periodogram
The periodogram method was introduced by Schuster in 1898. The power spectrum of a wss p p p process is the Fourier transform of its autocorrelation sequence:
Px ( e j ) =
k =
r (k ) e
x
jk
(8.4.1)
Therefore, spectrum estimation is, in some sense, an autocorrelation estimation problem. For an autocorrelation ergodic process and an unlimited amount of data, the autocorrelation may (theoretically) be determined with the time-average.
N 1 rx ( k ) = lim N xn+k xn* N 2 N + 1 n =
(8.4.2)
However, if xn is only available for a finite interval (say, [0, N-1]), the autocorrelation can be estimated, for example, with a finite sum
Summer II 2008
8/12/2008
The Periodogram
rx ( k ) = 1 N
x
n=0
N 1
n+k
* xn
(8.5.1)
To ensure that the values of xn outside the interval [0, N-1] are excluded from the sum, (8.5.1) must be modified as
rx ( k ) =
1 N
N 1 k
n =0
* xn + k xn ;
k = 0,1,..., N 1
(8.5.2)
Using the conjugate symmetry, the values of rx ( k ) for k < 0 can be defined as
rx ( k ) = rx* ( k )
Outside the interval:
(8.5.3)
rx ( k ) = 0
for k N
(8.5.4)
Summer II 2008
The Periodogram
Taking next the DTFT (actually, DFT) of the autocorrelation estimate leads to an estimate of the power spectrum called the periodogram:
Pper ( e j ) =
k = N +1
r ( k ) e
x
N 1
jk
(8.6.1)
It would be more convenient to express the periodogram in terms of the process itself rather then its autocorrelation. Let xN,n be the finite length N signal such that
x xN , n = n 0
0n< N
(8.6.2)
otherwise
Therefore, xN,n is the product of xn and a rectangular window:
xN ,n = WnR xn
(8.6.3)
Summer II 2008
8/12/2008
The Periodogram
Then, the estimated autocorrelation can be rewritten as
rx ( k ) =
1 N
n =
N ,n+ k
x* , n = N
1 x N , k x* , k N N
(8.7.1) (8 7 1)
Taking the Fourier transform and using the convolution theorem, the periodogram is
2 1 1 * Pper ( e j ) = X N ( e j ) X N ( e j ) = X N ( e j ) N N
(8.7.2)
where XN(ej) is the DTFT of the N-point data sequence xN,n
X N (e
)= x
n =
N ,n
jn
= xn e jn
n=0
N 1
(8.7.3)
Therefore, the periodogram is proportional to the squared magnitude of DFT of xn
xN , n X N , k
DFT
2 k 2 1 j X N ,k = Pper e N N
(8.7.4)
Summer II 2008
The Periodogram: Ex
Periodogram of white noise.
If xn is white noise with a variance of x2, then rx(k) = x2k and the power spectrum ( ) p p is a constant:
2 Px ( e j ) = x
(8.8.1)
A sample realization of unit variance white noise of length N = 32:
Summer II 2008
8/12/2008
The Periodogram: Ex
An autocorrelation estimate using (8.5.1). Although the autocorrelation is zero for |k| 32, it is nonzero for all other k.
The periodogram (solid) and the true power spectrum (dashed). Although the periodogram is approximately equal the true spectrum on average, there are visible variations.
Summer II 2008
10
The Periodogram
There is an interesting property. Let hi,n be an FIR filter of length N defined as follows: 1 in
hi ,n =
1 ini R e e Wn = N N 0
j
0n< N
(8.10.1)
otherwise
Its frequency response is
H i ( e j ) = hi ,n e jn = e
n =0
N 1
( i )( N 1)
2
sin N ( i ) 2 N sin ( i ) 2
(8.10.2)
which is a bandpass filter with a p center frequency i and a bandwidth that is approximately equal to
2 N
(8.10.3)
Summer II 2008
8/12/2008
11
The Periodogram
If a wss process xn is filtered with hi,n, the output process is
yi ,n = xn hi ,n =
k = n N +1
xk hi ,n k =
1 N
k = n N +1
xk e j ( n k )i
(8.11.1) (8 11 1)
Since |Hi(eji)|=i = 1, the power spectra of xn and yn are equal at frequency i:
Px e ji = Py e ji
( )
( )
1 Px ( e j ) = Px ( e j N 2
i
(8.11.2)
Furthermore, if the bandwidth of the filter is small enough so that the power spectrum of xn may be assumed as approximately constant over the filter passband, the th power in yi,n will b approximately i ill be i t l
E yi ,n
{ } = 21 P ( e
2
) H (e )
j i
(8.11.3)
and, therefore
Px e ji NE yi ,n
Summer II 2008
( )
{ }
2
(8.11.4)
12
The Periodogram
Thus, if we can estimate the power in yi,n, then the power spectrum at frequency i may be estimated as
Px e ji = NE yi ,n
E yi , n
From (8.11.1), this is equivalent to
2
( )
{ }
2
2
(8.12.1)
One simple approximation is to use a one-point sample average
{ }= y
2
i , N 1
(8.12.2)
yi , N 1
Therefore:
1 = 2 N
x e
k =0 k
N 1
2 jki
(8.12.3)
2 1 Px e ji = N yi , N 1 = N
( )
x e
k =0 k
N 1
2 jki
(8.12.4)
Which is equivalent to the periodogram.

Summer II 2008
8/12/2008
13
The Periodogram
Therefore, the periodogram may be viewed as power spectrum estimate via a filter bank of bandpass filters.
Summer II 2008
14
Performance of the Periodogram

The periodogram is proportional to the squared magnitude of DTFT of the finite length sequence xN,n. Therefore, from a computational point of view, the periodogram is easy to compute Let us evaluate properties of periodogram compute. periodogram.
1. Periodogram bias.
The expected value for the autocorrelation estimate is:
E {rx ( k )} =
1 N
N 1 k
n=0
* E { xn + k xn } =
1 N
N 1 k n =0
N k r (k ) = N r (k )
x x
(8.14.1)
For k N the expected value is zero. Using the conjugate symmetry of rx(k) N, zero
E {rx ( k )} = WkB rx ( k )
(8.14.2)
where
N k WkB = N 0
Summer II 2008
k N
(8.14.3)
k >N
8/12/2008
15

is a Bartlett (triangular) window. Therefore, the autocorrelation estimate is biased. The expected value of the periodogram is
N 1 N 1 E Pper ( e j ) = E rx ( k ) e jk = E {rx ( k )} e jk = rx ( k ) WkB e jk k = k = N +1 k = N +1
(8.15.1)
Since it is a Fourier transform of a product, using the frequency convolution, we obtain
1 E Pper ( e j ) = Px ( e j ) W B ( e j ) 2
where WB(ej) is the Fourier transform of the Bartlett window
(8.15.2)
1 sin ( N 2 ) W (e ) = N sin ( 2 )
B j
(8.15.3)
Summer II 2008
16

Thus, the expected value of the periodogram is the convolution of the power spectrum Px(ej) with the Fourier transform of a Bartlett window. Therefore, the periodogram is a biased estimate. However since WB(ej) converges to an impulse estimate However, as N goes to infinity, the periodogram is asymptotically unbiased
N
lim E Pper ( e j ) = Px ( e j )
(8.16.1)
For illustration, consider a random-phase sinusoid in white noise:
xn = A sin ( n + ) + vn
where is a uniform over [-,] random variable and vn is white noise with a variance v2. The power spectrum of xn is
(8.16.2) (8 16 2)
1 Px ( e j ) = v2 + A2 u0 ( 0 ) + u0 ( + 0 ) 2
(8.16.3)
Summer II 2008
8/12/2008
17

Therefore, the expected value of periodogram is
1 1 j j + E Pper ( e j ) = Px ( e j ) W B ( e j ) = v2 + A2 W B e ( 0 ) + W B e ( 0 ) (8.17.1) 2 4
The power spectrum Px(ej) for N = 64.
Summer II 2008
18

The expected value of the periodogram power spectrum estimate for N = 64.
There are two major effects: 1. The spectral smoothing produced by WB(ej) that leads to a spreading of the power in the sinusoid over a frequency band with bandwidth of approximately 4/N. 2. The power leakage through the sidelobes of the window creating secondary spectral peaks at frequencies
k 0
2 k N
(8.18.1)
Summer II 2008
8/12/2008
19

With A = 5, 0 = 0.4, and N = 64, 50 different realizations of (8.16.2) were generated and periodograms were estimated. Realizations and the average.
Increased the number of data values: N = 256. realizations and the average. The power is spread out over a narrower frequency band.
Summer II 2008
20

Smoothing introduced by the Bartlett window also limits the ability of the periodogram to resolve closely-spaced narrowband components in xn. Let a random process consists of two sinusoids in white noise:
xn = A1 sin ( n1 + 1 ) + A2 sin ( n2 + 2 ) + vn
(8.20.1)
where 1 and 2 are uncorrelated uniform over [-,] random variables and vn is white noise with a variance v2. The power spectrum of xn is
1 1 2 P ( e j ) = v2 + A2 u0 ( 1 ) + u0 ( +1 ) + A2 u0 ( 2 ) + u0 ( +2 ) (8.20.2) x 1 2 2
The expected value of periodogram is
1 E Pper ( e j ) = Px ( e j ) W B ( e j ) 2 1 1 2 j j + j j + = v2 + A12 W B e ( 1 ) + W B e ( 1 ) + A2 W B e ( 2 ) + W B e ( 2 ) 4 4
(8.20.3)
Summer II 2008
10
8/12/2008
21

The power spectrum Px(ej) for A1 = A2; N = 64.
p The expected value of the periodogram power spectrum for N = 64.
Summer II 2008
22

Since the width of the main lobe of WB(ej) increases as the data length decreases, there is a limit on how closely two sinusoids (or two narrowband processes) may be located, located for the given data length N in order to be resolved Usually the resolution N, resolved. Usually, limit is defined to be equal to the width of the main lobe of the spectral window at its half-power (-6 dB) point. For the Bartlett window:
Res Pper ( e j ) = 0.89
2 N
(8.22.1)
However, it is important to note that (8.22.1) is just a rule of thumb since it works on the th average and really depends on the phase b t d ll d d th h between t two sinusoids. i id With A = 5, 1 = 0.4, 2 = 0.45, 50 different realizations of (8.20.1) were generated and periodograms were estimated. According to (8.22.1), data length of N = 36 is required to resolve the frequency components.
Summer II 2008
11
8/12/2008
23

50 periodogram estimates and their average for N = 40. Barely resolved.
50 periodogram i d estimates and their average for N = 64. Clearly resolved.
Summer II 2008
24

2. Variance of the Periodogram.
The periodogram estimate is asymptotically unbiased. In order for it to be a p g y p y consistent estimate, it is necessary that the variance goes to zero as N . It is difficult to evaluate the variance of the periodogram for an arbitrary input process xn. The variance may be estimated for the special case of white noise xn with variance x2. In this case, the periodogram may be found as:
1 Pper ( e j ) = N = 1 N
xk e
k =0 N 1 N 1 k =0 l =0
N 1
2 jk
1 N 1 jk N 1 * jl xk e xl e N k =0 l =0
(8.24.1)
x x e
* j ( k l ) k l
Therefore, the 2nd moment of periodogram is
1 E Pper ( e j1 ) Pper ( e j2 ) = 2 N
E { x x x x } e
k =0 l =0 m=0 n =0 * * k l m n
Summer II 2008
N 1 N 1 N 1 N 1
j ( k l )1 j ( m n )2
(8.24.2)
12
8/12/2008
25

which actually depends on the 4th order moments of xn. However, since the process is Gaussian, we may use the moment factoring theorem to simplify these moments. For complex Gaussian variables the moment factoring theorem is variables,
* * * E { xk xl* xm xn } = E { xk xl*} E { xm xn } + E { xk xn } E { xm xl* }
(8.25.1)
Substitution (8.24.2) into (8.25.1), we observe that the first term simplifies to
1 N2
The second term becomes

k =0 m=0
2
N 1 N 1
4 x
4 =x
(8.25.2)
1 N2 =
x4e j( k l ) e j( k l ) =
1
N 1 N 1 k =0 l =0
x4
N
2
e
k =0
N 1
jk (1 2 )
e
l =0
N 1
jl (1 2 )
x4 1 e
jN (1 2 )
N 2 1 e j (1 2 )
1 e 4 sin N ( 1 2 ) 2 =x j (1 2 ) 1 e N sin (1 2 ) 2
jN (1 2 )
Summer II 2008
(8.25.3)
26

Therefore, the 2nd moment is
E Pper ( e
Since
j1
) P ( e )}
j2 per
sin N [ ] 2 2 1 2 = 1 + N sin [ ] 2 1 2
4 x
(8.26.1) (8 26 1)
Cov Pper ( e j1 ) Pper ( e j2 ) = E Pper ( e j1 ) Pper ( e j2 ) E Pper ( e j1 ) E Pper ( e j2 )

and
2 E Pper ( e j1 ) = x
} {
} {
} {
(8.26.2)
(8.26.3)
then, the covariance of the periodogram is
Cov Pper ( e
j1
) P ( e )}
j2 per
sin N (1 2 ) 2 = N sin (1 2 ) 2
4 x
(8.26.4)
Summer II 2008
13
8/12/2008
27

Finally, setting 1 = 2, we have for the variance
4 Var Pper ( e j ) = x
(8.27.1) (8 27 1)
Therefore, the variance does not go to zero as N and the periodogram is not a consistent estimate of the power spectrum. Since for the white noise
2 Px ( e j ) = x
(8.27.2)
the variance of the periodogram of white Gaussian process is
Var Pper ( e j ) = Px2 ( e j )
(8.27.3)
Summer II 2008
28

Let xn be white Gaussian noise with
Px ( e j ) = 1
( (8.28.1) )
It follows that the expected value is
E Px ( e j ) = 1
and the variance is
(8.28.2)
Var Px ( e j ) = 1
(8.28.3)
Therefore, although the periodogram is unbiased in this case, the variance equals a constant that is independent of the data length N. 50 realizations were generated and the periodograms were evaluated for them.
Summer II 2008
14
8/12/2008
29
Performance of Periodogram
N = 64
N = 128
N = 256 Variance does not decrease when the amount of data increases.
Summer II 2008
30

The analysis of non-Gaussian processes is more difficult. However, the variance of the periodogram for non-Gaussian processes can be approximately described by (8.27.3). (8 27 3)
Summary of the periodogram:
Summer II 2008
15
8/12/2008
31
The Modified Periodogram

The periodogram is proportional to the squared magnitude of the DTFT of the windowed signal xN,n = xnWnR. Would it be beneficial to use instead of a rectangular some other window? We examine next the effect of data window on the bias of the periodogram, which may be expresses as
2 1 1 Pper ( e j ) = X N ( e j ) = N N
n =
xW
n
R jn n
(8.31.1)
The expected value of the periodogram is

* ( e j ) = 1 E x W R e jn x W R e jm E Pper n n m m N n = m =
1 1 j nm * E xn xmWmR e ( ) = N m = n = N
m = n =
r ( n m )W
x
R j ( n m ) (8.31.2) m
Summer II 2008
32

Changing variables k = n-m
1 E Pper ( e j ) = N 1 = N
where
k = n =
r ( k )W
x
R jk j nk
1 N
k =
r (k ) W
x n =
R n
j WnR k e jk
(8.32.1)
k =
r ( k )W
x
B jk k
WkB = WkR WR = k
E Pper ( e j ) =
n =
R n
WnR k
(8.32.2)
is a Bartlett window. Using the frequency convolution theorem, the expected value: g q y , p
2 1 Px ( e j ) W R ( e j ) 2 N
(8.32.3)
where
W R ( e j ) =
sin ( N 2 ) j ( N 1) 2 e sin ( 2 )
Summer II 2008
(8.32.4)
16
8/12/2008
33

is the Fourier transform of the rectangular window. Therefore, the amount of smoothing in the periodogram is determined by the window that is applied to the data. data Although a rectangular window has a narrow main lobe compared to other windows and, therefore, produces the last amount of spectral smoothing, it has relatively large sidelobes that may lead to masking of weak narrowband components. For example, 2 sinusoids in white noise:
xn = 0.1sin ( n1 + 1 ) + sin ( n2 + 2 ) + vn
(8.33.1)
With 1 = 0.2, 2 = 0.3, and N = 128 the 128, expected value of periodogram (rectangular, left) and the Hamming window (right).
Summer II 2008
34

Using a Hamming window instead of the rectangular one allows to clearly observe the second frequency component due to the much lower sidelobes of Hamming windows. windows On the other hand this reduction in the sidelobe amplitude is hand, accompanied with an increase in the width of the mainlobe, which affects the resolution. The periodogram of a process that is windowed with a general window Wn is called the modified periodogram method:
1 PM ( e j ) = NU
where N is the length of the window and
n =
xWe
n n
2 jn
(8.34.1)
U=
is a constant.
1 N
W
n =0
N 1
2
(8.34.2)
Summer II 2008
17
8/12/2008
35

The expected value of the modified periodogram is
E PM ( e j ) =
We note that with
2 1 Px ( e j ) W ( e j ) 2 NU
(8.35.1) (8 35 1)
1 U= N
it follows:
W
n =0
N 1
1 = 2 N
W ( e j ) d
2
(8.35.2)
1 2 NU
W ( e j ) d = 1
2
(8.35.3)
and, with an appropriate window,
2 1 W ( e j ) NU
will converge to an impulse of unit area as N and the modified periodogram will be asymptotically unbiased.
Summer II 2008
36

Since the modified periodogram is simply the periodogram of a windowed data sequence, its variance is approximately the same as that for the periodogram:
Var PM ( e j ) Px2 ( e j )
(8.36.1)
Therefore, the modified periodogram is not a consistent estimate of the power spectrum and the data windows offer no benefit in terms of reducing the variance. The windows provide a trade-off between spectral resolution (main lobe with) and spectral masking (sidelobe amplitude). Properties of a few common time windows:
Summer II 2008
18
8/12/2008
37
Summary of the modified periodogram:
Summer II 2008
38
Bartletts method: averaging

Bartletts method of periodogram averaging produces a consistent estimate of the power spectrum. Averaging a set of uncorrelated measurements of a random variable x yields a consistent estimate of the mean E{x}. This suggests that we consider estimating the power spectrum of a random process by periodogram averaging. Let xi,n for i = 1,2,K be K uncorrelated realizations of a random process xn over the interval 0 n < L. With the periodogram of xi,n be
1 (i Pper) ( e j ) = L
x
n=0
L 1
i ,n
jn
i = 1, 2,..., K
(8.38.1)
The average of these periodograms is
1 Px ( e j ) = K
P (e )
K i =1 (i ) per j
(8.38.2)
Summer II 2008
19
8/12/2008
39

The expected value is
1 (i E Px ( e j ) = E Pper) ( e j ) = Px ( e j ) W B ( e j ) 2
} {
( (8.39.1) )
where WB(ej) is the Fourier transform of a Bartlett window that extends from -L to L. Therefore, the Bartletts estimate is asymptotically unbiased. Additionally, assuming that the data records are uncorrelated, the variance of the estimate is
1 1 (i ) Var Px ( e j ) = Var Pper ( e j ) Px2 ( e j ) K K
(8.39.2)
which goes to zero as K goes to infinity. Therefore, Bartletts estimate is consistent if both K and L approach infinity. The difficulty with this method is that uncorrelated estimates of the process are usually not available. In fact, there is typically a single realization of length N.
Summer II 2008
40

Therefore, the approach is to partition xn into K nonoverlapping sequences of length L where N = KL.
The Bartlett estimate is then computed as in (8.38.1) and (8.38.2) with
xi ,n = xn +iL
n = 0,1,..., L 1; i = 0,1,..., K 1
1 PB ( e j ) = N
(8.40.1)
Therefore, the Bartlett estimate is
x
i =0 n=0
K 1 L 1
n + iL
jn
(8.40.2)
Summer II 2008
20
8/12/2008
41

The expected value of Bartletts estimate is
1 E PB ( e j ) = Px ( e j ) WB ( e j ) 2
(8.41.1)
Therefore, the Bartletts estimate is asymptotically unbiased. Since the periodograms used in (8.40.2) are computed using sequences of length L, the resolution is
2 2 = 0.89 K Res PB ( e j ) = 0.89 L N
(8.41.2)
Which is K times worse than the periodogram Finally since the sequences xi,n are periodogram. Finally, generally correlated with one another (unless xn is white noise), the variance reduction will not be as large as in (8.39.2). However, for large N:
1 1 (i Var PB ( e j ) Var Pper) ( e j ) Px2 ( e j ) K K

(8.41.3)
Summer II 2008
42

Thus, if both K and L are allowed to go to infinity as N , the Bartletts estimate is consistent. Additionally, for a given value of N, Bartletts method allows to trade a reduction in spectral resolution in variance by changing the values K and L L.
Summary of the Bartletts method:
Summer II 2008
21
8/12/2008
43
Bartletts method
Estimate the power spectrum of 50 different unit variance white noise sequences of length N = 512 K = 1 (periodogram) K = 4, L = 128
K = 16, L = 32 Considerable decrease in variance.
Summer II 2008
44
Bartletts method
Consider a process consisting of 2 sinusoids with 1 = 0.2, 2 = 0.25 and unit variance noise. noise With N = 512 and for 50 realizations, the Bartletts estimates for K = 1, K = 4, and K = 16 were formed. Although the variance of the estimate decreases with K, the resolution decreases correspondingly as indicated by the broadening of spectral peaks.
Summer II 2008
22
8/12/2008
45
Welchs method
Welch proposed two modifications to Bartletts method: 1. Allow the sequences xi,n to overlap; 2. 2 Apply a data window wn to each sequence thereby producing a set of modified sequence, periodograms to be averaged. Assuming that successive sequences are offset by D points and that each sequence is L points long, then the ith sequence is given by
xi ,n = xn +iD
i = 0,1,..., L 1
(8.45.1)
Thus, the amount of overlap between xi,n and xi+1,n is L-D points, and if K sequences cover the entire N data points, then p ,
N = L + D ( K 1)
(8.45.2)
For example, with no overlap (D = L) we have K = N/L sections of length L as in Bartletts method. If the sequences are allowed to overlap by 50% (D = L/2), we may form N
K =2
(8.45.3)
Summer II 2008
46
Welchs method
sections of length L while maintaining the same resolution (section length) as Bartletts method and reducing the variance (doubling number of modified periodograms). periodograms) On the other hand with 50% overlap we could also form hand,
K=
N 1 L
(8.46.1)
sequences of length 2L, thus, increasing the resolution while maintaining the same variance as Bartletts method. Therefore, it is possible to trade a reduction in variance for a reduction in resolution by allowing the sequences to overlap. The Welchs estimates may be written as y
P ( e j ) = W
1 KLU
W x
i =0 n =0
K 1 i =0 (i ) M
K 1 L 1
n n + iD
jn
(8.46.2)
or in terms of modified periodograms
1 P ( e j ) = W K
P (e )
j
(8.46.3)
Summer II 2008
23
8/12/2008
47
Welchs method
The expected value of Welchs estimate is
E P ( e j ) = E PM ( e j ) = W
} {
2 1 Px ( e j ) W ( e j ) 2 LU
(8.47.1) (8 47 1)
where W(ej) is the Fourier transform of the L-point data window Wn used to form the modified periodograms. Therefore, Welchs method is asymptotically unbiased estimate of the power spectrum. The resolution depends on the data window. The variance is hard to evaluate, however, assuming the 50% overlap and the Bartlett window:
9 L 2 j Var P ( e j ) Px ( e ) W 16 N
(8.47.2)
It is possible to average more sequences for a given amount of data by increasing the amount of overlap. However, this would increase computational load and the correlation between sequences xi,n diminishing benefits of increasing overlap. The typical amount of overlap is either 50% or 75%.
Summer II 2008
48
Welchs method
Summary of the Welchs method:
Summer II 2008
24
8/12/2008
49
Welchs method
Considering the same process with two sinusoids in white noise and using Welchs method with N = 512, L = 128, 50% overlap (7 sections), a Hamming window, a plot of Welch s estimates for 50 realizations and the average are shown Welchs shown.
The variance and the resolution (rectangular vs. longer Hamming) are approximately the same as for Bartletts with K = 8 and L = 64. The gain is in the reduction of spectral leakage through the sidelobes.
Summer II 2008
50
Blackman-Tukey method: smoothing

Bartletts and Welchs methods reduce the variance of the spectral estimates by averaging periodograms and modified periodograms. Another method to decrease the statistical variability of periodogram is the periodogram smoothing referred to as the Blackman-Tukey method. The periodogram is a DTFT of a consistent estimate of the autocorrelation sequence. For any finite data record of length N, the variance of autocorrelation estimate will be large for values of lag k that are close to N. For example, the estimate of rx(k) at lag k = N-1 is
rx ( N 1) =
1 xN 1 x0 N
(8.50.1)
Since there is little averaging involved into formation of (8.50.1) for |k| N, no matter how large N becomes, these estimates will always be unreliable. Consequently, the only way to reduce the variance of the periodogram is to reduce the variance of these estimates or to reduce the contribution that they make to the periodogram. In B-T method, the variance of the periodogram is reduced by applying a window to the autocorrelation estimate to decrease the contribution of unreliable estimates to the periodogram.
Summer II 2008
25
8/12/2008
51

The Blackman-Tukey estimate is:
PBT ( e j ) =
k = M
r (k )W e
x k
jk
(8.51.1) (8 51 1)
where Wn is a lag window applied to the autocorrelation estimate. For example, if Wn is a rectangular window in [-M, M] with M<N-1, the estimates having the largest variance are decreased (or even set to zero), making a spectral estimate with smaller variance. However, this reduction in variance is accompanied by the reduction in resolution since a smaller number of autocorrelation estimates is used. Using the frequency convolution theorem, the Blackman-Tukey estimate is:
1 1 PBT ( e j ) = Pper ( e j ) W ( e j ) = 2 2
P ( e )W ( e ) du
ju j u per
(8.51.2)
Thus, the Blackman-Tukey estimate smoothes the periodogram by convolving it with the DTFT of the autocorrelation window.
Summer II 2008
52

The requirements to the lag window Wk are: 1. It should be conjugate symmetric so that its transform is real-valued; 2. 2 The windows should have a nonnegative Fourier transform, so that the spectral transform estimate is guaranteed to be nonnegative. The bias may be computed by taking the expected value of (8.51.2). It can be shown that, assuming M << N, the expected value is
1 E PBT ( e j ) Px ( e j ) W ( e j ) 2
(8.52.1)
where W(ej) is the DTFT of the lag window. The variance of the estimate may be approximated as
1 Var PBT ( e j ) Px2 ( e j ) N
k = M
2 k
(8.52.2)
provided that N >> M >> 1.

Summer II 2008
26
8/12/2008
53

Therefore, the trade-off is again between bias and variance. For a small bias, M should be large to minimize the width of the main lobe of W(ej), while M should be small to minimize the variance Generally it is recommended that M has a variance Generally, maximum value of N/5.
Summary of the Blackman-Tukey method:
Summer II 2008
54
Performance comparison
The selection of a non-parametric spectral estimator depends on the particular needs and requirements. To compare the performance of different estimator, two additional criteria are introduced. The variability of the estimate:
V
which is the normalized variance. The figure of merit of the estimate:
Var Px ( e j ) E
2 j x
{ } {P ( e )}
(8.54.1)
M V
which should be as small as possible.
(8.54.2)
Summer II 2008
27
8/12/2008
55
Performance comparison
The figure of merit is inversely proportional to the data length. The overall performance of classical estimators is fundamentally limited by the amount of data available.
Summer II 2008
56
More nonparametric methods

Another nonparametric approach to estimating the coherence function has been proposed by Thomson. This method is based on Thomsons spectral estimation procedure, procedure also referred to as the multiple taper method which is known to method, produce less biased spectral estimates in comparison with the periodogram method. This method was proposed for signals with complicated spectral densities. The appropriate procedure is briefly described next. Assuming an N-sample long time record xn with zero mean, the raw eigencoefficients are specified as a discrete Fourier transform of the input signal:
xk (e ) = xn vn ( k ) ( N , W ) e j n
j n= 0
(k )
N 1
(8.56.1)
where vn ( N , W ) is the discrete prolate spheroidal sequence, which is a Fourier transform of the discrete prolate spheroidal wave function U k ( N ,W ; f ) also called the Slepian function. These sequences are orthonormal providing spectral windows with well-concentrated energy, over which the data is observed. W denotes the bandwidth: 0 < W < 0.5.
Summer II 2008
28
8/12/2008
57
More nonparametric methods

The raw eigen-coefficients are usually weighted by k ( N ,W ) to form estimates of idealized eigen-coefficients
xk (e j ) = k ( N , W ) xk (e j )
Here k ( N , W ) are the eigenvalues of the N x N matrix
(8.57.1)
( N ,W ) mn =
sin 2 W (m n) ( m n)
m, n = 0,1,..., N 1
(8.57.2)
Thus, the spectral estimates are formed as follows
PT (e j ) =
1 2 NW
2 NW 1
k =0
xk (e j )
2
(8.57.3)
However, the Thomsons estimate still suffers from the nonparametric limitations: high resolution is possible for long data records only.
Summer II 2008
29

Lecture 08 - Non Parametric SE-Nice

Загружено:

Сведения о документе

Исходное описание:

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Lecture 08 - Non Parametric SE-Nice

Загружено:

Авторское право:

Доступные форматы

8/12/2008

Lecture 08: Spectrum estimation nonparametric methods

Material comes from Hayes

ELEN 5301 Adv. DSP and Modeling

ELEN 5301 Adv. DSP and Modeling

ELEN 5301 Adv. DSP and Modeling

ELEN 5301 Adv. DSP and Modeling

Therefore, xN,n is the product of xn and a rectangular window:

where XN(ej) is the DTFT of the N-point data sequence xN,n

Therefore, the periodogram is proportional to the squared magnitude of DFT of xn

A sample realization of unit variance white noise of length N = 32:

ELEN 5301 Adv. DSP and Modeling

Its frequency response is

ELEN 5301 Adv. DSP and Modeling

Since |Hi(eji)|=i = 1, the power spectra of xn and yn are equal at frequency i:

ELEN 5301 Adv. DSP and Modeling

One simple approximation is to use a one-point sample average

Which is equivalent to the periodogram.

ELEN 5301 Adv. DSP and Modeling

Performance of the Periodogram

ELEN 5301 Adv. DSP and Modeling

Performance of the Periodogram

Since it is a Fourier transform of a product, using the frequency convolution, we obtain

Performance of the Periodogram

For illustration, consider a random-phase sinusoid in white noise:

Performance of the Periodogram

ELEN 5301 Adv. DSP and Modeling

Performance of the Periodogram

Performance of the Periodogram

Performance of the Periodogram

ELEN 5301 Adv. DSP and Modeling

Performance of the Periodogram

p The expected value of the periodogram power spectrum for N = 64.

ELEN 5301 Adv. DSP and Modeling

Performance of the Periodogram

Res Pper ( e j ) = 0.89

ELEN 5301 Adv. DSP and Modeling

Performance of the Periodogram

50 periodogram i d estimates and their average for N = 64. Clearly resolved.

ELEN 5301 Adv. DSP and Modeling

Performance of the Periodogram

Therefore, the 2nd moment of periodogram is

Performance of the Periodogram

ELEN 5301 Adv. DSP and Modeling

Performance of the Periodogram

Cov Pper ( e j1 ) Pper ( e j2 ) = E Pper ( e j1 ) Pper ( e j2 ) E Pper ( e j1 ) E Pper ( e j2 )

then, the covariance of the periodogram is

ELEN 5301 Adv. DSP and Modeling

Performance of the Periodogram

the variance of the periodogram of white Gaussian process is

Var Pper ( e j ) = Px2 ( e j )

ELEN 5301 Adv. DSP and Modeling

Performance of the Periodogram

It follows that the expected value is

ELEN 5301 Adv. DSP and Modeling

ELEN 5301 Adv. DSP and Modeling

Performance of the Periodogram

Summary of the periodogram:

ELEN 5301 Adv. DSP and Modeling

The Modified Periodogram

The expected value of the periodogram is

ELEN 5301 Adv. DSP and Modeling

The Modified Periodogram