Digital Signal Processing 2/ Advanced Digital Signal Processing Lecture 14, Matched Filter, Prediction Gerald Schuller, TU Ilmenau

Prediction A system which produces the prediction error (for instance of part of an encoder) as an output is the following,

x ( n)
z 1

+
H ( z)

e ( n)= x ( n ) x ( n)

x ( n)

Here, x ( n) is the signal to be predicted, H ( z ) is our prediction filter which wor s on only past samples (that!s why we ha"e the delay element 1 by one sample, z , before it), x ( n) is our predicted signal, and e ( n)= x ( n ) x ( n) is our prediction error signal# Hence, our system which produces the prediction error has the $% domain transfer function of

H perr ( z )=1 z1H ( z )

&bser"e that we can reconstruct the original signal x ( n) in a decoder from the prediction error e ( n) , for instance in a decoder, with the following system,

e ( n)

+ + x ( n)

x ( n)=e ( n )+ x ( n) H ( z) z 1

The feedbac loop in this system is causal because it only uses past, already reconstructed samples' &bser"e that this decoders transfer function is

H prec =

1 1 = 1 z1H ( z ) H perr ( z )

which is e(actly the in"erse of the encoder, which was to be e(pected#

Octave/Matla !"ample Goal) *onstruct a prediction filter for our female speech signal of order +,, which minimi$es the mean%s-uared prediction error# .ead in the female speech sound)
x=wavread('fspeech.wav'); size(x) %ans = %207612 1 %listen to it: sound(x,32000)

/*onstruct our 0atri( A from ()

A=zeros(100000,10); for m=1 100000, A(m, )=x(m!1"(1 10))'; end; #$ons%ruc% our desired %ar&e% si&na' d, one samp'e in%o %he fu%ure d=x(11 100010); #$ompu%e %he predic%ion fi'%er h=inv(A'(A) ( A' ( d; f'ipud(h) ans = 0.900784 -0.776478 1.179245 -0.458494 0.622308 -0.320261 0.054122 -0.205571 -0.011090 -0.030701 p'o%(f'ipud(h))

The impulse response of the resulting prediction filter# &ur corresponding prediction error filter is H perr ( z )=1 z1H ( z ) , in &cta"e10atlab)
hperr=)1; !f'ipud(h)* hperr = 1.000000 -0.900784 0.776478 -1.179245 0.458494 -0.622308 0.320261 -0.054122 0.205571 0.011090 0.030701

/The prediction error e(n) is

e=fi'%er(hperr,1,x); #+he mean!s,uared error is e'(e-max(size(e)) %ans = 4.3287e-04

%Compare that with the mean sq are! si"nal power: x'(x-max(size(x)) %ans = 0.0069761 %#hi$h is more than 10 times as %i" as the pre!i$tion error& %whi$h shows that it wor's& %(isten to it: sound(e,32000) %)a'e a loo' at the si"nal an! it*s pre!i$tion error: p'o%(x) ho'd p'o%(e,'r')

Here we can see the original signal in blue, and it!s prediction error in red#

Online Adaptation The pre"ious e(ample calculated the prediction coefficients for the entire speech file# 2ut when we loo at the signal wa"eform, we see that its characteristics and hence its statistics is changing, it is not stationar#$ Hence we can e(pect a prediction impro"ement if we di"ide the speech signal into small pieces for the computation of the prediction coefficients, pieces which are small enough to show roughly constant statistics# In speech cosing, those pieces are usually of length 3, ms, and this approach is called Linear Predictive %oding &LP%'# Here, the predictions coefficients are calculated usually e"ery 3, ms, and then transmitted alongside the prediction error, from the encoder to the decoder#

Matla /Octave !"ample &ur speech signal is sampled at 43 H$, hence a bloc of 3, ms has (4) samples#
x=wavread('fspeech.wav'); size(x) %ans = #20./12 1 'en=max(size(x)); e=zeros(size(x)); #predic%ion error varia0'e ini%ia'iza%ion 0'oc1s=f'oor('en-/20); #%o%a' num0er of 0'oc1s s%a%e=zeros(1,10); #3emor4 s%a%e of predic%ion fi'%er #5ui'din& our 3a%rix A from 0'oc1s of 'en&%h /20 samp'es and process for m=1 0'oc1s, A=zeros(/30,10); #%ric1 up %o /30 %o avoid zeros in %he ma%rix for n=1 /30, A(n, )=x((m!1)(/20"n!1"(1 10))'; end; #$ons%ruc% our desired %ar&e% si&na' d, one samp'e in%o %he fu%ure d=x((m!1)(/20"(11 /20)); #$ompu%e %he predic%ion fi'%er h=inv(A'(A) ( A' ( d; hperr=)1; !f'ipud(h)*;
)e((m!1)(/20"(1 /20)),s%a%e*=fi'%er(hperr,1,x((m!1)(/20"(1 /20)),s%a%e);

end; #+he mean!s,uared error now is e'(e-max(size(e))

#ans = 1.136/e!02

#7e can see %ha% %his is on'4 a0ou% 1 - 2 of %he previous pred. 8rror9 %Compare that with the mean sq are! si"nal power: x'(x-max(size(x))
#ans = 0.00/:./1 x'(x-(e'(e) ans = 61.429

#;o our <=$ pred err ener&4 is more %han a fac%or of /1 sma''er %han %he si&na'9 #<is%en %o %he predic%ion error sound(e,32000)

%)a'e a loo' at the si"nal an! it*s pre!i$tion error: p'o%(x) ho'd p'o%(e,'r')

Here it can be seen that the prediction error is e"en smaller than before# 56* type coders are for instance speech coders, where usually +3 coefficients are used for the prediction, and transmitted as side information e"ery 3, ms# The prediction error is parameteri$ed and transmitted as parameters with a "ery low bit rate# This ind of system is used for instance in most digital cell phones systems#

Least Mean S*uares &LMS' Algorithm Unli e the 56* algorithm abo"e, which computes prediction coefficients for a bloc of samples and transmits these coefficients alongside the prediction error to the recei"er, the 50S algorithm updates the prediction coefficients after each sample, but based only on past samples# Hence here we need the assumption that the signal statistics does not change much from the past to the present# Since it is based on the past samples, which are also a"ailable as decoded samples at the decoder, we do not need to transmit the samples to the decoder# Instead, the decoder carries out the same computations in s#nchron# +ith the encoder# Instead of a matri( formulation, we use an iterati"e algorithm to come up with a solution for the prediction coefficients h # To show the dependency on the time n , we now call the "ector of prediction coefficients h ( n ) #

Again we would li e to minimi$e the mean -uadratic error,

( n )) ] = E ( x ( n ) hk ( n) x ( n k ))2 E [( x ( n) x
2 k=1

7e use the algorithm of Steepest 8escent (also called Gradient 8escent), see also http)11en#wi ipedia#org1wi i1Gradient9desce nt, to iterate towards the minimum, h ( n +1)= h ( n) f ( h ( n)) with optimi$ation function as our s-uared prediction error,
f ( h ( n ))=( x ( n) h k ( n) x ( n k ))2
k =1 L

7e ha"e
f ( h ( n )) f ( h ( n )) f ( n )= , ... , h1 ( n ) h L (n ) and we get f ( h ( n )) = 2e ( n )( x ( n k )) hk ( n) So together we obtain the LMS algorithm or update rule as hk ( n+1)= hk ( n)+e ( n)x ( n k ) for k =1,... L , where is a tuning parameter (the factor 3 is incorporated into ), with which we can trade off con"ergence speed and con"ergence accuracy#

In "ector form, this 50S update rule is h ( n +1)= h ( n)+e ( n)x ( n) where x ( n)=[ x ( n 1 ) , x ( n 2 ) , ... , x ( n L) ] #

LMS Octave/Matla
x=wavread('fspeech.wav'); size(x) e=zeros(size(x)); %ans = %207612 1 h=zeros(10,1); for n=11 max(size(x)), #predic%ion error and fi'%er e(n)=x(n) ! x(n!10"(0 :))' ( h; #<3; upda%e ru'e h= h " 1(e(n)((x(n!10"(0 :))); end e'(e-max(size(e)) %ans = 2.1586e-04

!"ample

/This is bigger than in the 56* case, but we also don!t need to transmit the prediction coefficients# 6lot a comparison of the original to the prediction error,
p'o%(x); ho'd p'o%(e,'r')

5isten to the prediction error,

sound(e,32000)

:or the decoder we get the reconstruction

h=zeros(10,1); xre1=zeros(size(x)); for n=11 max(size(x)), xre1(n)=e(n) " xre1(n!10"(0 :))' ( h; h= h " 1(e(n)((x(n!10"(0 :))); end #<is%en %o %he recons%ruc%ed si&na' sound(xre1,32000)