Proceedings of National Conference on Networking, Embedded and Wireless Systems, NEWS-2010, BMSCE

Transform domain robust Variable Step size Griffiths Adaptive Algorithm for System Identification
S.V. Narasimhan1 and S. Roopa2
Digital signal processing and systems Group Aerospace Electronics & Systems Division National Aerospace Laboratories, Bangalore-560017, (Council of Scientific & Industrial Research (CSIR), New Delhi)
E-mail: narasim@nal.res.in 2 roopa01@gmail.com
2 2 Department of E.C.E., B.M.S College of Engineering, Bnagalore-560018 1

Abstract A transform domain robust variable step size Griffiths LMS algorithm (TVGLMS) is proposed for system identification. For the TVGLMS, the robust variable step size has been achieved by using the Griffiths gradient which uses cross-correlation between the desired signal contaminated with observation noise and the input and the discrete cosine transform (DCT). The presentation considers both cases with and without desired signal decomposition by the DCT. The proposed algorithm is found to be having better convergence error/ misadjustment ( by 10 dB for colour and white observation noise) compared to that of ordinary transform domain LMS (TLMS) algorithm, both in the presence of white/ colored observation noise. The reduction in convergence error achieved by the new algorithm with desired signal decomposition is found to lower than that by without decomposition. (By 8 dB for colour and 3 dB for white observation noise)
Keywords: Robust Transform domain LMS algorithm, DCT

Griffiths LMS algorithm,, Variable step size LMS algorithm . 1. INTRODUCTION The LMS adaptive algorithm due to its simplicity is widely used for many applications like system identification, channel equalization, echo cancellation, noise removal, adaptive coding/compression and active noise control. However the LMS algorithm, suffers from slow convergence rate when the eigenvalue spread of the input autocorrelation matrix is large. To over come this, algorithms are proposed which orthogonalize the input bringing down the eigenvalue spread. In this direction transform domain approach is quite popular. In this, the orthogonalization of the input by the transform aligns the weight vector with the principal eigenvectors of the error surface there by removing the cross coupling which exists among the weights involved in adaptation and further the scaling of these orthogonal components results in reduction in eigenvalue spread. basically the orthogonalization enables to use different step sizes for the adaptive filters of different components of the input depending on their power (which is same as scaling). in view of this, the stepsize used for different components will be significantly large than that

used for the transversal LMS algorithm as it uses the total power of the input for stepsize normalization in the normalized LMS algorithm (NLMS).though the use of larger stepsize for different components results in faster convergence rate., it also results in higher convergence error/ misadjustment. Hence to have smaller misadjustment, it is necessity to reduce large values of the stepsize for different components as the convergence is approached. Therefore to achieve both fast convergence rate and low misadjustment, a variable step-size LMS (VSS) algorithm is required. In VSS, the step-size is varied from a large to the small value [1, 3, 46, 7] as the convergence is approached.. In [9], the step-size is made as a function of the error energy, however in the presence of observation noise this will result in larger stepsize. In [1], step-size is made proportional to the autocorrelation of the error and this is valid only for the white observation noise. The correlation LMS [6], uses crosscorrelation between input and error and this suffers from negative step-size and adaptation stalling. In Okellos [5] VSS, the step-size is a function of the sum of the squared cross-correlations between the error and the delayed inputs corresponding to the weights. This is free from the problems of [6] but may be sluggish for non stationary signal. In the recent algorithm by Zhang [7], step-size is made proportional to the squared norm of the smoothed gradient vector. All these algorithms attempt to make the step-size robust to observation noise, but not the gradient for the weight vector adaptation. A variable step size Griffiths LMS algorithm (VGLMS) which not only uses the step size but also gradient for weight vector which are robust to observation noise has been proposed [8]. The VGLMS achieves this by using the cross-correlation between the desired signal and the input. However, the VGLMS algorithm for a given maximum step size, has a slower convergence rate, due to replacement of instantaneous correlation between input and the error by the Griffiths averaged gradient. This motivates to apply VGLMS approach to algorithms which have faster convergence rate for inputs with large eigenvalue spread. Hence, it is of interest to extend this algorithm to transform domain adaptation as this will not only provide additional convergence speed over the variable step size but also a smaller misadjustment. A transform domain Griffiths algorithm has been applied for

105

Proceedings of National Conference on Networking, Embedded and Wireless Systems, NEWS-2010, BMSCE

adaptive blind demodulation of DS/CDMA signals [9]. However this is very specific to the application and cannot be use for system identification type of applications. In this paper, a transform domain robust variable step-size LMS (TVGLMS) algorithm has been proposed.. In the TVGLMS, the robust variable step size and the robust gradient are achieved by using the cross-correlation between the desired signal and different transform components of the input. In this, as a special case, use of transformed components of the desired signal instead of complete desired signal is also considered. The performance of the proposed algorithm is found to be better for both white and colored observation noise situations even at low SNR. Use of transformation for the desired signal has better performance as it reduces the observation noise for the individual components are reduced. II. BACKGROUND In this section the background material required viz., the system identification and the Griffiths variable step size algorithm will be briefly reviews for the purpose of reference and notational convenience. A. System identification Consider a system P ( z ) with impulse response P(n) = [ p(0) p (1) p ( M 1)] and with white noise input X (n) = [x(n) x(n 1)
M 1

Since e(n) = d (n) y (n) and d (n) = d (n) + o(n) ,, equation .(1a) becomes ( n) wk ( n + 1) = wk (n) + [d (n) y (n)]x(n k ) + ME x (n) (1b) ( n) o( n) x( n k ) ME x (n) In Equation. (1c), the presence of last term in is not desirable as only the expectation of the product,. E{o(n) x( n k )} = 0 Since o(n) and x(n) are uncorrelated to each other. However, the instantaneous value of the product, {o ( n ) x ( n k )} 0 . Hence in the LMS algorithm as it uses only instantaneous values of the variables involved, last term due to the observation noise o(n) will disturb the convergence process and even it make the adaptation to diverge. Hence the suppression of the observation noise for the LMS adaptation is essential. Further, a large step size results in larger convergence error /misadjustment, hence it is required to have a larger step size prior to convergence and a smaller step size after convergence. If the observation noise o(n) is not there or if it is of small magnitude compared to plant output d (n) , after many iterations W (n) will converge to P ( n) and will be a good estimate of P ( n) for a whit noise input x (n) . .However if

x(n M + 1)
and

]T .

For this, the

system output d (n) = noise

p (k ) x(n k )
k =0

the observation

o( n ) corrupts it to

P( z )
x( n )

d ( n )

+
y( n )

+ O( n )
d( n )

x(n) is not a white noise but colored characterized by fundamental and its harmonics, the system P ( z ) will be identified only at those frequencies. In view of this, W (n) will be different from P (n) and W ( z ) will match P( z ) only at the frequencies of the input. For such inputs the convergence of W (n) to P(n) will be slow due to large spectral range / eigenvalue spread. In such cases, it is required to orthogonalize the input to bring down the Eigen value spread using a orthogonal transform.
B. Griffiths variable step size algorithm (VGLMS) To remove the effect of observation noise on the adaptation process, Griffiths modified the NLMS algorithm. If

W( z )
Fig. 1 System Identification

+
e( n )

give a desired signal d ( n ) = d ( n ) + o( n ) .The normalized LMS (NLMS) adaptation rule for this is given by (n) wk (n + 1) = wk ( n) + e( n ) x ( n k ) (1a) ME x (n)

G k (n) = E [d (n) x(n k )] = E [{d ( n) + o(n)} x(n k )] , Then G k (n) , k = 0, , M 1 , represents the cross-correlation
between the desired signal and the input. Further as x (n) and

o( n) are independent,
E [o(n) x(n k ) ] = 0 . Therefore, G k ( n) = E[ d ( n) x( n k )], k = 0, , M 1
and E [e(n) x( n k )] = Gk (n) E [ y( n) x(n k )] . If Qk (n) = [G k (n) y (n) x (n k )] , k = 0, , M 1 Qk (n) is referred to as Griffiths cross-correlation and is free from the effect of observation noise component, o(n) .

E x (n) = x E x (n 1) + (1 x ) x 2 (n) , 0 < x < 1

M

y ( n) =

w ( n) x ( n k )
k k =0

E x (n) is power of the input computed recursively, y (n) is the adaptive filter output and (n) is the adaptation step size.

106

Proceedings of National Conference on Networking, Embedded and Wireless Systems, NEWS-2010, BMSCE

Therefore the LMS adaptation algorithm Equation n.(1a) with Griffiths cross-correlation is [4] (n)Qk ( n) wk (n + 1) = wk ( n) + (2a) ME x ( n) Okello [10] proposed an algorithm which provides a variable step-size based on the cross-correlation U k (n) , between input and the error.

wk ( n + 1 ) = wk ( n ) + ( n ) k ( n )
The second term of Eqn..5(d) is the product of

(5d)

(n) and the

weight vector adaptation gradient ( n) . The VGLMS algorithm uses both the noise step size and noise free gradient whereas other variable step size algorithms use only the noise free step size. III PROPOSED TRANSFORM DOMAIN VARIABLE STEP SIZE GRIFFITHS LMS ALGORITHM (TVGLMS) A TVGLMS without desired signal decomposition

U k (n) = E [e(n) x(n k )] = E [(d (n) y (n) )x(n k )] (3) Since E[o(n) x (n k )] = 0 , U k (n) is free from observation noise. In this, to avoid negative step-size, the variable stepsize (n) is given by [10]
M 1

(n) = ( n 1) +

k =0

2 Uk ( n)

0 < ( , ) < 1

(4)

. The expectation operation in U k (n) , reduces the effect of observation noise on step size. Further,

X 1( n )

w1 ( n)

U
k =0

2 k ( n)

0 in

DCT

M 1

X1( n ) X M 1( n )

y0 ( n )
w2 ( n )

Equation. (4), ensures positive step-size and the presence of ( n 1) avoids stalling [1]. This being a NLMS algorithm, for stability the initial value for (n) is set close to unity [7] and the lower limit for (n) is fixed to prevent adaptation stalling and to provide minimum misadjustment tolerable. The Griffiths and Okellos algorithms are combined to realize VGLMS algorithm []. In the VGLMS algorithm, U k (n) in Okellos algorithm is replaced by the Griffiths cross correlation Qk (n) = [G k (n) y (n) x(n k )] . The expectation operation for y (n) x( n k ) is removed due to use of instantaneous quantities in LMS algorithm and also as observation noise is not directly present in this term. It is important to note that as y ( n) is a time varying quantity and averaging of this quantity in Qk (n) makes the adaptation process sluggish, justifying the use of Qk (n) over U k (n) . The VGLMS rule for the individual coefficients is ( n) wk (n + 1) = wk (n) + G k ( n) y ( n) x ( n k ) (5a) ME x (n)

y1 ( n )
wM 1( n )

+ +

+
y( n ) _

y M 1 ( n ) +

x(n)

e(n)
P( z )
d ( n )
+

+
d (n)
o( n )

Fig.2 DCT transform domain LMS adaptive filter for system identification

The basic transform domain LMS algorithm using discrete cosine transform is shown in Fig. 2. For the input x (n) , the 2 M point DCT of produces orthogonal components X k ( n), k = 0, , M 1 (the remaining M points are symmetrical) and these form inputs to single coefficient adaptive filters Wk (n), k = 0, , M 1 . The output of these adaptive filters y k (n), k = 0, , M 1 are summed (with a gain factor of 2) to take care of symmetrical components) on sample to sample basis to get the estimate y (n) of the desired signal d ( n ) . The error signal e( n) = d (n) y (n) is used for the adaptation of the weights Wk (n), k = 0, , M 1 using the normalized LMS algorithm as 2 ( n) wk (n + 1) = wk (n) + 2 e( n) X k ( n ) , k ( n)

M 1

(n +1) = (n) +

(n) y(n)x(n k)] [ G

k k =0

(5b)

(n + 1) = G G k g k (n) + 1 g d (n) x(n k ), 0 < g < 1

Here

(5c)

decides the cut-off frequency and hence the degree

of rejection of the observation noise. If

k ( n) =

1 ( n) y ( n) x ( n k ) , G k MEv (n)

k = 0,1,2, M 1 (6)
2 2 2 k ( n) = k ( n 1) + (1 ) X k ( n)

0 < <1

Then Eqn. (5a) can be expressed as

(n) is the step size used for the adaptation. k2 (n) is the
input power to adaptive filter wk (n), k = 0, , M 1 and is

107

Proceedings of National Conference on Networking, Embedded and Wireless Systems, NEWS-2010, BMSCE

estimated recursively. The transform domain provides faster convergence compared to transversal filter since
M 1

individual errors ek (n) will be directly available. Use of these errors ek (n) is advantageous to that of overall error e( n ) . This is because; the overall error e(n) will contain not only the required error component but also other error components which act as observation noise for the weight adaptation under consideration. This not only facilitates faster convergence but also smaller convergence error/ misadjustment. In view of this, the application of VGLMS algorithm to the case where the desired signal is decomposed is of interest. Fig.3 shows the schematic for TVGLMS with desired signal decomposition. Here d (n) is decomposed by another DCT of same number of points ( 2 M ) to get the components d k ( n ) , k = 0, , M 1 . Further the outputs of the adaptive filters y k (n) are not summed but are remained to get the individual errors ek (n) as

(n) << (n)

2 k 2 k k =0

and the gradient step size will be

relatively large for the former. However due to large step size the convergence error o the transform domain LMS can be larger than that for transversal filter. Since e( n) = d (n) y (n) Equation. (6) Can be written as 2 ( n) wk (n + 1) = wk (n) + 2 [d (n) y (n)]X k (n) k ( n) Further as d (n) = d ( n) + o(n) ,
2(n) (7) [d(n) y(n)]Xk (n) + 2 o(n)Xk (n) 2 2 k (n) k (n) The adaptation for the weight wk (n) gets affected by the last term on the right hand side of Equation (7). Applying the Griffiths algorithm to this case, wk (n +1) = wk (n) + wk (n + 1) = wk (n ) + 2 (n)
2 k ( n)

o( n ) x(n)
P( z )
d (n)
+

[G (n) y(n) X (n)]

k k

(8)

d(n)

Where

X1(n)

_
w1(n) y ( n ) 0

G k ( n) = E[d (n) X k (n)]

+ d0 ( n )

DCT

= E[d (n) X k (n)] + E[o(n) X k (n)], k = 0, , M 1

E[o(n) X k (n)] = 0 as the observation noise

X1(n)

w2( n ) y ( n ) 1

+ +

o( n ) and input
XM1(n)

e1( n )

d1(n)

component X k (n) are uncorrelated. Hence use of Griffiths gradient as in Equation (8) will remove the effect of observation noise on the weight adaptation process. Further, the VGLMS can be used for reducing the step size as converges approaches. Defining

_
wM1( n )

yM1( n )

+ +
dM1( n )

eM1( n )

Qk (n) = Gk (n) y (n) X k (n) , k = 0, , M 1

(9a)

Fig.3 DCT transform domain LMS adaptive filter for System identification with desired signal decomposition

Qk (n) is Griffiths cross-correlation in the context of

transform domain LMS algorithm. Further Gk (n) is estimated recursively as ( n + 1) = G (n) + 1 d (n) X (n),0 < < 1 (9b) G k g k g k g

ek (n) = d k (n) y k (n) , k = 0, , M 1

Similar to VGLMS algorithm, the step size adapted as

M 1

( n )

can be

The TVGLMS algorithm (Equations. (8), (9), and (10) for this case is 2 (n) wk (n + 1) = wk (n) + 2 Gk (n) y k (n) X k (n) (11) k (n)

(n) = ( n 1) +

Q
k =0

2 k ( n)

0 < ( , ) < 1

(10)

Qk ( n ) = G k ( n ) y k ( n ) X k ( n ) , k

= 0, , M 1

( n + 1) = G G k g k ( n) + 1 g d k (n ) X k ( n), 0 < g < 1

This TVGLMS algorithm will be referred to as TVGLMS-1 algorithm.

B TVGLMS with desired signal decomposition For adaptation the weights wk (n) , if the desired signal is orthogonally decomposed by transform, then

The step size ( n) is adapted as

M 1

(n) = ( n 1) +

Q
k =0

2 k ( n)

0 < ( , ) < 1

108

DCT
(12a) (12b) (13)

e0( n )

Proceedings of National Conference on Networking, Embedded and Wireless Systems, NEWS-2010, BMSCE

This TVGLMS algorithm using components of the desire signal in addition the components of the input signal will be referred to as TVGLMS-2 algorithm. Both TVGLMS-1(Equations. (8), (9) and (10)) and TVGLMS-2 (Equations. (11), (12), and (13)) algorithms not only use a gradient but also use a variable step size which is robust to observation noise. In both cases, a high value of enables better tracking for non-stationary signal, whereas a small is acceptable for stationary signals. Further the noise free gradient and step-size, significantly improve the convergence performance. The TVGLMS-2 algorithm may perform better than that of TVGLMS-2, due to use of appropriate errors for the individual weight adaptations.

The convergence error for both white and colored observation noise is reduced by 10 dB by the proposed TVGLMS algorithm (without desired signal decomposition) over the TLMS algorithm. However, the convergence rate is somewhat reduced due to the averaging of the gradient involved in the proposed algorithm. The reduced convergence error is due to the robust variable step-size and this variation from a high value to a low value is shown in the figure. The convergence error for white and colored observation noise is reduced by 3 dB and 8 dB, respectively by the proposed TVGLMS algorithm (with desired signal decomposition) over the TLMS algorithm. However, the convergence rate is faster than that for without decomposition due to removal of other components which act as observation noise for the specific component. Even the noise also gets removed from the corresponding error due to decomposition. For the same reason the convergence error reduction is less compared to that by TVGLMS without desired signal decomposition. The reduced convergence error is due to the robust variable step-size and this variation from a high value to a low value is shown in the figure.

IV. SIMULATION RESULTS

The performance of the propose algorithms is illustrated for the system identification. The smoothed ensemble average square error (SEASE) expressed in dB, Sd ( n)

Sd (n) = 10 log10 ( S (n))

S (n) is a smoothed ( n) by a moving average filter.

( n) =

1 K

e i 2 ( n)
i =1

ei (n) = d i' (n) y i (n) - Error for ith realization and is derived using the system output without the observation noise and K is the number of realizations used in getting (n) . The plant chosen is an acoustic path of impulse response length N = 128 The DCT used is of 2 N points. The values of K = 50 . The observation noise used is both white (SNR=1dB)and coloured noise (SNR= -4dB). The coloured noise is generated by filtering white noise by a transfer function 1 . H (z) = 1 z 1 + 0.8 z 2 The input used is sum of three sinusoids of frequencies 50,100,150 Hz respectively and additive random noise ( SNR =18dB) . The parameters used for TVGLMS without and with desired signal decomposition are given in Table -1 and Table 2, respectively.
Table-1 Parameter Value Table-2 Parameter Value

min max

1e-4 0.99 1e-6 1e-4 0.99 1e-11 0.9999

min max

0.01 0.99 0.001 0.01 0.99 1e-11 0.9999 -

109

Proceedings of National Conference on Networking, Embedded and Wireless Systems, NEWS-2010, BMSCE

V. CONCLUSION

In this paper a new efficient robust transform domain Griffiths variable step size LMS (TVGLMS) algorithm was proposed for system identification both in the presence of white/ coloured observation noise. The new algorithm due to the input orthogonalization by the DCT, improves the slow convergence rate of VGLMS based on transversal adaptive filter. In this algorithm, both the gradient and the stepsize used are robust to observation noise. Further, as same gradient is used for both the weight vector and step size adaptation, it is computationally efficient. The TVGLMS algorithm results in a lower misadjustment as the gradient is free from observation

noise and also as the stepsize decreases towards convergence. The TVGLMS algorithm that decomposes the desired signal in addition to the input, has a better performance both in terms of convergence speed and the error magnitude, due to provision of removal observation noise for that component. The proposed algorithms are compared with the transform domain algorithm with out Griffiths gradient and without stepsize variation, and are found to be significantly superior to the latter in terms of misadjustment/ convergence error. The reduction in convergence error by the TVGLMS over TLMS, without desired signal decomposition is better than that provided by the one with decomposition. ACKNOWLEDGMENT I thank Ms Veena for her help in choosing the parameters for this new proposed algorithm.

REFERENCES [1] T.Abolnasar, K.M.Mayyas, A robust variable step-size LMS type algorithm: Analysis & simulation, IEEE Transactions on Signal Processing, Vol. 45(3), 1997, pp. 631637 [2] L. J. Griffiths, A simple adaptive algorithm for real time processing of in antenna arrays, Proc. IEEE, vol. 57, pp. 1696-1704, October 1967 [3] R. W. Harris, D. Chabries and F. Bishop, A variable step (VS) adaptive filter algorithm, IEEE Trans. Acoustics, Speech, and Signal Processing , Vol. APPS-34, No.2, pp. 309316, Apr. 1986. [4] R.H. Kwong and E.W. Johnston, "A variable step-size LMS algorithm", IEEE Trans. Signal Process., vol. 40, no. 7, pp. 1633-1642, July 1992. [5] James Okello et. al., A new modified variable step-size for the LMS algorithm, IEEE International Symposium on circuits and systems, pp. 170-173, Vol. 5, 1998 [6} T.J. Shan and T. Kailath, Adaptive algorithms with an automatic gain control feature, IEEE Trans. on Circuits And Systems, CAS-35, 1988, pp. 122-127 [7] Yonhhang Zhang, Ning Li, Jonathon. A. Chambers and Yanling Hao, New Gradient based Variable Step-Size LMS algorithms, EURASIP Journal on Advances in Signal Processing Vol. 2008, Article ID529480 [8] S.V.Narasimhan, S. Veena, Lokesha.Variable step-size Griffiths algorithm for improved performance of Feed forward/ Feedback Active Noise Control [9] Ho-Chi Hwang and Che-Ho Wei, Adaptive blind demodulation of DS/CDMA signals with transform domain Griffiths algorithm, IEEE International symposium on circuits and systems, (ISCAS-99) 30th May-2nd June 2, 1999

110