Adaptive Filter Algorithm

Phatheera Jaisin, Eakkathep Boonmee, and Witawat Ruengsukwanna

School of Information,Computer and Communication Technology (ICT)

Sirindhorn International Institute of Technology, Thammasat University
P.O. Box 22, Thammasat-Rangsit Post Office Pathum Thani 12121, Thailand
Tel: +66 (0)2986 9009, 2564 3221-9, Fax: +66 (0) 2986 9112-3
E − mail : phatheera j@hotmail.com

Abstract for manipulating the adaptive filter’s coefficient be

Adaptive filter is the filter that is extensively
used in the communication system. This paper
aim to show the algorithms for adaptive filter
such as least mean square (LMS), normalized LMS
(NLMS) and variable step-size LMS (VSLMS) al-
gorithms. After that apply these algorithms in
some application of adaptive filter and compare
the performance of these algorithms. VSLMS al-
gorithm has the fastest speed of convergence, but
it has the smallest MSE.
same as unknown system’s coefficient. The block
diagram of system identification is shown in Fig-
ure. 2. In this research, the unknown system is
used in system identification with noise. At the
beginning of research, the author try to research
and understand the process of adaptive filter. Af-
ter that, use different algorithms in system iden-
tification comparing the performance of algorithm
by getting small MSE and fast time-convergence.

1 Introduction
In communication system, the characteristics of
input signal and the noise change with time, there-
fore it need the filter that adjust with the change
and achieve the desired output. This is done by
varying the coefficients of the filter in response to
the various changes[1] that is called adaptive filter.
Adaptive filter is used for many kinds of applica- Figure 2: System Identification Block Diagram
tions. The simple block diagram of adaptive filter
is shown in Figure.1.
2 Adaptive Filter Algorithm
2.1 LMS algorithm
LMS algorithm was first proposed by Widrow
and Hoff in 1960. It is the most widely used for
adaptive filter algorithm. It is simply approximate
version of steepest descent method[1] and is used
for adjusting a set of adaptive filter’s coefficient.
The simple block diagram is shown in Figure. 3.
The process of this algorithm, it send filter input
Figure 1: Adaptive filter Block Diagram that is specified by x(n) passing through adaptive
filter. It give filter output is specified by y(n) The
In this paper, the authors research the algo- process in adaptive filter is assumed by this equa-
rithms that are used in adaptive filter such as LMS, tion.
NLMS, VSLMS and some application of adaptive y(n) = w(n)T x(n) (1)
T
filter. The application that is researched is system where w(n) is transpose and n is time-index. It
Identification application. This application is used compares the filter output to the desired signal that
 Table 1: LMS Algorithm

Initialization
w(0)=0
Algorithm
For n = 0,1,2,...
y(n) = w(n)T x(n)
e(n) = d(n)-y(n)
w(n + 1) = w(n) + µ e(n)x(n)
Figure 3: An N-tap transversal adaptive filter

is specified by d(n) and gets the error by this equa-

tion. that is used for adjusting the coefficient of adaptive
e(n) = d(n) − y(n) (2) filter. It can be calculated by this equation.
The LMS algorithm try to adjust the adaptive fil- µe(n)x
w(n + 1) = w(n) + (5)
ter coefficient that make e(n) be minimized, it is x(n)T x(n)
shown in function mean square error(MSE), it use
the error to adjust the next coefficient of adaptive The rate of step-size value must follow this equa-
filter by this equation. tion.
0<µ<2 (6)
w(n + 1) = w(n) + µe(n)x(n) (3)
The convergence rate of NLMS algorithm is po-
where µ is step-size. And The performance of al- tentially faster, but the computation complexity is
gorithm can be shown in two variables, rate of con- higher than the standard LMS algorithm. The al-
vergence and misadjustment. The algorithm uses gorithm can be summarized in Table. 2.
value of µ to decide the rate of convergence of LMS
algorithm. When µ increase, it effect to increase Table 2: NLMS Algorithm
the rate of convergence that is used for showing
the speed of the algorithm converges toward the Initialization
optimal solution. The value of step-size must be w(0)=0
lie in this range that is shown by this equation. Algorithm
1 For n=0,1,2,...
0<µ< (4) y(n)=w(n)T x(n)
tap-input power
e(n)=d(n)-y(n)
w(n + 1)=w(n)+ xµe(n) x(n)
where tap-input power is the sum of mean square (n)T x(n)
values of all the tap inputs in the transversal filter.
Misadjustment is the fraction between Excess MSE
and optimum MSE that is attained from Wiener
filer. Where Excess MSE is the different between
steady-state MSE that is attained by adaptive al-
gorithm minus optimum MSE. For the good per- 2.3 VSLMS algorithm
formance of adaptive algorithm, it need low misad- VSLMS algorithm is the algorithm that can ad-
justment and fast rate of convergence. The algo- just the step-size that increase or decrease as the
rithm can be summarized in Table. 1. MSE increase or decrease [3]. The step-size of this
algorithm is adjusted by this equation.
2.2 NLMS algorithm
µ(n + 1) = αµ(n) + γe(n)2 (7)
NLMS algorithm is viewed as a special imple-
mentation of the LMS algorithm which takes into for 0<α<1 and γ>0 It is controlled by the square
account the variation in signal level at the filter of the predictor error. A large prediction error in-
input and select a normalized step-size parameter creases the step-size to provide faster tracking. If
which results in a stable as well as fast converging the prediction error decreases, the step size will be
adaptive algorithm[6]. This algorithm was devel- decreased to reduce the misadjustment[3].
oped from the different viewpoint of Goodwin & In this algorithm, the initial value of step-size is
Sin in 1984. The process in adaptive filter is same usually taken to be µmax . A sufficient condition
as LMS algorithm, but it is different in the equation for µmax to guarantee bounded MSE is shown by
this equation.
2
µmax ≤ (8)
3tr(R)
The algorithm can be summarized in Table. 3.

Table 3: VSLMS Algorithm

Initialization
w(0)=0
Algorithm Figure 5: Varying step-size
For n=0,1,2,...
y(n)=w(n)T x(n)
e(n)=d(n)-y(n) it will decrease the speed of system(µ=0.9). For
w(n + 1) = w(n) + µ(n) e(n)x(n) changing the number of tap-weight(M), it is var-
µ(n + 1) = αµ(n) + γe(n)2 ied for 4, 5 and 7. The result is shown in Figure.
6. When increasing the number of tap-weight, it

3 Simulation Result
3.1 Researching and understanding
the algorithm of adaptive filter
In this part, the authors research the LMS al-
gorithm and try to understand how this algorithm
works. The adaptive filter try to minimize the error
between the desired signal and output from adap-
tive filter. The result is shown in Figure. 4. After Figure 6: Varying tap-weight

means increase the number of adaptive filter coef-

ficient. It will decrease the speed of system and
if decreasing the number of tap-weight less than
unknown system coefficient, it will make adaptive
filter cannot adjusting itself. In figure. 7. It is
shown the way to adjust the adaptive filter coef-
ficient. In the end, the final value must same as
transfer function of unknown system(matrix h).

3.2 Changing the algorithm of adap-

tive filter
Figure 4: Error between desired signal and output
In this part, the author change the algorithm
signal
from LMS to NLMS and VSLMS. The result is
shown in Figure. 8-10. The value of step-size in
that, the authors research the system identifica-
LMS and NLMS is set 0.6 and 0.9 respectively. And
tion, it has unknown system with transfer function
the value of α and γ is 0.99 and 6x10−9 .
that is shown in this matrix h=[0.2 -0.8 2.0 -1.2
The value of MSE is shown in Table. 4
1.7]. The authors observe the effect of changing the
value of step-size(µ) and tap-weight(M). For µ, it
is varied form 0.3, 0.6 and 0.9. The result is shown Table 4: Values of Final MSE
in Figure. 5. From this research, the simulation re- Algorithm Final MSE (dB)
sult occurs be in line with the theory. If increasing LMS -27.0489
the value of step-size, it mean increasing speed of NLMS -28.2383
convergence time(µ=0.3, 0.6), but if increasing the VSLMS -21.9672
value of step-size more than the appropriate value,

Figure 7: Final value of adaptive coefficient
Figure 8: MSE plot for LMS algorithm
4 Future plan
Month Activity
November Starting an investigation
into DFT and DCT have
been planned.
December Combining DFT and DCT
with LMS, NLMS and VS
LMS algorithm and really
implementing them using
MATLAB program have
been planned.
January Making decision which
method or technique is
better in terms of con-
vergence speed and MSE
have been planned.
February Making further improve-
ments by adjusting para-
meter involved, such as
adjusting step-size,etc
have been planned
March The project documentation
and finalization processes
have been planned.
5 Conclusion
In conclusion, this paper aim at making an inten-
sive investigation into algorithms that are used in
adaptive filter and some application of adaptive fil-
ter(System identification). The values of step-size
Figure 9: MSE plot for NLMS algorithm
Figure 10: MSE plot for VSLMS algorithm
and tap-weight are important to adaptive. They
use for adjusting the speed of convergence. If in-
creasing step-size, it makes the speed of conver-
gence increase. The best performance of the sys-
tem must choose the optimal value for each system.
If the system does not take the optimal value, it
will give worse performance of the system. In the
application with different algorithm, VSLMS and
NLMS algorithm give the better performance than
LMS algorithm in convergence time, but VSlMS
algorithm give the smallest MSE value.
References
[1] Tamol Bose, 2004, ”Digital Signal and Image
Processing”, John Wiley & Son, United States
of America.
[2] B.Widrow,J. M. McCool, M. G. Larimore, and
C. R. Johnson, Jr.,”Stationary and nonstation-
ary learning characteristics of the LMS adap-
tive filter” Proc. IEEE, vol. 64, pp. 1151-1162,
Aug 1976.
[3] Raymond H. Kwong & Edward W. John-
ston.,”A Variable Step Size LMS Algorithm”
Proc. IEEE, vol. 40, pp. 1633-1642, July 1992.
[4] Simon Haykin, 2002, ”Adaptive Filter The-
ory Forth Edition”, Prentice-Hall, New Jersey,
United States of America.
[5] B.Farhang Boroujeny,”Adaptive Filters Theory
And Application”, John Wiley & Son, United
States of America.