Printer Friendly


Byline: Rajesh Dey, Suman Kumar Laha, Pradip Kumar Sadhu, Ankur Ganguly and Achintya Das

ABSTRACT: The paper exhibits a relative study of different alternates of LMS adaptive procedure for structure identification and also it helps to compare different variants of LMS with the standard LMS by considering the minimum mean square error,convergence speed and tracking proficiency. The comparisons of various adaptive LMS algorithms are investigated through correlated and uncorrelated involvement of several signals in stationary situations. All the simulation plots for MSE are found by a lot of collective averaging of almost 250 self-governing simulation process.

The simulation results exhibit that the convergence speed of NLMS (Normalized Least Mean Square) algorithm is quicker if it is being analogous to usual LMS algorithm but becomes slower than RVSS (Robust Variable Step Size) algorithm i.e. it proves that RVSS algorithm has the fastest convergence speed among all proposed algorithm for uncorrelated and correlated inputs in stationary environment. The RVSS algorithm illustrates the fastest chasing proficiency when it is subjected to an abrupt disturbance environment.

Keywords: Least Mean Square (LMS), Variants of LMS, Mean Square Error (MSE), Adaptive Filter, Structure Identification


An adaptive filter is called a self-modifying digital filter because it is a structure with a linear filter which has a transfer function precise by different parameters and it is used to adjust those parameters permitting to an optimization algorithm that can adjusts its coefficients in order to minimize an error function stated as like the cost function, is a difference between the desired or reference signal and the output of the adaptive filter. The proposal of a time-varying (adaptive) filter is considerably much more challenging than to design an old-fashioned (time invariant) Wiener filter as it is required to establish an optimum coefficients wn,k for k = 0,1,...,p and for each value of n. The adaptive filter algorithm has been used extensively for last few decades in numerous fields of electronics electrical and computer science, such as signal processing, image processing, speech processing, noise and echo cancelling, channel equalization and communication [1].

The selection of adaptive algorithm for system identification is purely depends on many factors like computational complexity, convergence speed, maladjustment, tracking capability etc. Computational complexity depends on number of operations. It also required memory size and investment is necessary to program the algorithm on a computer. Convergence speed is defined as how fast the coefficients of an adaptive filter reach close enough" to the optimum Wiener solution process. Maladjustment is a degree of excess error related to how adjacent the coefficients of an adaptive filter (the probable and the best) are close to each other in steady-state time. Tracking capability refers to the ability of an algorithm to track statistical distinction in a non stationary atmosphere. The LMS adaptive algorithm is derived from the sharpest decent method and it is used to evaluate the gradient vector from the existing data.

LMS uses a negative feedback to diminish the error by altering the filter coefficients in the way of the negative of the gradient vector to get the least mean square error (MSE). The LMS algorithm is very widespread and has been extensively used due to its nature of simplicity. Its speed of convergence, however, is very much dependent and inversely proportional to the situation number I of the input-signal autocorrelation matrix, generalized as the ratio among the maximum and minimum number of eigenvalues of the matrix [2]. The step size parameter u is a very important factor which controls the convergence speed of the adaptive LMS algorithm. There is a trade-off between fast convergence rate and small mean square error or misadjustment. When u decreases, both the convergence rate and misadjustment decrease. On the other hand, when u increases, the convergence speed is also increases but it gives raise to the higher value of misadjustment.

However, when the value of u is very large, it indicates to the uncertainty of the adaptive algorithm [3].

The step size u of the LMS algorithm is restricted by its region of stability which is determine by the energy of the given input signal. When the input signal power varies with time then the convergence speed of the LMS algorithm will also vary by its simple nature. For small input signal power the convergence speed will slow down and for the large input signal power, the over-shoot error would increase. This problem can be overcome by altering the step size parameter u continuously with varying the input power. So the step size u is standardised by the current input power, results in the Normalized Least Mean Square algorithm (NLMS). In case of LMS or NLMS adaptive algorithm, misadjustement is directly relative to the step size u and variation rate is inversely proportional to the step size u. To eliminate the compromise between misadjustement and Convergence speed, the step size u must be made time varying during the adaptation process.

A huge step size has been applied here at the time of early stage of adaptation when the adaptive filter coefficients are far away from the optimum one. When the filter coefficients are close enough to the best wiener explanation, the step size must be made small to reduce the misadjustment. The RVSS algorithm is based on this time varying step size approach. In case of RVSS algorithm, the step size u is dependent on both data and error normalization. The RVSS algorithm customs an approximation of autocorrelation among the output error at adjacent time instants E(n) and E(n-1) to control the step size over a wide range.

In this paper, we have analyzed various kinds of LMS algorithm like NLMS algorithm, RVSS algorithm for system Identification, by considering the minimum mean square error, convergence speed and tracking capability.

The paper has been structured as follows:

In segment II, the model for structure identification has been explained. Section III describes briefly the conventional LMS algorithm, while section IV outlines the variants of LMS algorithm. Simulation results for system identification have been shown in section V. Lastly, in section VI conclusions has been drawn.


Structure identification is the development of mathematical system based on measured input-output data. The basic block diagram for system detection technique is revealed in Fig.1.

The filter with adaptive coefficient and the unknown model are placed in parallel with each other. The similar input X(n) is used to excite the filter and unknown structure. The unknown structure impulse response behaviour is denoted by H(n) which have to be estimated and the adaptive filter coefficients is denoted by W(n) initially set to zero. Here V(n) is an additive white Gaussian noise (AWGN) which is added with the unknown structure output during the process of plant identification. D(n) is the required reference signal which is compared with the approximate output of the filter Y(n) to generate error signal E(n). This error signal E(n) is applied by all the LMS adaptive algorithm to update the weights of the adaptive filter.The correction of the adaptive filter coefficients are sustained until the error E(n) nearly equals to zero.

The error signal is expressed as

E(n)=D(n)-Y(n) (1)

The unknown structure is assumed to be a time independent structure, which implies that the coefficients of its impulse behavior are constant and of finite extent. Hence, we can write


The response of the filter with the equal number of coefficients, n is obtained by convolving W (n) by X (n) and is expressed as


When E (n) = 0 then D (n) = Y (n)

Under these conditions the coefficients of the filter W (n) are roughly equal with the coefficients of the unknown structure H (n).


Widrow and Hoff pretended the Least Mean Square (LMS) adaptive algorithm in the year of 1960 which is the simplest, and robust algorithms for filtering with adaptive coefficient. Adaptive filtering structure uses LMS algorithm to find its coefficients that relate to generate the least mean squares of the error coefficient [4]. It is a stochastic gradient method which do the adaptation of the coefficients based on the error at that time.

LMS introduces an iterative process that results in consecutive corrections of weights to achieve minimum mean square error. This MSE is the main criterion for evaluating the optimum performance of filtering with adaptive coefficient variation. In LMS algorithm, the filter weights are updated according to the expression

W(n+1) = W(n) + uE(n)X(n) (4)

The parameter u is the step size that controls the convergence rate of the algorithm.

The cost function or MSE is given by

I3/4 = E [E2 (n)] (5)

As the convergence of the LMS algorithm mainly depend on the step size u so it is important to select u to ensure convergence of the algorithm must be chosen in the range of

0 < u (6)

Where I>> is the maximum Eigen value of the autocorrelation matrix(R) of input signal X(n).

The convergence of the Least Mean Square algorithm is inversely proportional to the condition number of t input-signal autocorrelation matrix, depend on the ratio of maximum and minimum Eigen values of this matrix.


1. NLMS Algorithm

Normalized Least Mean Square algorithm (NLMS) is an updated version of LMS algorithm. The severe disadvantage of the LMS algorithm is having a fixed step size u which is expressed by the energy of the excitation signal. The convergence speed of the LMS algorithm will also vary simultaneously with the input signal power. By adjusting the step size parameter u the above mentioned problem can be solved. So the step size u is updated by the current input power, results in the Normalized Least Mean Square algorithm (NLMS) [5].

In a stationary process, the criterion for LMS algorithm to converge in the mean if 0 < u , and in the mean-square if 0 < u or R must be approximated.

One way is to use the fact that, for stationary processes,


Where N is the length of the adaptive filter.

So, the condition of convergence may be given by


That may be calculated as


This estimate the step size u for mean-square convergence:


A time varying step size in the form to achieve the goal of


Where the term [beta] is an approximated step size with 0 < [beta] < 2.

Replacing u in the LMS weight variation with un leads to the Normalized LMS (NLMS) algorithm:


2. RVSS Algorithm

The drawbacks of LMS or NLMS adaptive algorithm are that there is a balance between step size adjustment and adaptation rate. Adjustments vary with proportional to the step size and adaptation rate is inversely proportional to the step size. A symmetry between adjustment and adaptation rate tends to find an alternate robust algorithm where the step size u made time varying during estimation process [6]. A step size with high value is used during the initial stage of estimation when the filter coefficients are far away from the desired one. This will minimize the transient which results higher value of convergence speed. When the filter coefficients are near to the desired wiener solution, the step size should be taken as small to decrease the maladjustment. The Robust Variable Step Size (RVSS) is a special type of LMS algorithm where variable step size (VSS) LMS algorithm based on the time estimated step size method.

In RVSS algorithm, the step size u is function of data and error estimation simultaneously [7].In RVSS algorithm autocorrelation between the error at time instants E (n) and neighboring time instants E(n-1) to control the step size. The changeable step size algorithm uses the updated coefficient equation in the form as


Where u (n) is a time changeable step size.

The following equations are given for updating the time variable step size -




With 0 < [alpha] <1, 0 < 0 being some constant parameters and the term umax and umin are the upper and lower limits of the time-varying step size. The exponential weighted coefficients [beta] regulates the factor of the estimating P (n).

The value of umax is generally selected near to the instability point of the general LMS is used to improve the adaptation rate while the minimum value of u is chosen to deliver a slight misadjustment at the time of steady state.


The simulation has been performed using MATLAB Version-10. In all simulated results or plots the length of the unknown System impulse is taken to be N=3 and a filter with adaptive coefficients of correct order is used to identify the unknown structure. In each cases, unknown structure noise components V(n) is taken as white Gaussian noise which has zero-mean and variance with a value of 0.02 or -17 dB. All of the simulation results for MSE are performed by ensemble averaging of 250 independent simulations process.

For White Gaussian Input:

A white Gaussian signal whose value considered with zero-mean and variance of unity is applied as input for both the e filter and unknown structure.

The impulse behaviour of the unknown structure is taken as H = [0.8 0.3 0.5] therefore length of the adaptive filter is also approximated as 3. The Step Size (u) used in each algorithms is selected such a manner to obtain exact value of misadjustment (M) equal to 2.2%. The calculated results of the step size parameter for the LMS, NLMS and RVSS algorithms are like this 0.0146, 0.0203 and 0.185.

For Correlated Input signal:

In this experiment, instead of using only a white Gaussian noise a correlated signal is used to excite both the adaptive filter and unknown structure which is generated by the following model:

X(n) = 0.9X(n-1) +F(n)

Where F(n) is used as a white Gaussian noise with zero-mean and unity variance independent with the plant internal noise. The correlated signal has larger total signal power compared to the power of the input signal which is used in the previous experiment. As the input signal is coloured signal so the Eigen values are widely spread which will make the convergence more difficult. The step size Parameter (u) in all algorithms is chosen to obtain the same exact value of misadjustment (M) equal to almost 7%. The calculated values of the step size parameter for the LMS, NLMS and RVSS algorithms are as follows 0.0466, 0.028 and 0.0125.

For Correlated Input with abrupt change in the plant parameters:

This experiment is same as the previous one only an abrupt change was made by multiplying all the system coefficients of the unidentified structure by -1 at the very middle of the adaptive procedure i.e., at iteration number 600.


Here we have studied and analyzed conventional adaptive LMS algorithm and its variants like NLMS, RVSS algorithm for linear structure identification by considering the low computational complexity, convergence speed, adaptation error and tracking capability for updating an adaptive filter. Simulation plots showed that the convergence speed of NLMS algorithm is quicker than conventional LMS algorithm but slower than RVSS algorithm as shown in Fig.2, Fig.3 and Fig.4. Though the RVSS algorithm has fastest convergence speed rather its complexity is much more than the conventional LMS and NLMS algorithm. From the Simulation plots, it is also seen that the RVSS algorithm converges faster than simple LMS and NLMS algorithm for both white Gaussian input and correlated input to obtain the same minimal MSE a shown in Fig.6 and Fig 7. The RVSS algorithm shows better tracking capability when introduced with an abrupt disturbance in the plant parameters as shown in Fig.8.

When convergence speed and tracking capability are crucial to the function, RVSS algorithm will always be a superior alternative than the other algorithms. The only drawbacks of RVSS algorithm is that it has a higher execution time rate with increasing complex nature than the other proposed algorithms.


[1] S.Heykin, Adaptive Filter Theory, Prentice-Hall, Upper Saddle River, NJ, 2001.

[2] KumpatiS.Narendra, andKannanParthasarathy, "Identification and Control of Dynamical Systems using neural networks", IEEE Transactions on Neural Networks, vol.1, No.1, March 1990.

[3] Symon Heykin, "Adaptive Filter Theory", Radial Basis Function. Networks, Chapter 20, pp 855-874, PHI Publication.

[4] B. Farhang - Boroujeny "On statistical efficiency of LMS algorithm system modeling," IEEE Trans. Signal Process. Vol. 41, pp. 1947-1951, 1993.

[5] Sanjeev Dhull, Sandeep Arya and O.P. Sahu, "Performance Variation of LMS And Its Different Variants", International Journal of Computer Science and Security, (IJCSS), Volume (4): Issue (5), pp 491-496.

[6] T. Aboulnasr and K. Mayyas, A Robust Variable Step-Size LMS-Type Algorithm: Analysis and Simulations, IEEE Trans. on Signal Proc., vol 45, No. 3, pp.631-639, March 1997.

[7] Z. Ramadan and A. Poularikas, "A robust variable step-size LMS algorithm using error-data normalization", Proc. of the IEEE SoutheastCon, pp. 219-224, 8-10 April 2005.

[8] Y. D. Landau and G. Zito, Digital Control Systems Design, Identification and Implementation. Berlin; New York: Springer, 2006.

[9] Andrew Dale, Zentralblatt MATH,"QRD-RLS adaptive filtering", vol 1170, 2009.

[10] Zayed Ramadan,"Error vector normalized adaptive algorithimapplied to adaptive noise canceller and system identification" American Journal of Engineering and Applied Sciences, Vol 3, Issue 4, 2010

[11] B.V. Uma, K.V. Padmaja, Dr. S. Ravishankar," Realization of Adaptive NEXT canceller for ADSL on DSP kit", Proceedings of the 5th WSEAS Int. Conf. on Data Networks, Communication and Computers, Bucharest, Romania, October 16-17, 2006.

[12] Z Ramadan,"A robust variable step size LMS algorithim using error data normalization adaptive filter application", Proceedings IEEE, 2005.

[13] Z Ramadan,"Performance analysis of a new variable step size LMS algorithm with error non linarites", Proceedings of IEEE thirty sixth southeastern symposium, 2004.

[14] Nagal and Rachana, Pradeep Kumar and Poonam Bansal,"An approach to implement LMS and NLMS adaptive noise cancellation in frequency domain", 5th international conference -confluence the next generation technology, 2014.

[15] Yonggang Zhang, Ning Li, Jonathon A Chambers and Yanling Hao,"New gradient based variable step size LMS adaptive algorithims", Journal on Advances in Signal Processing, 2008.

[16] R.A Haddad,"A variable step size LMS algorithim", Proceedings of the 33rd midwest symposium on circuits and systems, 1991.

[17] Rashi Mehrotra,"A framework for analysis of optimal power solutions for smart antennas", IEEE International conference on Microwaves communication antennas and Electronics systems, 2011.

[18] Simon, Haykin,"Adaptive system for signal process", Electrical Engineering and applied signal processing series, 2009.

[19] Zerguine A,"A noise constrained least mean fourth adaptive algorithim", 2011

[20] H.A.Leinhos,"Block adaptive decision feedback equalization with integral error correction for underwater acoustic communication", IEEE conference and Exihibitation conference proceedings, 2000.

[21] SM Kuo, DR Morgan,"Active noise control", Springer handbook of speech Processing, 2008.

[22] A Busboom,"Adaptive receiver for mobile DS-CDMA communication system IEEE 47th Vechuler technology conference Technology in motion, 1997.

[23] Vaibhav Narula,"Assesments of variants of LMS algorithims for noise cancellation in low and medium frequency signals", International conference on recent advancement in Electrical Electronics and Control Enginering, 2011.

[24] Al Saggaf, Ubaid M, Muhammad Mainuddin, Muhammad Arif and Azzedine Zerguine,"The q-Least Mean Square algorithim", Signal Processing, 2015.

[25] S.Dhar,"Comparision of different channel estimation algorithims from ITS prospective", IEE conference, 2011.
COPYRIGHT 2017 Asianet-Pakistan
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2017 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Dey, Rajesh; Laha, Suman Kumar; Sadhu, Pradip Kumar; Ganguly, Ankur; Das, Achintya
Publication:Science International
Article Type:Report
Date:Aug 31, 2017

Terms of use | Privacy policy | Copyright © 2022 Farlex, Inc. | Feedback | For webmasters |