Adaptive LMS Algorithm System Identification using

LabVIEW
E. Szopos1, M. Topa1, I. Dornean1, R. Groza1
1
Technical University of Cluj-Napoca, Faculty of Electronics, Telecommunications and Information Technology
[email protected], [email protected], [email protected], [email protected]

Abstract—System identification allows building up dk

mathematical models (transfer functions, systems of differential xk LMS yak yk +
+ -Σ
equations) for a dynamic system based on measured data. This a[k]
paper proposes a LabVIEW implementation of the Least Mean
ek
Squares (LMS) algorithm in the system identification problem.
The LMS was chosen for its computational simplicity and good LMS ybk
b[k]
convergence. The unknown system is modeled as a FIR or IIR
filter. The LabVIEW implementation was developed to identify
systems of any order and type. The error signal and the number
of iterations measure the performance of the algorithm.
Figure 1. The adaptive filter.
I. INTRODUCTION
where a[k], b[k] are the filter coefficients, M and N are the
System identification involves building mathematical models number of zeros and poles.
of a dynamic system based on a set of measured stimulus and
B. The Adaptive Algorithm
response data samples. The usage is in a wide range of
The identification consists in estimation of the coefficients
applications, including communications, mechanical
a[k], b[k] of the filter. Figure 1 shows an IIR filter, having the
engineering, biology, physiology, meteorology, economics and
coefficients updated using the LMS algorithm [1], [2], [3].
model-based control design.
Adaptive IIR filters are attractive due to the fewer coefficients
Identifying a model of a system is an iterative process; first,
to achieve the desired performance. However, it is more
raw data is acquired from a real-world system, then a
difficult to develop stable IIR algorithms than FIR ones, due to
mathematical algorithm is selected to process the data and
slowly convergence and local minim susceptibility.
model the system. The algorithm is run until the obtained
The output of the filter is [3], [4]:
model is below the desired error, if it fails, another algorithm is
selected and the procedure continues. The resulting
mathematical model is then used for the analysis of the system. Yk = ak T ⋅ X k + bk T ⋅ Yk -1 , (2)
Our paper aims to show the implementation in the LabVIEW
environment of the Least-Mean Square (LMS) algorithm used where Xk = [xk xk-1 ... xk-L+1]T is the data vector of the L most
for identification of systems of any order and type. The obtained recent input samples. The LMS algorithm starts by setting all
results will be further used in the research related to room the weights to zero at time k=0. The coefficient vectors are
transfer function identification [5]. updated with a small amount according to [1], [2]:
In section II the theoretical approach regarding the unknown
system model, the LMS algorithm and the problem of system ak = ak -1 + 2 μ ⋅ ek X k -1
identification are presented. Section III shows the results of , (3)
bk = bk -1 + 2 μ ⋅ ek Yk -1
several simulations made in LabVIEW. The last section is
devoted to the conclusions.
where dk is the desired output of the filter, μ is the convergence
II. PRESENTATION OF THE METHOD factor or step-size, ek is the error signal:

A. The Unknown System Model ek = d k − yk = d k − ( yak + ybk )

Let the unknown system be described by one of the
following transfer function [1], [5], [7]: ⎛ M -1 N ⎞, (4)
∑
= d k − ⎜ ai [k ] ⋅ x[k − i ] +
⎜ i =0 ∑ b [k ] ⋅ y[k − j ] ⎟⎟
j
M ⎝ j =1 ⎠
M ∑ a[k] ⋅ z -k The step size controls the stability and the convergence rate of
H FIR (z) = ∑ a[k] ⋅ z -k ; H IIR (z) = k=0
N
, (1)
the adaptive algorithm. To ensure convergence, μ must satisfy
k=0
1 - ∑ b[k] ⋅ z -k the condition:
k=1
 2 xk Unknown
0<μ< , (5)
total input power System
dk
yk ⎯
+
where the total input power refers to the sum of the mean- Ref. input Out 1 Σ
square values of the inputs xk, xk-1, ..., xk-L+1. IIR, FIR
Error ek
Model Adaptive
C. The System Identification Problem M, N select LMS Decision
COMM
block
Figure 2 shows an adaptive LMS filter playing the system μ
identification role [1], [7], [8], where the error signal ek is Error range
Out 2
obtained by subtracting the adaptive filter’s output yk, from dk. a[k], b[k]
The coefficients of the unknown system are continuously
updated until the error signal is in a range set by the user. After
Figure 2. The system identification block diagram.
adaptation the unknown system, labeled H(z) is modeled in the
sense that its transfer function is the same as the one of the The value of the step size was set to 1E-2 and for the error
adaptive system. range ±1E-11 for the cases (1), (4), 1E-8 for the case (2) and
The input xk applied to the adaptive and the unknown systems 1E-7 for (3).
should be a stochastic signal. In the very beginning of the Tables I and II show the performance of the identification
identification procedure, the user has to set up the model for the carried out and the location of the figures illustrating the
unknown system, the number of zeros and poles (M, N), the process. The quality of the process was measured in terms of
value of step size μ and the error range. At iteration k=0 the absolute error (ek) and number of iterations (no. iter.).
filter coefficient vectors a[k], b[k] are initialized to 0. The error
signal ek, is computed in each iteration (4), the filter coefficients TABLE I
are estimated and updated and then the error signal is compared THE PERFORMANCE OF IDENTIFYING AN IIR UNKNOWN SYSTEM
IIR with
to the acceptable error. If the error signal is not in the range, a Unknown
a[n]={9.02E-2, -2.708E-1, 2.708E-1, -9.02E-2}
new set of estimated coefficients is computed. The process system
b[n]={1, 0.6905, 0.8019, 0.3892}
continues until the error signal approaches the imposed range IIR ek = 8.98E-12, no. iter.=70793,
and then the Decision block generates a command signal to stop Identified (case 1) (Figures 4, 5, 6)
system FIR ek = -9.02E-9, no. iter.=1598,
the process. If the system was not identified, the error limits or (case 2) (Figures 7, 8)
the number of poles N and/or zeros M have to be increased,
otherwise the identification will fail again. TABLE II
THE PERFORMANCE OF IDENTIFYING A FIR UNKNOWN SYSTEM
III. IMPLEMENTATION AND SIMULATIONS Unknown FIR with
system a[n]={0.197, 0.201, 0.203, 0.201, 0.197}
LabVIEW has been on the market since the middle of 80’s IIR ek =9.53E-9, no. iter.=13609,
and is associated with virtual instrumentation. It is a block- Identified (case 3) (Figures 9, 10)
diagram approach to programming and is based on the system FIR
ek = 3.86E-12, no. iter.=2955.
graphical language. There is a wealth of information available (case 4)
on how to use LabVIEW to control a whole range of
instrumentation, but far less on the basics of signal processing
[9], [10].
This paper is not a detailed analysis of adaptive signal
processing, but rather an description of how LabVIEW can be
used as a scientific tool when implementing such algorithms.
Several simulations on identification of systems with the
adaptive LMS algorithm were carried out in LabVIEW. The
input xk is a wide band random signal: either a white or a
Gaussian noise [4], [6].
There are 4 different cases: (1) – the unknown IIR is
modeled as IIR; (2) – the unknown IIR is modeled as FIR; (3)
– the unknown FIR is modeled as IIR and (4) – the unknown
FIR is modeled as FIR.
To illustrate the capability of the algorithm an IIR filter of
3rd order was considered as unknown system for the cases (1) Figure 3. LabVIEW implementation of the LMS adaptive algorithm.
and (2) and a FIR of 4th order for the cases (3) and (4). The
number of samples L for the input signal was set to 1024. Figure 3 shows the LabVIEW implementation of the LMS
Many simulations were made using large values of the input algorithm in matrix form (see (3), (4)). The LMS inputs are
length, but the precision did not increase significantly and the described mathematically at time sample k as primary input
computation times became too long. (xk) and reference input (uak, ubk). The signal dk is the unknown
system’s output, which is also the desired output. The adaptive
algorithm is implemented for convenience as a sub vi (virtual
instrument file). It is contained within a While loop which runs
until the error ek reaches the imposed range. This loop contains
shift registers that are required to store past values of input,
output and coefficients (regressors). The input and output to the
LMS algorithm are both scalar quantities. As the library
function that generates Guassian or white noises is
fundamentally an array, it is necessary to make this of length
Figure 5. The error signal for case (1) of modeling.
one sample. Otherwise the noise generator output will stay as
type array and the connection to the LMS sub will not be
possible. The implementation enables both the FIR and IIR
identification by setting suitable the Model select input (Figure
2). If FIR is selected then only the A_block is enabled (Figure
3) and the unknown system is identified as a FIR one. If IIR is
selected then both blocks (A and B) are enabled and the
unknown system is IIR modelled. Each of the blocks represents
a FIR LMS and connecting them according (4) an IIR LMS is
obtained. This implementation permits to identify discrete
systems of any order and type by setting M, N, the Model select
input and the error range suitable.
Figures 4 and 5 show the evolution of the numerator’s
coefficients a[k], denominator’s coefficients b[k], the error
signal in the identification process for case (1). One should see
that all the coefficients vary with small ripples in the transient
regime and the error signal decreases slowly. Thereby the Figure 6. Frequency characteristics at iteration 27000 (M=N=4, ek= 4.38E-4)
number of iterations will increase. Figure 6 shows the in the IIR modeling process of an IIR system – case (1).
frequency characteristics close to the final iteration in the
identification process for case (1). One should see that for
relatively small values of ek the number of iterations is very
large, but M and N have lower values and could be proximate
(M≈N). The process continues until the matching situation is
reached, ek is very small (≈1E-11) and the magnitude and phase
characteristics fit well one to another.
In case (2), M was set to 64; the identification of the
unknown system is very fast (Figures 7, 8). If a low order FIR
is required, then a lower value should be set for M and the Figure 7. The error signal for case (2) of modeling
process should be repeated; if better precision is needed, the
order M must be larger (Figure 8).

Figure 8. Frequency characteristics at iteration 780 (M=64, ek= –7.42E-4) in

the FIR modeling process of an IIR system – case (2).

Figure 9 shows the error signal and Figure 10 the frequency

Figure 4. Evolution of the numerator and the denominator coefficients (ak, bk). plots in the FIR system modeling as IIR – case (3). Several IIR
versions of the unknown FIR system can be obtained for but the identified system’s order N is the smallest, in
different values set for M and N. For a better identification (low comparison with case (4) (FIR modeling of a FIR), which is the
value for ek, large values for M and N), the number of iterations fastest, but with the largest order M.
will increase. For a very low error range and very large values The convergence of the algorithm depends on the choices of
for M, N, the model will change from an IIR (case (3)) to a FIR the step sizeμ, the error range and the number of poles N, zeros
(case (4)). M. If μ is too large, the algorithm will converge very fast, if μ
is too small the convergence speed will be rather slow. As for
the error range, if it is chosen to be too large, the algorithm will
converge very fast and thereby the error signal will have large
values. When a too small error range is chosen, the algorithm
may not converge, if not so, the number of iterations will be
too large, some of the filter parameters having very small
values. Concerning the number of poles and zeros (N and M), if
these parameters have large values, the computational time will
increase considerably, but the system will be identified with
high precision. If N and M have small values the identification
Figure 9. The error signal – case (3). will be achieved with low accuracy, and thereby the error
signal will have large values.
Further research will be carried out for developing an
improved model of acoustic rooms for artificial reverberation.

REFERENCES
[1] B. Farhang-Boroujeny, “Adaptive filters. Theory and applications”, John
Wiley & Sons, West Sussex, England, 1999.
[2] G. Glentis, K. Berberidis, S. Theodoridis, “Efficient least squares
adaptive algorithms for FIR transversal filtering”, IEEE Signal
Processing Magazine, pp. 13-41, 1999.
[3] S. Zhang, D. Li, J. Sun, “Power system frequency tracking using an
adaptive lattice notch filter”, Trans Tech Publications, Switzerland, Vol.
295-296, pp. 673-680, 2005.
[4] E. Szopos, M. Ţopa, N. Toma, “System Identification with adaptive
algorithms”, IEEE 7-th Emerging Technologies Workshop: Circuit and
Systems for 4G mobile Wireless Communications-ETW, St. Petersburg,
Russia, pp. 64-68, 2005.
[5] N. Toma, M. Ţopa, V. Popescu, E. Szopos, “Comparative performance
Figure 10. Frequency characteristics at iteration 850 (M=N=4, ek= 1.28E-2) analysis of artificial reverberation algorithms”, IEEE-TTTC International
in the IIR modelling process of a FIR system – case (3). Conference on Automation, Quality and Testing, Robotics, Romania, pp.
138-142, 2006.
[6] H. Oba, M. Kim, H. Arai, “FPGA implementation of LMS and N-LMS
IV CONCLUSIONS processor for adaptive array applications”, IEEE International
Symposium on Intelligent Signal Processing and Communications
The paper presents the systems identification process using
Systems, Yonago Convention Center, Tottori, Japan, 2006.
the adaptive LMS algorithm implemented in the LabVIEW [7] E. Szopos, M. Topa, I. Lazar, “System identification with the Pade
environment. A FIR or IIR filter describes the unknown algorithm”, Proceedings of the International Conference on Applied
system. The implemented algorithm is capable to identify an Electronics, Plzen, Czech Republic, pp. 215-218, 2007.
[8] I. Dornean, M. Ţopa, B. Kirei, E. Szopos, "FPGA implementation of the
IIR or FIR system of any order. The input parameters for the
adaptive least mean square algorithm", Acta Technica Napocensis, Vol.
algorithm are the step-size μ, the number of poles N and zeros 48, No. 4, pp.19-22, Romania, 2007.
M, the error range. These parameters lead to several versions of [9] Johnson, G.W., LabVIEW Graphical programming, McGraw-Hill series,
the unknown system of different orders. The input signal is a NY, 1994.
[10] Chugani,M,L,Samant,A.R and Cerna,M., LabVIEW Signal Processing,
wide band random signal provided by LabVIEW. National Instruments Series, Prentice Hall, NJ, 1998.
The implemented algorithm has been tested for all the 4
possible cases of modeling IIR/FIR systems with IIR/FIR
models. The experimental results confirm that case (1) (IIR
modeling of an IIR) is the slowest regarding the error signal,