SET-MEMBERSHIP RECURSIVE LEAST-SQUARES ADAPTIVE FILTERING
ALGORITHM
Reza Arablouei* and Kutluyıl Doğançay†*
*
Institute for Telecommunications Research
†
School of Electrical and Information Engineering
University of South Australia, Mawson Lakes SA 5095, Australia
ABSTRACT references therein.
An SM filtering algorithm is typically formulated as a
A new set-membership adaptive filtering algorithm is set estimation problem and seeks solutions for a case that a
developed based on the exponentially-weighted RLS certain constraining assumption is made about the filter
algorithm with a time-varying forgetting factor that is output error. A usual assumption is a bounded magnitude for
optimized at each iteration by imposing a bounded- the filter output error. Several techniques have been
magnitude constraint on the a posteriori filter output error. proposed to estimate the target set of solutions, called
The new algorithm is designed to improve the numerical membership set, under the bounded error constraint. The
behavior of the previously proposed BEACON algorithm most prominent ones are the optimal bounding ellipsoid
while delivering the same convergence and tracking (OBE) algorithms that approximate the membership set by
performance as the BEACON algorithm. Simulation results tightly outer-bounding it with ellipsoids in the parameter
for a flat-fading MIMO channel estimation application space and optimize the size of the ellipsoid in some
demonstrate the superiority of the new algorithm over the meaningful sense. Different optimality criteria have led to
BEACON algorithm in terms of numerical stability. different OBE algorithms. The first OBE algorithm was
introduced in [4]. A thorough review of numerous further
Index Terms—set-membership adaptive filtering, RLS works developing the other members of the OBE family can
algorithm, BEACON algorithm, numerical stability, MIMO be found in [1].
channel estimation Among all the OBE algorithms, the Bounding
Ellipsoidal Adaptive CONstraind least-squares (BEACON)
1. INTRODUCTION algorithm [5] is particularly attractive since it shares many
of the desirable features of the various OBE algorithms.
Set-membership (SM) filtering algorithms are set-theoretic Furthermore, it incorporates simple but efficient innovation
estimation methods that unlike the traditional methods, e.g. check and optimal weight calculation processes, which
minimum mean square error (MMSE) or least-squares error make it computationally more efficient than other OBE
(LSE) filters, estimate sets of feasible solutions rather than algorithms.
single-point solutions. The SM approaches are of particular In this paper, we develop a new SM adaptive filtering
interest in signal processing applications because they algorithm based on the exponentially-weighted recursive
feature two major advantages over their traditional least-squares (EWRLS) algorithm with a time-varying
counterparts. First, they exhibit superior adaptation and forgetting factor that is optimized within the framework of
tracking properties. Second, they can effectively make use the SM filtering. In this sense, the proposed algorithm
of innovation in the data and improve computational differs from the OBE algorithms, which are based on the
efficiency by establishing a data-discerning update strategy weighted recursive least-squares (WRLS) algorithm with a
for the parameter estimates. More specifically, unlike the sequence of weights that does not have the functionality of a
traditional estimation schemes that implement a continuous sequence of forgetting factors. The proposed algorithm
update process regardless of the usefulness of the data, the enjoys an appreciably improved numerical behavior
SM algorithms assess the potential of the new data to compared to the BEACON algorithm while having the same
improve the quality of the estimate and weigh the data complexity and convergence performance as BEACON.
accordingly. This intelligent update strategy results in
discarding the data with unhelpful information content and 2. SET-MEMBERSHIP ADAPTIVE FILTERING
obviating the expense of updating when the data are
redundant. A more detailed and in-depth background on the Let us consider the affine-in-parameter model
SM filtering paradigm can be found in [1]-[3] and the
݀ሺ݊ሻ ൌ ܠ כሺ݊ሻ ݒሺ݊ሻ (1)
This work was supported in part by a Commonwealth Scientific and
Industrial Research Organisation (CSIRO) scholarship.
978-1-4673-0046-9/12/$26.00 ©2012 IEEE 3765 ICASSP 2012
Authorized licensed use limited to: SICHUAN UNIVERSITY. Downloaded on January 07,2025 at 12:45:49 UTC from IEEE Xplore. Restrictions apply.
where ݀ሺ݊ሻ אԧ is the reference signal at time index ݊, need to update it, i.e. ܟሺ݊ሻ ൌ ܟሺ݊ െ ͳሻ. Conversely,
אԧ is the column vector of the unknown system ȁ݁ሺ݊ሻȁ ߛ means that ܟሺ݊ െ ͳሻ is outside ሺ݊ሻ and needs
parameters, ܠሺ݊ሻ אԧ is the input vector, ݒሺ݊ሻ אԧ to be updated to a new vector ܟሺ݊ሻ that lies inside ሺ݊ሻ. In
accounts for measurement noise, ԧ denotes the set of this case, an update is carried out via (8) and (6) while the
complex numbers and superscript כstands for complex- optimum value for the weighting factor κሺ݊ሻ is found by
conjugate transposition. satisfying the bounded-error-magnitude constraint
Constraining the magnitude of the output estimation
ȁ݀ሺ݊ሻ െ כ ܟሺ݊ሻܠሺ݊ሻȁ ൌ ߛ (9)
error, ݁ሺ݊ሻ ൌ ݀ሺ݊ሻ െ ܠ כ ܟሺ݊ሻ, to be smaller than a pre-
determined threshold ߛ yields a specification on ܟ, which is which ensures that ܟሺ݊ሻ is a member of ሺ݊ሻ and
an estimate of . Consequently, there will be a set of consequently Ȳሺ݊ሻ. Multiplying both sides of (6) by כ ܠሺ݊ሻ
feasible solutions for ܟrather than a single estimate. The set and subtracting from ݀ כሺ݊ሻ yields
of all filter vectors ܟsatisfying the error constraint for all ͳ
possible input-desired output pairs in the model space ࣭ is ݀ሺ݊ሻ െ כ ܟሺ݊ሻܠሺ݊ሻ ൌ ݁ሺ݊ሻ (10)
called the feasibility set and is defines as ͳ κሺ݊ሻ࣡ሺ݊ሻ
where
ȣ ൌ ሩ ሼ א ܟԧ ǣ ȁ݀ െ ܠ כ ܟȁ ߛሽǤ (2)
ሺܠǡௗሻ࣭א
࣡ሺ݊ሻ ൌ כ ܠሺ݊ሻचሺ݊ െ ͳሻܠሺ݊ሻ. (11)
Direct calculation of ȣ is formidable and computationally Thus, by equating the RHSs of (9) and (10), κሺ݊ሻ is found
prohibitive. Hence, the adaptive SM filtering algorithms as
seek solutions that belong to a membership set Ȳሺ݊ሻ, which ͳ ȁ݁ሺ݊ሻȁ
is a superset of ȣ and is devised to be the minimal set κሺ݊ሻ ൌ ቆ െ ͳቇǤ (12)
࣡ሺ݊ሻ ߛ
estimate for ȣ at time instant ݊. The membership set is
defined by The BEACON algorithm is summarized in Table I. In this
algorithm, the norm of जሺ݊ሻ grows in time constantly
Ȳሺ݊ሻ ൌ ሩ ሺ݅ሻ (3) because of the accumulative term on the RHS of (5).
Growth of जሺ݊ െ ͳሻ decreases ࣡ሺ݊ሻ and so increases κሺ݊ሻ
ୀଵ
while a larger κሺ݊ሻ can in turn accelerate the growth of
where ሺ݊ሻ is the constraint set that contains all vectors ܟ जሺ݊ሻ. This positive feedback mechanism can typically push
satisfying the error bound at time instant ݊: the internal parameters, चሺ݊ሻ and κሺ݊ሻ, out of the realizable
ሺ݊ሻ ൌ ሼ א ܟԧ ǣ ȁ݀ሺ݊ሻ െ ܠ כ ܟሺ݊ሻȁ ߛሽ. (4) ranges in finite-precision implementations. The consequent
overflow/underflow of the parameter values can eventually
The membership set Ȳሺ݊ሻ is an ܮ-dimensional convex result in cessation of the adaptation since ՜ஶ चሺ݊ሻ ൌ .
polytope and still not easy to compute. Therefore, the OBE
algorithms estimate a sequence of ellipsoids instead that 4. THE SET-MEMBERSHIP RLS ALGORITHM
tightly outer-bound Ȳሺ݊ሻ.
Let us define
3. THE BEACON ALGORITHM
܀ሺ݊ሻ ൌ κିଵ ሺ݊ሻजሺ݊ሻ, (13)
Although BEACON is built upon the OBE concept, it can
be regarded as a WRLS algorithm with a time-varying ۾ሺ݊ሻ ൌ ି܀ଵ ሺ݊ሻ, (14)
weighting factor, κሺ݊ሻ, where the input autocorrelation and
matrix, जሺ݊ሻ, and the filter coefficients are updated via
κሺ݊ െ ͳሻ
जሺ݊ሻ ൌ जሺ݊ െ ͳሻ κሺ݊ሻܠሺ݊ሻ כ ܠሺ݊ሻ, (5) ߣሺ݊ሻ ൌ Ǥ (15)
κሺ݊ሻ
ܟሺ݊ሻ ൌ ܟሺ݊ െ ͳሻ κሺ݊ሻचሺ݊ሻܠሺ݊ሻ݁ כሺ݊ሻ (6) Multiplying both sides of (5) with κିଵ ሺ݊ሻ and using the
above definitions, we have
with the a priori estimation error being defined by
܀ሺ݊ሻ ൌ ߣሺ݊ሻ܀ሺ݊ െ ͳሻ ܠሺ݊ሻ כ ܠሺ݊ሻ, (16)
݁ሺ݊ሻ ൌ ݀ሺ݊ሻ െ כ ܟሺ݊ െ ͳሻܠሺ݊ሻ. (7)
In practice, चሺ݊ሻ ൌ जିଵ ሺ݊ሻ is updated rather than जሺ݊ሻ: ܟሺ݊ሻ ൌ ܟሺ݊ െ ͳሻ ۾ሺ݊ሻܠሺ݊ሻ݁ כሺ݊ሻ. (17)
כሺ݊ሻचሺ݊ Applying matrix inversion lemma to (16) yields
κሺ݊ሻचሺ݊ െ ͳሻܠሺ݊ሻܠ െ ͳሻ
चሺ݊ሻ ൌ चሺ݊ െ ͳሻ െ Ǥ (8)
ͳ κሺ݊ሻ ܠሺ݊ሻचሺ݊ െ ͳሻܠሺ݊ሻ
כ
۾ሺ݊ െ ͳሻܠሺ݊ሻ כ ܠሺ݊ሻ۾ሺ݊ െ ͳሻ
۾ሺ݊ሻ ൌ ߣିଵ ሺ݊ሻ ቆ۾ሺ݊ െ ͳሻ െ ቇǤ (18)
At time instant ݊, if ȁ݁ሺ݊ሻȁ ߛ, it is interpreted that ߣሺ݊ሻ כ ܠሺ݊ሻ۾ሺ݊ െ ͳሻܠሺ݊ሻ
ܟሺ݊ െ ͳሻ is inside the constraint set ሺ݊ሻ so there is no
3766
Authorized licensed use limited to: SICHUAN UNIVERSITY. Downloaded on January 07,2025 at 12:45:49 UTC from IEEE Xplore. Restrictions apply.
Table I, The BEACON algorithm. RLS
SM-RLS
- Initialization: BEACON
चሺͲሻ ൌ ߜ۷ where ߜ is a small positive number
and ۷ is the identity matrix
mean square error
ܟሺͲሻ ൌ 0
- At iteration ݊: 10
݁ሺ݊ሻ ൌ ݀ሺ݊ሻ െ כ ܟሺ݊ െ ͳሻܠሺ݊ሻ
if ȁ݁ሺ݊ሻȁ ߛ
࣡ሺ݊ሻ ൌ כ ܠሺ݊ሻचሺ݊ െ ͳሻܠሺ݊ሻ
ଵ ȁሺሻȁ
κሺ݊ሻ ൌ ቀ െ ͳቁ
࣡ሺሻ ఊ
-1
κሺሻचሺିଵሻܠሺሻ כ ܠሺሻचሺିଵሻ 10
चሺ݊ሻ ൌ चሺ݊ െ ͳሻ െ
ଵାκሺሻ כ ܠሺሻचሺିଵሻܠሺሻ
ܟሺ݊ሻ ൌ ܟሺ݊ െ ͳሻ κሺ݊ሻचሺ݊ሻܠሺ݊ሻ݁ כሺ݊ሻ 1 10 20 30 40 50 60 70 80
otherwise no. of iteration
Fig. 1. Mean square error performance of different algorithms.
चሺ݊ሻ ൌ चሺ݊ െ ͳሻ
ܟሺ݊ሻ ൌ ܟሺ݊ െ ͳሻ forgetting factor is optimized to satisfy the set-membership-
induced error bound, (9). Multiplying both sides of (17) by
Table II, The set-membership RLS algorithm.
כ ܠሺ݊ሻ and subtracting from ݀ כሺ݊ሻ gives
- Initialization:
ͳ
۾ሺͲሻ ൌ ߜ۷ ݀ሺ݊ሻ െ כ ܟሺ݊ሻܠሺ݊ሻ ൌ ݁ሺ݊ሻǤ (22)
ͳ ߣିଵ ሺ݊ሻܩሺ݊ሻ
ܟሺͲሻ ൌ
- At iteration ݊: Therefore, we can alternatively obtain (21) by equating the
݁ሺ݊ሻ ൌ ݀ሺ݊ሻ െ כ ܟሺ݊ െ ͳሻܠሺ݊ሻ RHSs of (22) and (9) and solving it with respect to ߣሺ݊ሻ.
if ȁ݁ሺ݊ሻȁ ߛ The BEACON and SM-RLS algorithms calculate the
ܩሺ݊ሻ ൌ כ ܠሺ݊ሻ۾ሺ݊ െ ͳሻܠሺ݊ሻ same filter coefficients though with a major difference in the
ீሺሻ way that they carry out the coefficient update process. In
ߣሺ݊ሻ ൌ ȁሺሻȁ BEACON, κሺ݊ሻ is a weighting factor, whereas in SM-RLS,
ቀ ିଵቁ
ം
۾ሺିଵሻܠሺሻ כ ܠሺሻ۾ሺିଵሻ ߣሺ݊ሻ acts as a forgetting factor. As a result, in SM-RLS,
۾ሺ݊ሻ ൌ ߣିଵ ሺ݊ሻ ቀ۾ሺ݊ െ ͳሻ െ ቁ increase of ܀ሺ݊ െ ͳሻ decreases ܩሺ݊ሻ and hence decreases
ఒሺሻା כ ܠሺሻ۾ሺିଵሻܠሺሻ
ܟሺ݊ሻ ൌ ܟሺ݊ െ ͳሻ כ
۾ሺ݊ሻܠሺ݊ሻ݁ ሺ݊ሻ ߣሺ݊ሻ, whereas a smaller ߣሺ݊ሻ leads to a smaller ܀ሺ݊ሻ.
otherwise Unlike in BEACON, this negative feedback mechanism
۾ሺ݊ሻ ൌ ۾ሺ݊ െ ͳሻ helps SM-RLS maintain its numerical stability.
ܟሺ݊ሻ ൌ ܟሺ݊ െ ͳሻ
5. SIMULATIONS
Substituting (12) into (15) gives
In this section, we compare performance of the SM-RLS
࣡ሺ݊ሻ algorithm with the conventional RLS algorithm and the
ߣሺ݊ሻ ൌ κሺ݊ െ ͳሻ Ǥ (19) BEACON algorithm for an application of flat-fading MIMO
ȁ݁ሺ݊ሻȁ
൬ െ ͳ൰ channel estimation studied in [6]. For this purpose, a MIMO
ߛ
communication system with four transmitter and four
By rewriting (11) as receiver antennas is considered. The sub-channels between
࣡ሺ݊ሻ ൌ כ ܠሺ݊ሻκିଵ ሺ݊ െ ͳሻ۾ሺ݊ െ ͳሻܠሺ݊ሻ, (20) all the transmitter and receiver pairs are independent
Rayleigh fading and vary in time based on Jakes model [7]
we can express (19) as with a normalized Doppler frequency ݂ ܶ௦ ൌ ͲǤͲͳ where ݂
ܩሺ݊ሻ is the maximum Doppler frequency shift and ܶ௦ is the
ߣሺ݊ሻ ൌ Ǥ (21) transmission symbol period. A sudden random change in the
ȁ݁ሺ݊ሻȁ
൬ െ ͳ൰ channel taps is also introduced halfway through the
ߛ
simulations. Four FIR filters each having four taps
where ܩሺ݊ሻ ൌ כ ܠሺ݊ሻ۾ሺ݊ െ ͳሻܠሺ݊ሻ. constitute the MIMO channel estimator. Similar to [6], in
The resultant new algorithm is summarized in Table II. the SM-RLS and BEACON algorithms, the norm of the
We call this algorithm set-membership recursive least- error vector composed by the errors of all the filters is used
squares (SM-RLS) algorithm. The reason for choosing this for the considered MIMO case in place of the absolute of
name is that the new algorithm is in fact an EWRLS the scalar error in the SISO case. The transmitted signal is
algorithm with a time-varying forgetting factor where the uncoded and modulated using QPSK scheme. It is grouped
3767
Authorized licensed use limited to: SICHUAN UNIVERSITY. Downloaded on January 07,2025 at 12:45:49 UTC from IEEE Xplore. Restrictions apply.
factor of the RLS algorithm. The important observation
from Figs. 2 and 3 is that, as predicted, the BEACON
norm of inverse autocorrelation matrix
0
10 algorithm is prone to numerical problems, in particular,
when its update frequency is high. It is evident from Figs. 2
and 3 that BEACON’s internal parameters, चሺ݊ሻ and κሺ݊ሻ,
-5
10 feature monotonic exponential increase/decrease in time.
This makes their dynamic range extremely wide and
consequently BEACON’s practicable run-time very limited.
-10 The experiment presented here shows that BEACON’s
10
optimal weight can grow from ͳ to ͳͲଵ in only ͺͲ
iterations. On the other hand, Figs. 2 and 3 show that the
RLS internal parameters of SM-RLS fluctuate around their
-15
10 SM-RLS
steady-state values and are of much smaller dynamic range
BEACON
rendering SM-RLS more suitable than BEACON for
1 10 20 30 40 50 60 70 80 practical applications.
no. of iteration
Fig. 2. The Frobenius norm of the inverse autocorrelation matrix
versus time for different algorithms. 6. CONCLUSION
A new set-membership adaptive filtering algorithm, called
RLS
15
10 SM-RLS
SM-RLS, was proposed. It was developed to improve
BEACON numerical behavior of the previously proposed BEACON
algorithm. Unlike the OBE algorithms that are known as set-
weighting/forgetting factor
10
membership weighted RLS algorithms, the proposed
10 algorithm is a set-membership exponentially-weighted RLS
algorithm with a time-varying forgetting factor that is
optimized within the set-membership adaptive filtering
5
10
context. Similar to BEACON, the proposed algorithm
exhibits a remarkable convergence and tracking
performance; however, it provides a dramatically improved
numerical behavior in comparison with BEACON.
0
10
7. REFERENCES
1 10 20 30 40 50 60 70 80
no. of iteration [1] J. R. Deller, Jr. and Y. F. Huang, “Set-membership
Fig. 3. Time evolution of the optimal weight of the BEACON identification and filtering for signal processing applications,”
algorithm and the optimal forgetting factor of the SM-RLS algorithm Circuits Systems Signal Process., vol. 21, no. 1, pp. 69-82,
together with the fixed forgetting factor of the RLS algorithm.
2002.
in packets of data each containing ͺͲ vectors of transmitted [2] J. R. Deller, Jr., M. Nayeri, and S. F. Odeh, “Least-square
symbols and these vectors are the common input to the identification with error bounds for real-time signal
processing and control,” Proc. IEEE, vol. 81, pp. 813–849,
filters of the MIMO channel estimator. For the RLS June 1993.
algorithm, a fixed forgetting factor of ͲǤͻ is used regarding
[3] J. P. Norton, Ed., Special issues on “Bounded-error methods
the assumed normalized Doppler frequency. For the SM-
in system identification,” Int. J. Automat. Contr. Signal
RLS and BEACON algorithms, the error threshold is set to Process., vol. 8, Jan./Feb. 1994; vol. 9, Jan./Feb. 1995.
ߛ ൌ ͲǤͳ. The energy per bit to noise power spectral density
[4] E. Fogel and Y. F. Huang, “On the value of information in
ratio is also ܧ Ȁܰ ൌ ͳʹ. system identification – bounded noise case,” Automatica, vol.
Fig. 1 compares mean square error (MSE) performance 18, no. 2, pp. 229–238, 1982.
of different algorithms when ensemble-averaged over ͳͲସ [5] S. Nagaraj, S. Gollamudi, S. Kapoor, and Y. F. Huang,
independent runs. As expected, Fig. 1 clearly shows that the “BEACON: An adaptive set-membership filtering technique
SM-RLS and BEACON algorithms perform similarly. It with sparse updates,” IEEE Trans. Signal Process., vol. 47,
should be noted that both the SM algorithms updated in no. 11, pp. 2928-2941, 1999.
average at ͺͲ percent of the iterations in this experiment. [6] T. Wang, R. C. De Lamare, and P. D. Mitchell, “Low-
Fig. 2 shows the Frobenius norm of the inverse complexity set-membership channel estimation for
autocorrelation matrix versus time for different algorithms. cooperative wireless sensor networks,” IEEE Trans. Veh.
Fig. 3 shows time evolution of the optimal weighting factor Technol., vol. 60, no. 6, pp. 2594–2607, 2011.
of the BEACON algorithm and the optimal forgetting factor [7] W. C. Jakes, Microwave Mobile Communications, New York:
of the SM-RLS algorithm together with the fixed forgetting Wiley, 1974.
3768
Authorized licensed use limited to: SICHUAN UNIVERSITY. Downloaded on January 07,2025 at 12:45:49 UTC from IEEE Xplore. Restrictions apply.
