Maximum Versoria Criterion-Based Robust Adaptive Filtering Algorithm
Maximum Versoria Criterion-Based Robust Adaptive Filtering Algorithm
Maximum Versoria Criterion-Based Robust Adaptive Filtering Algorithm
Abstract—Using the generalized Gaussian probability density the MMSE criterion do not perform well in nonlinear and non-
function as the kernel, a generalized correntropy has Gaussian interference situations, especially in the presence of
been proposed. A generalized maximum correntropy crite- non-Gaussian impulsive noises.
rion (GMCC) algorithm is then developed by maximizing the
Non-Gaussian interferences (including heavy-tailed non-
generalized correntropy. However, the GMCC algorithm has
a high steady-state misalignment and involves a high calcula- Gaussian interferences and light-tailed non-Gaussian interfer-
tion cost of the exponential term (generalized Gaussian kernel). ences) often occur in practical applications. Heavy-tailed (e.g.,
In this brief, we propose a maximum Versoria criterion (MVC) Laplace, α-stable, etc. distributions) non-Gaussian (impul-
algorithm, which is derived by maximizing the generalized sive) interferences produce more outliers than those assumed
Versoria function, to reduce steady-state misalignment and com- by Gaussian models. The characterization of a non-Gaussian
putational effort as compared to the GMCC algorithm. The MVC signal by its second order moment is no longer optimal
algorithm is then tested in system identification and acoustic
and many studies have shown that lower order statistic can
echo cancellation scenarios, which have demonstrated that the
proposed algorithm is robust against non-Gaussian impulsive lead to improved robustness against non-Gaussian impulsive
noises and performs much better than the LMP and GMCC interferences. In such situations, the sign algorithm (SA) [6],
algorithms. the affine projection sign algorithms (APSAs) [7]–[9], and
Index Terms—Adaptive filtering, Gaussian probability density
the continuous mixed p-norm (CMPN) algorithm [10] yield
function, generalized maximum correntropy criterion (GMCC), improved robustness against non-Gaussian impulsive noises.
maximum Versoria criterion (MVC). Using higher order moment algorithms to deal with light-
tailed (e.g., uniform, binary, etc. distributions) non-Gaussian
interferences are more suitable. For this situation, the least
I. I NTRODUCTION mean fourth (LMF) algorithm yields improved performance in
terms of the convergence rate and steady-state misalignment,
HE MINIMUM mean square error (MMSE) criterion
T is widely used in adaptive filtering due to its low
computational complexity and simplicity [1]. Based on the
as compared to the LMS algorithm [11].
Recent years, correntropy has been successfully applied
in non-Gaussian signal processing [12], [13]. The corren-
MMSE criterion, the least mean square (LMS) algorithm as tropy is a nonlinear similarity measure between two arbitrary
a method optimized for Gaussian models was developed. scalar random variables. Meanwhile it has a close relation-
The LMS algorithm is the most popular adaptive filtering ship with M-estimation [12]. Thus, the maximum correntropy
algorithm. To obtain improved performance as compared to criterion (MCC) algorithm is robust against non-Gaussian
the LMS algorithm, the variable-step-size LMS (VSS-LMS) impulsive noises. Lately, a generalized correntropy has been
and normalized LMS (NLMS) algorithms have been devel- proposed in [14]. Then, an adaptive filtering algorithm,
oped in [2] and [3], respectively. Furthermore, in order to called generalized maximum correntropy criterion (GMCC)
offer improved convergence performance and reduce compu- algorithm, was developed by maximizing the generalized
tational complexity relative to the NLMS algorithm, the non- correntropy [14]. The disadvantages of the GMCC algorithm
parametric VSS-NLMS (NPVSS-NLMS) and set-membership are high steady-state misalignment, and high calculation cost
NLMS (SMNLMS) algorithms were proposed in [4] and [5], of the exponential term (generalized Gaussian kernel) in the
respectively. Unfortunately, these algorithms developed under update equation of the weight vector. The Versoria function
Manuscript received December 13, 2016; revised February 1, 2017 has been successfully applied in the particle swarm opti-
and February 7, 2017; accepted February 14, 2017. Date of publication mization (PSO) algorithm for IIR adaptive filtering [15]. The
February 17, 2017; date of current version September 25, 2017. This work was PSO algorithm uses a modified Versoria function which is
supported by the National Natural Science Foundation of China under Grant
61271341 and Grant 61671392. This brief was recommended by Associate
employed to provide an adaptive inertial weight factor instead
Editor C.-T. Cheng. of the Sigmoid function for avoiding the exponential compu-
F. Huang and J. Zhang are with the Sichuan Province Key tation and ensuring the low steady-state misalignment. The
Laboratory of Signal and Information Processing, Southwest Jiaotong Versoria function in the PSO plays a similar role of vari-
University, Chengdu 611756, China (e-mail: [email protected];
[email protected]). able step-size. Unlike the traditional usage of Versoria, we use
S. Zhang is with the School of Computing, Engineering and Mathematics, the Versoria as the cost function in this brief. Thus, a max-
University of Western Sydney, Sydney, NSW 2751, Australia (e-mail: imum Versoria criterion (MVC) algorithm is developed by
[email protected]).
Color versions of one or more of the figures in this paper are available
maximizing the generalized Versoria function, which reduces
online at http://ieeexplore.ieee.org. the steady-state misalignment and the computational effort as
Digital Object Identifier 10.1109/TCSII.2017.2671521 compared to the GMCC algorithm. The performance of the
1549-7747 c 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY DELHI. Downloaded on May 22,2024 at 05:56:47 UTC from IEEE Xplore. Restrictions apply.
HUANG et al.: MVC-BASED ROBUST ADAPTIVE FILTERING ALGORITHM 1253
II. P ROPOSED M AXIMUM V ERSORIA Fig. 1. Comparison of steepness of the GPDF and the standard Versoria
C RITERION A LGORITHM function.
Define the input vector of an unknown system as xk = [xk ,
xk−1 ,. . . , xk−L+1 ]T , where k is the time index, L is the filter where p > 0 is the shape parameter, and τ = (2a)−p . In
length, and superscript T denotes vector transpose operation. this representation, as a special case, it includes the original
The output signal of the unknown system is modelled as Versoria function when p = 2.
In order to reduce the steady-state misalignment and the
dk = xTk wo + υk (1) computational effort as compared to the GMCC algorithm,
we here use the generalized Versoria function as cost function
where υk is the background measurement noise plus impulsive
interferences, and wo is an unknown weight vector that needs 1
J(wk−1 ) = E (6)
to be estimated. The system output error ek is defined as 1 + τ |ek |p
ek = dk − xTk wk−1 (2) where E[·] denotes the expectation operator. As can be seen
from (6), whenever an impulsive interference-corrupted ek
where wk−1 is the estimate of wo at iteration k − 1. appears, leads to J(wk−1 ) → 0 (the gradient of J(wk−1 )
The cost function of the maximum correntropy crite- approaches zero), which plays the role of suppressing impul-
rion (MCC) algorithm is the Gaussian probability density sive interferences. When the error ek is small, it is easy to
function (GPDF), which can be simply written as obtain J(wk−1 ) ≈ E[1 − τ |ek |p ]. In this case, the proposed
Versoria cost function works like the conventional cost func-
e2k tion of the LMP algorithm (note that the proposed algorithm
f (ek ) = exp − 2 (3)
2σ is based on the stochastic gradient ascent method).
A robust adaptive filtering algorithm, called maximum
where σ > 0 is the kernel width. The GPDF is high calcula- Versoria criterion (MVC) algorithm, can be developed by max-
tion cost when it is used in domains of machine learning and imizing the generalized Versoria cost function (6). Removing
signal processing, especially in adaptive filtering. For adaptive the expectation operator and computing the gradient of (6)
filtering, designing a suitable and high-efficiency cost frame- with respect to wk−1 , yields
work is an important issue. In order to draw forth the proposed
cost function, we first introduce the Versoria function, which 1
∇J(wk−1 ) = τ p |ek |p−1 sign(ek )xk (7)
is defined as (1 + τ |ek |p )2
8a3 1 Using the stochastic gradient ascent approach, the update
f (ek ) = = 2a (4) equation of the weight vector of the MVC algorithm can be
4a2 + e2k 1 + (ek /2a)2
obtained as
where a > 0 is the radius of the generating circle of Versoria. 1
The centroid of the generating circle is located at (0, a). wk = wk−1 + μ |ek |p−1 sign(ek )xk (8)
A larger value of the radius a leads to a steeper Versoria. (1 + τ |ek |p )2
As a comparison, Fig. 1 plots the GPDF (σ 2 = 0.5) and where μ is the step size. The Versoria term in (8) approaches
the standard Versoria function (a = 0.5). From this figure, we zero whenever an impulsive interference-corrupted ek appears.
can observe that the Versoria function is less steep than the Thus, this eliminates the likelihood of updating the weight vec-
GPDF for the large error ek while both the GPDF and the tor based on wrong information caused by the various outliers
Versoria function exhibit comparable steepness for the small including the impulsive interferences. When the system output
error ek . This means that the error along the direction of the error ek is small, the MVC algorithm works like the conven-
gradient ascent of the Versoria is faster to reach the optimal tional LMP algorithm. We can obtain different order’s MVC
point (i.e., zero error) than do the GPDF. From the perspective algorithms for different p values. Particularly, the generalized
of the adaptive filter, the gradient ascent algorithm based on MVC algorithm with p = 2 becomes
the Versoria function has a faster convergence rate to reach the
same steady-state error than do the GPDF-based algorithm; in 1
wk = wk−1 + μ 2 ek xk (9)
other words, the former yields a reduced steady-state error as 1 + τ e2k
compared to the latter under the same convergence rate.
Motivated by the generalized GPDF [14], a generalized which is standard MVC algorithm. When τ →0, the gen-
Versoria function can be simply regarded as eralized MVC algorithm reverts to the conventional LMP
algorithm [16]
1 1
f (ek ) = 2a = 2a (5)
1 + |ek /2a| p
1 + τ |ek |p wk = wk−1 + μ|ek |p−1 sign(ek )xk (10)
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY DELHI. Downloaded on May 22,2024 at 05:56:47 UTC from IEEE Xplore. Restrictions apply.
1254 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—II: EXPRESS BRIEFS, VOL. 64, NO. 10, OCTOBER 2017
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY DELHI. Downloaded on May 22,2024 at 05:56:47 UTC from IEEE Xplore. Restrictions apply.
HUANG et al.: MVC-BASED ROBUST ADAPTIVE FILTERING ALGORITHM 1255
Fig. 3. Misalignment curves of the SA, LMS, GMCC and MVC algorithms for different background√measurement
√ noise bk : (a) Laplace noise with zero-mean
and unit variance; (b) Gaussian noise with zero-mean and unit variance; (c) Uniform noise over [− 3, 3]; (d) Binary noise over {−1, 1} with probability
mass P(x = −1) = P(x = 1) = 0.5. The impulsive interferences occur with probability Pr = 0.1.
where the noise υk is assumed to be zero. It is not difficult IV. S IMULATION R ESULTS
to get Monte Carlo simulations are carried out for system iden-
1 1 tification and acoustic echo cancellation scenarios. The
0≤ |ek |p ≤ (23) performance of the adaptive algorithms is evaluated by the
(1 + τ |ek ) |p 2 4τ
normalized misalignment defined as: Misalignment(k) =
μ
If |vk−1 |2 ≥ 8τ , we have 10log10 (||w(k) − wo ||2 /||wo ||2 ). The misalignment learn-
1 ing curves are obtained from the ensemble averages of
0≤μ |ek |p |vk−1 |−2 ≤ 2 (24) 500 independent runs.
(1 + τ |ek |p )2
Then, we obtain
2 A. System Identification Scenario
1
|vk | = 1 − μ
2 −2
|e | |vk−1 | |vk−1 |2 ≤ |vk−1 |2
p In the system identification scenario, the length of the
2 k
(1 + τ |ek | )
p unknown system generated randomly and adaptive filters is
(25) 128 taps (L = 128). The input is a white Gaussian random
signal with zero-mean and unit variance. The system output is
Thus, the limit lim |vk |, if exists, is always bounded. This contaminated by a noise model as υk = bk + ωk nk , where bk
k→∞
means that in this case the MVC algorithm will never diverge is the background measurement noise, and ωk nk models the
(or the MVC has a zero POD), while the POD of the strong impulsive interferences, wherein ωk is an i.i.d. Bernoulli
LMF algorithm will be non-zero [19]. Comparing the con- process with a probability of success P(ωk = 1) = Pr and
vergent condition of the proposed MVC algorithm with that P(ωk = 0) = 1 − Pr (Pr is the probability of the impulsive
(|vk−1 |2 ≥ exp(−1)η/(2λ) [14]) of the GMCC algorithm, interferences occurring), and nk is a white Gaussian random
we obtain μ/(4τ ) = exp(−1)η/λ. As τ = λ, yields μ = sequence with zero-mean and variance σn2 = 1000σy2 . For
4exp(−1)η > η. That is, in this simple example, the step-size all simulations of this subsection, the parameter settings are:
bound of the MVC algorithm for converging is larger than that Pr = 0.1; other parameters are reported in the figures. As
of the GMCC algorithm. two examples, Fig. 2 shows the misalignment curves of the
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY DELHI. Downloaded on May 22,2024 at 05:56:47 UTC from IEEE Xplore. Restrictions apply.
1256 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—II: EXPRESS BRIEFS, VOL. 64, NO. 10, OCTOBER 2017
R EFERENCES
[1] A. H. Sayed, Fundamentals of Adaptive Filtering. Hoboken, NJ, USA:
Wiley, 2003.
Fig. 4. Misalignment curves of the SA, LMS, GMCC and MVC algorithms [2] R. H. Kwong and E. W. Johnston, “A variable step size LMS algo-
for acoustic echo cancellation application. rithm,” IEEE Trans. Signal Process., vol. 40, no. 7, pp. 1633–1642,
Jul. 1992.
[3] D. T. M. Slock, “On the convergence behavior of the LMS and the nor-
malized LMS algorithms,” IEEE Trans. Signal Process., vol. 41, no. 9,
proposed MVC algorithm with different constant a and adap- pp. 2811–2825, Sep. 1993.
tive ak , respectively. As we can see, a smaller value of a [4] J. Benesty, H. Rey, L. R. Vega, and S. Tressens, “A nonparametric
results in a lower steady-state misalignment but a slower con- VSS NLMS algorithm,” IEEE Signal Process. Lett., vol. 13, no. 10,
pp. 581–584, Oct. 2006.
vergence rate, and vice versa. In other words, a larger value [5] S. Gollamudi, S. Nagaraj, S. Kapoor, and Y.-F. Huang, “Set-membership
of τ (τ = (2a)−p ) yields a lower steady-state misalignment filtering and a set-membership normalized LMS algorithm with an adap-
but a slower convergence rate, and vice versa. One can also tive step size,” IEEE Signal Process. Lett., vol. 5, no. 5, pp. 111–114,
see that the proposed MVC algorithm with adaptive ak not May 1998.
[6] S. Zhang, J. Zhang, and H. Han, “Robust shrinkage normal-
only achieves a fast convergence rate, but also achieves a low ized sign algorithm in an impulsive noise environment,” IEEE
steady-state misalignment. In the following simulations, the Trans. Circuits Syst. II, Exp. Briefs, vol. 64, no. 1, pp. 91–95,
performance of the proposed MVC algorithm is compared with Jan. 2017.
that of the SA, LMS and GMCC algorithms. The computer [7] T. Shao, Y. R. Zheng, and J. Benesty, “An affine projection sign algo-
rithm robust against impulsive interferences,” IEEE Signal Process. Lett.,
simulation results are shown in Fig. 3, where the background vol. 17, no. 4, pp. 327–330, Apr. 2010.
measurement noise bk is (a) Laplace noise; (b) Gaussian [8] S. Wang, J. Feng, and C. K. Tse, “Kernel affine projection sign
noise; (c) Uniform noise; (d) Binary noise, respectively. As algorithms for combating impulse interference,” IEEE Trans. Circuits
Syst. II, Exp. Briefs, vol. 60, no. 11, pp. 811–815, Nov. 2013.
shown in Fig. 3, it is clearly seen that the proposed MVC [9] F. Huang, J. Zhang, and S. Zhang, “Combined-step-size affine projec-
( p > 1) algorithm outclasses the SA, LMS and GMCC algo- tion sign algorithm for robust adaptive filtering in impulsive interference
rithms in terms of the convergence rate and/or the steady-state environments,” IEEE Trans. Circuits Syst. II, Exp. Briefs, vol. 63, no. 5,
misalignment. Note that we do not show the misalignment pp. 493–497, May 2016.
[10] H. Zayyani, “Continuous mixed p-norm adaptive algorithm for
curves of the LMF since the LMF is not stable for any value system identification,” IEEE Signal Process. Lett., vol. 21, no. 9,
of the step-size when the noises include the strong impulsive pp. 1108–1110, Sep. 2014.
interferences. [11] E. Walach and B. Widrow, “The least mean fourth (LMF) adaptive
algorithm and its family,” IEEE Trans. Inf. Theory, vol. 30, no. 2,
pp. 275–283, Mar. 1984.
B. Acoustic Echo Cancellation Scenario [12] W. Liu, P. P. Pokharel, and J. C. Principe, “Correntropy: Properties
and applications in non-Gaussian signal processing,” IEEE Trans. Signal
Here, the performance of the SA, LMS, GMCC, and Process., vol. 55, no. 11, pp. 5286–5298, Nov. 2007.
proposed MVC algorithms is compared in an acoustic echo [13] T. Ogunfunmi and T. Paul, “The quarternion maximum correntropy
cancellation scenario, under the same initial convergence rate. algorithm,” IEEE Trans. Circuits Syst. II, Exp. Briefs, vol. 62, no. 6,
pp. 598–602, Jun. 2015.
The length of the impulse response of the acoustic channel [14] B. Chen, L. Xing, H. Zhao, N. Zheng, and J. C. Principe, “Generalized
is 512. The input signal is speech signal sampled at 8 kHz. correntropy for robust adaptive filtering,” IEEE Trans. Signal Process.,
The output signal is contaminated by υk = bk + ωk nk , where vol. 64, no. 13, pp. 3376–3387, Jul. 2016.
bk is Gaussian noise with zero-mean and unit variance, and [15] X. Yu, J. Liu, and H. Li, “An adaptive inertia weight particle swarm
optimization algorithm for IIR digital filter,” in Proc. Int. Conf. Artif.
ωk nk is the same as that in the system identification scenario. Intell. Comput. Intell., Shanghai, China, 2009, pp. 114–118.
The impulsive interferences occur with probability Pr = 0.1. [16] S.-C. Pei and C.-C. Tseng, “Least mean p-power error criterion for
The parameter settings are presented in the figure. As can adaptive fir filter,” IEEE J. Sel. Areas Commun., vol. 12, no. 9,
be seen from Fig. 4, the proposed MVC ( p > 1) algorithm pp. 1540–1547, Dec. 1994.
[17] O. Arikan, A. E. Cetin, and E. Erzin, “Adaptive filtering for non-
performs much better than the SA, LMS and corresponding Gaussian stable processes,” IEEE Signal Process. Lett., vol. 1, no. 11,
order’s GMCC algorithms. pp. 163–165, Nov. 1994.
[18] T. Y. Al-Naffouri and A. H. Sayed, “Transient analysis of adaptive filters
with error nonlinearities,” IEEE Trans. Signal Process., vol. 51, no. 3,
V. C ONCLUSION pp. 653–663, Mar. 2003.
[19] V. H. Nascimento and J. C. M. Bermudez, “Probability of divergence for
A maximum Versoria criterion (MVC) algorithm, which is the least-mean fourth algorithm,” IEEE Trans. Signal Process., vol. 54,
lower steady-state misalignment and less computational effort no. 4, pp. 1376–1385, Apr. 2006.
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY DELHI. Downloaded on May 22,2024 at 05:56:47 UTC from IEEE Xplore. Restrictions apply.