1549-7747 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSII.2017.2789207, IEEE
Transactions on Circuits and Systems II: Express Briefs
JOURNAL OF L
A
T
E
X CLASS FILES, VOL. , NO. , 2017 1
Robust constrained adaptive filtering under
minimum error entropy criterion
Siyuan Peng, Student Member, IEEE, Wee Ser, Senior Member, IEEE, Badong Chen, Senior Member, IEEE,
Lei Sun, Senior Member, IEEE, and Zhiping Lin, Senior Member, IEEE
Abstract—Minimum error entropy (MEE), as a robust adap-
tion criterion, has received considerable attention due to its broad
applicability especially in presence of non-Gaussian noises. In
this brief, we propose a constrained adaptive filtering algorithm
under minimum error entropy criterion, called CMEE, which is
derived by incorporating a set of linear equality constrains into
MEE to handle a constrained MEE optimization problem. In
addition, convergence analysis of the proposed CMEE including
the stability and steady-state mean square deviation (MSD) is
studied. Simulation results validate the theoretical conclusions,
and confirm the effectiveness of the new algorithm in non-
Gaussian noises.
Index Terms—constrained adaptive filtering, minimum error
entropy, convergence analysis, non-Gaussian noises.
I. INTRODUCTION
C
ONSTRAINED adaptive filtering algorithms have been
successfully used in a wide range of areas such as spec-
trum analysis and antenna arrays spatial-temporal processing
[1]–[6]. Indeed, most of the cost functions or optimization
criteria used in previous methods are based on the popular
mean square error (MSE) criterion, due to its attractive features
such as simplicity, smoothness, mathematical tractability, and
optimality under Gaussian assumption. However, in many
practical applications, the signal may be contaminated by some
non-Gaussian noises, especially impulsive noises, and in these
situations, the performance of the MSE based methods may
degrade seriously. To improve the robustness of constrained
adaptive filtering, the constrained maximum correntropy crite-
rion (CMCC) algorithm has recently been proposed for dealing
with the impulsive noises [7], [8]. However, the maximum
correntropy criterion (MCC) may not lead to superior perfor-
mance in non-Gaussian noises with a light-tail or multi-peak
distribution [9].
In recent years, information theoretic criteria that involve
higher-order statistics have received much attention [10].
Shannon’s entropy [11] provides an effective measure of
the average uncertainty in a given probability distribution
to calculate the expected value of information content in a
system. However, Renyi’s quadratic entropy [10] has been
widely used in information theoretic learning (ITL) due to its
S. Peng, Z. Lin and W. Ser are with the School of Electrical and Electronic
Engineering, Nanyang Technological University, 639798 Singapore (e-mail:
PENG0074@e.ntu.edu.sg, EZPLin@ntu.edu.sg, ewser@ntu.edu.sg).
B. Chen is with the Institute of Artificial Intelligence and Robotics, Xi’an
Jiaotong University, Xi’an 710049, China (e-mail: chenbd@mail.xjtu.edu.cn).
L. Sun is with the School of Information and Electronics, Beijing Institute
of Technology, Beijing 100081, China (e-mail: bitsunlei@126.com).
Manuscript received.
simplicity in estimation and calculation. In fact, the argument
of the logarithm in Renyi’s quadratic entropy can be simply
computed as a double summation over kernels. The minimum
error entropy criterion aims at minimizing the error’s entropy
such that the adaptive system can approach as close as possible
to the unknown system. Since MEE can capture higher-
order statistics of signals rather than simply their energy, it
is a powerful criterion to alleviate the negative effects of
various non-gaussian noises [9], [10], [12]. In general, the
MEE based adaptive filtering algorithms can achieve better
performance comparing to the MSE based and MCC based
methods, especially in multi-peak noises.
In this brief, we develop a robust MEE based constrained
adaptive filtering algorithm (CMEE), which in particular is
useful for handling different types of non-Gaussian noises. In
summary, the key contributions and the outline of this paper
are as follows:
• After briefly introducing the main notations, we derive
the CMEE algorithm in Section II.
• Based on the energy conservation relation and several
assumptions [13]–[17], we analyze in Section III the
mean square stability of the CMEE and derive an upper
bound on the step-size to ensure the mean square conver-
gence. Additionally, the theoretical value of the steady-
state mean square deviation (MSD) is also derived, which
is a solution of a nonlinear fixed-point equation.
• In Section IV, we firstly verify the theoretical results (e.g.,
the steady-state MSD) through simulations in Gaussian
and binary noises, and then demonstrate the desirable
performance of the CMEE comparing to conventional
MSE-based constrained adaptive algorithms and the CM-
CC algorithm in non-Gaussian noises.
• Finally, the conclusion is given in Section V.
II. CMEE ALGORITHM
A. Notations
In this brief, all vectors are column vectors, and the time
instant for vectors and scalars is placed as a subscript, for
example, w
n
and e
n
. Besides, tr {·}, (
ˆ
·), E[·], and T stand for
the trace operator, estimation operator, expectation operator,
and transpose operator, respectively. The squared Euclidean
norm of a vector w is defined as kwk
2
= w
T
w, and accord-
ingly, the weighted squared Euclidean norm can be expressed
as kwk
2
Σ
= w
T
Σw.