Preface
I
am indebted to the many readers and colleagues who have written to me on several occasions with
encouraging feedback
on
my earlier textbook
Fundamentals ofAdaptive Filtering
(Wiley,
NJ,
2003).
Their enthusiastic comments encouraged me to pursue this second project in an effort to create a
revised version for teaching purposes. During this exercise,
I
decided to remove some advanced
material and move select topics to the problems.
I
also opted to fundamentally restructure the entire
text into eleven consecutive
parts
with each part consisting of a series of focused lectures and ending
with bibliographic comments, problems, and computer projects. I believe this restructuring into
a sequence of lectures will provide readers and instructors with more flexibility in designing and
managing their courses. I also collected most background material on random variables and linear
algebra into three chapters at the beginning of the book. Students and readers have found
this
material
of independent interest in its own right. At the same time,
I
decided to maintain the same general style
and features of the earlier publication in terms of presentation and exposition, motivation, problems,
computer projects, summary, and bibliographic notes. These features have been well received by our
readers.
AREA
OF
STUDY
Adaptive filtering is a topic of immense practical relevance and deep theoretical challenges that
persist even to this date. There are several notable texts on the subject that describe many of the
features that have marveled students and researchers over the years. In this textbook, we choose
to step back and to take a broad look at the field. In
so
doing, we feel that we are able to bring
forth, to the benefit of the reader, extensive commonalities that exist among different classes of
adaptive algorithms and even among different filtering theories. We are also able to provide a uniform
treatment of the subject in a manner that addresses some existing limitations, provides additional
insights, and allows for extensions of current theory.
We do not have any illusions about the difficulties that arise in any attempt at understanding
adaptive filters more fully. This is because adaptive filters are, by design, time-variant, nonlinear,
and stochastic systems. Any one of these qualifications alone would have resulted in a formidable
system to study. Put them together and you face an almost impossible task. It is
no
wonder then that
current practice tends to study different adaptive schemes separately, with techniques and assump-
tions that are usually more suitable for one adaptation form over another. It is also no surprise that
most treatments of adaptive filters, including the one adopted in this textbook, need to rely on some
simplifying assumptions in order to make filter analysis and design a more tractable objective.
Still, in our view, three desirable features of any study of adaptive filters would be
(1)
to attempt
to keep the number of simplifying assumptions to a minimum,
(2)
to delay their use until necessary,
and
(3)
to apply similar assumptions uniformly across different classes of adaptive algorithms. This
last feature enables
us
to evaluate and compare the performance of adaptive schemes under similar
assumptions on the data, while delaying the use of assumptions enables
us
to extract the most infor-
mation possible about actual filter performance. In our discussions in this book we pay particular
attention to these three features throughout the presentation.
In
addition, we share the conviction that a thorough understanding of the performance and limi-
tations of adaptive filters requires a solid grasp of the fundamentals of least-mean-squares estimation
xvii