xvi Preface
in 1994, to more than 500 in 2004, and to more than 2000 in 2008). What has
been missing, however, is a book to not only gather key recent contributions
in one place, but also to serve as a starting point for those interested in this
field to begin learning about and exploring the state of the art. This is what
this book hopes to accomplish.
As is probably well-known by now, every super-resolution algorithm ever
developed is sabotaged by at least one spoke of our triumvirate “axis of evil”:
the need for (1) (subpixel) accurate motion estimation, (2) (spatially varying)
deblurring, and (3) robustness to modeling and stochastic errors. To be sure,
these are not independent problems and should ideally be treated in unison
(ambitious graduate students take note!). But realistically, each is sufficiently
complex as to merit its own section in the library, or at least a couple of nice
chapters in this book. This books gathers contributions that will present the
reader with a snapshot of where the field stands, a reasonable idea of where the
field is heading—and perhaps where it should be heading. Chapter 1 provides
an introduction to the history of the subject that should be of broad interest.
Indeed, the collection of citations summarized in this chapter is an excellent
wellspring for continued research on super-resolution.
One of the most active areas of work in image and video enhancement
in recent years has been the subject of locally adaptive processing methods,
which are discussed in Chapters 2, 3, 4, and 5. In contrast to globally optimal
methods (treated later in the book), these methods are built on the notion
that processing should be strongly tailored to the local behavior of the given
data. An explicit goal in some cases, and a happy consequence in others,
local processing enables us to largely avoid direct and detailed estimation of
motion. Readers interested in methods for explicit motion estimation will find
an excellent overview of modern techniques in Chapter 6.
While motion estimation is typically the first step in many super-resolution
algorithms, deblurring is typically the last step. Unfortunately, having been
relegated to the last position has meant that this important aspect of enhance-
ment has not received the respect and attention it deserves. Despite heavy
recent activity in both the image processing and machine vision community,
and some notable successes, deblurring even in its simplest (space-invariant,
known point-spread-function) form is still largely an unsolved problem. Inas-
much as we would like to hope, blur almost never manifests itself in a spatially
uniform fashion. In Chapter 7, the reader will find a well-motivated and di-
rect attack at this challenging problem. Despite our best efforts, a sequential
approach to super-resolution consisting of motion estimation, fusion, and de-
blurring will always be subject to the vagaries of the data, the models, and
noise. As such, building robustness into the reconstruction process, as treated
in Chapter 8, is vital if the algorithm is to be practically useful.
As with most inverse problems, super-resolution is highly ill-posed. In the
most general case, the motion between the frames, the blur kernel(s), and the
high-resolution image of interest are three interwoven unknowns that should
ideally be estimated together (rather than sequentially), and whose effect is