图像与视频超分辨率：技术、应用与未来发展

需积分: 9 143 浏览量更新于2024-07-18 收藏 18.4MB PDF 举报

《图像与视频的超分辨率》一书详细探讨了如何通过一组低分辨率（LR）观测数据恢复出具有高分辨率（HR）细节的图像或视频序列。超分辨率（SR）不仅局限于空间分辨率提升，也涉及时间分辨率增强和高光谱图像增强，特别是基于运动的空间分辨率增强。书中涵盖了预处理、视频序列压缩等关键技术，以及它们与信号处理中的图像恢复和运动估计等问题之间的紧密联系。作者们特别关注于解决无运动场景和高光谱图像的超分辨率问题，强调SR技术在诸如视频打印、高清电视、高清晰度LCD屏幕显示、CCD图像质量改善、视频监控、遥感、医学成像等领域的重要应用。随着SR技术的发展，它已成为图像和视频编解码器（如CODEC）设计的关键组成部分，有助于推动新型编码标准的诞生。本书深入浅出地介绍了超分辨率的基本原理、方法和实践，包括但不限于插值算法（如内插法、基于深度学习的模型）、多帧融合、自编码器架构以及对噪声抑制和细节增强的处理策略。此外，书中还探讨了如何将SR技术与其他视觉处理技术（如图像去模糊、降噪）结合起来，以达到更佳的视觉效果和性能。通过阅读这本书，读者不仅可以掌握基础的理论知识，还能了解到最新的研究进展和实际应用案例，对于从事图像处理、计算机视觉、信号处理或相关领域的研究人员和工程师来说，这是一本极具价值的参考书籍。

MOBK071-FM MOBKXXX-Sample.cls April 16, 2007 15:48

xvi SUPER RESOLUTION OF IMAGES AND VIDEO

It is our hope and expectation that this monograph will serve as a good reference to the

graduate student who would like to work in this area, but also to the practicing engineer, and

to the person completely outside of this research area who would like to “just ﬁnd out what is

this excitement all about!” As we already alluded above the material in this monograph can also

serve well the scientist in applying some of the tools and results to other related problems.

This has been a very stimulating and enjoyable (most of the time) project for the authors.

There were plenty of bits that crossed the Atlantic in the form of emails and CDs and faxes,

but also enjoyable meetings in Granada, Chicago, and a recent intense week in Athens, Greece

in trying to bring this project to completion (not even enough time to visit the Acropolis!).

There are a number of people we would like to thank for helping us bring this project

to completion. First of all, Al for asking us to undertake the project and for feedback along

the way and Joel Claypool for all the input but also his patience. We would like to thank

Prof. P. Milanfar for providing us with the software used to run the Bilateral Total Variation

example in Sections 4.5 and 5.1 and Javier Mart

ın, undergraduate student at the University

of Granada, for implementing as a ﬁnal year project the Baker-Kanade algorithm used in

Section 5.1. We would also like to thank the Ph.D students Derin Babacan and Bruno Amizic

(Northwestern University) and D

acil Barreto (University de las Palmas de Gran Canaria), and

Tom Bishop (University of Edinburgh), for their comments on the book (the last two while

visiting the University of Granada). Thank you is also due for the comments and images to

the Ph.D students at the University of Granada, L.D. Alvarez and Jaime Cancino. Finally, we

would like to thank our families for all their love and support (and also patience) over years:

Ereni, Zoe, Soﬁa, and Sally (AKK), Conchi and Paloma (RM), Marian, David, and Cristina

(JM).

An example in this book uses the FERET database of facial images collected under the

FERET program, sponsored by the DOD Counterdrug Technology Development Program

Ofﬁce [142, 143].

This work was supported by the “Comisi

on Nacional de Ciencia y Tecnolog

ıa” under

contract TIC2003-00880, by the Greece-Spain Integrated Action HG2004-0014, and by the

“Instituto de Salud Carlos III” project FIS G03/185.

Chicago Aggelos Katsaggelos

La Herradura Rafael Molina

Granada Javier Mateos

August, 2006

book Mobk071 April 16, 2007 17:4

CHAPTER 1

Introduction

1.1 WHAT IS SUPER RESOLUTION OF IMAGES AND VIDEO?

We use the term super resolution (SR) to describe the process of obtaining an high resolution

(HR) image or a sequence of HR images from a set of low resolution (LR) observations. This

process has also been referred to in the literature as resolution enhancement (RE) (henceforth,

we will be using the two terms interchangeably; we will also use the terms image(s), image

frame(s), and image sequence frame(s) interchangeably). SR has been applied primarily to

spatial and temporal RE, but also to hyperspectral image enhancement (a comprehensive

classiﬁcation of spatiotemporal SR problems is provided in [22, 24]). In this book we mainly

concentrate on motion-based spatial RE (see Fig. 1.1), although we also describe motion-free

[42] and hyperspectral [115, 118]imageSR problems. Furthermore, we examine the very recent

research area of SR for compression [13, 112], which consists of the intentional downsampling

(during pre-processing) of a video sequence to be compressed and the application of SR

techniques (during post-processing) on the compressed sequence.

In motion-based spatial RE,theLR-observed images are under-sampled and they are

acquired by either multiple sensors imaging a single scene (Fig. 1.1(a)) or by a single sensor

imaging a scene over a period of time (Fig. 1.1(b)). For static scenes the observations are related

by global subpixel displacements (due, for example, to the relative positions of the cameras, and

to camera motion, such as panning or zooming), while for dynamic scenes they are related by

local subpixel displacements due to object motion, in addition to possibly global displacements.

In both cases the objective again of SR is to utilize either the set of LR images (Fig. 1.1(a)) or

a set of video frames (Fig. 1.1(b)) to generate an image of increased spatial resolution.

Increasing the resolution of the imaging sensor is clearly one way to increase the resolution

of the acquired images. This solution, however, may not be feasible due to the increased

associated cost and the fact that the shot noise increases during acquisition as the pixel size

becomes smaller [135]. Furthermore, increasing the chip size to accommodate the larger number

of pixels increases the capacitance, which in turn reduces the data transfer rate. Therefore,

signal processing techniques, like the ones described in this book, provide a clear alternative for

increasing the resolution of the acquired images.

book Mobk071 April 16, 2007 17:4

INTRODUCTION 3

Continuous to

Discrete

Without

Aliasing

- Translation

- Rotation, etc.

Warping and blurring

Sampling

Downsampling

Desired HR Image

Noise

Observed LR Image

- Optical Blur

- Motion Blur

- Sensor PSF

- etc.

Continuous Scene

FIGURE 1.2: Block diagram of an SR system

warping—are applied. Both resulting systems, the so-called warp–blur and blur–warp models,

have been used and are analyzed in detail in Chapter 3. Similarly, there are various ways to

describe the downsampling block in Fig. 1.2, which will generate the aliased LR images of

the original HR image. Finally, noise is added to the output of the system introducing the

downsampling, resulting in the observed LR image. The noise component can model various

elements in the imaging chain, such as, the misregistration error and the thermal or electronic

noise, or the errors during storage or transmission. In the book we also consider the case when

the available LR images have been compressed. In this case, one more block representing

the compression system is added at the end of the block diagram of Fig. 1.2. Another noise

component due to quantization needs to be considered in this case, which is typically considered

to be the dominant noise component, especially for low bit-rates. As is clear from the block

diagram of Fig. 1.2, in a typical SR problem there are a number of degradations that need

to be ameliorated, in addition to increasing the resolution of the observations. The term RE

might be therefore more appropriate in describing the type of problems we are addressing in

this book.

Image restoration, a ﬁeld very closely related to that of SR of images, started with the

scientiﬁc work in the Space programs of the United States and the former Soviet Union

[11, 116]. It could be said that the ﬁeld of SR also started in the sky with the launching

of Landsat satellites. These satellites imaged the same region on the Earth every 18 days.

However, the observed scenes were not exactly the same since there were small displacements

among them [190]. Since then, a wealth of research has considered modeling the acquisition

of the LR images and provided solutions to the HR problem. Recent literature reviews are

provided in [22, 26, 41, 85, 123]. More recent work, e.g. [108, 133, 134, 166], also addresses

the HR problem when the available LR images are compressed using any of the numerous

剩余149页未读，继续阅读

zhaiduo1

粉丝: 1
资源: 1

图像与视频超分辨率：技术、应用与未来发展

Multi-Frame Video Super-Resolution Using Convolutional Neural Networks

Image Mosaicing and Super-Resolution

超分辨率重建代码 superresolution_v_2.0

Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform

写一段Super Resolution的代码

Super-Resolution

在 C# 中使用 Super Resolution GAN

Image Super Resolution是什么

Super Resolution GAN 模型结构的C#代码

如何介绍Super Resolution？

最新资源