"图像超分辨率SR论文总结与归纳：经典论文、网络分析、优劣评价与改善方向"

需积分: 5 162 浏览量更新于2024-01-28 2 收藏 25.74MB DOCX 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

图像超分辨率（SR）是一个重要的计算机视觉任务，旨在通过提高图像的空间分辨率来获取更多细节和清晰度。随着深度学习技术的发展，越来越多的研究者开始利用深度神经网络来解决图像超分辨率问题。本文对图像超分辨率任务的经典论文进行了总结与归纳，以帮助新手快速入门。我们主要关注了2015年CVPR会议上发表的一篇重要论文《Image Super-Resolution Using Deep Convolutional Networks》，该论文提出了SRCNN（Super-Resolution Convolutional Neural Network）模型，该模型基于卷积神经网络和稀疏编码技术，取得了显著的成果。论文提出的SRCNN模型直接学习低分辨率图像和高分辨率图像之间的端到端映射，避免了传统方法中复杂的流程和特征工程。该模型在学习映射过程中，使用了三个卷积层，分别用于提取图像的低级特征、非线性映射和最终的重建。通过大量的实验证明，SRCNN在提高图像质量和减少重建误差方面表现出色，成为了图像超分辨率领域的一个重要里程碑。除了介绍SRCNN模型的基本原理和结构外，我们还对该模型进行了深入的优劣评价。SRCNN的优点在于它简单直接的网络结构和良好的超分辨率效果，但它也存在一些缺点，比如对训练数据的依赖性较强，对噪声和运动模糊的鲁棒性较差。针对这些问题，我们提出了一些改进方向，例如引入对抗训练和增强模型的鲁棒性。在对SRCNN模型进行分析的基础上，我们还总结了其他经典论文在图像超分辨率领域的贡献和不足之处。除了SRCNN模型，还有一些其他有代表性的方法，如VDSR（Very Deep Super-Resolution）和ESPCN（Efficient Sub-Pixel Convolutional Neural Network），它们都在不同方面取得了显著的成果。通过对这些方法的深入分析，我们发现了图像超分辨率领域的研究热点和难点，并探讨了未来的研究方向，如如何更好地融合深度学习和传统图像处理技术，如何提高模型的鲁棒性和泛化能力等。综上所述，图像超分辨率是一个重要且具有挑战性的问题，深度学习技术的发展为这一问题的解决提供了新的思路和方法。通过对经典论文的总结与分析，我们可以更好地了解图像超分辨率领域的研究现状和动态，并为进一步的研究提供了有益的启示。我们相信，在不久的将来，图像超分辨率技术将会得到进一步的突破和应用，为图像处理和计算机视觉领域带来更多的价值和可能性。

资源详情

资源推荐

2016 CVPR Accelerating the Super-Resolution Convolutional

Neural Network（FSRCNN、对 SRCNN 进行加速）

Chao Dong, Chen Change Loy, and Xiaoou Tang

Department of Information Engineering, The Chinese University of Hong Kong

Code— http://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html

Abstract—As a successful deep model applied in image super-resolution (SR), the Super-

Resolution Convolutional Neural Network (SRCNN) [1,2] has demonstrated superior performance

to the previous hand-crafted models either in speed and restoration quality. However, the high

computational cost still hinders it from practical usage that demands real-time performance (24 fps).

In this paper, we aim at accelerating the current SRCNN, and propose a compact hourglass-shape

CNN structure for faster and better SR. We re-design the SRCNN structure mainly in three aspects.

First, we introduce a deconvolution layer at the end of the net- work, then the mapping is learned

directly from the original low-resolution im- age (without interpolation) to the high-resolution one.

Second, we reformulate the mapping layer by shrinking the input feature dimension before mapping

and expanding back afterwards. Third, we adopt smaller filter sizes but more map- ping layers. The

proposed model achieves a speed up of more than 40 times with even superior restoration quality.

Further, we present the parameter settings that can achieve real-time performance on a generic CPU

while still maintaining good performance. A corresponding transfer strategy is also proposed for

fast training and testing across different upscaling factors.

Contributions—Our contributions are three-fold: 1) We formulate a compact hourglass-shape

CNN structure for fast image super-resolution. With the collaboration of a set of deconvolution

filters, the network can learn an end-to-end mapping between the original LR and HR images with

no pre-processing. 2) The proposed model achieves a speed up of at least 40× than the SRCNN-Ex

[2] while still keeping its exceptional performance. One of its small-size version can run in real-

time (>24 fps) on a generic CPU with better restoration quality than SRCNN [1]. 3) We transfer the

convolution layers of the pro- posed networks for fast training and testing across different upscaling

factors, with no loss of restoration quality.

Conclusion—While observing the limitations of current deep learning based SR models, we

explore a more efficient network structure to achieve high running speed without the loss of

restoration quality. We approach this goal by re-designing the SRCNN structure, and achieves a

final acceleration of more than 40 times. Extensive experiments suggest that the proposed method

yields satisfactory SR performance, while superior in terms of run time. The proposed model can

be adapted for real-time video SR, and motivate fast deep models for other low-level vision tasks.

Work：该文章是对 SRCNN 的改进，SRCNN 重建速度过慢，这问题的大多数深度学习方法

不能使用池化技术。通过添加新的网络层来增加深度，基本上会引入更多的参数。可能会出

现两个问题。首先，很可能会过度拟合，需要更多的训练数据，使模型变得越来越大。

改进之处：1、并不是将三次插值后的图像当做输入，而是直接将 LR 图像丢入到网络

中，最后选用 deconv 进行放大

2、在映射 Layer 进行了改进，先 shrink 再将其复原

3、更多的映射 layer 和更小的 kernel

4、共享其中的 mapping layer，如果需要训练不同的 upscale model，最后仅需要 fine-tuning

最后的 deconvLayer

FSRCNN 包含如下五个主要处理步骤：

特征提取：利用一个 5x5 卷积代替 SRCNN 中的双立方差值

压缩：利用 1x1 卷积把特征层从 d 压缩为 s，其中 s 小于 d

非线性映射：多个 3x3 网络层代替单个的宽网络层

扩大： 1x1 卷积把特征层从 s 扩大为 d

反卷积：利用 9x9 过滤器重建 HR 图像

Activation：PReLu function（带参数的 ReLu，目的是比 ReLU 更好）

总结：SRCNN 和 FSRCNN 的三个区别：

1. 输入：SRCNN 需要对 LR 插值，FSRCNN 不需要，并用反卷积层来替换上采样层；

2. 非线性映射：SRCNN 非线性映射在 FSRCNN 中用压缩，映射，扩展，三步来代替；

3. 网络结构：SRCNN 为三层，FSRCNN 更深且层的尺寸更小。

2017 CVPR Deep Laplacian Pyramid Networks for Fast and

Accurate Super-Resolution（LapSRN、LP、残差学习）

Wei-Sheng Lai1 Jia-Bin Huang2 Narendra Ahuja3 Ming-Hsuan Yang1

1University of California, Merced 2Virginia Tech 3University of Illinois, Urbana-Champaign

Code—https://github.com/phoenix104104/LapSRN

Abstract—Convolutional neural networks have recently demonstrated high-quality reconstruction

for single-image super- resolution. In this paper, we propose the Laplacian Pyramid Super-

Resolution Network (Lap-SRN) to progressively re- construct the sub-band residuals of high-

resolution images. At each pyramid level, our model takes coarse-resolution feature maps as input,

predicts the high-frequency residuals, and uses transposed convolutions for up-sampling to the finer

level. Our method does not require the bicubic interpolation as the pre-processing step and thus

dramatically re- duces the computational complexity. We train the proposed Lap-SRN with deep

supervision using a robust Charbonnier loss function and achieve high-quality reconstruction. Fur-

thermore, our network generates multi-scale predictions in one feed-forward pass through the

progressive reconstruction, thereby facilitates resource-aware applications. Extensive quantitative

and qualitative evaluations on bench- mark datasets show that the proposed algorithm performs

favorably against the state-of-the-art methods in terms of speed and accuracy.

Contributions—Our algorithm differs from existing CNN-based methods in the following three

aspects:

(1) Accuracy. The proposed Lap-SRN extracts feature maps directly from LR images and

jointly optimizes the up-sampling filters with deep convolutional layers to predict sub- band

residuals. The deep supervision with the Charbonnier loss improves the performance thanks to the

ability to better handle outliers. As a result, our model has a large capacity to learn complicated

mappings and effectively reduces the undesired visual artifacts.

(2) Speed. Our Lap-SRN embraces both fast processing speed and high capacity of deep

networks. Experimental results demonstrate that our method is faster than several CNN based super-

resolution models, e.g., SRCNN [7], SCN [33], VDSR [17], and DRCN [18]. Similar to FSR- CNN

[8], our Lap-SRN achieves real-time speed on most of the evaluated datasets. In addition, our

method provides significantly better reconstruction accuracy.

(3) Progressive reconstruction. Our model generates multiple intermediate SR predictions in

one feed-forward pass through progressive reconstruction using the Laplacian pyramid. This

characteristic renders our technique applicable to a wide range of applications that require resource-

aware adaptability. For example, the same network can be used to enhance the spatial resolution of

videos depending on the available computational resources. For scenarios with limited computing

resources, our 8× model can still perform 2× or 4× SR by simply bypassing the computation of

residuals at finer levels. Existing CNN-based methods, however, do not offer such flexibility.

Conclusion—In this work, we propose a deep convolutional network within a Laplacian pyramid

framework for fast and ac- curate single-image super-resolution. Our model progressively predicts

high-frequency residuals in a coarse-to-fine manner. By replacing the predefined bicubic

interpolation with the learned transposed convolutional layers and optimizing the network with a

robust loss function, the proposed Lap-SRN alleviates issues with undesired artifacts and reduces

the computational complexity. Extensive evaluations on benchmark datasets demonstrate that the

proposed model performs favorably against the state-of-the-art SR algorithms in terms of visual

quality and run time.

Motivation：（1）目前大部分网络都在进行处理前，先对图像进行上采样操作，这样做不

仅会增加不必要的计算量，而且还可能会引入伪影。（2）目前大部分网络都用 L2 损失函

数来训练网络，由于 L2 损失函数不能捕捉多模型分布特征，这就会导致重建图像过平滑。

（3）目前大部分 SR 方法都采用一步的放大过程，这就会增加放大高倍数情况的复杂度。

解决：解决问题（1）的办法是直接将图像作为网络的 input，再进行上采样操作，这样就节

省了计算量。解决问题（2）的办法是使用 Charbonnier 损失函数。解决问题（3）的办法是

采用多步放大过程，即在生成高倍数图像前，先生成低倍数的图像，逐步地生成最后的高倍

数图像。

Work：本文提出的 LapSRN，网络以 LR 图片作为输入，以低分辨到高分辨的方式来预测

子带残差。在网络的每一层，一系列的卷积层被用来提取特征映射。此后，一个反卷积层用

来将特征映射上采样，然后传递到最后一层。最后用一个卷积层来预测子带残差（sub-band

residual，指各层上采样的结果与 ground-truth 的差别）。

整个网络可分为两部分，分别是（1）特征提取分支和（2）图像重建分支。特征提取分

支分为多个 level 层，每个 level 层都把上个 level 层的图像放大 2 倍，这就实现了逐步放大

的过程。在特征提取分支的每层 level 中，都有 d 个卷积层和 1 个逆卷积层。d 个卷积层的

目的是进行特征提取，1 个逆卷积层的目的是将图像放大 2 倍，即上采样操作。此时逆卷积

层的输出连接着两个地方：（1）作为特征提取分支 level+1 层的 input，重复 level 层的操作。

（2）连接一个卷积层生成残差图像，生成的残差图像传递到图像重建分支，与重建分支第

level 层的 input 进行逆卷积操作后的图像进行合成，得到 level 层的 SR 图像。此 SR 图像既

作为 level 层超分辨的结果，又是重建分支 level+1 层的 input。

剩余56页未读，继续阅读

爱吃柚子的小肖

粉丝: 1
资源: 2

会员权益专享

"图像超分辨率SR论文总结与归纳：经典论文、网络分析、优劣评价与改善方向"

基于深度学习的图像超分辨率算法论文合集2015-2019

腾讯开源图像超分辨率算法RealSR源码以及预训练模型已经训练、测试的图片

论文研究- 图像超分辨率重建技术综述 .pdf

SR3图像超分辨率模型

图像超分辨率重建python复现

python实现图像超分辨率重建并保存的代码

图像超分辨率技术 opencv python

基于卷积图像超分辨率重建去噪

彩色图像超分辨率matlab代码

机器学习图像超分辨率原理

神经网络的图像超分辨率算法代码

基于卷积神经网络的图像超分辨率重建Matlab代码

python超分辨率

POCS超分辨率代码实现

基于超分辨率对齿轮的图片处理使用SR-GAN模型比其他模型有什么优点

超分辨率领域基础知识

深度图像先验超分辨率算法训练代码

基于RP、CVT、DTCWT、NSCT-SR+DWT-SR+拉普拉斯金字塔算法-SR

maltab怎么保存得到的超分辨率图像？

用python写一个基于频域法的超分辨率重建程序

会员权益专享

最新资源