多尺度密集深度学习在泛锐化中的应用

95 浏览量更新于2024-08-27 收藏 359KB PDF 举报

"泛锐化的多尺度密集深度学习方法是一种用于图像处理的技术，旨在将低分辨率多光谱图像的光谱信息与高分辨率全色图像的空间信息融合，生成高分辨率多光谱图像。该方法通过利用深度学习，特别是多尺度特征提取和密集连接，提升了图像的清晰度和细节表现力。" 在图像处理领域，Pansharpening是一个关键任务，它涉及到将具有丰富空间信息但光谱分辨率较低的全色图像（PAN）与光谱分辨率高但空间分辨率低的多光谱图像（LRMS）进行融合，以创建出同时具有高空间和光谱分辨率的图像。这样的处理对于遥感、地理信息系统和环境监测等应用至关重要。本文提出的多尺度密集深度学习方法针对Pansharpening问题，构建了一个端到端的学习框架。这个框架由三个主要模块组成： 1. **多尺度特征提取**：使用并行的多尺度卷积层对PAN图像进行分析，这允许模型在不同层次上捕获图像的细节信息，从而更好地理解空间结构。多尺度特征提取有助于捕捉不同大小的特征，这对于融合不同分辨率的图像至关重要。 2. **全局标识分支结构**：为了保持输入图像的光谱特性不被破坏，模型采用了全局标识分支结构。这一设计允许原始光谱信息直接传递到输出，确保了光谱保真度，这是在进行图像融合时必须考虑的重要因素。 3. **密集学习块**：集成的密集学习块增强了模型在光谱-空间表达上的能力。密集连接的概念允许信息在不同层之间更自由地流动，增强了特征的复用，从而提高了模型的整体性能。实验结果显示，与现有的最新方法相比，该方法在视觉效果和量化指标上都表现出更高的清晰度和质量。这表明，提出的多尺度密集深度学习方法能够更有效地融合光谱和空间信息，生成高质量的高分辨率多光谱图像。这项工作在深度学习应用于Pansharpening技术上迈出了重要的一步，为未来在遥感图像处理和分析领域提供了新的工具和思路。通过结合深度学习的强大学习能力和多尺度、密集连接的网络结构，这种方法有望进一步提升图像融合的效果，服务于各种实际应用需求。

A MULTI-SCALE DENSELY DEEP LEARNING METHOD FOR PANSHARPENING

Zhikang Xiang

, Liang Xiao

1∗

, Pengfei Liu

, Yufei Zhang

School of Computer Science and Engineering,

Nanjing University of Science and Technology, Nanjing, China

School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing, China

xiaoliang@mail.njust.edu.cn

ABSTRACT

Pansharpening aims to produce a higher resolution multi-

spectral (HRMS) image by fusing the spectral information in

lower resolution multispectral (LRMS) image and the spatial

information in corresponding high resolution panchromatic

(PAN) image. In this work, we propose a multi-scale densely

deep learning based pansharpening method. Following an

end-to-end learning architecture, the proposed deep neural

network contains three modules: 1) a parallel multi-scale

convolutional layer is used to extract multiscale features of

PAN image; 2) a global identity branch structure is adopted to

preserve spectral structures; and 3) a dense learning block is

integrated to improve the spectral-spatial expressive power.

Compared with other state-of-the-art methods, experimen-

tal results obtained with our proposed method achieve high

pansharpening quality in visualization and quantification.

Index Terms— Pansharpening, Multiscale feature extrac-

tion, dense connection

1. INTRODUCTION

Since the remote sensing system is limited by optical diffrac-

tion, modulation transfer function, and signal-to-noise ratio, it

is difficult to obtain high resolution both in spatial and spec-

tral dimension. For this reason, pansharpening technique is

widely used to produce an HRMS image by fusing an LRMS

image and a corresponding PAN image.

In last decades, the researchers have developed various

pansharpening methods. Traditional pansharpening meth-

ods include component substitution (CS) and multiresolution

analysis (MRA) [1]. The former consists in transforming the

LRMS image in a suitable domain where one of the compo-

nents is replaced by the high-resolution PAN image, such as

This work has been supported by the National Natural Science Foun-

dation of China (Grant No. 61571230, 61871226, 61802202), the Na-

tional Major Research Plan of China (Grant No. 2016YFF0103604), the

Jiangsu Provincial Natural Science Foundation(Grant No. BK20161500,

BK20170905), the Jiangsu Provincial Key Developing Project(BE2018727),

the Postgraduate Research & Practice Innovation Program of Jiangsu

Province(KYCX18_0437). (corresponding author: Liang Xiao)

intensity-hue-saturation (IHS), principal component analy-

sis (PCA), and Gram-Schmidt (GS) [1]. These methods are

very popular owing to their easy implementation and fast

computation. The MRA based methods inject the resampled

LRMS band by spatial details obtained by multiresolution

decomposition of PAN image. The spatial details can be ex-

tracted according to approaches such as Laplacian pyramid

[3], wavelet transform [4] and nonseparable transform [1].

Compared with the CS methods, MRA based methods are

better in persevering spectral information, but worse in spa-

tial detail injecting. In summary, traditional methods usually

retain spatial information with the cost of spectral distortion.

Recently, convolutional neural network (CNN) has been

widely used in pansharpening. The advantage of CNN is the

fact that does not require manual priors, and it can learn the

mapping between input and output from the available LR and

HR remote sensing images. Existing CNN-based pansharp-

ening methods take the idea of single image super-resolution

as a reference, and can generally be divided into two groups.

The first group assumes that the relationship between LR/HR

MS image patches is same as LR/HR PAN image patches [5].

Actually, this assumption ignores the difference between PAN

image and each band of MS image, and makes no use of PAN

image in the reconstruction. The other one takes MS and

PAN images as input and trains an end-to-end network that

directly outputs the pan-sharpened image [6]. For example, a

deep pansharpening method by convolutional neural network

called PNN [7] adopts an architecture previously proposed for

image super-resolution and regards the problem as a simple

image regression with a black-box deep learning network.

In this paper, we proposed a multiscale deep neural net-

work shown in Fig. 1, which has the following features:

1.A parallel framework is used to extract the multiscale

features of PAN image together with high-pass information

of up-sampled LRMS image to reconstruct details of HRMS

image.

2.The network layers are densely connected to avoid the

vanishing-gradient problem during model training.

3.A global identity branch structure using up-sampled

LRMS image is adopted to preserve spectral structures.

下载后可阅读完整内容，剩余3页未读，立即下载

weixin_38659812

粉丝: 3
资源: 882

多尺度密集深度学习在泛锐化中的应用

通过稀疏回归进行泛锐化

泛锐化的新几何强制变分模型

MATLAB乘法运算在人工智能中的应用：深度学习模型的训练利器

MATLAB矩阵输入与深度学习的无缝结合：探索神经网络训练，提升模型性能

OpenCV图像锐化在虚拟现实中的应用：图像逼真度提升、沉浸感增强，打造身临其境的VR体验

【机器视觉缺陷检测流程】：优化方法与应用案例

机器学习赋能图像处理：MATLAB图像分类实战

机器学习分类问题新视角：特征提取的决定性作用

机器学习与图像处理结合：ImageFile库助力训练识别模型

最新资源