深度学习驱动的遥感图像融合方法

需积分: 10 59 浏览量更新于2024-08-05 收藏 1.78MB PDF 举报

"Remote_Sensing_Image_Fusion_With_Deep.pdf" 这篇论文《Remote Sensing Image Fusion With Deep Convolutional Neural Network》由Zhenfeng Shao和Jiajun Cai两位作者发表在2018年5月的IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing，卷11，号5。文章主要探讨了利用深度卷积神经网络（Deep Convolutional Neural Network, DCNN）进行遥感图像融合的技术。遥感图像通常包括具有高空间分辨率但低光谱分辨率的全色（PAN）图像和具有高光谱分辨率但低空间分辨率的多光谱（MS）图像。为了同时获取这两种图像的优势，即高空间分辨率和高光谱分辨率，图像融合技术被广泛应用于生成复合遥感图像。该研究提出了一种基于深度卷积神经网络的新方法，旨在从源图像中充分提取光谱和空间特征。论文的主要创新点在于设计了一个双分支网络结构，这个结构更深入，能够捕获源图像的显著特征。这种双分支网络允许分别处理空间和光谱信息，然后将这两个分支的结果有效地融合在一起。通过深度学习的方法，模型可以学习到不同分辨率图像之间的复杂关系，并自适应地优化融合过程，从而生成高质量的复合图像。深度卷积神经网络在图像处理领域已经证明了其强大的特征提取能力，它通过多层非线性变换自动学习图像的层次特征。在遥感图像融合中，DCNN可以逐层提取图像的空间细节和光谱特性，这在传统的融合方法中可能难以实现。此外，由于网络的深度，它可以捕获更复杂的上下文信息，这对于提高融合结果的视觉质量和分析性能至关重要。论文可能会详细介绍网络架构的设计，包括卷积层、池化层、激活函数的选择以及损失函数的定义。训练过程和验证策略也会被讨论，以展示所提方法的有效性和优越性。最后，可能会通过与其他融合技术的比较实验，展示新方法在保留细节、提高信噪比和增强图像理解等方面的性能提升。这篇研究为遥感图像融合提供了一种新的深度学习解决方案，有望在地球观测和环境监测等领域产生积极影响，提高遥感数据的利用效率和分析精度。

1658 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 11, NO. 5, MAY 2018

Let x

(i)

and y

(j )

denote the ith input feature map and jth

output feature map of a convolutional layer. Therefore, a con-

volutional operation with activation function applied to x

(i)

can

be expressed as

(j )

= f



(j )



(i)(j )

∗ x

(i)



(1)

where k

(i)(j )

is a convolutional kernel which is applied to ith

input feature map to obtain the jth output feature map, and b

(j )

indicates the bias. The symbol ∗ indicates the convolutional op-

erator and f indicates the activation function. If a convolutional

layer consists of M input feature maps and N output feature

maps, there will be N convolutional kernels with the size of

d × d × M, where d × d also indicates the size of local recep-

tive ﬁelds. Besides, each kernel contains a bias. Selection of

proper activation function is an essential part of a neural net-

work, and conventional activation functions include Sigmoid,

Tanh, and ReLU [35]. For example, (1) can be re-expressed as

the following by incorporating the nonlinear ReLU activation

function:

(j )

= max



0,b

(j )



(i)(j )

∗ x

(i)



. (2)

B. Remote Sensing Image Fusion Based on CNN

Most of current remote sensing image fusion methods usu-

ally contain two steps: Feature extraction and feature fusion.

For example, when the multiresolution analysis or sparse rep-

resentation is used to achieve image fusion, the ﬁrst step is to

express the source images via a series of base ﬁlters or atoms in

a dictionary. After the expressions are derived, the second step

is to choose appropriate strategies, such as weights differences,

to fuse expressions of source images so that expressions of the

fusion image can be generated. It is noteworthy that all of the

procedures can also be equalized to apply different convolu-

tional kernels to achieve feature extraction and feature fusion.

We will demonstrate this operation in Section III-A in detail.

Therefore, as convolutional layers can achieve the effect as same

as traditional fusion methods, it is reasonable and reliable to use

CNN to extract the characteristics of different remote sensing

images and fuse them to obtain f usion image.

The conventional CNN is usually adopted to solve the im-

age classiﬁcation problems [30], [54]. By putting an image into

networks, the output will be the probability of the image be-

longing to each category. While putting CNN to deal with im-

age super-resolution reconstruction issues, the designed CNN

removes the pooling procedure, and the output of the network is

reconstructed images which have the same size as input images.

Speciﬁcally, inputs and labels for network training are low-

resolution and high-resolution images respectively [36], [37].

In order to lower the difference between network outputs and

labels, the network will continuously learn parameters to ﬁt la-

bels. To utilize CNN to remote sensing image fusion, we adopt

the same thoughts from the ﬁeld of image super-resolution re-

construction. Fusion aims to generate a MS image with high

spatial resolution. Therefore, the label of the network is a high

spatial resolution MS image and the inputs are a PAN image

and a low spatial resolution MS image.

Presently, there are few deep learning based studies in re-

mote sensing image fusion. Masi et al. [39], Palsson et al. [40],

and Zhong et al. [55] directly adopted SRCNN [36], which is

a popular network for image super-resolution reconstruction,

to implement remote sensing image fusion. However, the ﬁrst

two methods only use three convolutional layers and cannot

adequately leverage the depth of the network to extract deep

features. Also, these methods regard the PAN images as a band

and overlay it on the MS images to train t he network, which

ignores distinctive characteristics of these two types of images.

While the third method only uses SRCNN to do image super-

resolution rather than image fusion. The fusion procedure is still

ﬁnished by traditional Gram-Schmidt transform method.

III. P

ROPOSED FUSION FRAMEWORK

In this section, we demonstrate our remote sensing image

fusion method in detail. The network architecture is shown in

Fig. 2, which uses the acronym RSIFNN in the f ollowing, mean-

ing CNN-based remote sensing image fusion.

A. Network Design

The proposed method contains the same procedures as the

classical remote sensing fusion methods: Feature extraction and

feature fusion. Different from other deep learning-based fu-

sion methods [39], [40], [55], the proposed method design two

branches to extract features of the MS and PAN images sep-

arately. Fig. 2 shows the whole training scheme of RSIFNN,

where the input MS and PAN images generated from the Quick-

Bird satellite are of 2.8 m and 0.7 m spatial resolution.

The output of network should be the MS image with same

spatial resolution as the PAN image. This image needs to be as

similar as possible to the ideal MS image (label) which obtained

by a sensor with same spatial resolution as the PAN image.

However, this ideal MS image does not exist. It will cause

troubles for our training and performance assessment if there

do not exist labels. Fortunately, this problem can be solved by

using Wald’s protocol [38]. In Wald’s protocol, the original MS

images are regarded as labels. For keeping the same spatial

resolution as the original MS image (labels), the original PAN

image needs to be down-sampled according to the ratio between

the resolution of MS image and PAN image. At the same time,

the input MS image with low spatial resolution is generated

by down-sampling the original MS image and interpolating the

down-sampled data. Fig. 3 shows the procedure of preparing

training samples. In the case of QuickBird images, we have MS

images and PAN images of 2.8 m and 0.7 m spatial resolution.

The inputs are obtained by down-sampling the original MS and

PAN images by factor 4 (the multiple relationships between the

original MS images and PAN images) to get MS images and

PAN images of 11.2 m and 2.8 m spatial resolution. Therefore,

we can transform original fusion task to obtain fused MS images

(2.8 m) by fusing low-resolution MS images (11.2 m) and PAN

images (2.8 m). In this way, we have labels (original MS images)

Authorized licensed use limited to: Hebei University of Technology. Downloaded on March 26,2021 at 10:51:46 UTC from IEEE Xplore. Restrictions apply.

剩余13页未读，继续阅读

23❤️

粉丝: 1

深度学习驱动的遥感图像融合方法

remotesensing-10-01662-v2.pdf

Comprehensive_survey_of_deep_learning_in_remote_sensing.pdf

remote-sensing-2.rar_matlab 遥感_remote_remote sensing_遥感

从https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes上爬取Indian Pines数据

从https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes上爬取Indian Pines数据，提供源代码

编写一段代码使用deeplabv3+网络实现遥感影像中建筑物的自动提取

SENSING_OPTION == ADC_BASED_SENSING是什么意思

Referring remote sensing image segmentation

最新资源