PATCH-BASED RESIDUAL NETWORKS FOR COMPRESSIVELY SENSED
HYPERSPECTRAL IMAGES RESTRUCTION
Xiaowei Hu
1
, Yang Xu
1
, Zhihui Wei
1*
, Hongyi Liu
2
, Ling Qian
3
1
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
2
School of Science, Nanjing University of Science and Technology, Nanjing 210094, China
3
School of Electronic and Optical Engineering, Nanjing University of Science and Technology, Nanjing 210094, China
ABSTRACT
Most traditional compressive sensing (CS) reconstruction
methods suffer from the intensive computation caused by
iterations. This paper aims at presenting a non-iterative
algorithm to reconstruct hyperspectral images (HSI) from
patch-based compressively sensed measurements. Our
method contains two residual convolutional neural networks.
One is reconstruction network for compressive sensing
reconstruction and the other is deblocking network for
removing the blocky effect, which is caused by patch-based
sampling. The reconstruction network can efficiently
reconstruct all the bands of HSI jointly, thus the spectral
correlation is well preserved. In addition, the deblock
performance is enhanced by combining more patches into a
larger patch in the deblocking network. Experimental results
verify that our method outperforms the state-of-the-art
compressive sensing reconstruction methods with patch-
based CS measurement.
Index Terms— residual network, compressive sensing,
image deblocking, hyperspectral images
1. INTRODUCTION
This work was supported in part by the National Natural Science
Foundation of China under Grant No. 61701238, 61772274, 61471199,
91538108, 61671243, 11431015, the Fundamental Research Funds for the
Central Universities under Grant No.30917015104, the Jiangsu Provincial
Natural Science Foundation of China under Grant BK20170858, the
Jiangsu Province Six Top Talents Project of China WLW-011, China
Postdoctoral Science Foundation under Grant 2017M611814, Jiangsu
Province Postdoctoral Science Foundation under Grant 1701148B.
*Corresponding author. Email: gswei@njust.edu.cn
These methods build an optimization problem which
requires iterative algorithms to solve.
In recent years, deep learning has made remarkable
achievements in image classification [4], image segmentati-
on [5] and other applications. Now, some researchers extend
it to the compressed sensing. For example, Kulkarni et al [6]
proposed a convolutional neural network architecture:
ReconNet. It takes in CS measurements of an image as input
and outputs an intermediate reconstruction, the intermediate
reconstruction is fed into an BM3D denoiser to obtain the
final reconstructed image. Ali Mousavi and Richard G.
Baraniuk [7] proposed a DeepInverse network. It learns the
inverse transformation from measurement vectors to signals
using a deep convolutional network. However, these
networks are designed for natural image and due to the high
dimension of the image, they often use patches to train and
test, thus reconstruction results of these methods suffer from
blocky artifacts.
Hyperspectral image (HSI) is a 3D data cube that
contains a series of 2D spatial images over continuous
spectral bands. Thus, it not only contains spatial structure,
but also has strong spectral correlations between different
bands. The traditional CS reconstruction reconstructs the
images band by band. Therefore, it cannot be used in HSI
CS which need to be processed all bands jointly. In our
paper, we propose a patch-based residual reconstruction
network for reconstructing all the bands of HSI jointly. The
spectral correlation is considered and reserved in the
reconstruction network. In addition, we combine the small
patches into a larger patch and put it into a deblocking
network to reduce the blocky effect.
2. PROPOSED METHOD
The proposed method follows the steps of the Fig.1 pipeline
showing. It mainly has 3 parts: patch based compressed
sensing, reconstruction network and deblocking network. In
the following of this chapter, we will introduce them in
details.
2.1. Patch Based HSI Compressed sensing