IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 53, NO. 3, MARCH 2015 1195
Improving the Spatial Resolution of Landsat
TM/ETM+ Through Fusion With SPOT5
Images via Learning-Based Super-Resolution
Huihui Song, Bo Huang, Member, IEEE, Qingshan Liu, and Kaihua Zhang
Abstract—To take advantage of the wide swath width of Land-
sat Thematic Mapper (TM)/Enhanced Thematic Mapper Plus
(ETM+) images and the high spatial resolution of Système Pour
l’Observation de la Terre 5 (SPOT5) images, we present a learning-
based super-resolution method to fuse these two data types. The
fused images are expected to be characterized by the swath width
of TM/ETM+ images and the spatial resolution of SPOT5 images.
To this end, we first model the imaging process from a SPOT image
to a TM/ETM+ image at their corresponding bands, by building
an image degradation model via blurring and downsampling oper-
ations. With this degradation model, we can generate a simulated
Landsat image from each SPOT5 image, thereby avoiding the
requirement for geometric coregistration for the two input images.
Then, band by band, image fusion can be implemented in two
stages: 1) learning a dictionary pair representing the high- and
low-resolution details from the given SPOT5 and the simulated
TM/ETM+ images; 2) super-resolving the input Landsat images
based on the dictionary pair and a sparse coding algorithm. It
is noteworthy that the proposed method can also deal with the
conventional spatial and spectral fusion of TM/ETM+ and SPOT5
images by using the learned dictionary pairs. To examine the
performance of the proposed method of fusing the swath width
of TM/ETM+ and the spatial resolution of SPOT5, we illustrate
the fusion results on the actual TM images and compare with
several classic pansharpening methods by assuming that the cor-
responding SPOT5 panchromatic image exists. Furthermore, we
implement the classification experiments on both actual images
and fusion results to demonstrate the benefits of the proposed
method for further classification applications.
Index Terms—Landsat Thematic Mapper (TM) or Enhanced
Thematic Mapper Plus (ETM+) image, spatial resolution,
Système Pour l’Observation de la Terre 5 (SPOT5) image,
super-resolution, swath width.
I. INTRODUCTION
W
ITH the emergence of a wide variety of remote sensing
instruments and the widespread application of remote
Manuscript received September 9, 2013; revised January 16, 2014 and
April 17, 2014; accepted June 3, 2014. The work of H. Song and B. Huang
was supported by the Hong Kong Research Grant Council under Grant CUHK
444612. The work of Q. Liu was supported in part by the National Natural
Science Foundation of China under Grant 61272223 and in part by the National
Science Foundation of Jiangsu Province under Grant BK2012045. The work of
K. Zhang was supported by the Nanjing University of Information Science and
Technology Scientific Research Foundation under Grant S8113049001.
H. Song, Q. Liu, and K. Zhang are with the Nanjing University of Informa-
tion Science and Technology, Nanjing 210044, China (e-mail: hhsongsherry@
gmail.com).
B. Huang is with the Chinese University of Hong Kong, Shatin, Hong Kong.
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TGRS.2014.2335818
sensing data, multisensor multiresolution image fusion is be-
coming increasingly more important [1]. A common feature
among these satellite sensors is that they trade off between
spatial resolution and other sensor properties, including spectral
resolution, temporal resolution, and swath width, caused by
technical and budget limitations. Two examples are the The-
matic Mapper (TM) or the Enhanced Thematic Mapper Plus
(ETM+) sensor aboard the Landsat satellite (we will take the
Landsat TM sensor as an example in this paper) and the High
Resolution Geometrical (HRG) instrument on the commercial
satellite Système Pour l’Observation de la Terre 5 (SPOT5).
Although launched by different agencies, SPOT (1–6 have
currently been launched) and Landsat (1–8 have currently been
launched) satellites have provided large amounts of data for
various fields since the launch of the first one. With the launch
of the new SPOT and Landsat satellites i n the future, studying
the fusion of data from these two platforms will greatly promote
their applications.
Landsat TM provides unique resources for global change
research and applications in such areas as agriculture, cartog-
raphy, and geology [2], with a spatial resolution of 30 m, at
most, of the seven bands (except band 6 with a spatial resolution
of 120 m) and a swath width of 185 km, whereas SPOT5
HRG aims for high-spatial-resolution images featured by four
multispectral bands [green, red, near-infrared (NIR) with a
spatial resolution of 10 m, and short-wave infrared (SWIR)
with a spatial resolution of 20 m] and one panchromatic band
with a swath width of 60 km. Obviously, these two sensors
are complementary in their spatial resolution and swath width
properties, i.e., the TM sensor is characterized by lower spatial
resolution but wider swath width, whereas the SPOT5 sensor
is characterized by higher spatial resolution but narrower swath
width. However, higher spatial resolution and a greater cover
area may be simultaneously needed in practical applications,
e.g., in large-area land cover/land use mapping with high spatial
resolution [3], [4] or in cases where higher spatial resolution
data are required but only low-resolution TM/ETM+ data are
available for the study scene. Thus, we propose to enhance the
spatial resolution of TM images by merging with an available
SPOT5 image.
In fact, the fusion of Landsat TM/ETM+ and SPOT5 images
has been explored by many researchers in the past two decades
[5]–[7]. However, most of them focused mainly on the fusion
of TM/ETM+ multispectral and SPOT5 panchromatic images
to integrate the good spectral property of the former and the
good spatial property of the latter, respectively. This spatial and
0196-2892 © 2014
IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.