338 CHINESE OPTICS LETTERS / Vol. 6, No. 5 / May 10, 2008
Region-based fusion of infrared and visible images using
nonsubsampled contourlet transform
Baolong Guo (
HHH
999
), Qiang Zhang (
ÜÜÜ
rrr
), and Ye Hou (
ûûû
)
ICIE Institute, School of Electromechanical Engineering, Xidian University, Xi’an 710071
Received October 9, 2007
With the nonsubsampled contourlet transform (NSCT), a novel region-segmentation-based fusion algorithm
for infrared (IR) and visible images is presented. The IR image is segmented according to the physical
features of the target. The source images are decomposed by the NSCT, and then, different fusion rules for
the target regions and the background regions are employed to merge the NSCT coefficients respectively.
Finally, the fused image is obtained by applying the inverse NSCT. Experimental results show that the
proposed algorithm outperforms the pixel-based methods, including the traditional wavelet-based method
and NSCT-based method.
OCIS codes: 100.0100, 350.2660, 110.3080.
Image fusion aims at synthesizing information from mul-
tiple source images to obtain a mor e ac curate, complete
and reliable description of the same scene. The fusion
of the infrared (IR) image and the visible image is an
increasingly important topic and is being employed in
many fields such as night vision, video surveillance, and
so on.
During the last deca de, a number of fusion algorithms
have been proposed, and the fusion methods based on
the multiscale transform (MST) are the most typical.
The co mmo nly used MST tools include the Laplacian
pyramid
[1,2]
and the wavelet transform (DWT)
[3]
. In
general, due to the perfect properties of the DWT such
as multiresolution, spatial and frequency localiza tion,
and direction, the DWT-based methods are superior to
the pyramid-based methods
[4]
. However, the DWT also
has some limitations such as limited directions and non-
optimal-sparse representation of image s. Thus, some
artifacts are easily introduced into the fused images
using the DWT-based methods, which will re duce the
quality of the resultant image consequently
[5]
. In 2006,
Cunha et al. proposed a novel multiscale geometric anal-
ysis (MGA) tool, namely, the nonsubsampled contourlet
transform (NSCT)
[6]
. The NSCT is not only with mul-
tiscale, localization, and multi-direction, but also w ith
properties of shift-inva riance and the same size between
each subband image and the original image. Therefore,
the NSCT is more suitable for image fusion.
In addition, most of the above fusion algorithms are
mainly pixel-based methods. However, for most im-
age fusion applications, it seems more meaningful to
combine objects rather than pixels
[7]
. Therefore, some
region-based fusion algorithms
[8−12]
have bee n proposed
in recent years. In this paper, we present a novel regio n-
based fusion algorithm (NSCT RG) for IR and visible
images using the NSCT. Segmentation is firstly per-
formed on the IR image and, consequently, the NSCT
coefficients from the target regions and the background
regions are merged sepa rately. Finally the fused image
is obta ined by performing inverse NSCT.
The NSCT is a shift-invar iant version of the contourlet
transform
[13]
. To achieve shift-invaria nce , the NSCT
eliminates the downsamplers and the upsamplers during
the decomposition and the reconstruction of the image,
and then it is built upon the nonsubsampled pyramid
filter banks (NSPFBs) and the nonsubsampled direc-
tional filter ba nk s (NSDFBs). Figure 1 displays the
construction of the NSCT.
The NSPFB and NSDFB, employed by the NSCT, are
both two-channel nonsubsampled filter banks (NSFBs).
And both of them satisfy the Bezout identity, which
guarantees the perfect reconstruction. To achieve the
multiscale decomposition, the two-channel NSPFB is it-
eratively used. Such expansion is conceptually similar to
the one-dimensional (1D) ‘`a trous’ wavelet transform. To
achieve finer direction decomposition, the NSDFB is also
iteratively used. For example, to achieve the four-channel
direction decomposition, the image is firstly filtered by
the original fan filters; Secondly, the filtered subband
images are respectively filtered by the upsampled filters,
in which the sampling matrix D is the quincunx matrix,
i.e., D =
1 −1
1 1
. Then the four -channel direction
decomposition is obtained.
The building block two-channel NSFBs in the NSPFB
and the NSDFB are invertible; therefore, the NSCT is
clearly invertible. As well, the NSCT can sa tisfy the
anisotropic scaling law, which is a key property in estab-
lishing the expansively nonlinear approximation behav-
ior. This property can be ensured by doubling the num-
ber of directions in the NSDFB expansion at every other
Fig. 1. Nonsubsampled contourlet transform.
1671-7694/2008/050338-04
c
2008 Chinese Optics Letters