An Infrared and Visible Image Fusion Method
Based on Non-Subsampled Contourlet Transform
and Joint Sparse Representation
Guiqing He*, Dandan Dong, Zhaoqiang Xia, Siyuan Xing, Yijing Wei
School of Electronics and Information
Northwestern Polytechnical University
Xi’an, Shaanxi, 710072, China
guiqing_he@nwpu.edu.cn
Abstract—In conventional fusion methods based on Non-
Subsampled Contourlet Transform (NSCT), low-frequency
subband coefficient of an image fails to express sparsely the
image’s low-frequency information, not in favor of extracting
source image features. To address this issue, an infrared and
visible image fusion method based on NSCT and joint sparse
representation (JSR) was proposed, in which, JSR transform of
the image’s low-frequency information is conducive to improving
sparsity of low-frequency subband containing main energy of the
image; as to high-frequency information, use of feature product
as a fusion rule is beneficial to extract detail feature of the source
image. experimental result indicates that, compared with
conventional multiscale transform-based DWT, NSCT-based
fusion method and sparse representation-based SR and JSR
algorithms, the method in this paper achieved better fusion effect,
capable of keeping target information of the infrared image and
background detail information (edge, texture, etc.) of the visible
image better.
Keywords—image fusion; infrared and visible images; non-
subsampled Contourlet transform; joint sparse; feature product
I. INTRODUCTION
Image fusion refers to integration of different images of
one object acquired by multiple sensors or multiple images of
it acquired by one sensor using a specific method in order to
obtain a more comprehensive and more accurate description
technique. Infrared and visible image fusion is an important
active research area in image fusion field, and had wide
application in a lot of military and civil fields such as military
reconnaissance and security monitoring. An infrared image is
acquired from thermal radiation information scattered or
reflected by target scene, basically unaffected by illumination
conditions, and reflects contour feature of the target scene, but
detail information is not rich; a visible image is acquired
according to spectral reflection characteristics of the target
scene, and contains the scene’s rich detail information
including edge and texture, but the image is much affected by
scene illumination. Fusing infrared and visible images is able
to take full advantage of mutual complementation of
information from both images, and describe scene information
more comprehensively and more accurately.
In the field of infrared and visible image fusion, the fusion
methods based on multiscale transform-domain analysis are
one kind of major methods, among which Non-Subsampled
Contourlet Transform (NSCT) [1][2] is widely focused and
studied by many scholars, as it has fast, stable, translational
invariant, multi-resolution and multi-directional image
representing capability, capable of capturing geometric
information of the natural image and overcoming a problem of
two or higher dimensional singularity that cannot be dealt with
by conventional wavelet transform, and it has been
successfully applied in image fusion field with relatively
optimal fusion effect. However, it is expected in image fusion
that an extracted image representation coefficient has excellent
sparsity and feature retention, so that relatively optimal fusion
result can be obtained by infusing a small number of
coefficients. Nonetheless, through NSCT decomposition, a
source image will generate a low-frequency subband and
multiple high-frequency directional subbands. The low-
frequency subband contains low-frequency information of the
source image, its coefficients approximated to zero are very
limited, that is, low-frequency information of the source image
cannot be represented sparsely, and thus direct fusion of it
would not be favorable to extract source image features. Given
that a low-frequency subband contains major energy of an
image and governs quality of fusion result to a very large
extent, so we hope to get more superior fusion result by
improving sparsity of low-frequency subband coefficient.
In this paper, an infrared and visible image fusion method
based on NSCT and joint sparse representation [3][4] is
proposed. Firstly, NSCT is used to extract effectively
information of low-frequency subband and high-frequency
directional subbands of a source image. Secondly, according
to joint sparse representation theory, low-frequency subband
coefficients are sparsely represented, thereupon common and
specific coefficients are extracted and fused; for high
frequency subbands, a fusion rule based on feature product is
used to capture detail features of the source image, such as
edge, linear feature and regional boundaries. Experimental
result indicates that, compared with other typical and
advanced fusion methods, the proposed method had improved