Hybrid fusion and demosaicing algorithm with near-infrared image
X. Y. Luo*
a
, J. Zhang
b
, Q. H. Dai
c
a
Image Processing Center, School of Astronautics, Beihang University, 100191 Beijing, China
b
School of Electronics and Information Engineering, Beihang University, National Key Laboratory
of CNS/ATM, 100191 Beijing, China
c
Department of Automation, Tsinghua University, Tsinghua National Laboratory for Information
Science and Technology (TNList), 100084 Beijing, China
ABSTRACT
In this paper, we propose a unified framework for color filter array (CFA) interpolation and visible/NIR image
combination. The proposed method aims to reconstruct a high quality image from raw CFA data and the corresponding
NIR image, similar to a multi-spectral fusion of color and NIR images. Based on NIR image, we impose a sparse
constraint of gradient difference to modify the traditional color interpolation. The experiments indicate the effectiveness
of our hybrid scheme to acquire joint color and NIR information in real-time, and show that this hybrid process can
generate a better color image when compared with treating interpolation and fusion separately.
Keywords: Color filter array, demosaicing, hybrid interpolation, image fusion
1. INTRODUCTION
Various demosaicing algorithms for color filter array (CFA) have been proposed. The simplest method interpolated each
color channel separately based on the inter-pixel correlation of neighboring pixel intensities. Depending on the statistical
characteristics of the neighborhood of the pixel, an adaptive demosaicing algorithm decided among three different
interpolation approaches [1]. The most conventional demosaicing algorithms applied edge-directed interpolation with the
inter-color correlation [2]-[3]. They obtained the edges via some pre-determined directional filters, so only the edges of
limited direction could be handled. To develop the edge detector, Chung and Chan proposed a new integrated gradient as
edge-sensing parameter to guide their decision-based demosaicing method [4]. Recently, a similarity-based demosaicing
algorithm was introduced to deal with edges of any direction [5]. Some studied the inter-channel correlation, and
recovered the missing color data referencing to another channel component [6]-[7]. These methods preserved better high-
frequency information of natural images. Gunturk et al. reported that high-frequency contents between the R, G, and B
channels hold high correlation [8]. Based on this correlation, they proposed a color interpolation algorithm via
alternating projections (AP) [8]. Owing to the good performance, this method has become a popular benchmark in CFA
research. Recently, the high correlation was extended to similarity [9], and Lu et al. simplified AP method to a one-step
implementation via analyzing its convergence [10].
Silicon-based digital sensors of most consumer use are also sensitive to the near-infrared (NIR) electromagnetic
spectrum. For the same scene, the visible and NIR image pairs can be captured via separating the spectrum or filtering
off one, such as a dual-camera system with a hot mirror [11]-[12], alternatively placing a NIR or visible blocking filter
on lens [13], and a 2CCD multi-spectral camera [14]. NIR is proximate to visible spectrum, which has different but
additional luminance and spatial information. Considering this property, jointing NIR and visible images has been
exploited to improve several vision and photography tasks in recent years. Generally, these applications focus on two
ways. One extracts more accurate information of original scene based on the extra information of NIR images, such as
illuminant detection [15] and material classification [16]. The other applications combine the color and NIR images to
enhance image. For example, Zhang et al. fused NIR and visible images to improve color rendering for a high dynamic
range [12]. The chrominance of color image was used to color NIR image [17]. Similarly, color image dehazing [13],
denoising [18], and deblurring [19] have been proposed.
*luoxy@buaa.edu.cn; tel.(86)-010-82338061
Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2014
Proc. of SPIE Vol. 9121, 91210J · © 2014 SPIE · CCC code: 0277-786X/14/$18
Proc. of SPIE Vol. 9121 91210J-1