68 D. Connah, M.S. Drew, and G.D. Finlayson
pyramids [4] and their variants [36], wavelets [24], complex wavelets [23], per-
ceptual transforms using centre-surround filters [38], bilateral filtering [21], or
multi-scale representations of the first fundamental form [32]. These methods
are often complex and intensive to compute, as well as being prone to generat-
ing artefacts when conflicting information appears in different image channels,
making them more suited to fusing pairs of images rather than multiple chan-
nels. Finally, the base layer of the pyramid, or wavelet decomposition, is often
a low-pass average image, which can lead to poor colour separation at edges for
low spatial scales.
Socolinsky and Wolff [33] cast image fusion as a variational problem, where
the goal is to find a greyscale output with gradient information as similar as
possible to the input image set. This approach solves the problem of greyscale
separation at low spatial scales, but can also be prone to warping artefacts
close to edges. These are exacerbated by the ambiguity of gradient ordering
at each pixel [15]. Piella [29] uses a variational approach to generate an output
that simultaneously preserves the underlying geometry of the multivalued image,
similarly to Socolinsky and Wolff, and performs an edge enhancement to improve
greyscale separation at object boundaries. The integration of gamut constraints
means that potential for artefacts is greatly reduced using this method, but
necessitates that the objective function is minimised using an iterative gradient
descent scheme, which restricts the speed of the method. As with the wavelet-
based approaches, the outputs are in greyscale only.
Several strategies exist for mapping high-dimensional images to RGB, rather
than just greyscale. Jacobson et al. [20] investigate different fixed projections;
these have an advantage over adaptive methods that colours remain fixed across
different visualisations, but the disadvantage that they preserve less information.
Adaptive approaches using standard decompositions such as PCA and ICA have
also proved popular. Tyo et al. [37] use PCA to extract a 3-D subspace from
the spectral data, and then rotate the basis of this space so that the final 3D
co-ordinates form a plausible RGB image. While this approach is information
preserving, the false coloured output can deviate from the ‘natural’ representa-
tion, and the global nature of the transform means that localised metamerism
may still be common.
In particular applications greyscale fusion schemes can also be applied to gen-
erate colour outputs. Schaul et al. [31] employ fusion of near-infrared (NIR)
and RGB images as part of a de-hazing scheme. They firstly decompose the
RGB image into an opponent-based representation and then use an edge-aware
multiscale representation to fuse the NIR and luminance channels into a single
greyscale. This greyscale is then swapped into the original image as the lumi-
nance component. Our approach differs in that it maps the contrast of each of
the R, G and B, channels as well as the NIR image, rather than just luminance
and NIR. Fay et al. [13] use dual-band RGB / long-wave infrared (LWIR) to im-
prove night-vision in low-light settings. This work, which results in fused colour
imagery, is specifically focused on a low-light-sensitive visible-light CCD imager.