An Improved Artifact Removal in Exposure Fusion with Local Linear
Constraints
Hai Zhang, Mali Yu
*
School of Information Science & Technology, Jiujiang University, Jiujiang 332005, China
ABSTRACT
In exposure fusion, it is challenging to remove artifacts because of camera motion and moving objects in the scene.
An improved artifact removal method is proposed in this paper, which performs local linear adjustment in artifact
removal progress. After determining a reference image, we first perform high-dynamic-range (HDR) deghosting to
generate an intermediate image stack from the input image stack. Then, a linear Intensity Mapping Function (IMF) in
each window is extracted based on the intensities of intermediate image and reference image, the intensity mean and
variance of reference image. Finally, with the extracted local linear constraints, we reconstruct a target image stack,
which can be directly used for fusing a single HDR-like image. Some experiments have been implemented and
experimental results demonstrate that the proposed method is robust and effective in removing artifacts especially in the
saturated regions of the reference image.
Keywords: Local linear constraints, deghosting, exposure fusion.
1. INTRODUCTION
Dynamic range of the real world exceeds that of the typical digital imaging devices. Hence, it is difficult to preserve
all details of the scenes with large amount of luminance levels in a photograph. To address this problem, the most
common method is to fuse differently exposed low-dynamic-range (LDR) images of the same scene to generate a high-
dynamic-range (HDR) image [1-4]. The main assumption of those methods is that the scene is completely static, so any
camera motion or moving objects in the scene will cause blurring and ghosting artifacts in the resulting image.
In the last decades, various HDR deghosting methods have been reported in literature, such as motion detection-
based methods [5-7], rigid registration-based methods [8-10], non-rigid registration-based methods [11-13]. A good
review of some of these methods can be seen in [14].
Among those methods, non-rigid registration-based methods are commonly used because of their remarkable effect
in dealing with complex movement and scene. Recently, non-rigid registration methods are grouped into two types:
pixel-pairwise matching and patch-pairwise matching. The typical method of pixel-pairwise matching is optical flow.
Hossain et al. first proposed to estimate the global intensity mapping and the dense displacement field by combining
weighted histograms and optical flow [11]. Zimmer et al. proposed an energy-based optic flow to estimate the dense
motion field with subpixel precision, which can be used for simultaneous super-resolution and HDR [12]. Hafner et al.
developed an energy minimization method to simultaneously reconstruct HDR image and estimate dense displacement
field [13]. The methods discussed above improve significantly registration quality, but wrong displacement fields exist
when the scene has large or complex displacements.
Recently patch-pairwise matching has achieved high performance in HDR imaging. Sen et al. integrated HDR
reconstruction and dense correspondence estimation in a PatchMatch-based method [15]. Qin et al. developed an
iterative optimization framework by combining the patch-pairwise matching and the exposure fusion based on the
random walker [16]. Hu et al. proposed to generate a aligned image stack from input images by using a new PatchMatch
model [17]. However, patch-pairwise matching methods require a reference image, so the performance highly depends
on the quality of reference image. Gallo et al. developed a locally non-rigid registration technique, which is less sensitive
to the reference image, but can only handle with small displacements [18].
Thus, we propose an improved artifact removal method, especially when the reference image has large saturated
regions. Our method is based on the observation that in a small window there is a linear mapping between the intensities
of two differently exposed images, which performs three steps. First, Hu et al. [17] is implemented, the output of which
is taken as the intermediate image stack. Then, in each small window, a linear Intensity Mapping Function (IMF) is