408 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 22, NO. 1, JANUARY 2013
[17] B. K. Gunturk, Y. Altunbask, and R. M. Mersereau, “Color plane
interpolation using alternating projections,” IEEE Trans. Image Process.,
vol. 11, no. 9, pp. 997–1013, Sep. 2002.
[18] D. D. Muresan and T. W. Parks, “Demosaicing using optimal recovery,”
IEEE Trans. Image Process., vol. 14, no. 2, pp. 267–278, Feb. 2005.
[19] X. Li, “Demosaicing by successive approximations,” IEEE Trans. Image
Process., vol. 14, no. 3, pp. 370–379, Mar. 2005.
[20] K.-H. Chung and Y.-H. Chan, “Color demosaicing using variance of
color differences,” IEEE Trans. Image Process., vol. 15, no. 10, pp.
2944–2955, Oct. 2006.
Two-Direction Nonlocal Model for Image Denoising
Xuande Zhang, Xiangchu Feng, and Weiwei Wang
Abstract—Similarities inherent in natural images have been
widely exploited for image denoising and other applications. In
fact, if a cluster of similar image patches is rearranged into a
matrix, similarities exist both between columns and rows. Using
the similarities, we present a two-directional nonlocal (TDNL)
variational model for image denoising. The solution of our model
consists of three components: one component is a scaled version
of the original observed image and the other two components
are obtained by utilizing the similarities. Specifically, by using
the similarity between columns, we get a nonlocal-means-like
estimation of the patch with consideration to all similar patches,
while the weights are not the pairwise similarities but a set of
clusterwise coefficients. Moreover, by using the similarity between
rows, we also get nonlocal-autoregression-like estimations for the
center pixels of the similar patches. The TDNL model leads to
an alternative minimization algorithm. Experiments indicate that
the model can perform on par with or better than the state-of-
the-art denoising methods.
Index Terms—Image denoising, similarity, two-direction non-
local model.
I. INTRODUCTION
Denoising is a fundamental and widely studied problem in image
processing. Various denoising methods have been proposed following
different disciplines such as statistics, variational theory, etc. Most of
these methods exploit the local correlation of image pixels. Recently,
the introduction of NLM opens the floodgate to the exploitation of
nonlocal similarities inherent in natural images for denoising and
other applications [1], [2]. The NLM estimates each pixel by the
weighted average of many pixels in the image, and the weights
are respectively evaluated according to pair-wise similarity between
two patches. The advantage of NLM is that it greatly reduces the
interference of noise and well preserves the details such as edges
and textures in the denoised image.
Manuscript received September 9, 2011; revised June 20, 2012; accepted
August 1, 2012. Date of publication September 19, 2012; date of current
version December 20, 2012. This work was supported by the National
Science Foundation of China under Grant 61001156, Grant 61105011, Grant
11101292, and Grant 60872138. The associate editor coordinating the review
of this manuscript and approving it for publication was Prof. Sina Farsiu.
X. Zhang is with the Department of Applied Mathematics, School of
Science, Xidian University, Xi’an 710071, China, and also with the School of
Mathematics and Computer Science, Ningxia University, Yinchuan 750021,
China (e-mail: love_truth@126.com).
X. Feng and W. Wang are with the Department of Applied Mathemat-
ics, School of Science, Xidian University, Xi’an 710071, China (e-mail:
xcfeng@mail.xidian.edu.cn; wwwang@mail.xidian.edu.cn).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TIP.2012.2214043
1057–7149/$31.00 © 2012 IEEE
Another line of work developed in recent years is built on sparse
representation. As early as in wavelet era, it is recognized that natural
image has a sparse representation in wavelet basis and its directional
extensions, including curvelet, contourlet and bandelet [3], while
noise does not. This discriminative property is exploited in various
wavelet shrinkage denoising methods. To alleviate the problems
caused by using the fixed transformation, several authors propose
to use learned dictionary, which is data adaptive and can characterize
image structures more efficiently. Elad et al. [4], [5] introduced the
K-SVD algorithm to learn the overcomplete dictionary for image
representation and denoising. Zhang et al. [6] presented an adaptive
shrinkage algorithm based on locally learned principle component
analysis (PCA) basis for image denoising. All these patch-based,
adaptive learning methods show promising denoising performance.
All the methods mentioned above use certain prior information
of the given data. For example, NLM exploits similarity inherent
in images, wavelet shrinkage and K-SVD utilize sparsity of images
in certain domain. By combining the nonlocal similarity and the
sparsity, Dabov et al. [7] presented a block matching based, two-stage
3D filtering algorithm (BM3D) for image denoisng, which achieves
so far the best denoising performance. In this algorithm, similar
patches are stacked into a 3D array, and the array is transformed
into wavelet domain by a 3D separable wavelet transform, then a
filtering is performed in the wavelet domain. The employment of Haar
wavelet on the third dimension contributes much to the denoising
performance of BM3D. Note that the Haar wavelet has only a zero-
order vanishing moment, which implies BM3D implicitly assumes
that similar patches have similar sparse representation. Explicitly
using this assumption, Mairal et al. [8] proposed a mixed-sparse
model with learned redundant dictionaries for image denoising and
color demosaicking. Dong et al. [9] proposed a centralized sparse
model with appropriately learned and chosen principle component
analysis (PCA) basis for image denoising. The latter two methods
achieve comparable denoising performances with BM3D.
In [10] and [11], the authors studied the performance bounds on
image denoising from an estimation theory perspective and provided
the fundamental limits of the problem. In [12], the same authors
proposed the patch-based locally optimal wiener filter (PLOW)
for image denoising with motivation to achieve the near optimal
performance. This method uses both geometrically and photomet-
rically similar patches to estimate the filter parameters and achieves
equal or slightly better performance than BM3D.
In this brief, we provide a different point of view. Similar to the
work in [7]–[9], for each patch centered at the pixel in consideration,
a cluster of similar patches is collected and rearranged into a matrix.
Then similarity between both columns and rows of the matrix will
be exploited in different ways. Specifically, the similarity between
columns is actually that between patches. We will design an NLM-
like filtering to get a denoised patch from all the columns. Different
from NLM, the weights will be jointly obtained by solving a
minimization problem involving all the similar patches. In this sense,
we say the weights are cluster-wise while the weights of NLM are
pair-wise similarities. Motivated by the assumption in [13], here we
assume that, the central pixel in each patch can be linearly represented
by neighboring pixels through autoregression, and furthermore, the
central pixels of all similar patches, corresponding to a row in the
matrix, have the same AR coefficients. The similarity between rows
needs to be understood in this sense. Combining the two-directional
similarity, we present a two-directional nonlocal (TDNL) variatiotnal
model for image denoising. The model can be viewed as a two-
direction regression method, an enhancement to NLM, a special
case of dictionary learning methods, or an improvement of PCA