A ROBUST SPARSE REPRESENTATION FRAMEWORK FOR DEPTH MAP RESTORATION
Xien Liu,Yanfeng Sun,Yongli Hu, Baocai Yin
Beijing Key Laboratory of Multimedia and Intelligent Software Technology
College of Computer Science and Technology
Beijing University of Technology
xienliu0201@gmail.com, {yfsun, huyongli, ybc}@bjut.edu.cn
ABSTRACT
Recently, techniques based on dictionary learning for sparse
representation have demonstrated promising results for depth
or disparity maps restoration. However, we show that these
methods are not robust due to the fact that depth or disparity
maps are not only slightly contaminated by additive Gaussian
noise but also seriously corrupted with outliers, occlusions, or
even variable uncertainties. These seriously corrupted pixels
not only lead to irregular structures obtained by dictionary but
also seriously deteriorate the sparse coding effectiveness.To
overcome these problems, in this paper we propose a new ro-
bust sparse representation framework to restore depth maps.
In our proposed framework, seriously corrupted pixels can be
automatically identified and their disturbance effects are grad-
ually diminished through a few iterations. Thus, our proposed
framework is more robust for depth restoration. Experimen-
tal results are presented to demonstrate the effectiveness of
the proposed framework.
Index Terms— Sparse coding, dictionary learning, depth
restoration, robust sparse representation
1. INTRODUCTION
Depth provides an important cue to understand real world
scenes. In the past decade, growing applications involves
depth data, such as view synthesis [1] and object tracking [2]
etc. However, depth data often cannot be used in real appli-
cations directly because they are often noisy. In this paper,
we consider using learning sparse representation methods to
restore corrupted depth or disparity maps.
Learning sparse representation plays an important role in
dealing with inverse problems, such as image restoration. We
usually assume that noise does not support sparsity, thus can
not be reconstructed in a sparse fashion. M. Elad and W. Dong
et al. have successfully used learning sparse representation
to restore noised images and achieved state-of-the-art results
[3][4][5][6]. In their methods, a set of overlapping patches
This work is supported by the National Basic Research Program of China
(973 Program)(No. 2011CB302703), NSFC (No. 61133003, 61171169), and
BJNSF(4132013, KZ201310005006).
directly taken from the given corrupted image were used to
train an overcomplete dictionary. And then all these patches
were approximated by sparse representations over the trained
dictionary. Lastly, a whole clean image was obtained by gath-
ering all these approximated patches.
Similar to natural images, Depth or disparity maps are al-
so spatial smoothness and support sparsity. Thus, it seems
that the above learning framework is also suitable for depth
restoration. More recently, M. Mahmoudi et al. have tried to
use this framework to restore depth maps [7][8], and promis-
ing results were obtained. However, we argue that such
framework is not robust to depth maps as it to images due
to the fact that besides additive Gaussian noise depth maps
are usually seriously corrupted with replacement noise, such
as occlusions, outliers and even uncertainties etc. These seri-
ously corrupted pixels (also called bad pixels) not only lead to
irregular structures obtained by dictionary but also seriously
deteriorate the sparse coding effectiveness.
To overcome these drawbacks, in this paper we propose a
new robust sparse representation framework for depth restora-
tion. In our proposed framework, bad pixels can be automati-
cally identified and their disturbance effects are gradually di-
minished through a few iterations; eventually, a high quality
depth map is recovered. It is worth to point out that there
does not need a special extra detecting process. Bad pixels in
depth maps can be detected simply based on the representa-
tion residuals. The detecting and the restoring fuse together
naturally into one coherent and iterative process.
In the next section we first summarize the learning sparse
representation framework and then analyze its drawbacks for
depth restoration. The proposed robust sparse representation
framework is presented in Section 3. And, in Section 4 some
experiments are conducted.
2. LEARNING SPARSE REPRESENTATION AND ITS
DRAWBACKS FOR DEPTH
The key idea of learning sparse representation is to find an
overcomplete dictionary D ∈ R
n×k
(with k > n)that can
”best” approximate given signals in a sparse fashion. Take