EDGE-PRESERVING INTERPOLATION FOR DOWN/UP SAMPLING-BASED DEPTH
COMPRESSION
Huiping Deng, Li Yu, and Zixiang Xiong
∗
Dept of Electronics and Information Engineering, Huazhong Univ. of Sci. & Tech., Wuhan, China
∗
Dept of ECE, Texas T&M University, USA
Email: denghuiping.hust@gmail.com,hustlyu@mail.hust.edu.cn,zx@ece.tamu.edu
ABSTRACT
Preserving the edges in depth compression is important for
improving the synthesized view quality, this paper presents
a novel edge-preserving depth up-sampling method for
down/up sampling-based depth coding using both the tex-
ture and depth information. We take into account the edge
similarity between depth maps and their corresponding tex-
ture images as well as the structural similarity among depth
maps to build a weight model. Based on the weight model,
the optimal MMSE up-sampling coefficients are estimated
from the local covariance coefficients of the down-sampled
depth map. Experimental results show that our proposed
interpolation method for down/up sampling-based depth cod-
ing improves both the coding efficiency and synthesized view
quality.
Index Terms— 3D video, down/up sampling-based depth
coding, view synthesis, and edge-preserving interpolation.
1. INTRODUCTION
R&D in 3D video has been rapidly growing in recent years
in both the industry and academia. An attractive 3D video
representation is to utilize depth information of the scene (on
top of the 2D texture information). With the help of depth
maps, many interesting applications such as glasses-free 3D
video, free-viewpoint television (FTV), and gesture/motion-
based human computer interaction are becoming possible;
in addition, an arbitrary number of intermediate views can
be synthesized with low-cost depth image-based rendering
(DIBR) techniques, but the quality depends on the accuracy
of the depth maps. Consequently, efficient depth coding is
one of the key issues in 3D video systems.
Depth maps generally have more spatial redundancy than
natural images. This property can be exploited to compress a
down-sampled depth map at the encoder, followed by decom-
pression and up-sampling at the decoder. MPEG 3DV exper-
iments demonstrate that this down/up sampling-based depth
coding approach can improve the depth map coding efficiency
[1]. Since the quality of the synthesized views depends on the
Work supported by the NSFC grants 60972016 and 60903172 and the
China-Finnish cooperation project 2010DFB10570.
accuracy of the depth information, depth coding-induced dis-
tortion not only affects the depth quality but also the synthe-
sized view quality. Therefore, depth up-sampling method at
the decoder needs to be carefully designed to guarantee syn-
thesized view quality.
Classical techniques, such as pixel repetition, bilinear
or bi-cubic interpolation cause jagged boundaries, blurred
edges, and annoying artifacts around edges. In [2, 3], a
median up-sampling method is applied to down/up sampling-
based depth map coding. In [4], Oh et al. proposed a depth
boundary reconstruction filter to correct depth coding errors;
the filter is designed on the basis of occurrence frequency,
depth difference, as well as pixel position distances which
are all related to the depth map itself. Liu et al. [5] designed
a trilateral in-loop filter to reconstruct the depth map that
takes into account both the similarity among depth samples
and that among corresponding texture pixels. Wildeboer
et al. [6, 7] proposed a joint bilateral up-sampling algo-
rithm by utilizing the high-resolution texture video in the
process of depth up-sampling; they calculated a weight-cost
based on pixel positions and intensity similarities. Although
these depth map reconstruction methods achieve good per-
formances, they do not consider any special characteristics
of the depth maps. Since depth maps contain sharp intensity
changes at object boundaries, depth coding-induced distor-
tion in the synthesized view is most pronounced along object
boundaries. Therefore, depth map up-sampling algorithm at
the decoder needs to be carefully designed to preserve depth
edges.
Edge-directed interpolation techniques recover sharp
edges while suppressing pixel jaggedness and blurring ar-
tifacts by imposing accurate source models. Li and Orchard
[8] proposed a new edge-directed interpolation (NEDI) al-
gorithm for natural images, which exploits image geometric
regularity by using the covariance of a low resolution im-
age to estimate that of a high resolution image. Asuni and
Giachetti [9] improved the stability of NEDI by using edge
segmentation. Zhang et al. [10] estimated the low resolution
covariance adaptively with improved non-local edge-directed
interpolation. Since NEDI needs a relatively large window