Enhanced Multi-view Prediction and Bit Allocation
1
Da Liu,
1
Li Wang,
1
Yuncai Hao
1
Beijing Institute of Control Engineering
Beijing, China
lovda@outlook.com
2
Fang Yin,
1
Chunyan Li,
1
Jun Zhang
2
Beihang University
Beijing, China
Abstract
—
In this paper, firstly an extended DFMC
Structure is proposed, then HQF jump period in extended DFMC
is presented. Considering temporal-view and interview prediction
structure, HQF location is determined. From the HQF, an
enhance LQF is proposed. Then considering the HQF and
enhance LQF, improved interview prediction is proposed.
Finally bit allocation in the proposed multi-view is proposed.
Experimental results show that the proposed method can achieve
better performance than the previous schemes.
Keywords-dual frame; motion compensation; multiview; bit
allocation
I. INTRODUCTION
i N
f
LQ
i N
f
LQ
i N
f
i
i
i
i
LQ
i N
f
Figure 1. Dual frame motion compensation.
In multi-view video coding, several cameras simultaneously
capture the video streams, thus multiple camera views of the
same scene is created. This approach needs large amounts of
data to transmit the video stream, so efficient compression
techniques are essential for this kind of application. In
temporal view, the frames relevance in a short period of time
is high, the coding efficiency can be further improved. In
inter-view, all cameras capture the same scene from different
viewpoints. A large amount of inter-view statistical
dependencies is existed. Improved temporal/inter-view motion
compensation can greatly reduce prediction error, thus coding
efficiency is improved.
In a single view, a kind of motion compensation method,
jump update dual frame motion compensation (JU-DFMC),
has been reported recently [1]-[3]. In JU-DFMC, two kinds of
frames are existed from the perspective of bit allocation. One
has relatively higher quality and is called high quality frame
(HQF), such as the i-1th frame
i
and the i-N-1th frame
HQ
i N
f
in Fig. 1; the other has relatively lower quality and is
called low quality frame (LQF), such as the i+kth frame (k=-
N, -N+1, -N+2, -N+3, …, -2, 0, 1). For each current frame, two
reference buffers are utilized for motion compensation, as
shown in Fig. 1, the first reference buffer contains the most
recently decoded frame, which is called short-term reference
frame (STR), and the second one contains a HQF from the
past, which is named as long-term reference frame (LTR).
A number of DFMC related work have been done. In [4],
bits were unevenly allocated among frames periodically, the
frames with higher bits utilized as the long-term reference
frame for the other frames. In the further work [1], the update
period of the LTR was set to ten frames. The PSNR of nine
frames that follow the LTR frame was utilized to determine
the bit allocation in the LTR. In [2], LTR was selected with
simulated annealing, the overall visual quality is better with
the cost that the computational complexity is relatively high.
At the same time, some DFMC related error resilient work has
also been done. In [5] and [6], the recursive optimal per-pixel
estimate (ROPE) algorithm was utilized in DFMC, and the
error propagation is restrained. In [7], a binary decision tree
was utilized in decoder to choose reference frame, the LTR
and STR are adaptively chose for error concealment. The
results show that it can bring better performance than just
using the short-term median MV block.
In recent years, some temporal/inter-view motion
compensation schemes have been reported. In [8], the problem
of coding N multiview video sequence was theoretically
studied. The impact of both inaccurate disparity compensation
and temporal GOP size K on the overall rate-distortion
efficiency was discussed. In [9], a geometric prediction
methodology for accurate disparity vector prediction was
proposed to reduce the disparity compensation cost. In [10], an
inter-view direct mode was proposed to signal the decoder that
the motion of a macroblock can be achieved from the coded
view without any coding bits. In [11], the statistical
dependencies from both temporal and inter-view reference
pictures for motion-compensated prediction are exploited. In
[12], the main view of the multiview video are encoded using
an MPEG-4 encoder and the auxiliary views are encoded by
joint disparity and motion compensation. In [13], view
temporal prediction structures that are adjusted to various
characteristics of general multi-view video were proposed.
In the paper, a multi-view coding structure with extend
DFMC is proposed. Firstly, the extend DFMC is proposed.
Secondly, HQF jump period in temporal view and further
adjustment for inter-view prediction are proposed. Thirdly, bit
allocation in the proposed multi-view structure is determined.
The rest of the paper is organized as follows. In Section II,