3D视频虚拟视图PSNR估计算法

需积分: 5 4 浏览量更新于2024-08-11 收藏 5.05MB PDF 举报

"该文提出了一种针对3D视频的虚拟视图PSNR（峰值信噪比）估计方法，旨在降低3D视频系统中的复杂性，无需实际渲染虚拟视图即可估算其失真或PSNR。文章深入探讨了3D视频的虚拟视图合成过程及其对质量差异的影响，并为评估3D视频编码系统的性能提供了理论基础。" 在3D视频技术中，通过多视角纹理视频和深度映射可以合成虚拟视图。这种技术为用户提供了一个更加沉浸式的观看体验。然而，评估3D视频系统性能或进行率-失真优化过程中，需要计算虚拟视图的失真或PSNR，这通常涉及将压缩3D视频与未压缩3D视频合成的虚拟视图进行比较，从而增加了系统的计算复杂度。本文作者Hui Yuan等人提出了一个创新的方法，直接估计虚拟视图的失真和PSNR，以减少3D视频系统的复杂性。这种方法的关键在于理解虚拟视图的合成过程以及失真如何从现有视图传播到虚拟视图。通过分析这个过程，他们能够开发出一个模型，无需实际生成虚拟视图，就能预测其质量指标。 PSNR是衡量视频质量的重要标准，它表示信号与噪声的比例。在3D视频中，较高的PSNR意味着更好的图像质量，更少的压缩失真。因此，准确估计虚拟视图的PSNR对于优化编码策略至关重要。该文的研究成果不仅有助于简化3D视频处理流程，还能为实时或近实时的3D视频应用提供高效的质量评估工具。文章中，作者可能探讨了以下几点内容： 1. 虚拟视图的合成算法：包括基于深度映射的视图合成技术，以及压缩与未压缩数据间的差异。 2. 失真传播模型：分析相邻视图之间的失真如何影响虚拟视图的质量。 3. PSNR估计模型：构建数学模型来预测虚拟视图的PSNR，而不实际执行合成步骤。 4. 实验验证：通过对比实验，展示所提方法的有效性和准确性，可能包括与其他方法的性能比较。这项工作对于3D视频编码、传输和播放技术的发展具有重要意义，因为它提供了一个更轻量级的解决方案，以支持高效能的3D视频系统设计。此外，这种方法也适用于其他依赖于视图合成和质量评估的多媒体应用。

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

IEEE TRANSACTIONS ON BROADCASTING 1

A Virtual View PSNR Estimation Method for 3-D Videos

Hui Yuan, Member, IEEE, Sam Kwong, Fellow, IEEE, Xu Wang, Student Member, IEEE,

Yun Zhang,

Member, IEEE, and Fengrong Li

Abstract—In three-dimensional videos (3-DVs) with n-view texture

videos plus n-view depth maps, virtual views can be synthesized from

neighboring texture videos and the associated depth maps. To evaluate

the system performance or guide the rate-distortion-optimization pro-

cess of 3-DV coding, the distortion/PSNR of the virtual view should

be calculated by measuring the quality difference between the virtual

view synthesized by compressed 3-DVs with one synthesized by uncom-

pressed 3-DVs, which increases the complexity of a 3-DV system. In

order to reduce the complexity of 3-DV system, it is better to esti-

mate virtual view distortions/PSNR directly without rendering virtual

views. In this paper, the virtual view synthesis procedure and the dis-

tortion propagation from existing views to virtual views are analyzed

in detail, and then a virtual view distortion/PSNR estimation method

is derived. Experimental results demonstrate that the proposed method

could estimate PSNRs of virtual views accurately. The squared correla-

tion coefﬁcient and root of mean squared error between the estimated

PSNRs by the proposed method and the actual PSNRs are 0.998 and

2.012 on average for all the tested sequences. Since the proposed method

is implemented row-by-row independently, it is also friendly for parallel

design. The execute time for each row of pictures with 1024×768 reso-

lution is only 0.079 s, while for pictures with 1920×1088 resolution it is

only 0.155 s.

Index Terms—

Distortion estimation, 3DV, video coding.

I. I

NTRODUCTION

ITH the improvements in high-speed networking, high-

capacity storage, and high-quality auto-stereoscopic display

technologies, extensive commercial applications of three-dimensional

Manuscript received June 18, 2015; revised September 30, 2015; accepted

October 13, 2015. This work was supported in part by the National Natural

Science Foundation of China under Grants 61571274, 61201211, 61471348,

and 61501299; in part by the Young Scholars Program of Shandong University

(YSPSDU) under Grant 2015WLJH39 in part by the Ph.D. Programs

Foundation, Ministry of Education of China under Grant 20120131120032;

in part by the Key laboratory of wireless sensor network and communica-

tion, Chinese Academy of Sciences under Grant 2013002; in part by the

Shenzhen Emerging Industries of Strategic Basic Research Project under

Grant JCYJ20150525092941043, in part by the City University of Hong Kong

Applied Research Grant 9667094; and in part by the City University of Hong

Kong Shenzhen Research Institute, Shenzhen, China.

H. Yuan is with the School of Information Science and

Engineering, Shandong University, Jinan 250100, China (e-mail:

yuanhui0325@gmail.com).

S. Kwong is with the Department of Computer Science, City University

of Hong Kong, Hong Kong, and also with the City University of Hong

Kong Shenzhen Research Institute, Shenzhen 5180057, China (e-mail:

cssamk@cityu.edu.hk).

X. Wang is with the College of Computer Science and Software

Engineering, Shenzhen University, Shenzhen 518060, China (e-mail:

wangxu@szu.edu.cn).

Y. Zhang is with the Shenzhen Institutes of Advanced Technology,

Chinese Academy of Sciences, Shenzhen 518055, China (e-mail:

yun.zhang@siat.ac.cn).

F. Li is with the Key Laboratory of Wireless Sensor Network

and Communication, Shanghai Institute of Microsystem and Information

Technology, Chinese Academy of Sciences, Shanghai 200050, China (e-mail:

lifengrongsim@mail.sim.ac.cn).

Color versions of one or more of the ﬁgures in this paper are available

online at http://ieeexplore.ieee.org.

Digital Object Identiﬁer 10.1109/TBC.2015.2492461

video (3DV) are becoming reality, e.g., the well-known 3D tele-

vision (3DTV) [1] and free viewpoint television (FTV) [2]. In

a typical 3DV system, which includes capture, storage, transmission,

and display, a 3D scene should ﬁrst be represented efﬁciently by

using a small amount of data [3]. Among 3D scene representation

technologies [3], representations made by n-view texture videos plus

n-view depth maps have been used extensively. For this kind of scene

representation, virtual views should be rendered from the acquired

n-view videos and their corresponding n-view depth maps by depth

image-based rendering (DIBR) [4].

In order to obtain the distortion or quality of the virtual view for

high-efﬁciency 3DV coding (3DVC) and 3DV quality assessment,

the distortion or PSNR of the virtual view can be calculated by com-

paring a virtual view synthesized by compressed 3DVs and the one

synthesized by uncompressed 3DVs, which increases the complexity

of a 3DV system. A more economical way is to estimate the value

of the virtual view’s distortion/PSNR directly. In [5], Zhang et al.

proposed a regional based virtual view distortion estimation method

for depth maps coding. In [6], a linear model based virtual view

distortion estimation method was proposed for depth maps cod-

ing. In our previous work [7], a planar model based virtual view

distortion estimation method was proposed for joint bit allocation

between texture videos and depth maps. Besides, similar distortion

models and applications can also be found in [8]–[10]. The existing

methods [5]–[10] can estimate the distortion variation tendency to

some extent, but the estimated virtual view distortion may not be close

to the actual distortion. In order to estimate the virtual view distor-

tion accurately, a synthesis distortion estimation method is proposed

in [11]. In this method, the effect of depth map distortion on synthesis

distortion is broken down into 2 parts, spatial variant region and spa-

tial invariant region, based on frequency domain analysis. However,

the model cannot be used easily due to its high computational

complexity.

From the basis of DIBR technology, it can be concluded that

the distortion of the virtual view depends only on the distortion

of the left and the right texture views and depth maps when the

camera systems are calibrated well [7]. Since the DIBR technol-

ogy is mathematically analytical, the distortion of virtual view can

also be mathematical derived from the distortion of the left and

the right texture views and depth maps. Motivated by this point,

a fast and accurate virtual view distortion/PSNR estimation method

is proposed by detailed analyzing of the virtual view synthesis

procedure.

To estimate the distortion/PSNR for virtual view accurately

with low complexity, DIBR procedure and distortion propagation

from existing views to virtual views are analyzed in detail in

Section II. During the analysis, it is necessary to mention that

the DIBR procedure is equal to disparity compensation when all

the cameras are well calibrated [7], thus a depth coding error

can only affect the horizontal position of the projected pixels

in the virtual view. In addition, for clear representation, a sum-

mary of some frequently used notations are given in Table I.

Experimental results and conclusions are given in Section III and IV

respectively.

0018-9316

 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.

See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

下载后可阅读完整内容，剩余6页未读，立即下载

weixin_38620893

粉丝: 4
资源: 888

3D视频虚拟视图PSNR估计算法

一种用于3-D视频的虚拟视图PSNR估计方法

网络游戏-基于3D卷积神经网络的虚拟现实视频质量评价方法.zip

一种用于3D-HEVC深度图编码的有效深度建模模式决策算法

高效的深度图压缩，可在3D视频中进行视图渲染

对3D Warping分解的高质量虚拟视点绘制方法

基于深度的3D视频质量评价

MeshStereo：具有网格对齐正则化的全局立体模型，用于视图插值

3D高效深度编码的渲染失真估计模型

利用边缘像素深度特性的深度估计方法 (2011年)

网格剖切功能强大的视图插值

最新资源