视频深度图恢复：bundle优化解决噪声与遮挡问题

4星 · 超过85%的资源需积分: 10 21 浏览量更新于2024-12-11 1 收藏 2.73MB PDF 举报

"《通过束优化恢复一致的视频深度图》该论文介绍了一种创新的视频深度图重建方法，主要关注于解决计算机视觉和三维视觉中的关键问题，如图像噪声和遮挡（occlusions）以及立体重建中的难题。作者Guofeng Zhang、Jiaya Jia、Tien-Tsin Wong和Hujun Bao来自浙江大学和香港中文大学的研究团队，他们的工作着重于利用深度信息的获取和整合，通过bundle optimization模型来提高视频深度图的质量。论文的核心内容包括使用颜色恒常性约束，这是一种常见的计算机视觉技术，它帮助确保不同帧的颜色一致性，即使在光照变化下也能保持物体的外观不变。然而，作者在此基础上进一步提出了一个几何一致性约束，这一约束将多个视频帧之间的几何关系考虑在内，能够自然地维护深度图的时间一致性。这种方法尤其在处理复杂遮挡场景，如交通标志的细杆、街灯以及道路的深度渐变时表现出色，能够准确地重建出高分辨率的深度图。具体来说，论文通过展示一个名为“Road”的视频序列的实例，展示了其深度重建成果。原始视频序列是由移动相机捕捉的，而经过作者方法处理后，生成的深度图不仅保留了细节，还准确地反映了物体间的深度关系，如车辆与道路的距离变化。这种方法对于增强虚拟现实、增强现实（AR）以及3D建模等领域具有重要意义，因为它提供了高质量的深度信息，有助于创建更真实、沉浸式的体验。这篇论文为深度图恢复提供了一种有效的策略，通过结合色彩和几何约束，并利用视频帧间的束优化技术，成功地解决了深度图重建中的一些关键挑战，为相关领域的研究者和实践者提供了有价值的参考。"

Recovering Consistent Video Depth Maps via Bundle Optimization

Guofeng Zhang

Jiaya Jia

Tien-Tsin Wong

Hujun Bao

State Key Lab of CAD&CG, Zhejiang University

The Chinese University of Hong Kong

{zhangguofeng, bao}@cad.zju.edu.cn {leojia, ttwong}@cse.cuhk.edu.hk

input sequence output video depth maps

Figure 1. High-quality depth reconstruction from the video sequence “Road” containing complex occlusions. Left: An input video sequence

taken by a moving camera. Right: Video depth maps automatically computed by our method. The thin posts of the trafﬁc sign and street

lamp, as well as the road with graduate depth change, are accurately constructed in the recovered depth maps.

Abstract

This paper presents a novel method for reconstruct-

ing high-quality video depth maps. A bundle optimization

model is proposed to address the key issues, including im-

age noise and occlusions, in stereo reconstruction. Our

method not only uses the color constancy constraint, but

also explicitly incorporates the geometric coherence con-

straint associating multiple frames in a video, thus can nat-

urally maintain the temporal coherence of the recovered

video depths without introducing over-smoothing artifact.

To make the inference problem tractable, we introduce an

iterative optimization scheme by ﬁrst initializing disparity

maps using segmentation prior and then reﬁning the dis-

parities by means of bundle optimization. Unlike previ-

ous work estimating complex visibility parameters, our ap-

proach implicitly models the probabilistic visibility in a sta-

tistical way. The effectiveness of our automatic method is

demonstrated using challenging video examples.

1. Introduction

Stereo reconstruction of dense depths from real images

has long been a fundamental problem in computer vision.

The reconstructed depths can be used by a wide spectrum

of applications including 3D modeling, robot navigation,

image-based rendering, and video editing. Although stereo

problem [14, 8, 15, 23] has been extensively studied during

the past decades, obtaining high-quality dense depth data is

still a challenging problem due to many inherent difﬁculties,

such as image noise, textureless pixels, and occlusions.

Given an input video sequence taken by a freely moving

camera, we propose a novel method to automatically con-

struct high-quality and consistent depth maps for all frames.

Our main contribution is the development of a global opti-

mization model based on multiple frames, which we called

bundle optimization, to resolve most of the aforementioned

difﬁculties in disparity estimation.

Our method does not explicitly model the binary visi-

bility (occlusion). Instead, the visibility is encoded nat-

urally in the energy deﬁnition. Our model also does not

distinguish among image noise, occlusions and estimation

errors, so as to achieve a uniﬁed framework in modeling

matching ambiguities. The color constancy constraint and

geometric coherence constraint linking different views are

combined in an energy minimization framework, reliably

reducing the inﬂuence of image noise and occlusions in a

statistical way. This process makes our optimization not

produce over-smoothing or blending artifact.

In order to deal with the disparity estimation in texture-

less region and alleviate the problem of segmentation es-

pecially on ﬁne object structures, we only use the image

segmentation prior in the disparity initialization. Then our

iterative optimization algorithm reﬁnes the segmented dis-

parities in a pixel-wise manner. Experiments show that this

is rather effective in estimating correct disparities in texture-

less regions while faithfully preserving the ﬁne structures of

object silhouettes.

Our method is very robust against occlusions, matching

ambiguities, and noise. We have conducted experiments on

a variety of challenging examples. Automatically computed

depth maps contain very little noise. Clear object silhou-

Authorized licensed use limited to: Shenzhen Institute of Advanced Technology. Downloaded on August 12, 2009 at 23:05 from IEEE Xplore. Restrictions apply.

下载后可阅读完整内容，剩余7页未读，立即下载

jajen

粉丝: 16
资源: 6

视频深度图恢复：bundle优化解决噪声与遮挡问题

Consistent-Video-Depth-Estimation

consistent_depth:我们从单眼视频（例如手持手机视频）中估计密集，无闪烁，几何上一致的深度

基于springboot的协同过滤算法商品推荐系统源代码（java+vue+mysql+说明文档）.zip

基于springboot的人事档案管理系统的设计与实现源代码（完整前后端+mysql+说明文档）.zip

易妆.rp

onvif device manager（ODM）工具 Last Update-version：2024-01-11

新建文本文档.html

基于springboot的Java Move体育商城源代码（完整前后端+mysql+说明文档）.zip

计算机网络信息安全与其防护对策.doc

基于springboot的Java Offer资讯交流Web系统源代码（完整前后端+mysql+说明文档）.zip

最新资源