分段差异细化：一种立体匹配遮挡处理方法

需积分: 16 185 浏览量更新于2024-09-07 收藏 29.38MB PDF 举报

"基于分段的用于立体匹配的遮挡处理的差异细化" 本文"Segment-based Disparity Refinement with Occlusion Handling for Stereo Matching"探讨了一种针对立体匹配的遮挡处理的差异细化方法，旨在改进传统的winner-take-all (WTA) 立体匹配结果。在立体匹配中，两个对应图像的对应像素之间的距离（即视差）被用来构建三维场景的深度信息。然而，遮挡、噪声和不精确的边缘检测可能导致匹配错误，特别是对于复杂的图像场景。首先，该论文指出，原始的WTA方法可能会因遮挡问题导致错误的视差估计。为了解决这个问题，文章提出通过超像素（superpixels）对参考图像进行过度分割。超像素是将图像中的像素组合成更大、更连贯的区域，这样可以更好地捕获图像的局部特性。然后，对于每个超像素，使用一种改进的随机样本共识（RANSAC）算法来拟合一个最佳的视差平面。RANSAC是一种常用的鲁棒模型估计方法，它可以容忍一定的异常值，从而提高模型拟合的质量。接着，文章设计了一个两层优化框架来精细化这些视差平面。第一层优化主要关注于减少单个超像素内部的视差不一致性，通过对超像素内的像素进行局部调整来提高匹配精度。第二层优化则考虑了相邻超像素间的连续性，利用马尔可夫随机场（Markov Random Field, MRF）模型来确保整个图像的视差场是平滑且一致的。MRF模型能够有效地捕捉图像的局部和全局上下文信息，从而减少因遮挡导致的视差不连续性。此外，该方法还考虑了遮挡处理，通过分析像素之间的依赖关系来识别和处理遮挡区域。在遮挡区域，由于缺少对应的像素，简单的WTA方法可能无法准确地估计视差。该方法利用遮挡信息来修正这些区域的视差，从而提高匹配的准确性。总结来说，这篇文章提出的差异细化方法通过结合超像素分割、改进的RANSAC视差平面拟合、以及MRF驱动的多层优化，有效地处理了立体匹配中的遮挡问题，提升了匹配结果的质量和稳定性。这一技术对自动驾驶、机器人导航、3D重建等领域具有重要的应用价值。

1057-7149 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIP.2019.2903318, IEEE

Transactions on Image Processing

IEEE TRANSACTIONS ON , VOL. **, NO. *, JANUARY 2019 3

III. OVERVIEW OF THE TWO LAYER

OPTIMIZATION

The matching cost volume C

(x) is generated by MC-

CNN

[6]. The disparity map d(x) is computed by winner-

take-all:

d(x) = arg min

∗

(x) (1)

The left reference image is segmented into superpixels {s

}

by the graph-based segmentation [37]. The workﬂow of our

method is shown in Fig. 1.

We propose a two-layer optimization to reﬁne the WTA

disparity map. In the global optimization layer (Section IV), a

front-parallel disparity map is estimated by MRF optimization.

The 3D neighborhood system N

is derived from superpixels

mean disparities {µ

}. In the local optimization layer (Section

V), slanted planes {π

} are ﬁtted for superpixels by RANSAC

and mean disparities of superpixels {µ

} are utilized to con-

straint the ﬁtting. The initial slanted disparity map is reﬁned

by a probabilistic model that exploits Bayesian inference and

Bayesian prediction in the 3D neighborhood system. Both

optimization layers operate at superpixel level and have high

efﬁciency.

IV. FRONT-PARALLEL DISPARITY MAP

We use the global MRF optimization to estimate a front-

parallel disparity map. Superpixels are formulated as graph

nodes. MRF optimization aims to minimize the following

energy:

E(µ) =

s∈Ω

(µ

) + λ

(s,t)∈N

(µ

, µ

), (2)

where µ

is the label, in our case it is the mean disparity

of superpixel s; Ω is the set of superpixels, Ω = {s

}, and

N represent the set of neighboring superpixels; and φ

(µ

)

is called the data term, ψ

(µ

, µ

) is called the smoothness

term and λ is a parameter to balance the inﬂuence of the

smoothness term. In contrast to 3D label MRF, optimizing 1D

label on superpixel level is efﬁcient (Section IV-B).

We propose a novel data term which is based on dispar-

ity distribution (Section IV-A) instead of matching cost or

similarity measure between left and right images. To handle

the foreground-background occlusions, the 3D neighborhood

system which represents depth discontinuities is derived by

{µ

} (Section IV-C). We also study a special case and prove

that the 1D label MRF formulation cannot model the highly

slanted surfaces (Section IV-D).

A. Disparity Distribution Interpretation

Segment-based stereo methods assume that disparities are

approximately linear within a segmentation. With the piece-

wise planar surfaces assumption, the disparity distribution

of a planar surface with appropriate boundaries shall be

evenly distributed. Considering the irregular boundary shape

Downloaded from https://github.com/t-taniai/LocalExpStereo

of superpixels, we model the disparity distribution within a

superpixel s a normal distribution

Norm

(µ

, σ

) =

√

2πσ

exp(−

(d − µ

)

2σ

), (3)

where d represents the disparity, µ

and σ

are disparity mean

and variance of superpixel s, respectively. Higher σ

indicates

a more slanted surface while for a front-parallel surface, σ

is approximately equal to zero. The data term of (2) is based

on disparity distribution histograms, as described in Section

IV-B.

B. MRF Optimization

To estimate a front-parallel disparity map, we estimate

mean disparities of superpixels. The front-parallel plane of

superpixel s can be obtained by π

= (0, 0, µ

). The data

term and smoothness term of (2) are deﬁned as follows:

1) Data Term: To measure the conﬁdence of disparity

centers, the disparity distributions of superpixels are divided

into histogram bins. We count the number that the WTA

disparity d

(x) in superpixel s falls into a bin B(µ

) with

bin-width L. The data term of s is deﬁned as

(µ

) = N

−

i=1

I(d

) ∈ B(µ

)), (4)

where N

is the number of pixels in superpixel s, µ

takes

discrete values, µ

= 0, L, 2L, ···, and lower data term

implies higher conﬁdence due to the negative sign. I is a

function of condition, deﬁned as

I(·) =

(

1, if · is true

0, if · is false

, (5)

and in (4) I indicates whether the disparity d

) falls into

bin B(µ

), i.e. d

) ∈ [µ

, µ

+ L).

The design of data term is voting-based. More observations

falling in the same bin results in a higher conﬁdence. The

WTA disparities in occluded regions are noise-corrupted and

it is hard for them to reach a consensus. Therefore, the data

term in occluded regions is relatively high and the label is

dominated by the smoothness term.

2) Smoothness Term: The smoothness term enforces the

similarity of disparity distribution centers among neighboring

superpixels, which is deﬁned as

(µ

, µ

) = max(ω

, )L(s, t)T (µ

, µ

), (6)

where ω

is a color-similarity weight which is deﬁned as

= e

−kI(s)−I(t)k

/γ

, (7)

where γ is a parameter that controls the inﬂuence of color

weight, and I(s) denotes the average color of superpixel s; 

is a lower-bound truncated value [29]; L(s, t) [38] is the shared

boundary length between neighboring superpixels s and t; and

T could be a metric or a semi-metric which will be deﬁned

in Section IV-C.

剩余11页未读，继续阅读

8BitCat

粉丝: 65
资源: 18

分段差异细化：一种立体匹配遮挡处理方法

基于分割的立体匹配及算法-Segment_Based_Stereo_Matching.part1.rar

基于分割的立体匹配及算法-Segment_Based_Stereo_Matching.part2.rar

有效的基于能量的多视图分段平面立体

基于区域的立体匹配算法介绍

基于多目立体匹配的深度获取方法

基于python的立体匹配基础算法SSD、SAD、ZNCC、BM、SGBM实现

论文研究-基于SLM的显微立体遮挡校正 .pdf

基于SUFT的双目立体匹配系统

基于双目视觉的立体匹配算法研究

基于图像分割的立体匹配论文合集

最新资源