高效准确的Semi-Global Matching: 基于Mutual Information的立体匹配方法

需积分: 10 6 浏览量更新于2024-09-12 收藏 1.85MB PDF 举报

本文主要探讨了精确立体匹配（Stereo Matching）的关键目标，特别是关注对象边界处的精度、对录制或光照变化的鲁棒性以及计算效率。作者提出了一个名为" Semi-Global Matching (SGM)"的方法，该方法专注于像素级别的匹配，利用互信息（Mutual Information）理论，并对全局平滑性的约束进行近似处理。这种方法的一个关键特性是能够检测和确定遮挡区域的精确深度，达到亚像素级别的准确度。 SGM方法的一大亮点是其设计的层次化互信息计算，这一创新使得匹配过程几乎与基于强度（Intensity-based）的匹配速度相当，显著提高了计算效率。这在处理大规模图像时尤为重要，因为通过分层处理，减少了不必要的计算负担，尤其是在处理复杂的场景中，如物体边缘和纹理丰富的区域。此外，文章还介绍了一个扩展，专门针对多基线立体图像。这种扩展进一步增强了SGM在处理不同视角之间深度信息的能力，从而增加了系统的通用性和适应性。这意味着它不仅可以在单视图立体匹配中表现出色，也能应对由多个摄像机角度提供的立体数据。另一个重要的贡献是提出的全局成本计算的近似方法，该方法的时间复杂度与像素数量和可能的深度差异成线性关系。这意味着随着图像尺寸的增大，该算法的运行时间保持相对较低，这对于实时或者对速度有严格要求的应用场景具有显著优势。这篇论文的核心内容是开发了一种高效且精确的立体匹配算法，它结合了互信息的统计特性与局部和全局信息的融合，有效地解决了对象边界识别、遮挡处理和计算效率提升等问题。由于其在实际应用中的广泛适用性和优越性能，SGM方法对于计算机视觉、机器人导航以及三维重建等领域具有重要意义。在当今技术日新月异的IT行业中，这样的研究成果无疑推动了相关领域的发展，尤其是在自动驾驶、无人机监控和虚拟现实等领域的实时三维重建中。

Accurate and Efﬁcient Stereo Processing by Semi-Global Matching and Mutual

Information

Heiko Hirschm¨uller

Institute of Robotics and Mechatronics Oberpfaffenhofen

German Aerospace Center (DLR)

P.O. Box 1116, 82230 Wessling, Germany

heiko.hirschmueller@dlr.de

Abstract

This paper considers the objectives of accurate stereo

matching, especially at object boundaries, robustness

against recording or illumination changes and efﬁciency of

the calculation. These objectives lead to the proposed Semi-

Global Matching method that performs pixelwise matching

based on Mutual Information and the approximation of a

global smoothness constraint. Occlusions are detected and

disparities determined with sub-pixel accuracy. Addition-

ally, an extension for multi-baseline stereo images is pre-

sented. There are two novel contributions. Firstly, a hierar-

chical calculation of Mutual Information based matching is

shown, which is almost as fast as intensity based matching.

Secondly, an approximation of a global cost calculation is

proposed that can be performed in a time that is linear to

the number of pixels and disparities. The implementation

requires just 1 second on typical images.

1. Introduction

Accurate, dense stereo matching is an important require-

ment for many applications, like 3D reconstruction. Most

difﬁcult are often the boundaries of objects and ﬁne struc-

tures, which can appear blurred. Additional practical prob-

lems originate from recording and illumination differences

or reﬂections, because matching is often directly based on

intensities that can have quite different values for corre-

sponding pixels. Furthermore, fast calculations are often

required, either because of real-time applications or because

of large images or many images that have to be processed

efﬁciently.

An application were all of the three objectives come to-

gether is the reconstruction of urban terrain, captured by an

airborne pushbroom camera. Accurate matching at object

boundaries is important for reconstructing structured envi-

ronments. Robustness against recording differences and il-

lumination changes is vital, because this often cannot be

controlled. Finally, efﬁcient (off-line) processing is neces-

sary, because the images and disparity ranges are huge (e.g.

several 100MPixel with 1000 pixel disparity range).

2. Related Literature

There is a wide range of dense stereo algorithms [8]

with different properties. Local methods, which are based

on correlation can have very efﬁcient implementations that

are suitable for real time applications [5]. However, these

methods assume constant disparities within a correlation

window, which is incorrect at discontinuities and leads to

blurred object boundaries. Certain techniques can reduce

this effect [8, 5], but it cannot be eliminated. Pixelwise

matching [1] avoids this problem, but requires other con-

straints for unambiguous matching (e.g. piecewise smooth-

ness). Dynamic Programming techniques can enforce these

constraints efﬁciently, but only within individual scanlines

[1, 11]. This typically leads to streaking effects. Global ap-

proaches like Graph Cuts [7, 2] and Belief Propagation [10]

enforce the matching constraints in two dimensions. Both

approaches are quite memory intensive and Graph Cuts is

rather slow. However, it has been shown [4] that Belief

Propagation can be implemented very efﬁciently.

The matching cost is commonly based on intensity dif-

ferences, which may be sampling insensitive [1]. Inten-

sity based matching is very sensitive to recording and il-

lumination differences, reﬂections, etc. Mutual Informa-

tion has been introduced in computer vision for matching

images with complex relationships of corresponding inten-

sities, possibly even images of different sensors [12]. Mu-

tual Information has already been used for correlation based

stereo matching [3] and Graph Cuts [6]. It has been shown

[6] that it is robust against many complex intensity transfor-

mations and even reﬂections.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, USA, June 20-26, 2005.

下载后可阅读完整内容，剩余7页未读，立即下载

hu3034212141

粉丝: 0
资源: 4

高效准确的Semi-Global Matching: 基于Mutual Information的立体匹配方法

"基于单片机的智能电子称设计及数据处理研究

"2022年徐州中能三原测控技术电子皮带秤说明书

ICL7106和ICL7107数字A/D转换器详解及特性介绍

Accurate and Efcient Stereo Processing

ALIKE: Accurate and Lightweight Keypoint Detection and Descripto

Live Accurate and Dense Reconstruction from a Handheld Camera

design of accurate and repeatable kinematics couplings.pdf

Kalign – an accurate and fast multiple sequence alignment algorithm

Accurate and efficient ground-to-aerial model alignment

Robust Methods for Accurate and Efficient 3D Modeling from Unstructured Imagery

最新资源