Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual
Information
Heiko Hirschm¨uller
Institute of Robotics and Mechatronics Oberpfaffenhofen
German Aerospace Center (DLR)
P.O. Box 1116, 82230 Wessling, Germany
heiko.hirschmueller@dlr.de
Abstract
This paper considers the objectives of accurate stereo
matching, especially at object boundaries, robustness
against recording or illumination changes and efficiency of
the calculation. These objectives lead to the proposed Semi-
Global Matching method that performs pixelwise matching
based on Mutual Information and the approximation of a
global smoothness constraint. Occlusions are detected and
disparities determined with sub-pixel accuracy. Addition-
ally, an extension for multi-baseline stereo images is pre-
sented. There are two novel contributions. Firstly, a hierar-
chical calculation of Mutual Information based matching is
shown, which is almost as fast as intensity based matching.
Secondly, an approximation of a global cost calculation is
proposed that can be performed in a time that is linear to
the number of pixels and disparities. The implementation
requires just 1 second on typical images.
1. Introduction
Accurate, dense stereo matching is an important require-
ment for many applications, like 3D reconstruction. Most
difficult are often the boundaries of objects and fine struc-
tures, which can appear blurred. Additional practical prob-
lems originate from recording and illumination differences
or reflections, because matching is often directly based on
intensities that can have quite different values for corre-
sponding pixels. Furthermore, fast calculations are often
required, either because of real-time applications or because
of large images or many images that have to be processed
efficiently.
An application were all of the three objectives come to-
gether is the reconstruction of urban terrain, captured by an
airborne pushbroom camera. Accurate matching at object
boundaries is important for reconstructing structured envi-
ronments. Robustness against recording differences and il-
lumination changes is vital, because this often cannot be
controlled. Finally, efficient (off-line) processing is neces-
sary, because the images and disparity ranges are huge (e.g.
several 100MPixel with 1000 pixel disparity range).
2. Related Literature
There is a wide range of dense stereo algorithms [8]
with different properties. Local methods, which are based
on correlation can have very efficient implementations that
are suitable for real time applications [5]. However, these
methods assume constant disparities within a correlation
window, which is incorrect at discontinuities and leads to
blurred object boundaries. Certain techniques can reduce
this effect [8, 5], but it cannot be eliminated. Pixelwise
matching [1] avoids this problem, but requires other con-
straints for unambiguous matching (e.g. piecewise smooth-
ness). Dynamic Programming techniques can enforce these
constraints efficiently, but only within individual scanlines
[1, 11]. This typically leads to streaking effects. Global ap-
proaches like Graph Cuts [7, 2] and Belief Propagation [10]
enforce the matching constraints in two dimensions. Both
approaches are quite memory intensive and Graph Cuts is
rather slow. However, it has been shown [4] that Belief
Propagation can be implemented very efficiently.
The matching cost is commonly based on intensity dif-
ferences, which may be sampling insensitive [1]. Inten-
sity based matching is very sensitive to recording and il-
lumination differences, reflections, etc. Mutual Informa-
tion has been introduced in computer vision for matching
images with complex relationships of corresponding inten-
sities, possibly even images of different sensors [12]. Mu-
tual Information has already been used for correlation based
stereo matching [3] and Graph Cuts [6]. It has been shown
[6] that it is robust against many complex intensity transfor-
mations and even reflections.
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, USA, June 20-26, 2005.
下载后可阅读完整内容，剩余7页未读，立即下载
hu3034212141
- 粉丝: 0
- 资源: 4
我的内容管理
展开
最新资源
- 李兴华Java基础教程：从入门到精通
- U盘与硬盘启动安装教程：从菜鸟到专家
- C++面试宝典：动态内存管理与继承解析
- C++ STL源码深度解析：专家级剖析与关键技术
- C/C++调用DOS命令实战指南
- 神经网络补偿的多传感器航迹融合技术
- GIS中的大地坐标系与椭球体解析
- 海思Hi3515 H.264编解码处理器用户手册
- Oracle基础练习题与解答
- 谷歌地球3D建筑筛选新流程详解
- CFO与CIO携手：数据管理与企业增值的战略
- Eclipse IDE基础教程：从入门到精通
- Shell脚本专家宝典：全面学习与资源指南
- Tomcat安装指南：附带JDK配置步骤
- NA3003A电子水准仪数据格式解析与转换研究
- 自动化专业英语词汇精华：必备术语集锦
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
