逆深度参数化单目SLAM：实时3D地图构建详解

1星需积分: 20 34 浏览量更新于2024-07-20 1 收藏 1.26MB PDF 举报

本论文深入探讨了视觉单目SLAM（Visual Mono-SLAM）算法，该算法由Davison等人开发，用于实时从手持单眼相机捕捉的图像中创建稀疏且一致的3D地图。论文的作者Sven Albrecht在2009年撰写硕士论文时，对单目SLAM的工作原理进行了详细的剖析。首先，论文介绍了相机模型和镜头畸变的基础知识，这对于理解单目SLAM的实施至关重要。摄像机模型描述了像素如何映射到三维空间中的真实世界坐标，而镜头畸变校正是为了纠正由于光学元件变形导致的图像失真，确保测量的精度。接着，作者重点讲解了扩展卡尔曼滤波器（Extended Kalman Filter，EKF）的应用。EKF是一种在存在噪声和不确定性的情况下进行状态估计的有效工具，它不仅能够提供摄像机的6度自由度（6DOF）姿态估计，即位置和旋转，还能为特征点在3D坐标系下的位置估计提供可靠的方法。在单目SLAM中，一个关键的创新是引入了逆深度编码。传统的SLAM通常依赖于深度信息来估计场景的立体结构，但在单眼相机下，缺乏直接的深度测量使得初始特征点定位变得困难。通过逆深度编码，论文提出了一种方法，即使在深度未知的情况下，也能立即初始化特征点的位置，这对于处理远距离或模糊的图像特征尤其有用。论文展示了这种逆深度编码在实际应用中的有效性，即使面对深度较大的特征点，也能保持良好的性能。这表明该方法对于单目SLAM的鲁棒性和实用性具有重要意义，尤其是在资源受限的便携式设备上，逆深度编码能简化深度估计过程，提高实时性能。这篇论文深入研究了视觉单目SLAM的内在机制，并通过逆深度编码技术优化了特征点的初始化过程，为单眼相机在移动机器人、增强现实（AR）和虚拟现实（VR）等领域中的应用提供了有价值的研究成果。

10 CHAPTER 2. CAMERA

image

scene

image sensor

ﬁeld of view

Figure 2.4: Field of View. The boundaries of the ﬁeld of view, determined by the size of the image sensor are

depicted in blue. The resulting volume could be extended further in direction of the positive X

and is just cut

for visualization purposes. The corresponding angles, deﬁning the ﬁeld of view are labeled as α

and α

(see

equations (2.3) and (2.4)), respectively. Please note that the origin for the resulting image (its upper left corner) is

placed at O

image

which is inverted from its corresponding real world position O

scene

2.2 A Simple Distortion Model

Section 2.1.2 introduced a ﬁrst step to adapt the basic pinhole camera model to represent real cameras

more closely, by means of the ﬁeld of view and the adaption of the origin to reﬂect digital image sen-

sors. However the adaptions discussed so far still describe a “perfect” camera. Each real camera tends

to have its own unique imperfections, introduced by imperfect manufacturing and assembly processes.

Especially in the low-cost sector individual cameras of the same type tend to vary in their camera proper-

ties. One possible error induced by the manufacturing process is depicted in Figure 2.5, where the image

sensor is not aligned perfectly parallel to the camera lens.

Since the displacement of the image sensor is unique for each produced camera a way to analyze

the inherent distortions and to model them is needed. A ﬁrst step is to further modify equation (2.6).

Equation (2.6) implies that the center of the image sensor is placed precisely at the ideal image center

R which is the intersection with the Z

-axis and the image plane. Considering the CMOS sensor of

the HERCULES WebCam Classic with size 3.6 mm × 2.7 mm yields 640 ×480 pixels of resolution, the

physical size for the sensor of one pixel on the chip is 0.005625 mm × 0.005625 mm = 5.625 µm ×

5.625 µm. So any displacement larger than

5.625

µm would mean that (2.6) is not accurate, still assuming

that the image sensor is perfectly parallel to the pinhole plane. If however the displacement of the center

of the image sensor is known, the adaption of (2.6) becomes quite simple:

d,i

+ u

+ v

(2.7)

where u

and v

denote the image coordinate, which corresponds to the image center R. Similar to

AN ANALYSIS OF VISUAL MONO-SLAM

2.2. A SIMPLE DISTORTION MODEL 11

cheap camera lens

cheap glue

cheap CMOS chip

(a) Exemplary cheap camera schematics (b) Tangential distortion

Figure 2.5: Exemplary result of production inaccuracy. (a) Illustrates a common displacement of the CMOS

sensor, which should ideally be parallel the the camera lens. Such a displacement induces tangential distortion,

depicted in (b): For an ideal camera the black dots positioned in a grid should be projected, preserving the undis-

torted uniform grid structure. The CMOS sensor displacement induces the projection of a distorted grid, shown by

dotted black lines. Correspondences between the original black dots and their projection are shown in blue.

equation (2.6) u

d,i

and v

d,i

denote integer values bounded by 0 and width and height respectively, while

, v

, u

and v

are given as real values. The coordinates of point P

= (x

, y

, z

)

are also speciﬁed by

real values, of course. The pair (u

d,i

, v

d,i

)

are referred to as the distorted image coordinates (marked

by subscript

), since they are still subject to lens distortion although they are calculated according to the

actual image center (u

, v

)

. Distortions like the one depicted in Figure 2.5 are more complicated to

deal with and will be discussed together with radial distortions in the following paragraph.

Flaws and imperfections in a camera are not only limited to the correct placement of the image

sensor, but can (and usually do) also occur in the lens or lens system of a camera. A compact overview

on properties of optical lenses like aberration or vignetting is presented in [22]. A far more detailed

discussion of these topics can be found in [25,27].

However once a suitable distortion model is applied on top of the current camera model lens effects

like aberration and vignetting can be ignored in the VISUAL MONO-SLAM context and are therefore

not discussed any further. The interested reader is referred to [22,25,27] for more information on these

topics.

Closely related to a distortion model is the camera calibration process. The distortion model de-

scribes how to calculate the ideal undistorted image coordinates (u

u,i

, v

u,i

)

which would be perceived

by an ideal camera from a pair of actually perceived distorted image coordinates (u

d,i

, v

d,i

)

for a given

point P

= (x

, y

, z

)

. Camera calibration describes techniques to estimate the unique camera param-

eters, used in a distortion model for an individual camera. A historical overview of different calibration

techniques and their corresponding models is presented in [14]. The distortion model used by Davison

et al. in [13,16, 20] is a simpliﬁed version of the “Brown-Conrady-model” (also known as “plumb bob

AN ANALYSIS OF VISUAL MONO-SLAM

12 CHAPTER 2. CAMERA

model”) proposed by Brown in 1965 [8]. This model uses 5 distortion coefﬁcients to describe the im-

perfections found in a camera image. If all 5 coefﬁcients are known it is quite simple to compute the

corresponding undistorted image coordinates of an ideal camera for a given pair of distorted image coor-

dinates. The next paragraph will present the whole model ﬁrst and offer some explanations concerning

the different components of the model. Subsequently the model will be simpliﬁed to the model used

in [13,16,20]. In section 2.3 the techniques used to estimate the distortion coefﬁcients will be shown as

well as some exemplary results of camera calibration.

The model of Brown distinguishes between two different kinds of distortions, namely radial distor-

tion and tangential distortion. Radial distortions are caused by the shape of the used lens and can cause

pincushion or barrel distortions of the image, where straight lines will be projected in a curved fashion.

Usually the effects of radial distortion become stronger the larger the distance between a projected point

= (u

, v

)

and the image center R = (u

, v

)

becomes. Tangential distortion generally refers to

distortions due to imperfections in the centering of the camera lens. To model radial distortion 3 distor-

tion coefﬁcients are used (k

, k

) while tangential distortion is described by 2 coefﬁcients (p

, p

). If

the radial distortion coefﬁcients are known, the correction terms (ˆu

, ˆv

)

for radial distortion are deﬁned

as:

ˆu

ˆv

d,i

− u

d,i

− v



1 + k

+ k



(2.8)

where

r =



((u

d,i

− u

)





d,i

− v

)



(2.9)

denotes the distance of pixel (u

d,i

, v

d,i

)

from the image center (u

, v

)

on the image sensor. To recap

how to calculate (u

d,i

, v

d,i

)

please refer to equation (2.7).

To compensate for tangential distortion effects the following terms are proposed, incorporating tan-

gential distortion coefﬁcients p

and p

˜u

˜v







d,i

+ p

+ 2u

d,i

)





+ 2v

d,i

) + 2p

d,i







(2.10)

where r is deﬁned as in equation (2.9).

The complete model, considering radial (2.8) and tangential (2.10) distortion deﬁnes the undistorted

image coordinates (u

u,i

, v

u,i

)

u,i

ˆu

+ ˜u

ˆv

+ ˜v

(2.11)







d,i

−u



1 + k

+ k



+ 2p

d,i

+ p

+ 2u

d,i

)





d,i

−v



1 + k

+ k



+ p

+ 2v

d,i

) + 2p

d,i













1 + k

+ k



+ 2p

+ p



+ 2f





1 + k

+ k



+ p



+ 2f



+ 2p





AN ANALYSIS OF VISUAL MONO-SLAM

2.2. A SIMPLE DISTORTION MODEL 13

Luckily in most cases this full distortion model is not actually needed, but can be a bit simpliﬁed.

As mentioned earlier Davison et al. [13, 16, 20] propose a simpliﬁed version of the general model.

For standard ﬁeld of view cameras (i.e. non-wide angle or “ﬁsh-eye” cameras) it is not recommended or

necessary to push the radial component of the distortion beyond the 4th order. Therefore only cameras

providing a highly distorted image actually need distortion coefﬁcient k

, in other cases it is advised to

assume k

= 0. Currently most manufactured cameras are assembled with very little imperfection in

centering the image sensor, so that tangential distortion becomes less important. This corresponds with

the observations by Zhang [50] who states that the distortion function is clearly dominated by coefﬁcients

and k

. In other words for most low-cost cameras the assumption p

= p

= 0 is valid. That leaves

a distortion model with just 2 radial distortion coefﬁcients (namely k

and k

) which corresponds to the

distortion model used in [50]:

u,i

d,i

− u

)



1 + k

+ k



d,i

− v

)



1 + k

+ k



(2.12)

The distortion model used by Davison et al. in [13, 16, 20] is quite similar to the distortion model

of Zhang [50]. Davison et al. deﬁne an undistortion function h

that maps a pair of distorted image

coordinates (u

d,i

, v

d,i

)

to a pair of undistorted image coordinates (u

u,i

, v

u,i

)

as follows:

u,i

= h

d,i





d,i

− u

)



1 +



d,i

− v

)



1 +







(2.13)

with

d,i

− u

))

+ (d

d,i

− v

))

(2.14)

Apart from the distance of the distorted image coordinates (u

d,i

, v

d,i

)

to the image center deﬁned in

equation (2.14) the distortion model (2.13) used in [13,16,20] corresponds to the distortion model (2.12)

of Zhang [50]. For the deﬁnition of distance r in Zhangs’s model, please refer to equation (2.9). However

the difference between r and r

also means that the distortion coefﬁcients are generally not equal (i.e.

and k

). From camera calibration (see 2.3) usually coefﬁcients k

and k

are obtained,

but not

. Since (2.12) and (2.13) should yield the same mapping for a pair (u

d,i

, v

d,i

)

of distorted

image coordinates the assumptions

= k

⇔

= k

and

= k

⇔

= k

are valid.

AN ANALYSIS OF VISUAL MONO-SLAM

剩余98页未读，继续阅读

小西服红色

粉丝: 24
资源: 7

逆深度参数化单目SLAM：实时3D地图构建详解

visual-slam-roadmap：2021年成为Visual-SLAM开发人员的路线图

liberation-mono-fonts-1.07.2-16.el7.noarch.rpm

删除 iberation-mono-fonts-1:1.07.2-15.el7.noarch

liberation-mono-fonts = 1:1.07.2-16.el7 is needed by liberation-fonts-1:1.07

推荐10个以上比较好的SLAM深度学习模型github源码？

E/mono-rt:是什么

ubuntu18.04安装orb-slam3

orb-slam2在ros下运行

ubuntu18.04运行orb-slam2

ubuntu18.04运行orb-slam3

最新资源