经典3D重建方法综述：多图像立体与结构运动

5星 · 超过95%的资源需积分: 50 26 浏览量更新于2024-07-26 2 收藏 5.96MB PDF 举报

本文档深入探讨了"3D Reconstruction from Multiple Images"这一主题，是关于三维重建的经典论文，总计180多页，对于该领域的初学者和研究者来说具有极高的参考价值。文章首先从3D获取的基本概念入手，对三维重建的各种方法进行了详细的分类概述。 1.1 部分介绍了三维重建方法的分类，包括但不限于： - (Passive) Stereo：被动立体视觉技术，通过两张或多张图像中的对应点来计算物体的深度信息，这是早期立体匹配的基础。 - Structure-from-Motion (SfM)：利用连续拍摄的图像序列，通过分析相机运动来推断场景的三维结构，常用于无人机航拍和视频监控中的三维重建。 - Active Triangulation：主动方式的三角测量，如激光扫描或时间飞行法（Time-of-Flight），通过直接测量距离获得三维点云。 - 其他方法还包括： - Shape-from-Shading and Photometric Stereo：基于表面光照变化和阴影信息推断形状，结合光谱信息进行三维重建。 - Shape-from-Texture and Shape-from-Contour：利用纹理和边缘信息恢复物体形状。 - Shape-from-Defocus：利用景深信息，如模糊边缘，进行形状恢复。 - Shape-from-Silhouettes：仅依赖于物体边缘轮廓进行形状估计。 - 还有Hybrid Techniques，即结合多种方法的混合策略，提高重建精度。 1.5 部分讨论了三维重建面临的挑战，包括但不限于：光照变化、遮挡、噪声、传感器限制以及计算复杂性等。第二部分详述了图像形成和相机模型，包括： - 2.2 部分重点讲解线性和非线性相机模型，如pinhole camera模型、投影方程以及与相机中心坐标系相关的矩阵表达。 - 2.3 Camera Calibration 涵盖内部校准（调整镜头参数）和外部校准（确定相机位置和姿态），这是保证三维重建准确性的关键步骤。第三部分探讨了三维重建的基本原理，这部分内容深入剖析了如何将理论知识转化为实际应用，涉及影像处理、特征匹配、优化算法等关键技术。这篇论文提供了一个全面的框架，涵盖了从三维重建的基本概念、方法到相机模型和校准的详细理论，以及实践中可能遇到的挑战和解决方案。对于希望在三维重建领域深入学习和研究的读者来说，这是一份不可多得的参考资料。

16 CHAPTER 1. INTRODUCTION TO 3D ACQUISITION

(a) (b)

Figure 1.9: (a) mini-dome with diﬀerent LED light sources, (b) scene with one of the LEDs

activated, (c) 3D reconstruction of cuneiform tablet, without texture, and (d) same tablet textured

using an Image-Based Rendering technique.

1.4. OTHER METHODS 17

Figure 1.10: Left: the regular texture yields a clear perception of a curved surface. Right: the

result of a shape-from-texture algorithm.

Observing an ellipse, the assumption can be made that it actually is a circle, and the slant and

tilt angles of the plane can be determined. For instance, in the shape-from-texture ﬁgure we have

visualized the local surface orientation via ellipses. This 3D impression is compelling, because

we tend to interpret the elliptical shapes as projections of what in reality are circles. This is

an example of shape-from-contour as applied by our brain. The circle – ellipse relation is just

a particular example, and more general principles have been elaborated in the literature. An

example is the maximization of area over perimeter squared, as a measure of shape compactness,

over all possible deprojections, i.e. surface patch orientations. Returning to our example, an

ellipse would be deprojected to a circle for this measure, consistent with human vision. Similarly,

symmetries in the original shape will get lost under projection. Choosing the slant and tilt angles

that maximally restore symmetry is another example of a criterion for determining the normal

to the shape. As a matter of fact, the circle – ellipse case also is an illustration for this measure.

Regular ﬁgures with at least a 3-fold rotational symmetry yield a single orientation that could

make up for the deformation in the image, except for the mirror reversal with respect to the image

plane (assuming that persp ective distortions are too small to be picked up). This is but a special

case of the more general result, that a unique orientation (up to mirror reﬂection) also results when

two copies of a shap e are observed in the same plane (with the exception where their orientation

diﬀers by 0

or 180

in which case nothing can be said on the mere assumption that both shapes

are identical). Both cases are more restrictive than skewed mirror symmetry (without perspective

eﬀects), which yields a one-parameter family of solutions only.

1.4.4 Shape-from-Defocus

Cameras have a limited depth-of-ﬁeld. Only points at a particular distance will be imaged with a

sharp projection in the image plane. Although often a nuisance, this eﬀect can also be exploited

because it yields information on the distance to the camera. The level of defocus has already

been used to create depth maps. As points can be blurred because they are closer or farther from

the camera than at the position of focus, shape-from-defocus methods will usually combine more

than a single image, taken from the same position but with diﬀerent focal lengths. This should

disambiguate the depth.

1.4.5 Shape-from-Silhouettes

Shape-from-silhouettes is a passive, multi-directional approach. Suppose that an object stands on

a turntable. At regular rotational intervals an image is taken. In each of the images, the silhouette

18 CHAPTER 1. INTRODUCTION TO 3D ACQUISITION

Figure 1.11: The ﬁrst three images show diﬀerent backprojections from the silhouette of a teapot

in three views. The intersection of these backprojections form the visual hull of the object, shown

in the bottom right image.

of the object is determined. A way to ease this process is by providing a simple background, like a

homogeneous blue or green cloth (‘blue keying’ or ‘green keying’). Initially, one has a virtual lump

of clay, larger than the object and fully containing it. From each camera orientation, the silhouette

forms a cone of projection rays, for which the intersection with this virtual lump is calculated. The

result of all these intersections yields an approximate shape. Figure 1.11 illustrates the process.

One has to be careful that the silhouettes are extracted with good precision. Once a part of

the lump has been removed, it can never be retrieved in straightforward implementations of this

idea. Also, cavities that do not show up in any silhouette will not be removed. For instance,

the eye sockets in a face will not be detected with such method and will remain ﬁlled up in the

ﬁnal model. On the other hand, the hardware needed is minimal, and very low-cost shape-from-

silhouette systems can be produced. If multiple cameras are placed around the object, the images

can be taken all at once and the capture time can be reduced. This will increase the price, and

also the silhouette extraction may become more complicated. In the case video cameras are used,

a dynamic scene like a moving person, can be captured in 3D over time. An example is shown

in Figure 1.12, where 15 video cameras were set up in an outdoor environment. Of course, in

order to extract precise cones for the intersection, the relative camera positions and their internal

settings have to be known precisely. This can be achieved with the same self-calibration methods

expounded in the following chapters.

1.4.6 Hybrid Techniques

The aforementioned techniques often have complimentary strengths and weaknesses. Therefore,

several systems try to exploit multiple techniques in conjunction. A typical example is the com-

bination of shape-from-silhouettes with stereo. Both techniques are passive and use multiple

cameras. The visual hull produced from the silhouettes provides a depth range in which stereo

can try to reﬁne the surfaces in between the rims. Similarly, one can combine stereo with struc-

tured light. Rather than trying to generate a depth map from the images pure, one can project a

20 CHAPTER 1. INTRODUCTION TO 3D ACQUISITION

random noise pattern, to make sure that there is enough texture. As still two cameras are used,

the projected pattern doesn’t have to be analyzed in detail. Local pattern correlations may suﬃce

to solve the correspondence problem. One can project in the near-infrared, to simultaneously take

color images and retrieve the surface texture without interference from the projected pattern. So

far, the problem with this has often been the weaker contrast obtained in the near-infrared band.

Many such integrated approaches can be thought of.

This said, there is no single 3D acquisition system to date that can handle all types of objects

or surfaces. Transparent or glossy surfaces (e.g. glass, metals), ﬁne structures (e.g. hair or wires),

and repetitive structures (e.g tiles on a ﬂoor) may cause problems, depending on the system that

is being used. The next section discusses still existing challenges in a bit more detail.

1.5 Challenges

The production of 3D models has been a popular research topic already for a long time now,

and important progress has indeed been made since the early days. Nonetheless, the research

community is well-aware of the fact that still much remains to be done. In this section we list

some of these challenges.

As seen in the previous section, there is a wide variety of techniques for creating 3D models,

but depending on the geometry and material characteristics of the object or scene, one technique

may be much better suited than another. For example, untextured objects are a nightmare

for traditional stereo, but too much texture may interfere with the patterns of structured-light

techniques. Hence, one would seem to need a battery of systems to deal with the variability of

objects – e.g. in a museum – to be modeled. As a matter of fact, having to model the entire

collection of diverse museums is a useful application area to think about, as it poses many of the

pending challenges, often several at once. Another area is 3D city modeling, which has quickly

grown in importance over the last years. It is another extreme in terms of conditions under

which data have to be captured, in that cities represent an absolutely uncontrolled and large-scale

environment. Also in that application area, many problems remain to be resolved.

Here is a list of remaining challenges, which we don’t claim to be exhaustive:

• Many objects have an intricate shape, the scanning of which requires great precision com-

bined with great agility of the scanner to capture narrow cavities and protrusions, deal with

self-occlusions, ﬁne carvings, etc.

• The types of objects and materials that potentially have to be handled are very diverse,

ranging from metal coins to woven textiles; stone or wooden sculptures; ceramics; gems in

jewellery and glass. No single technology can deal with all these surface types and for some

of these types of artifacts there are no satisfactory techniques yet. Also, apart from the 3D

shape the material characteristics may need to be captured as well.

• The objects to be scanned range from tiny ones like a needle to an entire landscape or city.

Ideally, one would handle this range of scales with the same techniques and similar protocols.

• For many applications, data collection may have to be undertaken on-site under potentially

adverse conditions or implying transportation of equipment to remote sites.

• Objects are sometimes too fragile or valuable to be touched and need to be scanned ’hands-

oﬀ’. The scanner needs to be moved around the object, without it being touched, using

portable systems.

• Masses of data often need to be captured, like in our museum collection or city modeling

examples. Eﬃcient data capture and model building is essential if this is to be practical.

• Those undertaking the digitisation may or may not be technically trained. Not all applica-

tions are to be found in industry, and technically trained personnel may very well not be

剩余187页未读，继续阅读

激情青春

粉丝: 1
资源: 3

经典3D重建方法综述：多图像立体与结构运动

3D Reconstruction From Multiple Imaging Planes

3D reconstruction from images

multiple-Select.css

超声CT图像重建matlab代码-Image-Reconstruction-from-multiple-OCT-Partial-Images:

Multiple View Geometry in Computer Vision

Multiple View Geometry in Computer Vision Second Edition

多平面成像重建3D骨肿瘤MRI研究

【Basic】Image Reconstruction in MATLAB: Reconstructing Images from Projection Data

[Practical Exercise] MATLAB Camera Calibration Images

Image Fusion in MATLAB: Fusion Techniques for Multi-source Images

最新资源