时间飞行与深度成像技术：传感器、算法与应用概览

1星需积分: 9 169 浏览量更新于2024-07-22 收藏 31.55MB PDF 举报

身份认证购VIP最低享 7 折!

30元优惠券

"Time of Flight and Depth Imaging" 是一本2014年出版的书籍，主要集中在TOF（Time-of-Flight）相机与深度成像技术的应用。这本书是GCPR会议集的一部分，由Marcin Grzegorzek、Christian Theobalt、Reinhard Koch和Andreas Kolb等人编辑。它涵盖了传感器、算法以及这些技术的实际应用，提供了当时该领域的最新状态调查。 TOF相机是一种利用光飞行时间原理来测量距离的设备。这种技术基于光从发射到反射回相机所需的时间来计算物体的距离。TOF相机在计算深度信息方面具有较高的精度和实时性，使其在各种应用中受到青睐，比如3D建模、机器人导航、增强现实、安全监控以及人机交互等。深度成像是TOF相机的核心功能之一，它通过生成3D点云数据来提供场景的深度信息。这些数据可以用于创建真实世界的数字模型，对物体和环境进行精确分析。深度图像的生成依赖于高效的算法，这些算法处理TOF相机捕获的原始信号，将其转化为具有深度信息的图像。这涉及到信号处理、图像校正、噪声过滤等多个步骤。本书《Time-of-Flight and Depth Imaging》可能详细探讨了以下知识点： 1. TOF相机的工作原理：解释光飞行时间的概念，如何通过脉冲激光或红外光源测量光往返的时间。 2. TOF传感器的设计与实现：包括不同类型的传感器架构，如像素阵列和信号处理电路。 3. 深度图像生成算法：涵盖从原始数据到深度图像的转换过程，包括信号恢复、相位解码、深度映射等。 4. 应用案例：展示TOF和深度成像在各个领域的实际应用，如工业自动化、医疗影像、虚拟现实等。 5. 技术挑战与解决方案：讨论TOF相机的局限性，如光照条件的影响、动态范围限制以及噪声处理等。 6. 未来发展趋势：展望TOF和深度成像技术的未来研究方向和潜在的创新点。作为LNCS（Lecture Notes in Computer Science）系列的一部分，本书也反映了计算机科学领域对这种技术的关注，特别是在计算机视觉、图形学和机器学习等子领域中的研究进展。此外，提及的Dagstuhl 2012研讨会和GCPR 2013工作坊表明了学术界对该主题的持续讨论和合作。

资源详情

资源推荐

Technical Foundation and Calibration Methods for Time-of-Flight Cameras 7

Beside integration time, that directly inﬂuences the signal-to-noise ratio (SNR)

of the measurement and consequently the variance of the measured distance, the

user can inﬂuenced the quality of the measurements made by setting the f

value

to ﬁt the application. As stated by Lange [1], as f

increases the depth resolution

increases but the non ambiguity range decreases.

2.1 Systematic Distance Error

Systematic errors occur when the formulas used for the reconstruction do not

model all aspects of the actual physical imager. In CWIM cameras a prominent

such error is caused by diﬀerences between the actual modulation and correlation

functions and the idealized versions used for calculations. In case of a sinusodial

modulation Sec. 1.1 , higher order harmonics in the modulating light source

(Fig. 2.1) induce deviations from a perfect sine function.Use of the correlation

of the physical light source with the formulas 1.1 lead to a periodic ”wiggling”

error which causes the calculated depth to oscillate around the actual depth.

The actual form of this oscillation depends on the strength and frequencies of

the higher order harmonics. [1,3].

Fig. 2. Left:Measured modulation of the PMD light source: Right: Mean depth devia-

tion as a function of the real distance. Images from [4].

There are two approaches for solving this problem. The ﬁrst approach is to

sample the correlation more phase shifts and extend the formulas to incorporate

higher order harmonics[5]. With current 2-tap sensor this approach induces more

errors when observing dynamic scenes. The second approach which we will fur-

ther discuss in 3.2 is to keep the formulas as they are and estimate the residual

error between true and calculated depth [6,7] . The residual can then be used

in a calibration step to eliminate the error. Finally, [8] employ a phase modu-

lation of the amplitude signal to attenuate the higher harmonics in the emitted

amplitude.

8D.Leﬂochetal.

2.2 Intensity-Related Distance Error

In addition to the wiggling systematic error, the measured distance is greatly

altered by an error dependent of the total amount of incident light recieved by

the sensor. Measured distance of lower reﬂectivity objects appear closer to the

camera (up to 3cm drift for the darkest objects). Fig. 3 highlights this error

eﬀect using a simple black-and-white checkerboard pattern. The described error

is usually known as Intensity-related distance error and its cause is not fully

understood yet [9].

Nevertheless, recent investigation [10] shows that ToF sensor has a non-linear

response during the conversion of photons to electrons. Lindner et al. [9] claims

that the origin of the intensity-related error is assumed to be caused by non-

linearities of the semi conductor hardware.

A diﬀerent point of view would be to consider the eﬀect of multiple returns

caused by inter-reﬂections in the sensor itself (scattering between the chip and

the lens). Since the signal strength of low reﬂectivity objects is considerably

weak, they will be more aﬀected by this behavior than for high signal strength

given by brighter objects. For more information about multi-path problems on

ToF cameras, referred to Sec. 2.5.

Fig. 3. Impact of the intensity-related distance error on the depth measurement: Left

image shows the intensity image given by a ToF camera. Right image shows the surface

rendering obtained by the depth map colored by its corresponding intensity map. Note

that those images were acquired using a PMD CamCube 2.0, PMDTechnologies GmbH

2.3 Depth Inhomogeneity

An important type of errors in ToF imaging, the so-called ﬂying pixels, occurs

along depth inhomogeneities. To illustrate these errors, we consider a depth

boundary with one foreground and one background object. In the case that the

solid angle extent of a sensor pixel falls on the boundary of the foreground and

the background, the recorded signal is a mixture of the light returns from both

areas. Due to the non-linear dependency of the depth on the raw channels and

Technical Foundation and Calibration Methods for Time-of-Flight Cameras 9

the phase ambiguity, the resulting depth is not restricted to the range between

foreground and background depth, but can attain any value of the camera’s

depth range. We will see in Section 4.1, that is important to distinguish between

ﬂying pixels in the range of the foreground and background depth and outliers.

The fact that today’ ToF sensors provide only a low resolution promotes the

occurrence of ﬂying pixels of both kinds.

We remark that the problem of depth inhomogeneities is related to the mul-

tiple return problem, since also here light from diﬀerent paths is mixed in one

sensor cell. In the case of ﬂying pixels, however, local information from neigh-

boring pixels can be used to approximately reconstruct the original depth. We

refer to Section 4.1 for details.

2.4 Motion Artifacts

As stated in 1.1, CWIM ToF imagers need to sample the correlation between

incident and reference signal at least using 3 diﬀerent phase shifts. Ideally these

raw images would be acquired simultaneously. Current two tap sensors only

allow for two of these measurements to be made simultaneously such that at

least one more measurement is needed. Usually, further raw images are acquired

to counteract noise and compensate for diﬀerent electronic charateristics of the

individual taps. Since these (pairs) of additional exposures have to be made

sequentially, dynamic scenes lead to erroneous distance values at depth and

reﬂectivity boundaries.

Methods for compensating motion artifacts will be discussed in Section 4.2.

2.5 Multiple Returns

The standard AMCW model for range imaging is based on the assumption that

the light return to each pixel of the sensor is from a single position in the scene.

This assumption, unfortunately, is violated in most scenes of practical interest,

thus multiple returns of light do arrive at a pixel and generally lead to erroneous

reconstruction of range at that pixel. Multiple return sources can be categorised

due to two primary problems. Firstly, the imaging pixel views a ﬁnite solid angle

of the scene and range inhomogeneities of the scene in the viewed solid angle

lead to multiple returns of light—the so-called mixed pixed eﬀect which results in

ﬂying pixels (see Section 2.3 above). Secondly, the light can travel multiple paths

to intersect the viewed part of the scene and the imaging pixel—the multipath

inteference problem. Godbaz [11] provides a thorough treatment of the multiple

return problem, including a review covering full ﬁeld ToF and other ranging

systems with relevant issues, such as point scanners.

In a ToF system light returning to the sensor is characterised by amplitude A

and phase shift φ. The demodulated light return is modelled usefully as the

complex phasor

η = Ae

jφ

, (8)

Technical Foundation and Calibration Methods for Time-of-Flight Cameras 11

the ToF camera is viewing the ﬂoor and wall a hole is reconstructed in the ﬂoor,

where the position of the hole aligns with the light path from camera to wall,

wall to ﬂoor, and then back to the camera. The distance into the hole is due

to the phase of the total measured phasor and is determined by the relative

amplitude and phase of the component returns, as per Eq. 9. Another example

that exhibits strong multipath interference is the sharp inside corner junction

between two walls [12]. The light bouncing from one wall to the other causes the

camera to measure an erroneous curved corner.

Multipath interference can also occur intra-camera due to the light refraction

and reﬂection of an imaging lens and aperture [13,14,15]. The aperture eﬀect

is due to diﬀraction which leads to localised blurring in the image formation

process. Fine detail beyond the limits in angular resolution is greatly reduced,

causing sharp edges to blur. Aberrations in the lens increase the loss in resolu-

tion. Reﬂections at the optical boundaries of the glass produce what is commonly

referred to as lens ﬂare [16], which causes non-local spreading of light across the

scene. In ToF imaging the lens-ﬂare eﬀect is most prominent when a bright fore-

ground scatterer is present. The foreground object does not need to be directly

visible to the camera, as long as the light from the source is able to reach that

object and reﬂect, at least in part, back to the lens [17]. Such light scattering

leads to distorted reconstructed ranges throughout the scene with the greatest

errors occurring for darker objects.

2.6 Other Error Sources

ToF camera sensors suﬀer from the same errors as standard camera sensors. The

most important error source in the sensor is a result of the photon counting

process in the sensor. Since photons are detected only by a certain probability,

Poisson noise is introduced. We refer to Seitz [18] and the thesis by Schmidt [10,

Sect. 3.1] for detailed studies on the Poisson noise. An experimental evaluation

of noise characteristics of diﬀerent ToF cameras has been performed in by Erz

&J¨ahne [19]. Besides from that other kinds of noise, e.g. dark (ﬁxed-pattern)

noise and read-out noise, occur.

In ToF cameras, however, noise has a strong inﬂuence on the estimated scene

depth, due to the following two issues

– The recorded light intensity in the raw channels is stemming from both active

and background illumination. Isolating the active part of the signal reduces

the SNR. Such a reduction could be compensated by increasing the integra-

tion time, which on the other hand increases the risk of an over-saturation of

the sensor cells, leading to false depth estimation. As a consequence, a trade-

oﬀ in the integration time has to be made, often leading to a low SNR in the

raw data, which occurs especially in areas with extremely low reﬂectivity or

objects far away from the sensor.

– Since the estimated scene depth depends non-linearly on the raw channels

(cf. Eqs. 4 and 7), the noise is ampliﬁed in this process. This ampliﬁcation is

typically modeled ([1,20]) by assuming Gaussian noise in the raw data and

剩余323页未读，继续阅读

界明城

粉丝: 193
资源: 2

时间飞行与深度成像技术：传感器、算法与应用概览

Close-Range Photogrammetry and 3D Imaging

Describe the background information of Significance of analyzing metal-transfer images for quality control and process optimization in detail

the early second paper about Optical computational imaging

帮我检索癫痫核磁共振近五年发表的文献，并简单说明文献内容

virtual imaging technology

Contrast recovery

telecommunications and imaging laboratory的vibe代码怎么运行

cannot import name _imaging from PIL

自旋回波序列扫描需要几分钟

depth_image = Image.fromarray(depth_data)

ImportError: DLL load failed while importing _imaging

计算机视觉审稿快的OA期刊

Voxel-based morphometric magnetic resonance imaging (MRI) postprocessing

balanced multiwavelets

Infrared Simulation Library

zemax raid

a= ['magnetic', 'resonance' ,'imaging']依次取出 imaging,resonance imaging

Write a paper about Deep-learning based analysis of metal-transfer images in GMAW process

single depth slice

基于arm嵌入式人体心率血氧的监测的参考文献

最新资源