微软Kinect传感器在增强计算机视觉中的应用综述

需积分: 9 105 浏览量更新于2024-07-21 收藏 437KB PDF 举报

"本文是对使用微软Kinect传感器增强计算机视觉技术的一次全面回顾，探讨了该传感器在机器视觉领域的广泛应用和潜力。" 随着微软Kinect传感器的诞生，低成本、高分辨率的深度感知和视觉（RGB）传感技术得以普及。Kinect传感器提供的深度和视觉信息的互补性为解决计算机视觉中的基础问题开辟了新的途径。文章详细回顾了近年来基于Kinect的计算机视觉算法和应用，将所审查的方法按照它们能解决或增强的视觉问题类型分类。首先，文章讨论了预处理方法，这是计算机视觉流程的关键步骤。Kinect传感器获取的原始数据通常需要经过去噪、校正和平滑等预处理操作，以提高后续分析的准确性和效率。这些方法包括对深度图像的滤波处理和RGB图像的颜色校正，以及对不规则或缺失的数据进行插值。其次，是对象跟踪和识别领域。Kinect的实时三维信息使得物体跟踪更加精确，即使在复杂背景下也能实现。同时，结合深度信息的物体识别算法能够更好地理解物体的形状和位置，提高了识别的鲁棒性。这在机器人导航、安防监控和人机交互等领域有广泛应用。接下来，文章深入到人类活动分析。通过Kinect传感器，可以捕捉并分析人体关节的运动，从而识别出各种动作和行为。这些技术在体育训练、医疗康复和游戏交互中有着广泛的应用前景。此外，手部姿态分析也是重点之一。由于Kinect能提供精细的手部深度信息，因此可以实现精确的手势识别，这对于无触控交互和手语翻译等应用具有重要意义。最后，文章还涵盖了室内3D映射。利用Kinect构建三维环境模型成为可能，这对于建筑规划、室内设计和虚拟现实体验等领域提供了强大的工具。这篇综述展示了Kinect传感器在计算机视觉领域的广泛影响力，揭示了其在各个方向上的创新应用，并为未来的研究方向提供了指导。通过对各种算法和技术的总结，该文为研究人员和开发者提供了宝贵的参考资料，有助于推动相关领域的进一步发展。

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE TRANSACTIONS ON CYBERNETICS

Fig. 4. Example for hole-ﬁlling based on the bilateral ﬁlter [25]. (a) Raw

depth image. (b) Depth image after ﬁltering.

turns out that Kinect is able to capture the relative 3-D

coordinates of markers with minor errors (< 1cm) in case the

sensor is positioned in an ideal range (1m to 3m) and with an

effective ﬁeld of view. In [15], authors examine the accuracy

of joint localization and the robustness of pose estimation

with respect to more realistic setups. In the experiment, six

exercises are conducted, in which the subject is either seated

or positioned next to a chair. The exercise is generally chal-

lenging for human pose recognition since the self-occlusion

appears frequently and the capturing view angle is changed

over time. The acquired 3-D location of each joint is then

compared to the data generated by a marker-based motion

capture system, which can be considered as ground truth data.

According to the results, Kinect has a signiﬁcant potential as

a low-cost alternative for real-time motion capturing and body

tracking in healthcare applications. The accuracy of the Kinect

joint estimation is comparable to marker-based motion capture

in a more controlled body pose (e.g., standing and exercising

arms). However, in general poses, the typical error of Kinect

skeletal tracking is about 10 cm. Moreover, the current Kinect

algorithm frequently fails due to occlusions, nondistinguishing

depth (limbs close to the body) or clutter (other objects in the

scene).

III. Preprocessing

The data obtained with Kinect normally cannot be directly

fed into the designed computer vision algorithms. Most of

the algorithms take advantage of rich information (RGB and

depth) attached to a point. In order to correctly combine the

RGB image with the depth data, it is necessary to spatially

align the RGB camera output and the depth camera output. In

addition, the raw depth data are very noisy and many pixels

in the image may have no depth due to multiple reﬂections,

transparent objects or scattering in certain surfaces (such as

human tissue and hair). Those inaccurate/missing depth data

(holes) need to be recovered prior to being used. Therefore,

many systems based on Kinect start with a preprocessing mod-

ule, which conducts application-speciﬁc camera recalibration

and/or depth data ﬁltering.

A. Kinect Recalibration

In fact, Kinect has been calibrated during manufacturing.

The camera parameters are stored in the device’s memory,

which can be used to fuse the RGB and depth information.

This calibration information is adequate for casual usage, such

as object tracking. However, it is not accurate enough for

reconstructing a 3-D map, for which a highly precise cloud of

3-D points should be obtained. Moreover, the manufacturer’s

calibration does not correct the depth distortion, and is thus

incapable of recovering the missing depth.

Zhang et al. [16] and Herrera et al. [17] develop a cali-

bration board based technique, which is derived from Zhang’s

camera calibration technique used for the RGB camera [18].

In this method, 3-D coordinates of the feature points on the

calibration card are obtained from the RGB camera’s coordi-

nate system. Feature-point matching between the RGB image

and the depth image is able to spatially correlate those feature

points between two different images. This spatial mapping

helps feature points to get their true depth values based on

the RGB camera’s coordinate system. Meanwhile, the depth

camera measures 3-D coordinates of those feature points in the

IR camera’s coordinate system. It assumes that the obtained

depth values by the depth camera can be transformed to the

true depth values by an afﬁne model. As a result, the key is

to estimate the parameters of the afﬁne model, which can be

done by minimizing the distances between the two point sets.

This technique combined with a calibration card allows users

to recalibrate the Kinect sensor in case the initial calibration is

not accurate enough for certain applications. The weakness of

this method is that it does not speciﬁcally pay attention to the

depth distortion. Correcting the depth distortion may become

unavoidable for most 3-D mapping scenarios.

There are a few publications that discuss solutions for

Kinect depth distortion correction. Smisek et al. [11] discover

that the Kinect device has shown radially symmetric distor-

tions. In order to correct this distortion, a spatially varying

offset to the calculated depth is applied. The offset at a given

pixel position is calculated as the mean difference between

measured depth and expected depth in metric coordinates.

In [19], a disparity distortion correction method is proposed

based on the observation that a more accurate calibration can

be made by correcting the distortion directly in disparity units.

An interesting paper [20] deals with more practical issues,

which investigates a possible inﬂuence of thermal and envi-

ronmental conditions when calibrating Kinect. The experiment

turns out that variations of the temperature and air draft have a

notable inﬂuence on Kinect’s images and range measurements.

Based on the ﬁndings, temperature-related rules have been

established in the paper, which reduce errors in the calibration

and measurement process of the Kinect.

B. Depth Data Filtering

Another preprocessing step is depth data ﬁltering, which can

be used for depth image denoising or missing depth (hole)

recovering. A naive approach considers the depth data as a

monochromatic image and thus applies existing image ﬁlters

on it, such as a Gaussian ﬁlter. This simple method works only

for regions where the signal statistics is in favor of the underly-

ing ﬁlter. A more sophisticated algorithm [21] investigates the

speciﬁc characteristics of a depth map created by Kinect, and

ﬁnds out that there are two types of occlusions/holes caused

by different reasons. The algorithm automatically separates

剩余16页未读，继续阅读

ailesargentees

粉丝: 0
资源: 1

微软Kinect传感器在增强计算机视觉中的应用综述

Kinect的详细介绍

Beginning Kinect Programming with the Microsoft Kinect SDK

Enhanced-Microsoft-Band-Alarms:在游戏商店！

enhanced-media-reviews-app:具有增强功能的评论应用程序

蔡氏电路matlab仿真代码-tree_enhanced_embedding_model:TEM：可解释建议的树增强嵌入模型，WWW2018

enhanced-cooja-radiologger-headless:增强型cooja-radiologger-headless-开源

enhanced-selenium-user-extensions:用于在 MotherApp 中使用的 selenium 的 user-extensions.js

enhanced-steam-image-gallery:微小的Chrome扩展程序，可在Steam创意工坊的图像库中添加键盘支持（前进键）

Review on Enhanced GPSR protocol For Wireless Sensor Networks.pdf

Surface-enhanced Raman spectroscopy measurement of cancerous cells with optical fiber sensor

最新资源