基于图的Kinect视频人脸识别与分割技术

需积分: 0 88 浏览量更新于2024-08-12 收藏 425KB PDF 举报

"这篇研究论文提出了一种名为集成半监督图（IntSSG）的人脸分割方法，适用于Kinect视频。该方法结合了肤色检测、在线SIFT匹配、自适应深度阈值处理以及半监督图框架，能够在大姿态变化和光照变化等复杂条件下准确地进行人脸识别和分割。" 在当前的IT领域，尤其是在计算机视觉和人工智能应用中，人脸分割是一项重要的技术。本文主要关注的是在Kinect设备获取的色深视频中实现这一目标。Kinect是一种能够同时捕捉彩色图像和深度信息的设备，为三维人脸识别和分割提供了可能。首先，IntSSG方法通过肤色检测来初步识别可能属于人脸的部分。肤色检测是利用颜色模型来区分肤色像素，通常涉及到RGB、YCbCr或HSV色彩空间的分析。接着，采用在线Scale-Invariant Feature Transform (SIFT)匹配进一步定位人脸特征。SIFT是一种强大的特征检测算法，即使在尺度变化、旋转和光照变化下也能保持稳定，因此它有助于在视频序列中稳定地初始化人脸和非人脸像素。然后，IntSSG应用自适应深度阈值策略来细化这些像素的标签。这种方法考虑了深度信息，以更精确地分割出人脸区域，特别是在背景复杂或者与周围环境相似的情况下。深度信息可以提供额外的维度来区分前景（如人脸）和背景。最后，论文引入了一个半监督图框架，通过传播经过精细化的标签来分割剩余的像素。在半监督学习中，部分像素的标签是已知的，而其他像素的标签则通过考虑全局结构和邻域信息来推断。这种图割或能量最小化的过程使得IntSSG能够处理未标注数据，并在大姿态变化和光照变化等困难场景中保持良好的分割性能。实验结果显示，IntSSG在处理复杂情况时具有高精度，证明了该方法的有效性。该技术对于人机交互、视频编辑和虚拟现实等领域的应用具有显著价值，因为它能够实时并准确地识别人脸，即便是在极具挑战性的环境下。关键词：肤色检测，人脸识别，深度信息，半监督学习，图割，Kinect视频。

AN INTEGRATED GRAPH-BASED FACE SEGMENTATION APPROACH FROM KINECT

VIDEOS

Jixia Zhang

, Haibo Wang

, Shaoguo Liu

, Jiangyong Duan

, Ying Wang

, Chunhong Pan

National Laboratory of Pattern Recognition, CASIA

The Robotics Research Center, Shandong University

jixiazhang@gmail.com

ABSTRACT

In this paper, we present an Integrated Semi-Supervised Graph

(IntSSG) approach to automatically segment face from color-depth

video. In the ﬁrst step, IntSSG performs skin color detection and

online SIFT matching to initialize some face and non-face pixels.

Then, the labels of these pixels are reﬁned by conducting adap-

tive depth thresholding. Finally, based on a semi-supervised graph

framework, IntSSG segments face by propagating the reﬁned labels

to other pixels. Experimental results show that IntSSG is able to

accurately segment faces in difﬁcult situations such as large pose

changes and illumination variations.

Index Terms— Skin detection, face segmentation, depth

1. INTRODUCTION

Face segmentation from video has wide applications in many ﬁelds,

such as human computer interaction, video editing [1], virtual real-

ity [2]. However, this remains a challenging problem. On the one

hand, efﬁcient face segmentation requires a full face output without

any holes and noises. On the other hand, face segmentation must

overcome the problems of large head pose variation, occlusion and

illumination changes that often exist throughout a video sequence.

Face segmentation was usually conducted through the stepwise

use of face color detection and region segmentation [3, 4, 5, 6]. Chai

et al. [7] detected skin color pixel with a generic skin model, and

then performed region-based regularization to link separated regions

and ﬁnd the most likely facial region. By contrast, in [5], following

the classiﬁcation of each pixels, a binary partition tree is utilized to

merge detected face pixels or regions. In [6], color Gaussian Mixture

Models are adopted to decide the possibilities of each pixel belong-

ing to face or background, which is then followed by a graph-cut

based segmentation to segment face. Although skin detection is in-

sensitive to head pose variations, it tends to fail when the illumina-

tion condition is different to the one when skin detector is trained.

Some approaches employ a top-down scheme to conduct face

segmentation. These approaches adopt a global face model to local-

ize face ﬁrst, and then segment face boundary. For example, Luo

et. al [8] utilized a blob model to locate head-and-shoulder and then

segment face with a shape model. However, the speciﬁed blob and

shape models limit its practical applications. In [9], face is localized

This work was supported in part by the National Natural Science

Foundation of China under Grants 61175025, 61272049, 61005013, and

61203279, and the Strategic Priority Research Program of the Chinese A-

cademy of Sciences (Grant No. XDA06030300).

with a learned frontal face detector. As reported in [9], this approach

may fail when face is not near-frontal.

In this paper, we present IntSSG, an effective approach to auto-

matically segment faces from color-depth videos. By treating face

segmentation as a semi-supervised graph problem, IntSSG success-

fully integrates face pixel detection and segmentation in a single

computation framework. In its ﬁrst step, IntSSG jointly employs

skin color detection and online SIFT matching to detect possible face

and non-face pixels. Second, IntSSG takes advantage of depth cue

to remove the potential wrong labeled pixels in them, by which only

most reliable pixels are preserved as prior labels. Finally, with the

preserved labeled pixels as prior, IntSSG relies on graph propagation

to ﬁnd all face pixels [10]. The main novelties of IntSSG lies in (1)

face pixel classiﬁcation and region segmentation are uniﬁed in a s-

ingle computational framework, without any pre- or post-processing

steps, and (2) skin color, temporal coherence, and depth cues are

employed together to guarantee that prior face pixels are correctly

detected before propagating to other pixels.

The remainder of this paper is organized as follows. The pro-

posed segmentation method is presented in Section 2. In Section 3,

experimental results and analysis are reported. Finally, conclusions

are drawn in Section 4.

2. AN INTEGRATED SEMI-SUPERVISED GRAPH-BASED

FACE SEGMENTATION APPROACH

This section presents details of the Integrated Semi-Supervised

Graph-based (IntSSG) face segmentation approach. With color and

depth images from Kinect, IntSSG integrates skin detection, SIFT

matching, depth reasoning and smoothness propagation in a uniﬁed

semi-supervised graph framework.

2.1. Main Framework

Let I

be the color image frame obtained at time t, and D

be the

corresponding depth image. Denote the color value of one pixel by

x ∈ R

, and its class label by y ∈ {1, −1} (1 for face class and

−1 for non-face class). The goal of IntSSG face segmentation is to

classify each pixel as either face or non-face class, i.e., estimating

the labels Y



, y

, . . . , y



for all the n pixels.

IntSSG follows a semi-supervised graph-based scheme [10] to

obtain Y

: ﬁrst annotate a small portion of pixels as prior labels,

then propagate the prior labels to all pixels. More precisely, all the

labels are estimated by minimizing the following energy function:

+ λE

, (1)

下载后可阅读完整内容，剩余4页未读，立即下载

weixin_38601499

粉丝: 2
资源: 938

基于图的Kinect视频人脸识别与分割技术

体感编程系列（一）Kinect开发入门理论知识点概述

Kinect官方文档中文翻译.rar

kinect studio录制视频

基于kinect的图片缩放代码

matlab图像身高测量,基于Kinect景深图像的快速身高和体重测量方法

跌倒行为检测方法 kinect

KinectopenCV 读取Kinect彩图、深度图

基于python获取Kinect2的内部参数的方法以及代码

Azure Kinect DK手势识别

matlab中实现kinect深度图像坐标

最新资源