![](https://csdnimg.cn/release/download_crawler_static/18881713/bg1.jpg)
AN INTEGRATED GRAPH-BASED FACE SEGMENTATION APPROACH FROM KINECT
VIDEOS
Jixia Zhang
1
, Haibo Wang
2
, Shaoguo Liu
1
, Jiangyong Duan
1
, Ying Wang
1
, Chunhong Pan
1
National Laboratory of Pattern Recognition, CASIA
1
The Robotics Research Center, Shandong University
2
jixiazhang@gmail.com
ABSTRACT
In this paper, we present an Integrated Semi-Supervised Graph
(IntSSG) approach to automatically segment face from color-depth
video. In the first step, IntSSG performs skin color detection and
online SIFT matching to initialize some face and non-face pixels.
Then, the labels of these pixels are refined by conducting adap-
tive depth thresholding. Finally, based on a semi-supervised graph
framework, IntSSG segments face by propagating the refined labels
to other pixels. Experimental results show that IntSSG is able to
accurately segment faces in difficult situations such as large pose
changes and illumination variations.
Index Terms— Skin detection, face segmentation, depth
1. INTRODUCTION
Face segmentation from video has wide applications in many fields,
such as human computer interaction, video editing [1], virtual real-
ity [2]. However, this remains a challenging problem. On the one
hand, efficient face segmentation requires a full face output without
any holes and noises. On the other hand, face segmentation must
overcome the problems of large head pose variation, occlusion and
illumination changes that often exist throughout a video sequence.
Face segmentation was usually conducted through the stepwise
use of face color detection and region segmentation [3, 4, 5, 6]. Chai
et al. [7] detected skin color pixel with a generic skin model, and
then performed region-based regularization to link separated regions
and find the most likely facial region. By contrast, in [5], following
the classification of each pixels, a binary partition tree is utilized to
merge detected face pixels or regions. In [6], color Gaussian Mixture
Models are adopted to decide the possibilities of each pixel belong-
ing to face or background, which is then followed by a graph-cut
based segmentation to segment face. Although skin detection is in-
sensitive to head pose variations, it tends to fail when the illumina-
tion condition is different to the one when skin detector is trained.
Some approaches employ a top-down scheme to conduct face
segmentation. These approaches adopt a global face model to local-
ize face first, and then segment face boundary. For example, Luo
et. al [8] utilized a blob model to locate head-and-shoulder and then
segment face with a shape model. However, the specified blob and
shape models limit its practical applications. In [9], face is localized
This work was supported in part by the National Natural Science
Foundation of China under Grants 61175025, 61272049, 61005013, and
61203279, and the Strategic Priority Research Program of the Chinese A-
cademy of Sciences (Grant No. XDA06030300).
with a learned frontal face detector. As reported in [9], this approach
may fail when face is not near-frontal.
In this paper, we present IntSSG, an effective approach to auto-
matically segment faces from color-depth videos. By treating face
segmentation as a semi-supervised graph problem, IntSSG success-
fully integrates face pixel detection and segmentation in a single
computation framework. In its first step, IntSSG jointly employs
skin color detection and online SIFT matching to detect possible face
and non-face pixels. Second, IntSSG takes advantage of depth cue
to remove the potential wrong labeled pixels in them, by which only
most reliable pixels are preserved as prior labels. Finally, with the
preserved labeled pixels as prior, IntSSG relies on graph propagation
to find all face pixels [10]. The main novelties of IntSSG lies in (1)
face pixel classification and region segmentation are unified in a s-
ingle computational framework, without any pre- or post-processing
steps, and (2) skin color, temporal coherence, and depth cues are
employed together to guarantee that prior face pixels are correctly
detected before propagating to other pixels.
The remainder of this paper is organized as follows. The pro-
posed segmentation method is presented in Section 2. In Section 3,
experimental results and analysis are reported. Finally, conclusions
are drawn in Section 4.
2. AN INTEGRATED SEMI-SUPERVISED GRAPH-BASED
FACE SEGMENTATION APPROACH
This section presents details of the Integrated Semi-Supervised
Graph-based (IntSSG) face segmentation approach. With color and
depth images from Kinect, IntSSG integrates skin detection, SIFT
matching, depth reasoning and smoothness propagation in a unified
semi-supervised graph framework.
2.1. Main Framework
Let I
t
be the color image frame obtained at time t, and D
t
be the
corresponding depth image. Denote the color value of one pixel by
x ∈ R
3
, and its class label by y ∈ {1, −1} (1 for face class and
−1 for non-face class). The goal of IntSSG face segmentation is to
classify each pixel as either face or non-face class, i.e., estimating
the labels Y
t
=
y
t
1
, y
t
2
, . . . , y
t
n
for all the n pixels.
IntSSG follows a semi-supervised graph-based scheme [10] to
obtain Y
t
: first annotate a small portion of pixels as prior labels,
then propagate the prior labels to all pixels. More precisely, all the
labels are estimated by minimizing the following energy function:
E
s
+ λE
f
, (1)
2733978-1-4799-2341-0/13/$31.00 ©2013 IEEE ICIP 2013