An Automatic Multi-sample 3D Face Registration Method Based On Thin Plate
Spline And Deformable Mod l
Wenyu Qin, Yongli Hu, Yanfeng Sun and Baocai Yin
Beijing Key Laboratory of Multimedia and Intelligent Software Technology
College Of Computer Science and Technology, Beijing University Of Technology
Beijing, China
mrpomelo@sina.com
, huyongli@bjut.edu.cn, yfsun@bjut.edu.cn, ybc@bjut.edu.cn
Abstract—Non-rigid registration of 3D facial surfaces is a cru-
cial step in a variety of computer vision tasks. In this paper, we
present a fully automatic 3D face registration method based on
the thin plate spline (TPS) and deformable model. To model
the non-rigid modality of complex 3D facial surfaces, the thin
plate spline is adopted to represent the transformation between
3D faces. The farthest point sampling (FPS) method is used to
generate the control points for the thin plate spline transfor-
mation automatically. There are two phases for 3D face regis-
tration. Firstly, the preliminary registration is obtained by
closest points searching between the thin plate spline trans-
formed reference and the target. Then the multi-sample regis-
tration is implemented to improve the precision of the registra-
tion by using a dynamical reference produced based on de-
formable model. To eliminate outliers countermeasures are
presented in both phases. The experiments on Bu-3dfe and
Bjut-3d face databases show that the proposed method is effec-
tive and robust.
Keywords-3D nod-rigid registration, multi-sample, thin plate
spline, deformable model, outliers
I. INTRODUCTION
With the development of the 3D data acquisition tech-
nology and computer visualization technology, it is conve-
nient to acquire high resolution 3D faces which provide ab-
undant data for the research of 3D face modeling, 3D face
recognition. However, the original 3D face scans generally
have different topological structures with varying numbers of
vertices. The basic and vital problem of data registration
should be conquered. Because of the complexity of geometry
and the huge volume of data of the high resolution 3D face
scans, it is a challenge to develop automatic registration me-
thod with high efficiency, robustness, and precision.
For the 3D facial data, the registration is a mapping from
one object to another object, in which each point gets the
corresponding point according to its inherent property. The
methods proposed to solve the registration problem can be
classified into rigid and non-rigid types. Reference [1] intro-
duced an iterative closest point (ICP) algorithm based on the
rigid transformation. But the disadvantage of this method is
that it demands adequate pre-registration and consumes a
good chunk of running time. Moreover, the rigid transforma-
tion is unsuitable for many cases of non-rigid deformation,
such as with 3D faces. So many non-rigid methods have
been proposed to deal with the problem of non-rigid defor-
mation. Reference [2] introduced a registration approach
based on coherent point drift (CPD) algorithm. It formulated
the alignment of two point sets as a probability density esti-
mation problem. The reference point set is represented as
gaussian mixture model (GMM) centroid and fitted to
the target point set by the expectation maximization (EM)
algorithm. But EM converges so slowly when the volume of
the data is large. Reference [3] proposed a semi-automatic
algorithm based on thin plate spline [4] (SA-TPS), which
selects some feature points as the control ones for TPS trans-
formation by manual interactive tools [5]. Nevertheless, get-
ting adequate feature points is a time consuming procedure
with subjective errors. To improve the efficiency, Reference
[6] proposed a random method to produce the TPS control
points automatically (Rd-TPS). But a problem with this me-
thod is that the distribution of the control points is not uni-
form on the surface. As a result, the deformations of some
local shapes are not satisfactory, and the registration error
increases as the number of samples grows. Reference [7]
proposed a deformation model for 3D face matching. The
TPS transformation is used to represent the deformation of
3D faces too, but the landmarks are manually localized and
the deformation model is learned from a small number of
face samples. In order to improve the precision of point-to-
point registration of 3D faces, we propose an automatic non-
rigid registration method based on TPS transformation and
deformable model, named as DMTPS. The registration pro-
cedure of DMTPS can be divided into two phases. The first
phase is the registration of two samples, in which the refer-
ence face is deformed and registered to the target face by
TPS. So the preliminary correspondence can be achieved by
the reference registration. The second phase is the multi-
sample registration. Based on the preliminary results, we
build a deformable model which is different from the paper
[7] and is constructed from all of the 3D face samples to ob-
tain a dynamic reference for every sample. As the dynamic
reference is more similar to the target comparing with the
fixed reference, the registration result would be improved if
the dynamic reference is aligned to the target. To eliminate
the influence of the outliers, we deal with the outliers in both
phases. The experiments carried out on Bu-3dfe [8] and
Bjut-3D [9] face databases show that our method is more
accurate and satisfactory for 3D face registration.
The rest of this paper is organized as follows. Section II
introduces the non-rigid registration method based on TPS
transformation. In Section III, we present a new automatic
control point selection method for TPS transformation. In
section IV, a deformable model is constructed to produce
dynamic reference for the aligning face, and a multi-sample
e
2012 IEEE International Conference on Multimedia and Expo Workshops
978-0-7695-4729-9/12 $26.00 © 2012 IEEE
DOI 10.1109/ICMEW.2012.85
453