Automatic 3D Face Detection, Normalization and Recognition
Ajmal Mian, Mohammed Bennamoun and Robyn Owens
School of Computer Science and Software Engineering
The University of Western Australia
35 Stirling Highway, Crawley, WA 6009, Australia
{ajmal, bennamou, robyn.owens}@csse.uwa.edu.au
Abstract
A fully automatic 3D face recognition algorithm is pre-
sented. Several novelties are introduced to make the recog-
nition robust to facial expressions and efficient. These nov-
elties include: (1) Automatic 3D face detection by detecting
the nose; (2) Automatic pose correction and normalization
of the 3D face as well as its corresponding 2D face using
the Hotelling Transform; (3) A Spherical Face Representa-
tion and its use as a rejection classifier to quickly reject a
large number of candidate faces for efficient recognition;
and (4) Robustness to facial expressions by automatically
segmenting the face into expression sensitive and insensi-
tive regions. Experiments performed on the FRGC Ver 2.0
dataset (9,500 2D/3D faces) show that our algorithm out-
performs existing 3D recognition algorithms. We achieved
verification rates of 99.47% and 94.09% at 0.001 FAR and
identification rates of 98.03% and 89.25% for probes with
neutral and non-neutral expression respectively.
1. Introduction
Face recognition is a challenging problem because of
the ethnic diversity of faces and variations caused by ex-
pressions, gender, pose, illumination and makeup. Appear-
ance based (2D) face recognition algorithms were the first
to be investigated due to the wide spread availability of cam-
eras. One of the classic face recognition algorithms uses the
eigenface representation of Turk and Pentland [15] which
is based on the Principal Component Analysis (PCA). Lin-
ear Discriminate Analysis (LDA) [16], Independent Com-
ponent Analysis (ICA) [3], Bayesian methods [11] and Sup-
port Vector Machines (SVM) [14] have also been success-
fully used for appearance based face recognition. Zhao et
al. [17] give a detailed survey of 2D face recognition algo-
rithms and conclude that existing algorithms are sensitive
to illumination and pose. Therefore, researchers are now
investigating other data acquisition modalities of the face
to overcome these limitations. One of the most promis-
ing modalities is the 3D shape of the face. A 3D face is
a three dimensional vector [x
i
,y
i
,z
i
]
of the x, y and z co-
ordinates of the pointcloud of a face (i =1...n, where n is
the number of points). A 2D face on the other hand is a five
dimensional vector [u
i
,v
i
,R
i
,G
i
,B
i
]
where u, v are the
pixel coordinates and R, G and B are their corresponding
red, green and blue components. When the 3D face and the
2D face are registered, the pixel coordinates u, v of the 2D
face can be replaced with the absolute coordinates x, y of
its corresponding 3D face.
Bowyer et al. [6] present a comparative survey of
3D face recognition algorithms and conclude that 3D face
recognition has the potential to overcome the limitations of
its 2D counterpart. Especially, the 3D shape of a face can be
used to correct the pose of its corresponding 2D facial im-
age which is one of the major contributions of our paper. We
present a fully automatic algorithm for pose correction of a
3D face and its corresponding 2D colored image. Existing
techniques perform pose correction by manually identify-
ing landmarks on the faces (e.g. [7]). Our approach is to
automatically detect the nose tip and correct the pose using
the Hotelling transform [8]. The pose correction measured
from the 3D face is also used to correct the 3D pose of its
corresponding 2D face. Since 2D face recognition is a well
studied area [17], we will only demonstrate the pose cor-
rection of the 2D faces (along with the 3D faces) and then
focus on 3D face recognition alone.
Another major contribution of our paper is an efficient
3D Spherical Face Representation (SFR) based rejection
classifier which quickly eliminates a large number of ineli-
gible candidate faces from the gallery. The remaining faces
are then verified using a novel recognition algorithm which
is robust to facial expressions. Robustness to facial expres-
sions is achieved by automatically segmenting the face into
expression sensitive and insensitive regions and using the
latter for recognition.
2. Three-Dimensional Face Normalization
Blanz et al. [5] used morphable models to deal with
pose variations in 2D facial images. We use 3D face data