Face pose normalization and simulation methods based on multi-
view face alignment
Changsong Liu
1,2,3
and Liting Wang
1
; 1Department of Electronic Engineering, Tsinghua University, Beijing 100084, China, 2State Key
Laboratory of Intelligent Technology and Systems,3Tsinghua National Laboratory for Information Science and Technology (China)
Abstract
In this paper, we propose a face pose normalization and
simulation methods based on multi-view face alignment that can
enhance the performance of the face recognition algorithm
towards large pose variation. The proposed method includes two
steps: 1) multi-view face alignment, 2) face pose normalization and
simulation methods. Multi-view face alignment algorithm is
inspired by the design idea of the Supervised Descent Method
(SDM) which is considered the state-of-the-art in face alignment.
The proposed method modified the algorithm to adapt multi-view
problems by changing the histogram of gradient feature to
projection of gradient feature in order to adapt large pose
variance. In addition, the feature scale also can be adaptive
adjusted towards different part of face, for example, eyes, mouth,
eyebrows, etc. Based on the multi-view face alignment results, 2D
face normalization and simulation methods are proposed.
Experimental results over many images with obvious pose changes
have shown our method can significantly normalize the multi-view
pose face and improve the accuracy of the existing common face
recognition method when faces of probe sets have large pose
variation.
Key words: face pose normalization; face simulation; multi-view
face alignment; face recognition
Introduction
Over the past few decades, face recognition has become a
popular area of research in pattern recognition and computer vision
due to its wide range of commercial and law enforcement
applications, such as biometric authentication, video surveillance,
and information security. Until now, a great number of face
recognition methods have been proposed and the system has been
claimed widely used. However, pose variation is a challenge and
the performance of current methods is still unsatisfactory. There is
a wide variety of studies related to pose-robust face recognition
when each identifier has only one frontal training sample [1].
Methods deal with large pose problems can be divided into three
categories: the first category is the pose invariant feature extraction
method. But the main disadvantage of this method is that it is
difficult to extract the pose-invariant features. The second category
is based on the multi-view face image solution. For example, the
method in [2] extended the traditional subspace to multi-subspace.
The main drawback of this method is it is very difficult to
absolutely divide different face into accurate pose sets, and the
errors of the pose estimation will reduce the performance of face
recognition. The third category is the method based on 3D face
model [3], based on 3D face model, the virtual face images are
generated in solving the problem which has achieved good results.
But a disadvantage is that the computational cost is large.
Robust face alignment is a pre-processing which could output
a reliable and correct correspondence to face recognition
normalization. Usually, we locate face landmarks and normalize
face to extract features and train classifiers. Face alignment method
under the most popular Active Shape Models (ASMs)[4] and the
Active Appearance Models (AAMs) [5,6] has some disadvantages,
the Supervised Descent Method (SDM)[7] refined the fixed shape
PCA model, which is considered the state-of-the-art in face
alignment. When it is utilized to multi-view and some large scale
and more flexible situations, still, some improvement should be
made to get a robust and reliable performance towards multi-view
face pose changes. Face normalization relied on face alignment
result, which is very important step for face recognition classifier
training. There are kinds of face normalization towards pose
changes by both 2D and 3D methods. How to normalize multi-
view face image and get a better face recognition result is still a
matter of concern.
The main objective of this proposed method is to design a
face normalization and simulation method to improve the face
recognition towards multi-view faces. The first proposed step is
robust multi-view face alignment method. Though the Supervised
Descent Method (SDM) provides a framework of face alignment, it
still has some difficulties when it is utilized to multi-view and
some large scale and more flexible situations. Compared with the
generative feature, the discriminative feature shows good results in
the past ASM and AAM framework. Under the SDM framework,
the feature and feature scale could be chosen and discussed to
improve the face alignment accuracy in multi-view cases. The
second proposed step is face normalization method. Relied on face
alignment results, multi-view face is normalized to frontal face
template.
Proposed face pose normalization and
simulation methods framework
Fig.1. Proposed algorithm framework dealing with multi-view face pose
variation
The basic idea proposed in this study is to deal with multi-
view face recognition with the help of face alignment results. In
face alignment step, each identifier’s multi-view face is located by
88 feature points. Then, the shape and texture of both the multi-