Discriminant Analysis on Riemannian Manifold of Gaussian Distributions
for Face Recognition with Image Sets
Wen Wang
1,2
, Ruiping Wang
1
, Zhiwu Huang
1,2
, Shiguang Shan
1
, Xilin Chen
1
1
Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS),
Institute of Computing Technology, CAS, Beijing, 100190, China
2
University of Chinese Academy of Sciences, Beijing, 100049, China
{wen.wang, zhiwu.huang}@vipl.ict.ac.cn, {wangruiping, sgshan, xlchen}@ict.ac.cn
Abstract
This paper presents a method named Discriminant Anal-
ysis on Riemannian manifold of Gaussian distributions
(DARG) to solve the problem of face recognition with image
sets. Our goal is to capture the underlying data distribution
in each set and thus facilitate more robust classification. To
this end, we represent image set as Gaussian Mixture Model
(GMM) comprising a number of Gaussian components with
prior probabilities and seek to discriminate Gaussian com-
ponents from different classes. In the light of information
geometry, the Gaussians lie on a specific Riemannian man-
ifold. To encode such Riemannian geometry properly, we in-
vestigate several distances between Gaussians and further
derive a series of provably positive definite probabilistic k-
ernels. Through these kernels, a weighted Kernel Discrim-
inant Analysis is finally devised which treats the Gaussians
in GMMs as samples and their prior probabilities as sam-
ple weights. The proposed method is evaluated by face i-
dentification and verification tasks on four most challenging
and largest databases, YouTube Celebrities, COX, YouTube
Face DB and Point-and-Shoot Challenge, to demonstrate its
superiority over the state-of-the-art.
1. Introduction
In contrast to traditional face recognition task based on
single-shot images, image-set based face recognition prob-
lem attracts more and more attention recently. For the task
of image-set based face recognition, both the gallery and
probe samples are image sets, each of which contains many
facial images or video frames belonging to one single per-
son. Compared with recognition from single image, the nu-
merous images in each set naturally cover more variations
in the subject’s face appearance due to changes of pose, ex-
pression and/or lighting. Therefore, how to represent the
variations and further discover invariance from them are the
key issues of image-set based face recognition [23, 36, 34].
To represent the variations in an image set, a probabilis-
tic model seems a natural choice. Among many others,
Gaussian Mixture Model (GMM) can precisely capture the
data variations with a multi-modal density, by using a vary-
ing number of Gaussian components. Theoretically, after
modeling image set by GMM, the dissimilarity between any
two image sets can be computed as the distribution diver-
gence between their GMMs. However, divergence in distri-
bution is not adequate for classification tasks that need more
discriminability. Especially, when the gallery and probe sets
have weak statistical correlations, larger fluctuations in per-
formance were observed [23, 36, 6, 18, 13].
To address the above problem, in this paper we propose
to learn a discriminative and compact representation for
Gaussian distributions and thus measure the dissimilarity of
two sets with the distance between the learned representa-
tions of pair-wise Gaussian components respectively from
either GMM. However, Gaussian distributions lie on a spe-
cific Riemannian manifold according to information geom-
etry [1]. Therefore, discriminant analysis methods devel-
oped in the Euclidean space cannot be applied directly. We
thus propose a novel method of Discriminant Analysis on
Riemannian manifold of Gaussian distributions (DARG). In
this method, by exploring various distances between Gaus-
sians, we derive corresponding provably positive definite
probabilistic kernels, which encode the Riemannian geom-
etry of such manifold properly. Then through these kernels,
a deliberately devised weighted Kernel Discriminant Anal-
ysis is utilized to discriminate the Gaussians from different
subjects with their prior probabilities incorporated.
1.1. Previous work
For face recognition with image sets, a lot of relevant
approaches have been proposed recently. According to how
to model the image sets, these approaches can be roughly
classified into three categories: linear/affine subspace based
methods, nonlinear manifold based methods and statistical
2048978-1-4673-6964-0/15/$31.00 ©2015 IEEE