Similarity Metric Learning for Face Recognition
Qiong Cao, Yiming Ying
Department of Computer Science
University of Exeter, UK
{qc218,y.ying}@exeter.ac.uk
Peng Li
Department of Engineering Mathematics
University of Bristol, UK
lileopold@gmail.com
Abstract
Recently, there is a considerable amount of efforts de-
voted to the problem of unconstrained face verification,
where the task is to predict whether pairs of images are
from the same person or not. This problem is challenging
and difficult due to the large variations in face images. In
this paper, we develop a novel regularization framework to
learn similarity metrics for unconstrained face verification.
We formulate its objective function by incorporating the ro-
bustness to the large intra-personal variations and the dis-
criminative power of novel similarity metrics. In addition,
our formulation is a convex optimization problem which
guarantees the existence of its global solution. Experiments
show that our proposed method achieves the state-of-the-art
results on the challenging Labeled Faces in the Wild (LFW)
database [10].
1. Introduction
Face recognition has attracted increasing attentions due
to its applications in biometrics and surveillance. Re-
cently, considerable research efforts are devoted to the un-
constrained face verification problem [8, 17, 18, 20, 23, 24],
the task of which is to predict whether two face images rep-
resent the same person or not. The face images are taken
under unconstrained conditions and show significant varia-
tions in complex background, lighting, pose, and expression
(see e.g. Figure 1). In addition, the evaluation procedure for
face verification typically assumes that the person identities
in the training and test sets are exclusive, requiring the pre-
diction of never-seen-before faces. Both factors make face
verification very challenging.
Similarity metric learning aims to learn an appropriate
distance or similarity measure to compare pairs of exam-
ples. This provides a natural solution for the verification
task. Metric learning [5, 7,22,25,26] usually focuses on the
(squared) Mahalanobis distance defined, for any x, t ∈ R
d
,
by d
M
(x, t) = (x − t)
T
M(x − t), where M is a posi-
tive semi-definite (p.s.d.) matrix. It was observed in [8, 27]
Figure 1: Example images from the Labeled Faces in the
Wild (LFW) database exhibit large intra-personal varia-
tions: each column is a pair of images from the same person.
that directly applying metric learning methods only yields
a modest performance for face verification. This may be
partly because most of such methods deal with the specific
tasks of improving kNN classification, which may be not
necessarily suitable for face verification. Similarity learn-
ing aims to learn the bilinear similarity function [3, 19]
defined by s
M
(x, t) = x
T
Mt or the cosine similarity
CS
M
(x, t) = x
T
Mt
√
x
T
Mx
√
t
T
Mt
[14], which has
successful applications in image searching and face verifi-
cation.
In this paper, we build on previous studies [7, 8, 11, 14,
22, 25, 27] to show the great potential of similarity met-
ric learning methods to boost the verification performance
using low-level feature descriptors such as Scale-Invariant
Feature Transform (SIFT) [8] and Local Binary Pattern
(LBP) [16]. To this end, we develop a novel regulariza-
tion framework to learn similarity metrics for unconstrained
face verification, which is referred to as similarity metric
learning over the intra-personal subspace. We formulate its
objective function by considering both the robustness to the
large intra-personal variations and the discriminative power,
a property that most metric learning methods do not hold.
In addition, our formulation is a convex optimization prob-
lem, and hence a global solution can be efficiently found by
existing algorithms. This is, for instance, not the case for
the current similarity metric learning model [14].
We report experimental results on the Labeled Faces in
the Wild (LFW) [10] dataset, a standard testbed for un-
4321