To appear in the proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures
(AMFG 2003), October 2003, Nice, France.
PCA-Based Face Recognition in Infrared Imagery:
Baseline and Comparative Studies
Xin Chen Patrick J. Flynn Kevin W. Bowyer
Department of Computer Science and Engineering
University of Notre Dame
Notre Dame, IN 46556
{xchen2, flynn, kwb}@nd.edu
Abstract
Techniques for face recognition generally fall into global
and local approaches, with the principal component analy-
sis (PCA) being the most prominent global approach. This
paper uses the PCA algorithm to study the comparison and
combination of infrared and typical visible-light images for
face recognition. This study examines the effects of light-
ing change, facial expression change and passage of time
between the gallery image and probe image. Experimen-
tal results indicate that when there is substantial passage of
time (greater than one week) between the gallery and probe
images, recognition from typical visible-light images may
outperform that from infrared images. Experimental results
also indicate that the combination of the two generally out-
performs either one alone. This is the only study that we
know of to focus on the issue of how passage of time affects
infrared face recognition.
1. Introduction
Although current face recognition systems have achieved
good results for images that are taken in a controlled en-
vironment, they perform poorly in less controlled situa-
tions [1]. Infrared imagery (IR) may offer better perfor-
mance than other modalities due to robustness to environ-
mental effects and deliberate attempts to obscure identity.
The anatomical informationwhich is utilized by IR involves
subsurface features thought to be unique to each person.
Also, IR provides a capability for identification under all
lighting conditions including total darkness [2].
However, face recognition in the thermal domain has re-
ceived relatively little attention in the literature in compari-
son with recognition in visible imagery. This is mainly be-
cause of the lack of widely available IR image databases.
Previous work in this area shows that well-known face
recognition techniques, for example PCA, can be success-
fully applied to IR images, where they perform as well on
IR as on visible imagery [3] or even better on IR than on vis-
ible imagery [4] [5]. However, in all of these studies [3] [4]
[5], the gallery and probe images of a subject were acquired
in the same session, on the same day. In our current study,
we also examine performancewhen there is substantial time
between gallery and probe.
The performance evaluation methodology employs the
concept of a training image set used to develop the iden-
tification technique, a gallery image set that embodies the
set of persons enrolled in the system, and a probe image
set containing images to be identified. We employ a closed
universe assumption, i.e. each probe image will have a cor-
responding match in the gallery. Identification of a probe
image yields a ranked set of matches, with rank 1 being the
best match.
Socolinsky and Selinger [4] [5] used 91 subjects and
the gallery and probe images were acquired within a very
short period of time. These experiments will be called same
session recognition. Experiments in which the probe and
gallery images are acquired on different days or weeks will
be called time-lapse recognition. Socolinsky and Selinger
used a sensor capable of imaging both modalities (visible
and IR) simultaneously through a common aperture. This
enabled them to register the face with reliable visible im-
ages instead of IR images. They emphasized the IR sensor
calibration and their training set is the same as the gallery
set. In their experiments, several face recognition algo-
rithms were tested and the performance using IR appears
to be superior to that using visible imagery.
Wilder et al. [3] used 101 subjects and the images
were acquired without time lapse. They controlled only
for expression change. Several recognition algorithms were
tested and they concluded that the performance is not sig-
nificantly better for one modality than for another.
This study examines more varied conditions and uses
a relatively larger database, in both the number of images
and the number of subjects, compared with the databases
used by Wilder et al. and Socolinsky et al. [3] [4] [5].
We consider the performance of the PCA algorithm in IR,
including the impact of illumination change, facial expres-