Illumination Invariant Face Recognition Using Thermal Infrared Imagery
∗
Diego A. Socolinsky† Lawrence B. Wolff‡ Joshua D. Neuheisel† Christopher K. Eveland‡
‡Equinox Corporation †Equinox Corporation
9 West 57th Street 207 East Redwood Street
New York, NY 10019 Baltimore, MD 21202
{diego,wolff,jneuheisel,eveland}@equinoxsensors.com
Abstract
A key problem for face recognition has been accurate iden-
tification under variable illumination conditions. Conven-
tional video cameras sense reflected light so that image
grayvalues are a product of both intrinsic skin reflectivity
and external incident illumination, thus obfuscating the in-
trinsic reflectivity of skin. Thermal emission from skin, on
the other hand, is an intrinsic measurement that can be iso-
lated from external illumination. We examine the invari-
ance of Long-Wave InfraRed (LWIR) imagery with respect
to different illumination conditions from the viewpoint of
performance comparisons of two well-known face recogni-
tion algorithms applied to LWIR and visible imagery. We
develop rigourous data collection protocols that formalize
face recognition analysis for computer vision in the thermal
IR.
1 Introduction
The potential for illumination invariant face recognition us-
ing thermal IR imagery has received little attention in the
literature [1, 2]. The current paper quantifies such invari-
ance by direct performance analysis and comparison of face
recognition algorithms between visible and LWIR imagery.
It has often been noted in the literature [3, 2, 4] that vari-
ations in ambient illumination pose a significant challenge
to existing face recognition algorithms. In fact, a variety
of methods for compensating for such variations have been
studied in order to boost recognition performance, including
among others histogram equalization, laplacian transforms,
gabor transforms and logaritmic transforms. All these tech-
niques attempt to reduce the within-class variability intro-
duced by changes in illumination, which severly degrades
classification performance. Since thermal IR imagery is in-
∗
This research was supported by the DARPA Human Identification at a
Distance (HID) program, contract # DARPA/AFOSR F49620-01-C-0008.
dependent of ambient illumination, such problems do not
exist.
To perform our experiments, we have developed a spe-
cial Visible-IR sensor capable of taking simultaneous and
co-registered images with both a visible CCD and a LWIR
microbolometer. This is of particular significance for this
test, since we are testing on exactly the same scenes for
both the visible and IR recognition performance, not a bore-
sighted pair of images.
In order to perform proper invariance analysis, it is nec-
essary that thermal IR imagery be radiometrically cali-
brated. Radiometric calibration achieves a direct relation-
ship between the grayvalue response at a pixel and the ab-
solute amount of thermal emission from the correspond-
ing scene element. This relationship is called responsivity.
Thermal emission is measured as flux in units of power such
as W/cm
2
. The grayvalue response of thermal IR pixels for
LWIR cameras is linear with respect to the amount of inci-
dent thermal radiation. The slope of this responsivity line
is called the gain and the y-intercept is the offset. The gain
and offset for each pixel on a thermal IR focal plane array
is significantly variable across the array. That is, the linear
relationship can be, and usually is, significantly different
from pixel to pixel. This is illustrated in Figure 1 where
both calibrated and uncalibrated images are shown of the
same subject.
While radiometric calibration provides non-uniformity
correction, the relationship back to a phisical property of
the imaged object (its emmisivity) provides the further ad-
vantage of data where environmental factors contribute to a
much lesser degree to within-class variability.
An added bonus of radiometric calibration for thermal
IR is that it simplifies the problem of skin detection in clut-
tered scenes. The range of human body temperature is quite
small, varying from 96
◦
F to 100
◦
F. We have found that skin
temperature at 70
◦
F ambient room temperature to also have
a small variable range from about 79
◦
F to 83
◦
F. Radiomet-
ric calibration makes it possible to perform an initial seg-
1