1 Introduction 3
image against an enrollment face image whose identity is being claimed. Person
verification for self-serviced immigration clearance using E-passport is one typical
application.
Face identification involves one-to-many matching that compares a query face
against multiple faces in the enrollment database to associate the identity of the
query face to one of those in the database. In some identification applications, one
just needs to find the most similar face. In a watchlist check or face identification
in surveillance video, the requirement is more than finding most similar faces; a
confidence level threshold is specified and all those faces whose similarity score is
above the threshold are reported.
The performance of a face recognition system largely depends on a variety of
factors such as illumination, facial pose, expression, age span, hair, facial wear, and
motion. Based on these factors, face recognition applications may be divided into
two broad categories in terms of a user’s cooperation: (1) cooperative user scenarios
and (2) noncooperative user scenarios.
The cooperative case is encountered in applications such as computer login,
physical access control, and e-passport, where the user is willing to be coopera-
tive by presenting his/her face in a proper way (for example, in a frontal pose with
neutral expression and eyes open) in order to be granted the access or privilege.
In the noncooperative case, which is typical in surveillance applications, the user
is unaware of being identified. In terms of distance between the face and the camera,
near field face recognition (less than 1 m) for cooperative applications (e.g., access
control) is the least difficult problem, whereas far field noncooperative applications
(e.g., watchlist identification) in surveillance video is the most challenging.
Applications in-between the above two categories can also be foreseen. For ex-
ample, in face-based access control at a distance, the user is willing to be coopera-
tive but he is unable to present the face in a favorable condition with respect to the
camera. This may present challenges to the system even though such cases are still
easier than identifying the identity of the face of a subject who is not cooperative.
However, in almost all of the cases, ambient illumination is the foremost challenge
for most face recognition applications.
1.3 Processing Workflow
Face recognition is a visual pattern recognition problem, where the face, represented
as a three-dimensional object that is subject to varying illumination, pose, expres-
sion, and other factors, needs to be identified based on acquired images. While two-
dimensional face images are commonly used in most applications, certain applica-
tions requiring higher levels of security demand the use of three-dimensional (depth
or range) images or optical images beyond the visual spectrum. A face recognition
system generally consists of four modules as depicted in Fig. 1.2: face localiza-
tion, normalization, feature extraction, and matching. These modules are explained
below.