FACE HALLUCINATION VIA SEMI-KERNEL PARTIAL LEAST SQUARES
Qiang Zhang, Fei Zhou, Fan Yang and Qingmin Liao,IEEE Member
Shenzhen Key Lab of Information Sci. &Tech., Shenzhen Engineering Lab of IS &DRM,
Department of Electronics Engineering / the Graduate School at Shenzhen, Tsinghua University,
China
zqtsinghua@126.com; flying.zhou@163.com; fyang@sz.tsinghua.edu.cn;
liaoqm@sz.tsinghua.edu.cn
Keywords-face super-resolution (SR); collaborative representation (CR); semi-kernel partial least
squares (semi-KPLS);
Abstract—In this paper, a patch-based super-resolution (SR) method is proposed to hallucinate
facial images. Two steps are involved in this method. In the first step, semi-kernel partial least
squares (semi-KPLS) algorithm is used to generate a nonlinear correlative space. In the second step,
we use collaborative representation (CR) to infer a high-resolution (HR) face. The experiments
conducted on FERET database demonstrate the proposed algorithm can generate better results, in
comparison with some state-of-the-art approaches.
Introduction
The details of facial features are crucial for identifying an individual from surveillance video, but
the resolution of a facial region from surveillance videos is usually very low. Therefore, in order to
obtain detailed facial features, it is necessary to infer a high-resolution (HR) face image from the
low-resolution (LR) one. This technique is called face hallucination or face super-resolution (SR).
According to the available inputs, existing SR methods can roughly be divided into two classes:
multi-frames SR [1-4], and single-frame SR [5-11]. The inspiration of multi-frames SR is to retrieve
high-frequency details from complementary multi-images. This type of method suffers from ill-
conditioned registration and is limited to small increase in image spatial resolution [15]. Instead,
single-frame SR prefers to infer the HR counterpart of a single image by obtaining extra
information from training samples and we mainly focus on this approach in this paper. In [14],
Freeman et al. propose an example-based method using a Markov Random Field (MRF) with belief
propagation, and HR images are generated by building a one-to-one relation between HR and LR
patches in training samples. In [18], based on the principle of locally linear embedding (LLE), HR
patches are constructed by a linear combination of several neighbors. However, most SR methods
are developed for the purpose of super-resolving general images.
For face SR, the utilization of the properties of face is conductive to generate the high-resolution
face images. Baker et al. [19] were the first to develop a hallucination method under a Bayesian
formulation and proposed the term “face hallucination”. In this method, it generates the high
frequency details from a parent structure with the assistance of training samples. This work is
further extended by Su et al. [23], where a steerable pyramid is used to provide larger search scope
for local best match approach. Eigen-transformation is introduced to hallucinating face in [5],
whose conclusion results in the example-based paradigm. That is hallucinated face can be
represented by a linear combination of training samples. In [24], canonical correlation analysis is
employed to maximize the correlation coefficients between LR and HR data. However, subspace
transformations may reduce the discrimination of hallucinated faces. Liu et al. [25] developed a
two-step approach integrating a global parametric model with Gaussian assumption and a local non-
parametric model based on Markov random field (MRF). Zhuang et al. [6] use locality preserving
projection and neighbor embedding to hallucinate a high-resolution face. More recently, the work in
[8] shows that patch-based method can achieve plausible results without the residue compensation.
In fact, the residue compensation is merely indispensable for the methods that include the step of
International Conference on Intelligent Systems Research and Mechatronics Engineering (ISRME 2015)
© 2015. The authors - Published by Atlantis Press