AN ADAPTIVE GROUP LASSO BASED MULTI-LABEL REGRESSION APPROACH
FOR FACIAL EXPRESSION ANALYSIS
Kaili Zhao, Honggang Zhang, Jun Guo
Beijing University of Posts and Telecommunications, China
ABSTRACT
In the realm of facial expression analysis, numerous attempt-
s have been made to link each facial picture to one affective
category. Nevertheless, in our daily life, few of the facial ex-
pressions are exactly one of the predefined affective states.
Therefore, to analyze the facial expressions more effectively,
this paper proposes an Adaptive Group Lasso based Multi-
label Regression approach, which depicts each facial expres-
sion with multiple continuous values of predefined affective
states. Adaptive Group Lasso is adopted to depict the rela-
tionship between different labels which different facial ex-
pressions share some same affective facial areas (patches).
Moreover, to solve the multi-label regression problem, a con-
vex optimization formulation is presented, which would guar-
antee a global optimal solution. The experiment results based
on JAFFE dataset have verified the superior performance of
our approach.
Index Terms— Facial Expression Analysis, Multi-label
Regression, Adaptive Group Lasso
1. INTRODUCTION
Nowadays, facial expression analysis techniques are becom-
ing increasingly important in our daily life. Most previous at-
tempts depict each facial picture with one of the predefined af-
fective labels, such as the 6 basic affective states of happiness,
sadness, surprise, fear, anger and disgust [1, 2]. Numerous
recently researches have been focused on feature extraction
methods [3, 4] or classifiers [5, 6] and satisfied single-label
recognition accuracy has already been obtained.
However, one common concern of the previous approach-
es is the assumption that each facial picture is linked to only
one affective label, which tends to be over simplified. As we
know, one can produce complex expressions. In 1971, Ekman
and Friesen [7] describe which blends of the basic emotions
occur and what these blends look like universally. For in-
stance, when one gets an unexpected award, such as the ICIP
Best Student Paper, he would be both surprised and happy.
JAFFE dataset [8], which is one the most widely used dataset
in the realm of facial expression analysis, is originally de-
scribed via multi-label, that is the continuous values of the
6 basic affective states, which represents the mean semantic
ratings of each picture by 60 Japanese female students. Mean-
while, no pictures in the dataset can be lossless described by
one of the predefined affective states
Therefore, to depict the facial expression more effective-
ly, this paper proposes an Adaptive Group Lasso (AGL) based
multi-label regression approach for facial expression analysis.
With each picture describing by continuous values of all the
6 affective states, we obtain a far more accurate way to repre-
sent facial expressions. AGL is adopted to depict the relation-
ship between different labels: different affective states share
nearly the same facial affective areas, which is the facial parts
to express emotions. In the past decades, there are many stud-
ies focusing on facial expression analysis with Action Units
which depicts facial muscle distributions [2, 9]. This makes
us consider an interesting problem how to utilize facial affec-
tive areas to analyze facial expressions. For example, though
happiness and sadness are totally different affective states, the
same areas of the face, such as mouth, eyebrow and jaw, are
the facial parts to express both emotions. Thus, by placing
the weights, which are extracted from the same position for
different labels, into the same group, the AGL regularization
term in the objective function would encourage them to be ze-
ro or non-zero simultaneously. Therefore, the nonzero-weight
areas of different labels would turn to be the same. Mean-
while, by solving the multi-label regression problem via a
convex optimization formulation, our AGL method enjoys a
global optimal solution during the training process.
To evaluate the performance of our multi-label regression
algorithm, we adopt the original multi-label data of JAFFE
and compare our algorithm against the start-of-the-art algo-
rithm of Support Vector Regeression (SVR), as well as the
baseline algorithm of Linear Regression (LR). According to
the evaluation criterions of Sum Absolute Error (SAE) and
Sum Squared Error (SSE), our algorithm obtains superior per-
formance.
The remainder of the paper is organized as follows. To
start with, Section 2 illustrates that facial expressions recog-
nition should be modeled as a multi-label regression problem.
In Section 3, AGL and its relationship with facial expression
analysis is described. Then, Section 4 introduces our multi-
label regression algorithm based on AGL. In Section 5, the
experiment results are presented and Section 6 draws the con-
clusion.