Look at Boundary: A Boundary-Aware Face Alignment Algorithm
Wayne Wu
∗ 1,2
, Chen Qian
2
, Shuo Yang
3
, Quan Wang
2
, Yici Cai
1
, Qiang Zhou
1
1
Tsinghua National Laboratory for Information Science and Technology (TNList),
Department of Computer Science and Technology, Tsinghua University
2
SenseTime Research
3
Amazon Rekognition
1
wwy15@mails.tsinghua.edu.cn
1
caiyc@mail.tsinghua.edu.cn
1
zhouqiang@tsinghua.edu.cn
2
{qianchen, wangquan}@sensetime.com
3
shuoy@amazon.com
Abstract
We present a novel boundary-aware face alignment al-
gorithm by utilising boundary lines as the geometric struc-
ture of a human face to help facial landmark localisation.
Unlike the conventional heatmap based method and regres-
sion based method, our approach derives face landmarks
from boundary lines which remove the ambiguities in the
landmark definition. Three questions are explored and an-
swered by this work: 1. Why using boundary? 2. How to
use boundary? 3. What is the relationship between bound-
ary estimation and landmarks localisation? Our boundary-
aware face alignment algorithm achieves 3.49% mean error
on 300-W Fullset, which outperforms state-of-the-art meth-
ods by a large margin. Our method can also easily integrate
information from other datasets. By utilising boundary in-
formation of 300-W dataset, our method achieves 3.92%
mean error with 0.39% failure rate on COFW dataset, and
1.25% mean error on AFLW-Full dataset. Moreover, we
propose a new dataset WFLW to unify training and testing
across different factors, including poses, expressions, illu-
minations, makeups, occlusions, and blurriness. Dataset
and model will be publicly available at https://wywu.
github.io/projects/LAB/LAB.html
1. Introduction
Face alignment, which refers to facial landmark detec-
tion in this work, serves as a key step for many face appli-
cations, e.g., face recognition [76], face verification [49, 50]
and face frontalisation [21]. The objective of this paper
is to devise an effective face alignment algorithm to han-
dle faces with unconstrained pose variation and occlusion
across multiple datasets and annotation protocols.
∗
This work was done during an internship at SenseTime Research.
!"# !$#
!%#
&''()!*+),-./01#)
234()! 56),-./01#)
748()!96),-./01#)
Figure 1: The first column shows the face images from different
datasets with different number of landmarks. The second column
illustrates the universally defined facial boundaries estimated by
our methods. With the help of boundary information, our approach
achieves high accuracy localisation results across multiple datasets
and annotation protocols, as shown in the third column.
Different to face detection [46] and recognition [76],
face alignment identifies geometry structure of human face
which can be viewed as modeling highly structured out-
put. Each facial landmark is strongly associated with a
well-defined facial boundary, e.g., eyelid and nose bridge.
However, compared to boundaries, facial landmarks are
not so well-defined. Facial landmarks other than corners
can hardly remain the same semantical locations with large
pose variation and occlusion. Besides, different annotation
schemes of existing datasets lead to a different number of
landmarks [29, 5, 67, 31] (19/29/68/194 points) and anno-
tation scheme of future face alignment datasets can hardly
be determined. We believe the reasoning of a unique facial