
A Morphable Face Albedo Model
William A. P. Smith
1
Alassane Seck
2,3
Hannah Dee
3
Bernard Tiddeman
3
Joshua Tenenbaum
4
Bernhard Egger
4
1
University of York, UK
2
ARM Ltd, UK
3
Aberystwyth University, UK
4
MIT - BCS, CSAIL & CBMM, USA
william.smith@york.ac.uk, alou.kces@live.co.uk, {hmd1,bpt}@aber.ac.uk, {jbt,egger}@mit.edu
Statistical Diffuse Albedo Model Statistical Specular Albedo Model Combined Model Rendering
1 2 3
+
–
Mean
1 2 3
+
–
Mean
Mean
1 2 3
+
–
Figure 1: First 3 principal components of our statistical diffuse (left) and specular (middle) albedo models. Both are visualised
in linear sRGB space. Right: rendering of the combined model under frontal illumination in nonlinear sRGB space.
Abstract
In this paper, we bring together two divergent strands of
research: photometric face capture and statistical 3D face
appearance modelling. We propose a novel lightstage cap-
ture and processing pipeline for acquiring ear-to-ear, truly
intrinsic diffuse and specular albedo maps that fully factor
out the effects of illumination, camera and geometry. Using
this pipeline, we capture a dataset of 50 scans and com-
bine them with the only existing publicly available albedo
dataset (3DRFE) of 23 scans. This allows us to build the
first morphable face albedo model. We believe this is the
first statistical analysis of the variability of facial specular
albedo maps. This model can be used as a plug in replace-
ment for the texture model of the Basel Face Model and we
make our new albedo model publicly available. We ensure
careful spectral calibration such that our model is built in a
linear sRGB space, suitable for inverse rendering of images
taken by typical cameras. We demonstrate our model in a
state of the art analysis-by-synthesis 3DMM fitting pipeline,
are the first to integrate specular map estimation and out-
perform the Basel Face Model in albedo reconstruction.
1. Introduction
3D Morphable Models (3DMMs) were proposed over 20
years ago [4] as a dense statistical model of 3D face geom-
etry and texture. They can be used as a generative model of
2D face appearance by combining shape and texture param-
eters with illumination and camera parameters that are pro-
vided as input to a graphics renderer. Using such a model
in an analysis-by-synthesis framework allows a principled
disentangling of the contributing factors of face appearance
in an image. More recently, 3DMMs and differentiable ren-
derers have been used as model-based decoders to train con-
volutional neural networks (CNNs) to regress 3DMM pa-
rameters directly from a single image [29].
The ability of these methods to disentangle intrinsic (ge-
ometry and reflectance) from extrinsic (illumination and
camera) parameters relies upon the 3DMM capturing only
intrinsic parameters, with geometry and reflectance mod-
elled independently. 3DMMs are usually built from cap-
tured data [4, 22, 5, 7]. This necessitates a face capture
setup in which not only 3D geometry but also intrinsic face
reflectance properties, e.g. diffuse albedo, can be measured.
A recent large scale survey of 3DMMs [10] identified a lack
of intrinsic face appearance datasets as a critical limiting
factor in advancing the state-of-the-art. Existing 3DMMs
are built using ill-defined “textures” that bake in shading,
shadowing, specularities, light source colour, camera spec-
tral sensitivity and colour transformations. Capturing truly
intrinsic face appearance parameters is a well studied prob-
lem in graphics but this work has been done largely inde-
pendently of the computer vision and 3DMM communities.
In this paper we present a novel capture setup and pro-
cessing pipeline for measuring ear-to-ear diffuse and spec-
ular albedo maps. We use a lightstage to capture multiple
photometric views of a face. We compute geometry using
uncalibrated multiview stereo, warp a template to the raw
5011