I. Ihrke et al. / Transparent and Specular Object Reconstruction 2405
Figure 4: The principle of shape-from-distortion based measurements. (a) The setup consists of a single or a number of
patterns in fixed world positions. The pattern is illuminating a specular object diffusely while a camera takes images. (b)
Captured example images. (c) The observed patterns encode one world position for every pixel of the camera. (d) From this
information depth and normal can be extracted. (e) A resulting surface scan [TLGS05].
object, resulting in the depth-normal ambiguity mentioned
before. The principle of the experimental setup for shape-
from-distortion approaches is shown in Figure 4.
2.1.1. Calibrated patterns
One of the earliest approaches of specular surface acquisi-
tion based on shape-from-distortion was proposed by Schultz
[Sch94]. The pattern is assumed to be known and consists of
a partial radiance map of the sky-sphere. The author develops
an algorithm based on information propagation from known
seed points. Reflections on refractive and mirror-like sur-
faces are simulated from four viewpoints and the algorithm
is evaluated on synthetic data.
Halstead et al. [HBKM96] present a shape-from-distortion
approach for the measurement of the human cornea. They
describe a one-view setup where the camera is placed at
the tip of a conically shaped pattern. By observing the re-
flections in the human eye, and employing an inverse ray-
tracing approach the authors reconstruct 3D surface models
of the human eye. The reconstruction approach is iterative
and performs normal fitting using a spline representation of
the surface followed by a refinement step.
Bonfort and Sturm [BS03] develop a multi-view technique
based on specularly reflected observations of a calibrated
world pattern. The method is voxel-based and is similar to
space carving techniques [KS00]. The algorithm first com-
putes a normal for every voxel in every view. This is possible
because of the depth-normal ambiguity in the one-view case
where only one reflected world point is measured. In the
second phase the algorithm determines the object surface by
voxel colouring, the voxels with the most consistent normals
for different views are considered to be surface voxels.
Nehab et al. [NWR08] also define a consistency measure
for normal directions. They use this measure to replace the
matching cost in standard stereo algorithms, an approach in-
troduced by Sanderson et al. [SWN88]. However, because
the matching lacks robustness, the authors anisotropically
diffuse the error landscape to remove spurious mismatches.
Additionally, an analysis of reconstruction ambiguities is pre-
sented.
Tarini et al. [TLGS05] present a one-view approach where
different patterns at the same world location are used to com-
pute pixel to world plane correspondences with sub-pixel ac-
curacy. The patterns are generated using a computer monitor.
Since the monitor is placed in close proximity of the object
the inherent depth-normal ambiguity has to be considered.
The authors resolve it using an iterative approach. An initial
guess for the depth value is propagated and corresponding
normals are computed. The normal field is then integrated to
obtain an updated depth estimate from which updated nor-
mals are computed. The process is iterated until the surface
shape converges. The approach is unique in that it includes
attenuation of the reflected pattern as is, for example the
case in coloured metals like copper and gold. An overview
of the approach which exemplifies the shape-from-distortion
framework is shown in Figure 4.
2.1.2. Theoretical analysis
A theoretical analysis of shape-from-distortion for specular
surfaces has been presented by Oren and Nayar [ON96] and
Savarese et al. [SP01, SP02, SCP05].
Oren and Nayar [ON96] consider specular surface recon-
struction in a structure-from-motion [HZ00] setting. The ap-
parent motion of features in the image plane of a moving
camera is analysed. The authors develop a classification be-
tween ‘real’ features, that is, world points not reflected by
a specular object and ‘virtual’ features, that is, features in-
fluenced by specular reflection. The theory is based on en-
velopes of reflected rays, that is, caustic curves. It is shown
that in the case of co-planar camera movement with respect
to the surface, a profile can be computed from just two spec-
ularly reflected features. For 3D profiles, tracking of a single
specular feature from the occluding boundary of the object is
c
2010 The Authors
Computer Graphics Forum
c
2010 The Eurographics Association and Blackwell Publishing Ltd.