A Factorization Method for Multiple Perspective Views via
Iterative Depth Estimation
Toshio Ueshiba and Fumiaki Tomita
Electrotechnical Laboratory, Tsukuba, Japan 305-8568
SUMMARY
This paper proposes a factorization method that re-
constructs camera motion and scene shape based on the
matching of multiple images under the condition that the
camera captures a perspective view. Starting from the affine
projection camera model, the projection depth is iteratively
estimated until the measurement matrix has rank 4. Then,
the obtained measurement matrix is factorized to restore the
three-dimensional information of the scene in the projec-
tion space. This approach eliminates noise sensitive proc-
esses, such as the calculation of the fundamental matrix,
that are required in the factorization for the conventional
perspective projection image, and a stable reconstruction is
realized. Furthermore, the metric constraint in the conven-
tional affine model is extended, and the metric constraint in
the perspective projection condition is derived. It is shown
that the reconstruction in Euclidean space is realized if the
internal parameters of the camera are given. © 2000 Scripta
Technica, Syst Comp Jpn, 31(13): 8795, 2000
Key words:
Affine projection; perspective projec-
tion; factorization; reconstruction of three-dimensional in-
formation; metric constraint.
1. Instruction
The problem in which the relative locations of the
cameras and the three-dimensional information of the scene
are simultaneously reconstructed based on multiple images
obtained from various viewpoints is called the structure
from motion and is one of the essential problems in
computer vision. A large number of algorithms for this
problem have been proposed. Among them, the factoriza-
tion method proposed by Tomasi and Kanade [1] is an
excellent method that is simple and highly stable.
Their method is based on the property that the meas-
urement matrix composed of the two-dimensional coordi-
nates of the feature points observed in the image can be
decomposed into the product of two matrices representing
the camera motion and the three-dimensional positions of
the feature points, respectively, under the assumption that
the camera executes the affine projection. Tomasi and
Kanade used orthographic projection as the camera model.
Several extensions of the method were subsequently made
to the weak perspective camera model and the paraperspec-
tive camera model [2].
Recently, several methods were proposed to extend
the factorization method to the case of the perspective
projection camera model [38]. A difficulty in applying the
factorization method to the perspective projection images
is that the structural parameter of the scene called projective
depth is unknown. Thus, the measurement matrix is also
unknown, and the factorization cannot directly be applied.
The crucial point is therefore to determine projective depth
by some means.
Christy and Horaud presented the shape reconstruc-
tion method [3, 4], which starts from the paraperspective
camera model and makes the perspective projection model
approach the measurement matrix by iteratively applying
linear factorization. In this method, a constraint called
metric constraint is handled in each step of the factorization
© 2000 Scripta Technica
Systems and Computers in Japan, Vol. 31, No. 13, 2000
Translated from Denshi Joho Tsushin Gakkai Ronbunshi, Vol. J81-D-I, No. 8, August 1998, pp. 17181726
87