ular to the line defined by the viewpoints O and O
c
. The angle
u
is
called the view angle of m. The image formation process can be
explicitly expressed as follows [39]:
km ¼ K ðX
s
þð0; 0; nÞ
T
Þð2Þ
K ¼
rf s u
0
0 f
v
0
001
0
B
@
1
C
A
; X
s
¼
RX þ t
kRX þ tk
;
k ¼
n þ
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
n
2
m
T
K
T
K
1
mðn
2
1Þ
q
m
T
K
T
K
1
m
ð3Þ
where (R,t), called the extrinsic parameters, is the rotation and
translation which relates the world coordinate system to the view
sphere coordination system O xyz, and K is the camera intrinsic
matrix, with f the focal length, r the aspect ratio, s the skew factor
and p =(u
0
,
v
0
,1)
T
is defined as the homogenous coordination of
the principal point. n is usually called as the mirror parameter,
which is the distance from O to O
c
.Ifn = 1, the used mirror is a
paraboloid (i.e. the camera is paracatadioptric). If 0 < n < 1, the mir-
ror is an ellipsoid or a hyperboloid (i.e. the camera is hypercatadiop-
tric). Since the calibration of a catadioptric camera with n = 0 is the
same with that of a pinhole camera, in this work we only concen-
trate on the calibration of central catadioptric camera with
0<n 6 1.
For the revolution conic section mirror, the mirror parameter n
satisfies [35,1]
n ¼
2
e
1 þ
e
2
ð4Þ
where
e
is the eccentricity of the conic. The relationship between
eccentricity
e
and the mirror parameter n for different types of cen-
tral catadioptric cameras is shown in Table 1 [35]. Generally speak-
ing, the mirror parameter n can be computed easily with the
provided eccentricity
e
from manufactures [1,15,12]. Therefore, n
is assumed to be known in this paper.
For a central catadioptric camera, the principal point can be eas-
ily calibrated using the center of the bounding ellipse [13,35,12] or
line images [32,39,38], and then the origin of the image can be
translated to p by a linear transformation:
T
p
¼
10u
0
01
v
0
00 1
0
B
@
1
C
A
ð5Þ
Hence, the image coordination m can be translated to
~
m by
~
m ¼ T
p
m ¼ðu u
0
;
v
v
0
; 1Þ
T
, and p is translated to
~
p ¼ð0; 0; 1Þ
T
.
Denote
e
K as
e
K ¼
rf s 0
0 f 0
001
0
B
@
1
C
A
ð6Þ
The image formation process (2) can be described by
k
~
m ¼
e
KðX
s
þð0; 0; nÞ
T
Þð7Þ
The matrix
e
K
T
e
K
1
is in the following form:
k
1
k
2
=20
k
2
=2 k
3
0
001
0
B
@
1
C
A
ð8Þ
where
k
1
¼ 1=ðr
2
f
2
Þ; k
2
¼2s=ðr
2
f
3
Þ; k
3
¼ðr
2
f
2
þ s
2
Þ=ðr
2
f
4
Þð9Þ
In this paper, we assume the principal point of catadioptric
camera is precalibrated using the center of the bounding ellipse
like in [39]. Hereafter, denote u , u u
0
;
v
,
v
v
0
; m ,
~
m and
p ,
~
p for simplicity.
2.3. Projective reconstruction by factorization
Suppose there are N perspective cameras P
i
, i =1,..., N and M
3D points X
j
=(X
j
, Y
j
, Z
j
,1)
T
, j =1,..., M. The image coordinates
are represented by m
ij
=(u
ij
,
v
ij
,1)
T
. The image formation process
can be described as follows:
k
ij
m
ij
¼ P
i
X
j
ð10Þ
where k
ij
is a non-zero scale factor, commonly called as the projec-
tive depth.
We stack Eq. (10) of all the cameras into a matrix W
3NM
, which
can be factorized as follows:
k
11
m
11
k
12
m
12
... k
1M
m
1M
k
21
m
21
k
22
m
22
... k
2M
m
2M
... ... ... ...
k
N1
m
N1
k
N2
m
N2
... k
NM
m
NM
0
B
B
B
@
1
C
C
C
A
|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}
W
3NM
¼
P
1
...
P
N
0
B
@
1
C
A
|fflfflfflffl{zfflfflfflffl}
M
3N4
ðX
1
; ...; X
M
Þ
|fflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflffl}
S
4M
ð11Þ
W
3NM
¼ M
3N4
S
4M
ð12Þ
where W is the scaled measurement matrix, M
3N4
¼
P
T
1
; P
T
2
; ...; P
T
N
T
is the projective matrix, and S
4M
=(X
1
, ...,X
M
)is
the projective shape matrix. We use Sturm and Triggs’ method
[28] for the computation of {k
ij
}, and recovery of projective
reconstruction fP
i
g
N
i¼1
by factorizing W. Obviously, the factorization
in (11) is a projective reconstruction up to a 4 4 homography
matrix H
44
. That is, W can also be factorized as W ¼
ðM
3N4
H
44
Þ H
1
44
S
4M
. In order to get the Euclidean reconstruc-
tion, metric constraint should be used to recover the 4 4 homog-
raphy matrix H [16].
3. Calibration method
In this paper, we assume the principal point of catadioptric
camera is precalibrated using the center of the bounding ellipse
Fig. 1. Image formation of a central catadioptric camera.
Table 1
The relationship between eccentricity
e
and mirror parameter n.
Ellipsoidal Paraboloidal Hyperboloidal Planar
e
0<
e
<1
e
=1
e
>1
e
? 1
n 0<n <1 n =1 0<n <1 n =0
X. Deng et al. / Computer Vision and Image Understanding 116 (2012) 715–729
717