![](https://csdnimg.cn/release/download_crawler_static/15122847/bg4.jpg)
WU et al.: ROBUST DEPTH-BASED RE-ID 2591
TABLE I
T
ERMS AND DEFINITIONS FOR SECTION III
Fig. 3. Illustration of body self-occlusion and feature extraction region. (a) is
a sample RGB image, (b) is the corresponding point cloud, (c) is the within-
voxel feature extraction region and (d) is the between-voxel feature extraction
region.
A preliminary version of this work appeared in [76]. In this
work, apart from providing more in-depth discussion on the
proposed Eigen-depth feature and the depth-based person
re-identification framework, a kernelized implicit feature trans-
fer scheme is proposed to learn the relation between depth
features and RGB features so as to estimate depth features
in RGB images when depth sensor is not ready. In addition,
more extensive experiments have been conducted.
III. D
EPTH VOXEL COVARIANCE AND
EIGEN-DEPTH FEATURE
This section will present the extraction of depth
voxel covariance descriptor and locally rotation invariant
Eigen-depth feature. Our descriptors are extracted from point
cloud, a set of points on object surface expressed by 3D
coordinate (x, y, z) in real world converted from depth image.
We first tabulate the notations defined in this section in Table I.
A. Basic Feature Extraction
We first extract basic features of point cloud. We assume
another kind of biometric cue, skeleton joints of pedestrian
body, is also available along with depth images (e.g., when
using Kinect). We intend to extract features on the body parts
whose surfaces are more invariant and reliable. As shown in
Figure 3 (a) and (b), due to pose difference, sometimes a part
of limb surface is not observed under self-occlusion, so the
surface shapes of arms and legs contain more noises rather
than valuable information. Therefore, we divide each point
cloud of the whole body by two shoulder joints and two hip
joints and only the points of head and torso are used for feature
extraction, while the four limbs are not.
For each point in the point cloud, a normal vector [77]
is computed as basic feature. The direction of normal vector
describes the shape of a small neighbour region of that point.
For a point x, k nearest neighbourhoods of x are found, and
then the direction on which data is least scattered is computed
by PCA [78] as the unit normal vector direction (n
x
, n
y
, n
z
).
For each point (x, y, z) (unit: mm), a feature vector F(x, y, z)
is composed of the coordinate and the unit normal vector
F(x, y, z) =[x, y, z, n
x
, n
y
, n
z
]
T
. (1)
B. Depth Voxel Covariance Descriptor
To depict the variation of local feature vectors and alleviate
noises, we exploit two types of covariance matrices, namely
within-voxel covariance and between-voxel covariance.
1) Within-Voxel Covariance: We divide a point cloud into
rectangular voxels (e.g., 6 × 2 voxels in our case) with 50%
overlap, and an example is shown in Figure 3 (c). In each
voxel, within-voxel covariance matrix is computed to describe
the shape. For a voxel R
1
,let{f
1,i
}
m
i=1
be the 6-dimensional
feature vectors inside R
1
. Within-voxel covariance matrix C
W1
is then defined as follows:
C
W1
=
1
m − 1
m
i=1
(f
1,i
− µ
1
)(f
1,i
− µ
1
)
T
, (2)
where µ
1
is the mean of the feature vectors of R
1
.
2) Between-Voxel Covariance: While within-voxel covari-
ance describes shape in a voxel, the differences of shapes
between voxels also contain discriminative information. Sim-
ilar to standard covariance, we define a novel between-voxel
covariance to represent the relation between different voxels.
As shown in Figure 3 (d), the point cloud is divided
into 6 × 2 voxels without overlap. Between-voxel covariance
matrices are computed for each pair of 8-adjacent voxels. For
two adjacent voxels R
1
and R
2
,let{f
1,i
}
m
i=1
and {f
2, j
}
n
j=1
be
the 6-dimensional feature vectors inside R
1
and R
2
, respec-
tively. We define the between-voxel covariance matrix C
B1,2
as follows:
C
B1,2
=
1
mn
m
i=1
n
j=1
(f
1,i
− f
2, j
)(f
1,i
− f
2, j
)
T
. (3)
For a depth image, the unification of the within-voxel and
between-voxel covariance matrices of all voxels is called the
depth voxel covariance descriptor (DVCov).
C. Local Rotation Invariance of Eigenvalues
Assume p, q are voxels of a person in depth camera K .
Let {f
K
p,i
}
N
i=1
denote the feature vectors, C
K
Wp
denote the
within-voxel covariance matrix of voxel p and C
K
Bp,q
denote
between-voxel covariance matrix between voxels p and q.
We assume only viewpoint rotation and location change take
place between two different camera views A and B and the
rotation is local so that the body part within voxels p and q
will not become invisible due to self-occlusions. To express
the transformation from camera view A to camera view B,
let R
AB1
denote the rotation transformation matrix of point
coordinate (x, y, z), R
AB2
denote the rotation transforma-
tion matrix of unit normal vector (n
x
, n
y
, n
z
),andf
AB
=
[x
s
, y
s
, z
s
, 0, 0, 0]
T
denote the shift of pedestrian location.