Cross-Domain 3D Model Retrieval via Visual Domain Adaptation
Anan Liu, Shu Xiang, Wenhui Li
∗
, Weizhi Nie and Yuting Su
School of Electrical and Information Engineering, Tianjin University, China
liwenhui@tju.edu.cn
Abstract
Recent advances in 3D capturing devices and 3D
modeling software have led to extensive and di-
verse 3D datasets, which usually have different dis-
tributions. Cross-domain 3D model retrieval is be-
coming an important but challenging task. How-
ever, existing works mainly focus on 3D model
retrieval in a closed dataset, which seriously con-
strain their implementation for real applications. To
address this problem, we propose a novel cross-
domain 3D model retrieval method by visual do-
main adaptation. This method can inherit the ad-
vantage of deep learning to learn multi-view vi-
sual features in the data-driven manner for 3D
model representation. Moreover, it can reduce
the domain divergence by exploiting both domain-
shared and domain-specific features of different do-
mains. Consequently, it can augment the discrimi-
nation of visual descriptors for cross-domain simi-
larity measure. Extensive experiments on two pop-
ular datasets, under three designed cross-domain
scenarios, demonstrate the superiority and effec-
tiveness of the proposed method by comparing
against the state-of-the-art methods. Especially, the
proposed method can significantly outperform the
most recent method for cross-domain 3D model re-
trieval and the champion of Shrec’16 Large-Scale
3D Shape Retrieval from ShapeNet Core55.
1 Introduction
The rapid development of 3D techniques for modeling, re-
construction, printing has led to huge deluge of 3D content.
3D model retrieval is becoming mandatory in diverse do-
mains, such as e-business, digital entertainment, medical di-
agnosis and education
[
Liu et al., 2017; Cheng et al., 2017;
Nie et al., 2016; Tang et al., 2017; He et al., 2017
]
. Espe-
cially, effective methods for cross-domain 3D model retrieval
play an important role on real applications in virtual and aug-
mented reality, shape completion and scene synthesis. It has
become one of the hot research topics in both computer vision
and machine learning.
∗
Corresponding author
1.1 Motivations
3D model retrieval aims to search the relevant candidates
from the assigned dataset given a query 3D model. Although
much work has been done for 3D model retrieval, there still
exist two critical problems:
1) How to make good use of the current small-scale 3D
model datasets to augment the generalization of algo-
rithms. Compared with millions of 2D image datasets, e.g.
ImageNet and MSCOCO, the current 3D model datasets only
contain limited samples. The most recent 3D datasets, such as
ModelNet40
[
Wu et al., 2015
]
and ShapeNetCore55
[
Chang
et al., 2015
]
, only contain 12311 and 51300 models, respec-
tively. Although the deep learning methods, e.g. Multi-
View Convolutional Neural Network (MVCNN)
[
Su et al.,
2015
]
which won the first prize of Shrec’16 Large-Scale 3D
Shape Retrieval from ShapeNet Core55, achieved significant
improvement for the task under the identical-domain sce-
nario, they cannot work well for the real applications when
the source and target come from different domains. Theo-
retically, deep learning is highly dependent on big data. In
essence, the current deep learning methods can be regarded as
overfitting with respect to individual datasets. Therefore, it is
necessary to develop sophisticated methods of visual domain
adaptation to integrate these small-scale datasets and improve
the generalization of 3D model retrieval methods.
2) How to retrieve 3D models from different datasets with
diverse data distributions. In the past few years, multiple
3D modeling devices have been widely applied in human life.
Diverse 3D datasets have been released for the research on
3D understanding. For example, there are multiple RGB-D
data captured in real world by depth sensors, e.g. Microsoft
Kinect, Intel RealSense. Meanwhile, there are many new 3D
datasets, such as ShapeNet and 3D warehouse, which con-
sist of 3D CAD models. For the real applications, the target
and source 3D models usually come from different datasets,
even different modalities. 3D models may have different vi-
sual and structural information even though they belong to the
same category. However, there are limited works to address
the challenging task of cross-domain 3D model retrieval.
To handle the problems mentioned above, we propose a
novel cross-domain 3D model retrieval method via visual do-
main adaptation as shown in Fig. 1. First, MVCNN is uti-
lized to extract the visual features for the multi-view images
of each 3D model. Then, visual domain adaptation is imple-
Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18)
828