多模态3D模型检索算法：结构与视觉信息融合

需积分: 16 62 浏览量更新于2024-08-26 1 收藏 2.11MB PDF 举报

随着计算机视觉技术的快速发展，三维模型在虚拟现实、医学手术、地理信息系统等多个领域得到了广泛应用。随着3D模型数据量的爆炸性增长，如何高效地管理和检索这些模型变得至关重要。本文介绍了一种基于多模态三维模型数据的检索算法，旨在解决这一问题。该研究由天津大学电子信息技术工程学院的Anan Liu、Wenhui Li、Weizhi Nie和Yuting Su团队提出，于2017年发表在《神经计算》(Neurocomputing)期刊第259期，176-182页。文章收录于ScienceDirect平台上，该期刊的主页可访问网址为www.elsevier.com/locate/neucom。研究的核心内容首先涉及从每个虚拟3D模型中提取结构信息和视觉信息。这一步骤通过利用先进的图像处理和特征提取技术来获取模型的关键几何特征和纹理特征，以便于后续的分析和比较。结构信息可能包括形状、尺寸、拓扑关系等，而视觉信息则包含色彩、纹理、光照等视觉特征。接下来，论文提出了一个通用的图匹配方法，用于处理不同模态（如结构与视觉）之间的相似度测量。图匹配技术在这个阶段发挥了关键作用，它能够有效地将不同模态下的特征映射到一个共享的空间，以便进行跨模态的相似性评估。这种方法旨在克服单一模态信息的局限性，提高检索的准确性和鲁棒性。最后，论文所述的算法综合考虑了结构信息和视觉信息的融合，通过结合两者的权重或特征组合，设计了一个融合策略，以得出最终的模型相似度评分。这种融合方式有助于提高检索结果的质量，使得即使在单个模态信息不足的情况下，也能通过其他模态的信息来辅助检索。总结来说，这篇研究论文提供了一种创新的3D模型检索方法，通过多模态数据的协同处理，提高了3D模型检索的效率和准确性，为3D数据管理提供了新的解决方案。这对于推动3D模型在更多领域的实际应用具有重要意义，特别是在对精度和速度有高要求的场景下。

Neurocomputing 259 (2017) 176–182

Contents lists available at ScienceDirect

Neurocomputing

journal homepage: www.elsevier.com/locate/neucom

3D models retrieval algorithm based on multimodal data

Anan Liu, Wenhui Li, Weizhi Nie

∗

, Yuting Su

The School of Electronic Information Engineering, Tianjin University, China

a r t i c l e i n f o

Article history:

Received 24 January 2016

Revised 2 June 2016

Accepted 18 June 2016

Available online 14 February 2017

Keywords:

3D model retrieval

Multimodal

Multimodal fusion

a b s t r a c t

With the development of computer vision in recent year, 3D models have been utilized in many appli-

cations, such as virtual reality, me dical surgical, geographic information system. With the growth of 3D

models, it is necessary to develop effective 3D model retrieval methods for data management. In this pa-

per, we proposed a novel algorithm based on multimodal 3D model data to handle model retrieval prob-

lem. First, we extract structure information and visual information from each virtual 3D model. Then, a

universal graph matching is employed to handle similarity measure in different modals respectively. Fi-

nally, a simple statistical model is utilized to handle similarity measure and ﬁnish retrieval process. The

ﬁnal comparing experiments demonstrate the superiority of our approach.

1. Introduction

The rapid development in computer vision has made it more

practicable to make use of the 3D object information. The key of

3D object utilization is 3D object retrieval and recognition, and ef-

fective algorithms for them are increasingly demanded. In recent

years, many algorithms are proposed to handle 3D model retrieval

and recognition problem [19,21] .

Sundar et al . [28] proposed the 3D model retrieval method

based on skeletal information. They encoded the geometric and

topological information in the form of a skeletal graph. Then, graph

matching method is utilized to handle similarity measure. Gao

et al. [10] proposed a 3D model descriptor, Spatial Structure Cir-

cular Descriptor (SSCD) which contains the spatial structure of a

3D model described by 2D projection images. The SSCD can effec-

tively preserve the global spatial structure of 3D models and guar-

antee the accuracy of similarity measure. Liu et al. [20] proposed a

graph-based method for 3D model retrieval. This method used the

grab clustering method for representation view extraction and the

random-walk algorithm is leveraged to update the weight of each

representation view. Then the similarity measurement of two mod-

els was converted into graph matching problem by considering the

view set as a graph model.

However, all of these methods only focus on the structure in-

formation of 3D model, while ignoring the visual information. In

this paper, we proposed a novel method, which can effectively uti-

lize structure and visual information to handle 3D model retrieval

∗

Corresponding author.

E-mail addresses: truman.nie@gmail.com , weizhinie@tju.edu.cn (W. Nie).

problem. First, we extract a set of 2D views of 3D model from dif-

ferent angles. At the same time, K -means is utilized to extract a set

of key points of 3D model from three-dimensional space. Second,

different modals are utilized to construct different graph models

in order to represent the structure and visual information of 3D

model respectively. Finally, the high-order graph matching, and SM

algorithm are applied to compute the similarity of structure graphs

and the similarity between visual graph model respectively, which

are leveraged to get the ﬁnal similarity between different models

and handle retrieval problem.

The contributions of this paper are followed as:

• We proposed an effective 3D retrieval method based on mul-

timodal data containing the visual information and the spatial

information. For the visual information, we use the classic SM

method to leverage the similarity problem. For spatial informa-

tion, we cluster the points of the 3D models and use the loca-

tions of keypoints to represent the model and handle the sim-

ilarity between the different models by using computing the

similarity of the clusters from the different models.

• We proposed a novel high-order graph matching to handle sim-

ilarity between different 3D models in three-dimensional space.

Compared to the traditional two-order graph matching method

which only considers the correlation between pairwise and ig-

nores the spatial information among points which are impor-

tant for matching. So we propose to use tensors to solve the

three-order spatial graph matching problem.

• We successfully utilized multimodal information of 3D model

to guarantee the accuracy of similarity. The ﬁnal comparison

experiments also demonstrate the superiority of the retrieval

framework.

http://dx.doi.org/10.1016/j.neucom.2016.06.087

下载后可阅读完整内容，剩余6页未读，立即下载

weixin_38653691

粉丝: 7
资源: 961

多模态3D模型检索算法：结构与视觉信息融合

非刚性三维模型检索特征提取技术研究.pdf

基于深度学习算法的脑肿瘤CT图像特征分割技术改进.pdf

Cross-Modal-Center-Loss:用于3D跨模态检索的跨模态中心损失

基于matlab网络安全相关的密码学、网络攻防、安全分析等教程 .txt

三维点云场景数据获取及其场景理解关键技术综述_李勇1

深度学习驱动的非刚性三维模型特征提取技术综述

多模态文本处理技术综述

人脸识别算法评估与测试：精确保留评价指标与方法

OpenCV SIFT特征提取的扩展与改进：算法演进与最新进展

【深度解析】：图像识别算法的5大核心原理，专家级教程！

最新资源