感知组织方法：计算机视觉与机器学习的Tensor Voting探讨

tensor

voting

需积分: 9 159 浏览量更新于2024-07-21 1 收藏 6.46MB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

"Tensor Voting: A Perceptual Organization Approach to Computer Vision and Machine Learning" 是一本由 Philippos Mordohai 和 Gérard Medioni 联合撰写的专著，它探讨了计算机视觉和机器学习领域中的一种创新理念——感知组织方法。本书作为 Morgan & Claypool 出版社系列“合成讲座：图像、视频和多媒体处理”中的第8讲，旨在深入理解视觉系统如何通过模拟人类对环境的理解来处理复杂的数据。在计算机视觉领域，Tensor Voting（张量投票）是一种利用多模态数据和多尺度特征来构建局部结构表示的技术。这种方法借鉴了人类大脑中感知物体的方式，通过组合和聚合来自不同视角或特征的局部信息，形成全局的结构理解。这不仅有助于物体识别和场景分析，还对诸如行人检测、目标跟踪、图像分割等任务具有重要意义。该技术的核心思想是通过构建张量（在数学中是高阶数组，代表多个向量的集合），每个张量元素代表一个局部特征点的投票。这些投票可以是深度信息、颜色、纹理或者形状特征。通过计算张量的内在结构，如最大值、平均值或奇异值分解，可以提取出关键的结构信息，帮助系统识别出物体的关键部分及其关系。 Tensor Voting 的优点在于其鲁棒性和适应性，即使在噪声环境中也能保持稳定的表现，因为它是基于局部特征的组合，而不是单一特征的依赖。此外，这种方法还能处理各种尺度的问题，因为张量可以适应不同大小的局部结构。在机器学习方面，Tensor Voting 可以视为一种非监督学习方法，用于特征选择和降维，减少了对大量标注数据的依赖。通过这种方式，系统可以在没有明确标签的情况下，从大量图像数据中自动发现重要的视觉模式。 "Tensor Voting: A Perceptual Organization Approach to Computer Vision and Machine Learning" 是一本富有洞察力的著作，它不仅提供了深入理解计算机视觉中感知组织理论的框架，也展示了这一理论如何在实际应用中转化为强大的算法工具，推动了计算机视觉和机器学习领域的前沿研究。对于那些对视觉感知建模和机器学习算法有兴趣的科研人员和工程师来说，这本书无疑是一本不可或缺的参考资料。

资源详情

资源推荐

P1: IML/FFX P2: IML/FFX QC: IML/FFX T1: IML

MOBK039-01 MOBK039-Median.cls November 9, 2006 21:33

6 TENSOR VOTING: A PERCEPTUAL ORGANIZATION APPROACH

(a) Input data (b) Inferred curve

FIGURE 1.5: Tensor voting is able to infer the correct intrinsic dimensionality of the data, which is

1D, despite the fact that it appears as 2D if observed globally. The correct perceptual structure, a curve,

is inferred without having to examine potential surface hypotheses.

Additional advantages brought about by the representation and voting schemes are noise

robustness and the ability to employ a least-commitment approach. As shown in numerous

experiments and publications [25, 60, 104], tensor voting is robust to very large percentages of

outliers in the data. This is due to the fact that random outliers cast inconsistent votes, which do

not affect the solution signiﬁcantly. This does not hold when there is a systematic source of errors,

as is thecase in manycomputer vision problems. Examplesof such problems are shown in theap-

propriate chapters. Regarding the avoidance ofprematuredecisions, the capability of the second-

order tensor toencompass saliencies for all structure types allowsus not having to decidewhether

a token is an inlier or an outlier before all the necessary information has been accumulated.

Finally, we believe that a model-free, data-driven design is more appropriate for a general

approach since it is easier to generalize to new domains and more ﬂexible in terms of the types

of solutions it can infer. The main assumption in the algorithms described here is smoothness,

which is a very weak and general model. Moreover, the absence of global computations increases

the amount of data that can be processed since computational and storage complexity scale

linearly with the number of tokens, if their density does not increase.

1.3 OUTLINE

The book is organized in three parts:

•

The original tensor voting framework in 2D and 3D and its application to computer

vision problems.

•

Tensor voting in high dimensions with applications to machine learning problems.

•

First-order voting for boundary inference and its application to ﬁgure completion.

P1: IML/FFX P2: IML/FFX QC: IML/FFX T1: IML

MOBK039-01 MOBK039-Median.cls November 9, 2006 21:33

INTRODUCTION 7

We beginby illustrating thetensor voting framework in 2D and 3D.We show how tokens,

which are simple primitives that potentially belong to perceptual structures, can be represented

by second-order, symmetric, nonnegative deﬁnite tensors. Then, we present the mechanism

for information propagation we have designed. The results of this propagation are structural

saliency values for each potential structure type along with the preferred normal and tangent

orientations at each token. Experimental results demonstrate the effectiveness of our approach,

as well as its robustness to noise.

A large part of our research efforts is devoted to the development of a stereo reconstruction

approach, which is presented in Chapter 3. Stereovision can be cast as a perceptual organization

problem under the premise that solutions must comprise coherent structures. These structures

become salient due to the alignment of potential pixel correspondences reconstructed in a 3D

space. Tensor voting is performed to infer the correct matches that are generated by the true

scene surfaces as inliers of smooth perceptual structures. The retained matches are grouped

into smooth surfaces and inconsistent matches are rejected. Disparity hypotheses for pixels

that remain unmatched are generated based on the color information of nearby surfaces and

validated by ensuring the good continuation of the surfaces via tensor voting. Thus, information

is propagated frommore to less reliablepixels considering both geometric and color information.

Arecent,major enhancement of the frameworkisanefﬁcient N D implementation, which

is described in Chapter 4. We present a new implementation of tensor voting that signiﬁcantly

reduces computational time and storage requirements, especially in high-dimensional spaces,

and thus can be applied to machine learning problems, as well as a variety of new domains.

This work is based on the observation that the Gestalt principles still apply in spaces of higher

dimensionality. The computational and storage requirements of the original implementation

prevented its wide application to problems in high dimensions. This is no longer the case with

the new implementation which opens up an array of potential applications mostly in the ﬁeld

of instance-based learning.

Chapter 5 presents our approach to machine learning problems. We address unsuper-

vised manifold learning from observations in high-dimensional spaces using the new efﬁcient

implementation of tensor voting. We are able to estimate local dimensionality and structure,

measure geodesic distances, and perform nonlinear interpolation. We ﬁrst show that we can

obtain reliable dimensionality estimates at each point. Then, we present a quantitative eval-

uation of our results in the estimation of a local manifold structure using synthetic datasets

with known ground truth. We also present results on datasets with varying dimensionality and

intersections under severe noise corruption, which would have been impossible to process with

current state-of-the-art methods. We also address function approximation from samples, which

is an important problem with many applications in machine learning. We propose a noniter-

ative, local, nonparametric approach that can successfully approximate nonlinear functions in

P1: IML/FFX P2: IML/FFX QC: IML/FFX T1: IML

MOBK039-02 MOBK039-Median.cls November 9, 2006 21:34

CHAPTER 2

TENSOR VOTING

The tensor voting framework is an approach for perceptual organization that is able to infer

salient structures based on thesupport thetokens which comprise them receive from their neigh-

bors. It is based on the Gestalt principles of proximity and good continuation and can tolerate

very large numbers of outliers. Data tokens are represented by tensors and their saliency is com-

puted based on informationpropagatedamong neighboring tokensvia tensorvoting.The tokens

can be any type of local primitive, such as points or surfels, that is localized in space and poten-

tially has orientation preferences associated with it, but no dimensions. The framework has been

developed over the past several years beginning in the work of Guy [24] which was followed by

the work of Lee [49] and Tang [103]. Parts of the work presented here can also be found in [64].

A shortcoming of the original framework was its inability to detect terminations of

the inferred perceptual structures. This has been addressed with the addition of ﬁrst order

information to the framework [112]. To avoid confusion we make the distinction between ﬁrst

and second order information throughout, even though the description of the ﬁrst additions

comes a later in Chapter 6. We begin by brieﬂy going over other perceptual organization

approaches and proceed to describe the original, second order formulation of tensor voting in

2-D and 3-D.

2.1 RELATED WORK

Perceptual organization has been an active research area since the beginning of the previous

century based on the work of the Gestalt psychologists [42, 43, 118]. Important issues include

noise robustness, initialization requirements, handling of discontinuities, ﬂexibility in the types

that can be represented, and computational complexity. This section reviews related work which

can be classiﬁed in the following categories. More detailed descriptions can be found in [60, 64]

where work on perceptual organization based on regularization, relaxation labeling, level set

methods, clustering and robust estimation is also presented.

•

symbolic methods

•

methods based on local interactions

•

methods inspired by psychophysiology and neuroscience.

P1: IML/FFX P2: IML/FFX QC: IML/FFX T1: IML

MOBK039-02 MOBK039-Median.cls November 9, 2006 21:34

10 TENSOR VOTING: A PERCEPTUAL ORGANIZATION APPROACH

Symbolic Methods. Following the paradigm set by Marr [58], many researchers developed

methods for hierarchical grouping of symbolic data. Lowe [56] developed a system for 3-D

object recognition based on perceptual organization of image edgels. Groupings are selected

among the numerous possibilities according to the Gestalt principles, viewpoint invariance and

low likelihood of being accidental formations. Later, Mohan and Nevatia [63] and Dolan and

Riseman [17] also proposed perceptual organization approaches based on the Gestalt principles.

Both are symbolic and operate in a hierarchical bottom-up fashion starting from edgels and

increasing the level of abstraction at each iteration. The latter approach aims at inferring curvi-

linear structures, while the former aims at segmentation and extraction of3-D scene descriptions

from collations of features that have high likelihood of being projections of scene objects. Along

the same lines is Jacobs’ [34] technique for inferring salient convex groups among clutter since

they most likely correspond to world objects. The criteria to determine the non-accidentalness

of the potential structures are convexity, proximity and the contrast of the edgels.

Methods Based on Local Interactions. Shashua and Ullman [99] ﬁrst addressed the issue of struc-

tural saliency and how prominent curves are formed from tokens that are not salient in isolation.

They deﬁne a locally connected network that assigns a saliency value to every image location

according to the length and smoothness of curvature of curves going through that location. In

[79], Parent and Zucker infer trace points and their curvature based on spatial integration of

local information. An important aspect of this method is its robustness to noise. This work was

extended to surface inference in three dimensions by Sander and Zucker [86]. Sarkar and Boyer

[89] employ a voting scheme to detect a hierarchy of tokens. Voting in parameter space has to

be performed separately for each type of structure, thus making the computational complexity

prohibitive for generalization to 3-D. The inability of previous techniques to simultaneously

handle surfaces, curves and junctions was addressed in the precursor of our research, the work

of Guy and Medioni [25, 26]. A uniﬁed framework where all types of perceptual structures

can be represented is proposed along with a preliminary version of the voting scheme presented

here. The major advantages of [25, 26] are noise robustness and computational efﬁciency, since

it is not iterative. How this methodology evolved is presented in the remaining sections of this

chapter.

Methods Inspired by Psychophysiology and Neuroscience. Finally, there is an important class of

perceptual organization methods that are inspired by human perception and research in psy-

chophysiology and neuroscience. Grossberg and Mingolla [22] and Grossberg and Todor-

ovic [23] developed the Boundary Contour System and the Feature Contour System that can

group fragmented and even illusory edges to form closed boundaries and regions by feature

cooperation in a neural network. Heitger and von der Heydt [29], in a classic paper on neural

剩余135页未读，继续阅读

lligen-dai

粉丝: 40
资源: 14

感知组织方法：计算机视觉与机器学习的Tensor Voting探讨

tensor voting理论详解（含有伪代码实现）

2017 Machine Learning for OpenCV Intelligent image processing with Python

Visual Knowledge Discovery and Machine Learning-Springer(2018).pdf

Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 586–595 (2018)

att&ck 知识库(企业)中文版

Charbonnier loss

Baseline: On-the-fly stylization

if train_opt.get('perceptual_opt'): percep_type = train_opt['perceptual_opt'].pop('type') cri_perceptual_cls = getattr(loss_module, percep_type) self.cri_perceptual = cri_perceptual_cls( **train_opt['perceptual_opt']).to(self.device) else: self.cri_perceptual = None代码中文含义

bottom-up inference

GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection

perceptual loss 没有梯度

Perceptual GAN的框架

python pesq

颜色衰减先验去雾代码matlab

Perceptual loss

Perceptual Loss pytorch

最新资源