积分算子特征值扰动引导的内核选择新方法

23 浏览量更新于2024-08-26 收藏 477KB PDF 举报

积分算子的特征值摄动在内核选择中的应用是一项前沿的研究课题，特别是在基于核方法的现代研究和实践中占据重要地位。本文的主要贡献者是Yong Liu、Shali Jiang和Shizhong Liao，他们来自天津大学计算机科学与技术学院。他们的研究论文探讨了如何通过理解核矩阵（在统计学习中，核函数将数据映射到高维空间的“核技巧”所生成的矩阵）与连续积分算子之间的联系来改进内核选择。通常，内核选择的过程涉及优化泛化误差的估计或其他性能度量。作者指出，核矩阵可以被视为积分算子的样本版本，其特征值在一定条件下会收敛到积分算子的固有值。在这个背景下，他们提出了一种新的内核选择准则，该准则依赖于积分算子的特征值扰动。特征值扰动衡量了核矩阵特征值与积分算子固有值之间的差异，这是评估核函数性能的重要指标。论文深入探讨了特征值扰动与泛化误差之间的关系，即特征值的稳定性和模型的预测能力之间存在关联。作者利用这种联系，通过最小化由特征值扰动导出的泛化误差界限，提出了新的内核选择策略。这意味着，根据他们提出的准则选择的核函数能够保证良好的泛化性能，从而在实际应用中提升模型的预测准确性和稳定性。这项工作不仅提供了理论上的洞察，还提供了一种实用的方法来优化内核选择过程，这对于机器学习特别是支持向量机（SVM）、核主成分分析（KPCA）等算法的性能提升具有重要意义。通过对积分算子的特征值扰动进行有效控制，研究者能够更好地理解并利用核方法的优势，进一步推动了这一领域的发展。

Eigenvalues Perturbation of Integral Operator

for Kernel Selection

Yong Liu Shali Jiang Shizhong Liao

∗

School of Computer Science and Technology

Tianjin University, Tianjin 300072, P. R. China

szliao@tju.edu.cn

ABSTRACT

Kernel selection is one of the key issues both in recent re-

search and application of kernel methods. This is usually

done by minimizing either an estimate of generalization error

or some other related performance measure. It is well known

that a kernel matrix can be interpreted as an empirical ver-

sion of a continuous integral operator, and its eigenvalues

converge to the eigenvalues of integral op erator. In this pa-

per, we introduce new kernel selection criteria based on the

eigenvalues perturbation of the integral operator. This per-

turbation quantiﬁes the diﬀerence between the eigenvalues

of the kernel matrix and those of the integral operator. We

establish the connection between eigenvalues perturbation

and generalization error. By minimizing the derived gener-

alization error bounds, we propose the kernel selection cri-

teria. Therefore the kernel chosen by our proposed criteria

can guarantee good generalization performance. To compute

the values of our criteria, we present a method to obtain the

eigenvalues of integral operator via the Fourier transform.

Experiments on benchmark datasets demonstrate that our

kernel selection criteria are sound and eﬀective.

Categories and Subject Descriptors

I.2.6 [Artiﬁcial Intelligence]: Learning—Parameter Learn-

ing; I.5.2 [Pattern Recognition]: Design Methodology—

Classiﬁer Design and Evaluation; H.2.8 [Database Man-

agement]: Database Applications—Data Mining

General Terms

Algorithms, Theory, Experimentation

Keywords

Kernel Selection, Eigenvalues Perturbation, Integral Opera-

tor, Generalization Error.

∗

corresponding author

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for proﬁt or commercial advantage and that copies bear this notice and the full cita-

tion on the ﬁrst page. Copyrights for components of this work owned by others than

ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-

publish, to post on servers or to redistribute to lists, requires prior speciﬁc permission

and/or a fee. Request permissions from Permissions@acm.org.

CIKM’13, October 27–November 01, 2013, San Francisco, CA, USA

http://dx.doi.org/10.1145/2505515.2505584.

1. INTRODUCTION

Kernel methods [33, 29, 10, 30, 31] have been widely used

in pattern recognition and machine learning. Because the

performance of kernel methods greatly depends on the choice

of the kernel function, the kernel selection becomes an im-

portant topic in kernel methods. A related problem is the

evaluation of the generalization ability of learning algorithm-

s. In fact, it is common to select the optimal kernel function

by choosing the one with the lowest generalization error.

Obviously, the generalization error is not directly com-

putable, as the probability distribution generating the data

is unknown, therefore it is necessary to resort to estimates

of its value. The generalization error can be estimated ei-

ther via theoretical bounds or testing on some unused data

(hold-out testing or cross validation). To estimate the upper

bounds of the generalization error, some complexity mea-

sures are introduced: such as VC dimension [33], Rademach-

er complexity [3], maximal discrepancy [2], regularized risk

[29], radius-margin bound [33] and compression coeﬃcien-

t [24]. However, for most of these complexity measures,

proposed to derive theoretical generalization error bounds,

it is diﬃcult to compute their values [25, 26], which make

them hard to be used for kernel selection in practice. Min-

imizing the empirical estimate of the generalization error is

an alternative to kernel selection. K-fold cross-validation

(KCV) and leave-one-out cross-validation (LOO) [9, 23] are

two popular empirical estimates. Although KCV and LOO

are widely used in many ﬁelds, they have their dark sides:

(a) the overall learning problem may over-ﬁtting the cross-

validation error [6, 7]; (b) high computational cost. For the

sake of eﬃciency, some approximate KCV and LOO criteria

are given: such as generalized cross-validation (GCV)[19],

generalized comparative Kullback-Liebler distance (GCKL)

[34], generalized approximate cross-validation (GACV) [35],

span bound [8, 9] and inﬂuence function [15].

Based on the similarity, Cristianini et al. [14] present a

kernel selection criterion called kernel target alignment (K-

TA). Nguyen and Ho [25, 26] point out several drawbacks

of the KTA, and propose a surrogate measure (called FSM)

to evaluate the goodness of a kernel function via the data

distribution in the feature space. Similar to KTA, Cortes et

al. [12] present a centered kernel target alignment criteri-

on (CKTA) with a centered kernel matrix. Although KTA,

CKTA and FSM are widely used, the connection between

these criteria and generalization error for speciﬁc learning

algorithms has not been established, so the kernels chosen

by these criteria may not guarantee good generalization per-

formance.

2189

下载后可阅读完整内容，剩余9页未读，立即下载

weixin_38691703

粉丝: 2
资源: 961

积分算子特征值扰动引导的内核选择新方法

广义特征值的摄动问题 (1978年)

广义Kato型算子及具有广义(ω)性质算子的摄动 (2012年)

边界与算子双摄动的四阶拟线性椭圆型方程的奇摄动 (2013年)

3类新型Banach空间的Semi-Fredholm算子摄动与G猜想 (2011年)

微分算子含双参数的高阶椭圆型方程的一般边值问题的奇摄动 (1983年)

完全奇异积分方程关于积分曲线摄动的稳定性 (2005年)

通过手性重叠算子对A和A☆进行单环摄动耦合

一类奇摄动积分微分方程边值问题 (2005年)

精细积分法求解奇异摄动边值问题的高效算法

新方法：特征值上界估计——摄动离散Riccati矩阵方程

最新资源