改进的多核K-means聚类：矩阵诱导正则化提升性能

55 浏览量更新于2024-08-27 收藏 714KB PDF 举报

矩阵诱导正则化的多核k均值聚类（MultipleKernelk-Means Clustering with Matrix-Induced Regularization）是一项针对多核学习背景下提高聚类性能的研究方法。传统多核k-means (MKKM)算法试图通过结合一组预定义的内核来优化聚类效果。然而，现有算法在处理这些内核之间的关联性时存在不足，往往可能导致选择冗余内核，这不仅降低了信息源的多样性，而且最终对聚类结果产生负面影响。本文的核心贡献在于提出了一种创新的MKKM聚类算法，引入了矩阵诱导正则化策略。这种正则化技术旨在减少内核间的冗余，并增强所选内核的多样性。通过理论分析，作者揭示了这种矩阵诱导正则化的内在联系，即它与“正则化”的经典概念相契合，有助于防止过度拟合，促进模型的泛化能力。矩阵诱导正则化的具体实现包括以下步骤： 1. 定义一个目标函数，该函数同时考虑了数据的多模态表示以及内核之间的相关性约束。 2. 通过引入正则化项，限制了内核权重向量中的元素大小，避免过高的权重集中在几个相似内核上，从而鼓励算法探索更多元化的内核组合。 3. 使用优化算法（如梯度下降或EM算法的变体）迭代求解最优内核权重，使得每个类的样本点在其对应类中心的多核空间中尽可能紧密，同时保持内核间的多样性。这种正则化策略的优势在于： - 提升聚类的稳健性：通过减少内核之间的冗余，提高了模型对噪声和异常值的抵抗能力。 - 增强泛化能力：利用多种内核的互补特性，使得算法在未知数据上的表现更加出色。 - 更好地利用信息源：利用矩阵诱导正则化的多样性促进，能够挖掘出更丰富的特征组合，进而提升聚类的准确性和效率。本文的研究对于改进多核k-means聚类方法具有重要意义，它提供了一种有效的手段来克服内核选择中的问题，从而在实际应用中提升聚类任务的性能。未来可能的研究方向包括扩展到非线性模型、自动选择内核及适应性调整正则化参数等。

Multiple Kernel k-Means Clustering with Matrix-Induced Regularization

Xinwang Liu, Yong Dou, Jianping Yin

School of Computer

National University

of Defense Technology

Changsha, China, 410073

Lei Wang

School of Computer Science

and Software Engineering

University of Wollongong

NSW, Australia, 2522

En Zhu

School of Computer

National University

of Defense Technology

Changsha, China, 410073

Abstract

Multiple kernel k-means (MKKM) clustering aims to opti-

mally combine a group of pre-speciﬁed kernels to improve

clustering performance. However, we observe that existing

MKKM algorithms do not sufﬁciently consider the corre-

lation among these kernels. This could result in selecting

mutually redundant kernels and affect the diversity of in-

formation sources utilized for clustering, which ﬁnally hurts

the clustering performance. To address this issue, this pa-

per proposes an MKKM clustering with a novel, effective

matrix-induced regularization to reduce such redundancy and

enhance the diversity of the selected kernels. We theoreti-

cally justify this matrix-induced regularization by revealing

its connection with the commonly used kernel alignment cri-

terion. Furthermore, this justiﬁcation shows that maximiz-

ing the kernel alignment for clustering can be viewed as a

special case of our approach and indicates the extendability

of the proposed matrix-induced regularization for designing

better clustering algorithms. As experimentally demonstrated

on ﬁve challenging MKL benchmark data sets, our algorithm

signiﬁcantly improves existing MKKM and consistently out-

performs the state-of-the-art ones in the literature, verifying

the effectiveness and advantages of incorporating the pro-

posed matrix-induced regularization.

Introduction

Clustering algorithms aim to partition a group of samples

into k clusters, where the similarity of samples from intra-

clusters shall be greater than that from inter-clusters (Har-

tigan 1975). As one of the classical clustering algorithms,

k-means provides an intuitive and effective way to perform

clustering. In speciﬁc, the k-means clustering is composed

of (i) calculating k prototypes (i.e., centres of k clusters)

given an assignment of samples to clusters and (ii) updating

the assignment matrix by minimizing the sum-of-squares

cost given the prototypes. These two steps are alternately

performed until convergence. Due to its conceptual simplic-

ity, easy-implementation and high efﬁciency, k-means clus-

tering has been intensively studied and extended (Yu et al.

2012; G

onen and Margolin 2014; Cai, Nie, and Huang 2013;

Du et al. 2015). As an important extension, kernel k-means

ﬁrst maps data onto a high-dimensional space through a fea-

 2016, Association for the Advancement of Artiﬁcial

ture mapping and then conducts a standard k-means cluster-

ing in that space (Sch

olkopf, Smola, and M

uller 1998). This

enables kernel k-means to handle the linearly non-separable

problem in an input space that k-means suffers from.

In many practical applications of clustering, samples are

represented by multiple groups of features extracted from

different information sources. For example, three kinds of

feature representations

, colour, shape and texture, are ex-

tracted to distinguish one ﬂower from another (Nilsback and

Zisserman 2006). These different sources usually provide

complementary information, and it is better to let learning

algorithms optimally combine them in order to obtain excel-

lent clustering. This line of research is known as multiple

kernel (view) clustering in the literature.

Many efforts have been devoted to improving multiple

kernel clustering from all kinds of aspects (Zhao, Kwok, and

Zhang 2009; Lu et al. 2014; Xia et al. 2014; Zhou et al. 2015;

Kumar and Daum

e 2011). In this paper, we explore a bet-

ter way to combine a set of pre-speciﬁed kernels for clus-

tering. The existing research on this aspect can roughly be

grouped into two categories. The ﬁrst category learns a con-

sensus matrix via low-rank optimization (Xia et al. 2014;

Zhou et al. 2015; Kumar and Daum

e 2011). In (Xia et al.

2014), a transition probability matrix is constructed from

each single view, and they are used to recover a shared

low-rank transition probability matrix as a crucial input to

the standard Markov chain method for clustering. The work

in (Zhou et al. 2015) proposes to capture the structures of

noises in each kernel and integrate them into a robust and

consensus framework to learn a low-rank matrix. The al-

gorithm (Kumar and Daum

e 2011) learns the clustering in

one view and uses it to “label” the data in other views to

modify a similarity matrix. By following multiple kernel

learning (MKL) framework, the other category optimizes a

group of kernel coefﬁcients, and uses the combined kernel

for clustering (Yu et al. 2012; G

onen and Margolin 2014;

Du et al. 2015; Lu et al. 2014). The work in (Yu et al. 2012)

proposes a multiple kernel k-means clustering algorithm.

Similar work has also been done in (G

onen and Margolin

2014), with the difference that the kernels are combined in a

localized way to better capture the sample-adaptive charac-

teristics of data. Differently, by replacing the squared error

In literature, each representation is also termed as a view.

Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16)

1888

下载后可阅读完整内容，剩余6页未读，立即下载

weixin_38618140

粉丝: 9
资源: 908

改进的多核K-means聚类：矩阵诱导正则化提升性能

具有自适应权重的增强型正则化k均值类型聚类算法

K均值聚类Matlab分类算法.rar

通过概念分解与局部流形正则化进行多视图聚类

K-means.rar_K means matlab_K 聚类_K均值聚类算法_k均值聚类_向量 聚类

Matlab基于K均值聚类的神经网络-基于K均值聚类的ＲＢＦ神经网络.rar

KMA.rar_K._k均值聚类_均值聚类_聚类算法 VC

K-means.rar_K._K均值；聚类_k-means k均值_k-means 聚类_k均值聚类matlab

k均值聚类

K均值聚类算法

3D正则化稀疏子空间聚类实现光谱图像精确土地覆盖分割

最新资源

K-means.rar_K means matlab_K 聚类_K均值聚类算法_k均值聚类_向量聚类