多核极小学习机：优势与优化策略

192 浏览量更新于2024-08-27 1 收藏 605KB PDF 举报

标题："Multiple Kernel Extreme Learning Machine" 是一篇研究论文，主要探讨了多核极大学习机（Multiple Kernel Extreme Learning Machine，MKELM）这一主题。该论文由Xinwang Li、Lei Wang、Guang-Bin Huang、Jian Zhang和Jianping Yan等人合作撰写，分别来自国防科技大学计算机科学学院、澳大利亚伍伦贡大学计算机科学与软件工程学院、新加坡南洋理工大学电气与电子工程学院以及悉尼科技大学工程与信息技术学院。文章背景：在过去的十年里，由于其高效性、易于实现、分类与回归任务的一体化以及二元和多元学习任务的统一处理，极端学习机（Extreme Learning Machine，ELM）成为了研究领域的一个重要课题。然而，尽管ELM算法结合了许多优点，现有的研究在优化选择内核方面并未给予充分重视。这表明了MKELM作为一种改进方法的必要性和潜力。主要内容： MKELM论文旨在解决这个问题，通过引入多核学习的概念，它扩展了ELM的灵活性，允许根据数据特性动态选择或组合多个内核函数。这种灵活性有助于提高模型的性能，适应不同类型的输入数据，并可能在复杂非线性问题上表现出色。作者们可能对不同内核（如高斯核、多项式核、sigmoid核等）的权重分配、内核函数的参数调整以及它们如何协同工作进行了深入研究。研究方法：论文可能涉及以下几个关键步骤： 1. **内核空间转换**：通过多核技术，将原始数据映射到多个相关的特征空间，每个空间对应一个内核函数。 2. **参数优化**：寻找最优的内核组合权重，以平衡各个空间的贡献，同时保持整体模型的简单性和泛化能力。 3. **模型构建**：利用优化后的内核函数，构建简单且有效的隐层神经网络，其中权值通常通过线性计算得出。 4. **性能评估**：通过实验验证，比较MKELM与传统ELM及其他相关方法在各种数据集上的性能，以证明其优势。结论与未来方向：该论文可能展示了多核极大学习机在实际问题中的优越性能，以及它在处理高维、非线性和小样本数据时的潜力。此外，作者还可能讨论了进一步研究的方向，如更复杂的内核集成策略、MKELM在其他机器学习任务（如深度学习融合）中的应用，以及理论分析以更好地理解其内在机制。 "Multiple Kernel Extreme Learning Machine"这篇论文是关于如何通过多核技术提升极端学习机性能的研究，它强调了内核选择优化在提升学习机效果中的关键作用，并展示了在实际问题中的有效应用。对于那些关注机器学习效率和灵活性的科研人员来说，这篇论文提供了有价值的新视角和实践指导。

3. Multiple kernel extreme learning machine

In [6],akernelELMisﬁrst proposed, in which a Gaussian kernel

and a polynomial kernel are empirically speciﬁed. Such speciﬁed

kernels may not be suitable for v arious applications. This moti v at es

us to design a learning algorithm which is able to automatically learn

a data-dependent optimal kernel for ELM in different applications.

Inspired by the idea of MKL, we assume that the optimal kernel can

be expressed as a linear combination of base kernels, and jointly

learn the structural parameters of ELM and the optimal kernel

combination coefﬁcients. This e xt ension makes ELM able to handle

different heterog eneous data integrations, and this extends the ELM

to a wider range of applications. Following the research on SVM

based MKL, we ﬁrst design a sparse MK -ELM algorithm, and general-

ize it to the non-sparse case. After that, a radius-incorporated variant

is proposed. Three efﬁcient algorithms are given to solve the

corresponding kernel learning problems.

3.1. The sparse MK-ELM

By incorporating the base kernel combination weights into

ELM, and imposing an ℓ

-norm and non-negative constraint on

the base kernel weights, we obtain the objective function of the

proposed sparse MK-ELM as follows:

min

β;ξ

‖β‖

∑

i ¼ 1

‖ξ

i

‖

s:t: β

ϕðx

; γÞ¼y

ξ

i

; 8i; ∑

p ¼ 1

¼1; γ

Z 0; 8p; ð6Þ

where β ¼½β

; … ; β

A R

ðjϕ

ðÞjþ⋯ þjϕ

ðÞjÞT

, β

A R

jϕ

ðÞjT

ðp ¼ 1; …; mÞ

is the pth component corresponding to the pth base kernel. Recall

that ξA R

Tn

is the training error matrix on training data,

i

¼½ξ

; ξ

; … ; ξ



ð1r i r nÞ is the ith column of ξ.

As observed, Eq. (6) optimizes the structural parameter of

ELM β and the kernel combination coefﬁcients γ jointly. We now

show how to solve the objective function in Eq. (6) efﬁciently.

By substituting Eq. (5) into Eq. (6), Eq. (6) can be rewritten as

min

β;ξ

‖β‖

∑

i ¼ 1

‖ξ

i

‖

s:t: ∑

p ¼ 1

ﬃﬃﬃﬃﬃ

ðx

Þ¼y

ξ

i

; 8i; ∑

p ¼ 1

¼1; γ

Z 0; 8p:

ð7Þ

After deﬁning

β ¼½

;

; …;

; ð8Þ

where

ﬃﬃﬃﬃﬃ

; p ¼1; …; m, Eq. (7) can be equivalently refor-

mulated as

min

β;ξ

∑

p ¼ 1

‖

∑

i ¼ 1

‖ξ

i

‖

s:t: ∑

p ¼ 1

ðx

Þ¼y

ξ

i

; 8i; ∑

p ¼ 1

¼1; γ

Z 0; 8p: ð9Þ

It is not difﬁcult to verify that Eq. (9) is a joint-convex optimization

problem [34], and its Lagrangian function is

Lð

β; ξ; γÞ¼

∑

p ¼ 1

‖

∑

i ¼ 1

‖ξ

i

‖

 ∑

t ¼ 1

∑

i ¼ 1

∑

p ¼ 1

ðx

Þy

þξ

þτ ∑

p ¼ 1

1

;

ð10Þ

where α A R

nT

and τ are the Lagrange multipliers. In Eq. (10),we

omit the non-negative constraints on γ

; ðp ¼1; …; mÞ since the

newly updated kernel combination weights are automatically kept

non-negative at each iteration, as will be validated later.

We can have the KKT optimality conditions of Eq. (10) as

follows:

¼γ

∑

t ¼ 1

∑

i ¼ 1

ðx

Þ; 8p ð11Þ

; 8t 8i ð12Þ

∑

p ¼ 1

ðx

Þ¼y

ξ

i

; 8i; ð13Þ

which can be rewritten into a matrix form as

Kð; ; γÞþ



α ¼ Y

; ð14Þ

where Kðx

; x

; γÞ¼ϕðx

; γÞ

ϕðx

; γÞ¼∑

p ¼ 1

ðx

; x

Þ. Y ¼½y

; … ;

A R

Tn

is the label matrix. From Eq. (14), the α , which cor-

responds to the structural parameter of ELM, can be obtained by

α ¼ Kð; ; γÞþ



1

: ð15Þ

We then show how to update the kernel combination coefﬁ-

cients γ efﬁciently. By taking the derivative of Eq. (10) w.r.t

ðp ¼ 1; …; mÞ and let it vanish, we obtain that the new kernel

combination weights γ

new

are updated by

new

‖

∑

p ¼ 1

‖

; 8p; ð16Þ

where

‖

¼γ

ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ

∑

s;t ¼ 1

∑

i;j ¼ 1

ðx

; x

: ð17Þ

and γ

; ðp ¼1; …; mÞ is the pth kernel combination weight in the

last iteration. As seen from Eqs. (16) and (17), the newly updated

new

ðp ¼ 1; …; mÞ are kept non-negative at each iteration, which

automatically satisﬁes the non-negative constraint. The detailed

derivation of updating the kernel combination weights is provided

in the Appendix.

The overall optimization algorithm for solving sparse MK-ELM

is presented in Table 1.

Algorithm 1. The sparse MK-ELM.

Input: fK

p ¼ 1

, y and C.

2: Output: α and γ.

Initialize γ ¼γ

and t¼0.

4: repeat

Compute Kð; ; γÞ¼∑

p ¼ 1

Update α

by solving Eq. (15).

Update γ

t þ1

by Eq. (16).

8: t ¼ t þ1.

until maxfjγ

t þ1

γ

jgr 1e 4

3.2. Non-sparse MK-ELM

Recent research on SVM based MKL has indicated that non-

sparse MKL algorithms can usually outperform the sparse alter-

natives [20,35] by arguing that some complementary information

may be lost due to the sparsity constraint. In the following part, we

ﬁrst design a non-sparse MK-ELM algorithm, and propose an

optimization algorithm to solve the resulting kernel learning

X. Liu et al. / Neurocomputing 149 (2015) 253–264 255

剩余11页未读，继续阅读

weixin_38717169

粉丝: 4
资源: 947

多核极小学习机：优势与优化策略

极限学习机源码

Python知识点3——列表操作

extreme learning machine 代码

matlab心的代码-Multi-Kernel-Extreme-Learning-Machine:“多核极限学习机”的Matlab代码

ELM based Multiple Kernel -means with Diversity-Induced Regularization

115157679elm_kernel.zip

Elm_KElm.7z

什么是极限学习机

核极限学习机

极限学习机的代码

最新资源