新元胞生长算法：神经网络训练效率与质量提升

需积分: 50 129 浏览量更新于2024-07-18 收藏 677KB PDF 举报

本文主要探讨了神经网络训练中的优化算法性能，特别是在监督学习领域的重要性。研究者旨在通过比较新提出的metaheuristic蝙蝠算法（BA）与其他标准算法，如反向传播（Back Propagation）、Levenberg-Marquardt、蝙蝠算法（Bat Algorithm）、遗传算法（Genetic Algorithm）和粒子群优化（Particle Swarm Optimization），展示BA在时间效率和解决方案质量上的优势。实验在标准数据集Proben1上进行，该数据集源自医学问题，以便于未来的研究者进行对比。 Proben1作为选定的基准测试集，其包含了实际应用中的医学问题，使得结果具有广泛的实际意义。文章特别提到了一种名为"ANew Adaptive Merging and Growing Algorithm for Designing Artificial Neural Networks"（2009年7月在《IEEE Transactions on Systems, Man, and Cybernetics – Part B》发表）的算法，它是一种自适应合并和生长算法，用于人工神经网络的设计。虽然没有直接提到AMGA算法，但可以推测这个算法可能与神经网络结构的自适应调整有关，可能是作者们用来改进神经网络设计的一种方法。文章作者包括Kazi Md. Rokibul Alam、Md Ashiqul Islam、Faijul Amin等，他们在神经网络领域的项目有系列工作，例如Kazi Md. Rokibul Alam的时序预测项目和Faijul Amin关于神经网络的应用。此外，还有Kazuyuki Murase教授，他在大学的科研产出丰硕，对神经网络优化算法有着深入的研究。值得注意的是，论文内容已经接受了期刊的最终版发表，但可能在页面布局上还存在后续编辑的可能。总体来说，这篇论文不仅关注神经网络的训练优化，而且展示了新算法在实际问题中的竞争力，为其他研究人员提供了有价值的数据和比较点，对于提高eLearning环境下的神经网络训练效率具有实际指导意义。

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

ISLAM et al.: NEW ADAPTIVE MERGING AND GROWING ALGORITHM FOR DESIGNING ANNs 707

Fig. 1. Typical constructive algorithm in designing ANNs.

Fig. 2. Typical pruning algorithm in designing ANNs.

the initial conditions in constructive algorithms. For example,

the number of hidden layers and neurons can simply be set

to one. Constructive algorithms are computationally efﬁcient

because they always search small solutions ﬁrst. Furthermore,

they are very easy to implement. There are also hurdles that

constructive algorithms need to overcome [2]. For example, one

has to determine when to add hidden neurons and when to stop

the neuron addition process.

B. Pruning Algorithm

Unlike constructive algorithms, a pruning algorithm starts

with an oversized ANN architecture, for example, a single-

hidden-layered ANN with a large number of hidden neurons.

Most pruning algorithms start the removal of irrelevant param-

eters (connections and/or hidden neurons) after training an

oversized ANN to a minimum error (see the review paper [3]).

They generally prune one neuron in each pruning step. There

are only few recent works that prune several neurons in each

pruning step during training (e.g., [12] and [47]). The deci-

sion to prune hidden neurons is taken based on the neuron’s

signiﬁcance or relevance. A number of pruning criteria have

been proposed to decide which hidden neurons are to be pruned

from a trained ANN [see the review paper [3] and some recent

works (e.g., [12] and [26])]. When an ANN is trained, each

of its hidden neurons learns some information during training,

although they may be insigniﬁcant for some neurons. The ANN

is usually retrained after pruning so that its existing hidden

neurons can adjust their weights to compensate the roles played

by pruned hidden neurons (e.g., [25], [26], and [29]).

Fig. 2 shows how a pruning algorithm deletes hidden neurons

from an oversized ANN either during training (e.g., [12] and

[47]) or after converging to a minimum error (e.g., [24]–[26]).

There are several advantages of a pruning algorithm. This

algorithm can learn the target function very quickly and has a

lesser chance to trap into local minima [3], [26], [48]. These two

facilities are obtained due to extra parameters (e.g., hidden neu-

rons) in the initial ANN architecture of the pruning algorithm.

Authorized licensed use limited to: IEEE Xplore. Downloaded on February 10, 2009 at 05:11 from IEEE Xplore. Restrictions apply.

剩余18页未读，继续阅读

orange_258312241

粉丝: 1

新元胞生长算法：神经网络训练效率与质量提升

神经网络结构搜索【ICCV 2019】.zip

基于遗传算法的BP神经网络优化算法

神经网络结构优化方法及应用.pdf

基于输出敏感度方差重要性的RBF神经网络结构优化算法研究

基于遗传算法的神经网络结构优化

基于遗传算法的BP神经网络优化算法_基于遗传算法的BP神经网络优化算法_matlab_遗传神经网络_遗传算法_优化MATLAB_

基于参数优化的RBF神经网络结构设计算法.pdf

基于遗传算法的前向神经网络结构优化

用于神经网络结构优化的改进遗传算法.pdf

基于进化算法的神经网络结构优化研究_杨英

最新资源