进化神经网络结构：NEAT算法优化

需积分: 12 102 浏览量更新于2024-09-09 收藏 676KB PDF 举报

"ENN理论，即Efficient Evolution of Neural Network Topologies，是一种利用遗传算法优化神经网络权重和结构的方法。由Kenneth Stanley和Risto Miikkulainen提出的这种方法旨在解决神经网络架构演化的问题，以获得在强化学习任务中的优势。与传统的固定拓扑结构的神经网络相比，NEAT（NeuroEvolution of Augmenting Topologies）在具有隐藏状态信息的强化学习任务上表现出色。其效率提升归功于三个方面：(1)采用有原则的跨不同拓扑结构的交叉方法；(2)使用物种分化保护结构创新；(3)从最小结构逐步增量增长。通过一系列的消融研究，证实了这些组件对于整个系统及其相互之间的重要性，从而实现了显著更快的学习速度和更优的解决方案。" 在神经网络的优化中，传统的优化方法如梯度下降法虽然广泛应用，但在处理复杂的非线性问题时可能会陷入局部最优，无法找到全局最优解。ENN理论，特别是NEAT算法，引入了遗传算法的思想，用生物进化的过程来指导神经网络的优化。这种方法允许网络的结构和权重同时进行演化，增加了搜索空间的灵活性。首先，NEAT的交叉操作针对不同的网络拓扑结构设计，它能够在保持原有结构优点的同时，融合不同个体的特性，创造出可能更适应环境的新结构。这种策略有助于跳出局部最优，发现更高效的设计。其次，物种保护机制是NEAT的核心之一。在进化过程中，新产生的网络结构可能与现有结构差异较大，若直接合并可能导致优秀特征的丧失。通过物种分化，可以将具有相似结构的个体归为一类，保护它们的创新不被过早淘汰，促进多样性的保持和长期的进化。最后，NEAT的增量增长策略从最简单的网络结构开始，逐渐添加节点和连接，使得网络结构能够根据任务需求自然生长，避免一开始就构建过于复杂的网络，从而减少过拟合的风险。一系列的消融实验验证了这三个组件的必要性，每个部分都对系统的整体性能和组件间的协同作用产生了积极影响。通过这种方式，NEAT不仅能够找到更优的解决方案，而且在学习速度上也有了显著提升，尤其适用于那些需要处理复杂环境和隐藏状态信息的强化学习任务。因此，ENN理论是神经网络优化领域的一种创新性方法，为深度学习和人工智能的发展提供了新的思路。

Efficient Evolution of Neural Network Topologies

Kenneth

Stanley and Risto Miikkulainen

Department of Computer Sciences

The University of Texas at Austin

Austin, TX 78712

kstanley,

risto@cs.

utexas.edu

Abstract

Neuroevolution, i.e. evolving artificial neural networks

with genetic algorithms, has been highly effective in rein-

forcement learning

tasks,

particularly those with hidden

state information. An important question in neuraevolu-

tion is how to gain an advantage from evolving neural net-

work topologies along with weights. We present a method,

NeuroEvolution of Augmenting Topologies (NEAT) that

outperforms the best fixed-topology methods on a challeng-

ing benchmark reinforcement learning

task.

We claim that

the increased efficiency is due to

(1)

employing a principled

method of crossover of different topologies,

(2)

protecting

structural innovation using speciation, and

(3)

incremen-

tally growing from minimal structure. We test this claim

through a series of ablation studies that demonstrate that

each component is necessary to the system

a whole and

to each other. What results is significantly

faster

learning.

NEAT is also an important contribution to GAS because

it shows how it is possible for evolution to both optimize

and

complexify

solutions simultaneously, making it possible

evolve increasingly complex solutions over time, thereby

strengthening the analogy with biological evolution.

I. INTRODUCTION

Neuroevolution (NE), the artificial evolution of neural net-

works using genetic algorithms, has shown great promise

in reinforcement learning tasks. NE outperforms standard

reinforcement learning methods in many benchmark tasks

[6,

10,

Neural networks are a good class of decision

making systems to evolve because they are capable of repre-

senting solutions to many different kinds of problems, and the

mapping from genotype to phenotype is generally efficient.

NE is particularly well suited to reinforcement learning tasks

because NE does not require supervision.

A major question in NE is how to gain an advantage from

evolving topology in addition to connection weights. On one

hand, evolving topology might overcomplicate the search. On

the other, it can also save time by finding the right number of

hidden neurons

for

a particular problem automatically [7].

A previous study showed that fixed-topology NE can out-

perform a topology-evolving system on the benchmark dou-

ble pole balancing task

[6].

This finding is important because

pole balancing has been a benchmark task in NE and rein-

forcement learning for over

years

[l,

91,

and

dou-

ble pole balancing is challenging to even the best of modem

methods. Doing well at this important benchmark suggests

that a method will do well in other tasks as well. Whether

Topology and Weight Evolving Artificial Neural Networks

(TWEANNs) can enhance the performance

NE remains

an open question.

In this article, we aim to show that evolving topology can

indeed increase performance. We present a new

TWEANN,

NeuroEvolution of Augmenting Topologies (NEAT), that sig-

nificantly outperforms the fixed-topology NE method that

currently takes the fewest evaluations

the double pole

balancing task. We identify three major challenges for

TWEA”s and present solutions

each of them:

(1)

there

a genetic representation that allows disparate topologies to

crossover in a meaningful way?

Our

solution is to use his-

torical markings to line up genes with the same origin. (2)

How can topological innovation that needs a few generations

to optimize be protected

that it does not disappear from the

population prematurely?

Our

solution is to separate each in-

novation into a different species.

(3)

How can topologies

minimized

throughout evolution

without the need for a spe-

cially contrived fitness function that measures complexity?

Our

solution is to start from a minimal structure and grow

only when necessary. This paper establishes that each of our

solutions is necessary by showing that

performance sig-

nificantly declines with the ablation

any of the major

so-

lution components. Working together in NEAT these compo-

nents constitute a promising new approach to difficult rein-

forcement learning tasks.

We begin by describing the NEAT method, including re-

sults showing that NEAT is significantly faster than other NE

methods on the hardest pole balancing benchmark. We then

present ablation studies designed to explain NEAT’S perfor-

mance in terms of its components.

11. NEUROEVOLUTION

AUGMENTING

Genetic

Encoding

NEAT is designed specifically to address the three challenges

raised in the introduction. Each genome includes a list of

connection genes,

each of which refers

two

node genes

be-

ing connected (figure

l).

Each connection gene specifies the

in-node, the out-node, the weight of the connection, whether

TOPOLOGIES (NEAT)

0-7803-7282-4/02/$10.00 02002

IEEE

1757

下载后可阅读完整内容，剩余5页未读，立即下载

weixin_39960221

粉丝: 0
资源: 1

进化神经网络结构：NEAT算法优化

PSO训练BP神经网络

ENNlib:简易神经网络库

enn-policy:实施Kubernetes网络策略

Load forcasting 2019_loadforecasting_enn_prediction.zip

基于改进ENN2 聚类算法的多故障诊断方法

2019年电力负荷预测方法与ENN模型分析

EKF-ENN融合算法提升瓦斯涌出量预测精度

改进ENN2聚类算法在多故障诊断中的应用

CIPSO-ENN耦合算法：非线性系统参数高效识别

SMOTE ENN python

最新资源