机器学习驱动的进化策略

需积分: 12 185 浏览量更新于2024-07-19 收藏 5.57MB PDF 举报

"Machine Learning for Evolution Strategies" by Oliver Kramer, from Informatik Universität Oldenburg, Germany. 本书"机器学习与进化策略"深入探讨了如何将机器学习技术应用于进化策略这一领域。进化策略是一种优化方法，源自生物进化理论，常用于解决多目标优化问题，尤其是在复杂环境中。这种策略模拟了自然界中的物种进化过程，通过随机变异和选择机制来逐步优化解决方案。在机器学习领域，特别是深度学习和强化学习等子领域，进化策略已经被证明是有效的模型参数优化手段。它们可以替代传统的梯度下降方法，尤其在存在局部最优或计算梯度困难的情况下。通过利用进化算法的无监督特性，这些策略能够探索更广阔的解决方案空间，从而找到可能更好的模型配置。书中可能涵盖了以下关键知识点： 1. **进化算法基础**：解释了遗传算法、粒子群优化、差分进化等基础进化策略的原理和操作步骤。 2. **适应度函数**：讨论如何设计和选择适应度函数以衡量解决方案的质量，并与机器学习模型的性能指标相结合。 3. **种群管理和选择策略**：详述如何维护和更新种群，以及选择优秀个体进行繁殖的策略，如轮盘赌选择、锦标赛选择等。 4. **变异和交叉操作**：阐述了在机器学习上下文中如何应用变异和交叉操作，以保持种群多样性并推动进化过程。 5. **协同进化**：介绍如何利用协同进化思想，让不同模型或组件相互竞争和合作，以提升整体性能。 6. **强化学习中的应用**：探讨进化策略如何在强化学习中作为策略优化算法，如ES（进化策略）和CMA-ES（高斯协方差矩阵适应演化策略）。 7. **并行与分布式计算**：讨论如何利用并行和分布式计算架构加速进化策略的执行，特别是在处理大数据集时。 8. **实际应用案例**：提供实际的机器学习和数据科学项目案例，展示如何将进化策略应用于真实世界的问题，如网络流量预测、图像分类或自动驾驶汽车控制。 9. **最新研究进展**：可能会涵盖最新的研究工作，如结合神经网络的新型进化策略，或者在应对高维复杂问题上的改进方法。 10. **未来挑战与发展方向**：作者可能会讨论当前存在的挑战，如收敛速度、早熟现象，以及未来可能的研究方向。通过本书，读者不仅可以了解到进化策略的基础知识，还能掌握将其应用于机器学习实践的技巧，为解决复杂优化问题提供新的思路。对于从事机器学习研究、工程开发或对优化方法感兴趣的读者来说，这是一份宝贵的参考资料。

1.6 Overview 7

introduction for readers, who are not familiar with machine learning modeling and

machine learning algorithms.

Chapter 5

Chapter 5 gives an introduction to scikit-learn, a machine learning library for

Python. With a strong variety of efﬁcient ﬂexible methods, scikit-learn imple-

ments algorithms for pre-processing, model selection, model evaluation, supervised

learning, unsupervised learning, and many more.

Chapter 6

In Chap. 6, we show that a reduction of the number of ﬁtness function evaluations of

a (1+1)-ES is possible with a combination of a k-nearest neighbor (kNN) regression

model, a local training set of ﬁtness function evaluations, and a convenient meta-

model management. We analyze the reduction of ﬁtness function evaluations on a

small set of benchmark functions.

Chapter 7

In the line of research on ﬁtness function surrogates, the idea of meta-modeling the

constraint boundary is the next step we analyze in Chap. 7. We employ support vector

machines (SVMs) to learn the constraint boundary as binary classiﬁcation problem.

A new candidate solution is ﬁrst evaluated on the SVM-based constraint surrogate.

The constraint function is only evaluated, if the solution is predicted to be feasible.

Chapter 8

In Chap. 8, we employ the concept of bloat by optimizing in a higher-dimensional

solution space that is mapped to the real-solution space with dimensionality reduc-

tion, more speciﬁcally, with principal component analysis (PCA). The search in a

space that employs a larger dimensionality than the original solution space may be

easier. The solutions are evaluated and the best w.r.t. the ﬁtness in the original space

are inherited to the next generation.

Chapter 9

The visualization of evolutionary blackbox optimization runs is important to under-

stand evolutionary processes that may require the interaction with or intervention by

the practitioner. In Chap. 9, we map high-dimensional evolutionary runs to a two-

dimensional space easy to plot with isometric mapping (ISOMAP). The ﬁtness of

embedded points is interpolated and visualized with matplotlib methods.

Chapter 10

In multimodal optimization, the task is to detect most global and local optima in

solution space. Evolutionary niching is a technique to search in multiple parts of the

solution space simultaneously. In Chap. 10, we present a niching approach based on

an explorative phase of uniform sampling, selecting the best solutions, and applying

clustering to detect niches.

8 1 Introduction

Chapter 11

In Chap. 11, we summarize the most important ﬁndings of this work. We give a

short overview to evolutionary search in machine learning and give insights into

prospective future work.

1.7 Previous Work

Parts of this book built upon previous work that has been published in peer-reviewed

conferences. An overview of previous work is the following:

• The covariance matrix estimation approach of Chap. 3 based on Ledoit-Wolf esti-

mation has been introduced on the Congress on Evolutionary Computation (CEC)

2015 [5] in Sendai, J apan.

• The nearest neighbor meta-model approach for ﬁtness function evaluations pre-

sented in Chap. 6 is based on a paper presented at the EvoApplications conference

(as part of EvoStar 2016) [6] in Porto, Portugal.

• The PCA-based dimensionality reduction approach that optimizes in high-

dimensional solution spaces presented in Chap. 8 has been introduced on the Inter-

national Joint Conference on Neural Networks (IJCNN) 2015 [7] in Killarney,

Ireland.

• The ISOMAP-based visualization approach of Chap. 9 has also been introduced

on the Congress on Evolutionary Computation (CEC) 2015 [8] in Japan.

Parts of this work are consistent depictions of published research results presenting

various extended results and descriptions. The book is written in a scientiﬁc style

with the use of “we” rather than “I”.

1.8 Notations

We use the following notations. Vectors use small bold latin letters like x, scalars

small plain Latin or Greek letters like σ . In optimization scenarios, we use the variable

x = (x

,...,x

)

∈ R

for objective variables that have to be optimized w.r.t. a

ﬁtness function f . In machine learning, concepts often use the same notation. Patterns

are vectors x = (x

,...,x

)

of attributes or features x

with j = 1,...,d for

machine learning models f . This is reasonable as candidate solutions in optimization

are often treated as patterns in this book. Patterns lie in an d-dimensional data space

and are usually indexed from 1 to N , i.e., a data set consists of patterns x

,...,x

They may carry labels y

,...,y

resulting in pattern-label pairs

{(x

, y

),...,(x

, y

)}.

1.8 Notations 9

When the dimensionality of objective variables or patterns differs from d (i.e., from

the dimensionality, where the search actually takes place), we employ the notation

x. In Chap. 8,

x represents an abstract solution of higher dimensionality, in Chap. 9

it represents a visualizable low-dimensional pendant of solution x.

While in literature, f stands for a ﬁtness function in optimization and for a super-

vised model in machine learning, we resolve this overlap in the following. While

being used this standard way in the introductory chapters, Chap. 6 uses f for the

ﬁtness function and

f for the machine learning model, which is a surrogate of f .

In Chap. 7, f is the ﬁtness function, while g is the constraint function, for which a

machine learning surrogate ˆg is learned. In Chaps. 8 and 9, f is the ﬁtness function

and F is the dimensionality reduction mapping from a space of higher dimensions

to a space of lower dimensionality. In Chap. 10, f is again the ﬁtness function, while

c delivers the cluster assignments of a pattern.

A matrix is written in bold large letters, e.g., C for a covariance matrix. Matrix

is the transpose of matrix C. The p-norm will be written as ·

p with the

frequently employed variant of the Euclidean norm written as ·

1.9 Python

The algorithms introduced in this book are based on Python and built upon various

well-known packages including Numpy [9], SciPy [10], Matplotlib [11], and

scikit-learn [12]. Python is an attractive programming language that allows func-

tional programming, classic structured programming, and objective-oriented pro-

gramming. The installation of Python is usually easy. Most Linux, Windows, and

Mac OS distributions already contain a native Python version. On Linux systems,

Python is usually installed in the folder /usr/local/bin/python. If not,

there are attractive Python distributions that already contain most packages that are

required for fast prototyping machine learning and optimization algorithms. First

experiments with Python can easily be conducted when starting the Python inter-

preter, which is usually already possible, when typing Python in a Unix or Windows

shell.

An example for a typical functional list operation that demonstrates the capabilities

of Python is:

[f(x)*math.pi for x in range(1,10)]

This expression generates a list of function values with arguments 1,...,10 of func-

tion f(x), which must be deﬁned before, multiplied with π .

A very frequently employed package is Numpy for scientiﬁc computing with

Python. It comprises powerful data structures and methods for handling

N -dimensional array objects and employs outstanding linear algebra and random

number functions. SciPy provides efﬁcient numerical routines, e.g., for numer-

ical integration, for optimization, and distance computations. Matplotlib is a

10 1 Introduction

Python plotting library that is able to generate high-quality ﬁgures. scikit-learn

is a Python framework for machine learning providing efﬁcient tools f or data min-

ing and data analysis tasks. The scikit-learn framework will be introduced in more

detail in Chap. 4.

References

1. Beyer, H., Schwefel, H.: Evolution strategies—a comprehensive introduction. Nat. Comput.

1(1), 3–52 (2002)

2. Rechenberg, I.: Evolutionsstrategie—Optimierung technischer Systeme nach Prinzipien der

biologischen Evolution. Frommann-Holzboog, Stuttgart (1973)

3. Schwefel, H.-P.: Numerische Optimierung von Computer-Modellen mittel der Evolution-

sstrategie. Birkhaeuser, Basel (1977)

4. Holland, J.H.: Adaptation in Natural and Artiﬁcial Systems. University of Michigan Press, Ann

Arbor (1975)

5. Kramer, O.: Evolution strategies with ledoit-wolf covariance matrix estimation. In: Proceedings

of the IEEE Congress on Evolutionary Computation, CEC 2015, pp. 1712–1716 (2015)

6. Kramer, O.: Local ﬁtness meta-models with nearest neighbor regression. In: Proceedings of

the 19th European Conference on Applications of Evolutionary Computation, EvoApplications

2016, pp. 3–10. Porto, Portugal (2016)

7. Kramer, O.: Dimensionality reduction in continuous evolutionary optimization. In: 2015 Inter-

national Joint Conference on Neural Networks, IJCNN 2015, pp. 1–4. Killarney, Ireland, 12–17

July 2015

8. Kramer, O., Lückehe, D.: Visualization of evolutionary runs with isometric mapping. In: Pro-

ceedings of the IEEE Congress on Evolutionary Computation, CEC 2015, pp. 1359–1363.

Sendai, Japan, 25–28 May 2015

9. van der Walt, S., Colbert, S.C., Varoquaux, G.: The NumPy array: a structure for efﬁcient

numerical computation. Comput. Sci. Eng. 13(2), 22–30 (2011)

10. Jones, E., Oliphant, T., Peterson, P., et al.: SciPy: open source scientiﬁc tools for Python (2001–

2014). Accessed 03 Oct 2014

11. Hunter, J.D.: Matplotlib: a 2D graphics environment. Comput. Sci. Eng. 9(3), 90–95 (2007)

12. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M.,

Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher,

M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res.

12, 2825–2830 (2011)

剩余119页未读，继续阅读

qq_41714974

粉丝: 0
资源: 8

机器学习驱动的进化策略

Real-Time Machine Learning Model Update Strategies: 3 Tips to Keep Your Model Ahead

Deep Learning for Beginners

DDoS.Attacks.Evolution.Detection.Prevention.Reaction.and.Tolerance

Optimization of Machine Learning Using MATLAB Genetic Algorithms: Strategies of Integration and ...

Applications of MATLAB Optimization Algorithms in Machine Learning: Case Studies and Practical Guide

【Machine Learning Time Series Forecasting: From Beginner to Expert】: Mastering Core Applications

Integration Learning Methods: Master These 6 Strategies to Build an Unbeatable Model

MATLAB Genetic Algorithm for Solving Scheduling Problems: Practical Strategies and Case Studies

MATLAB Supply Chain Management Optimization: Strategies for Enhancing Efficiency and Case Studies

In-depth Analysis of Frame Buffer Management and Data Caching Strategies for ST7789 Displays

最新资源