神经网络入门指南：理解与实践

需积分: 0 200 浏览量更新于2024-08-02 收藏 1.18MB PDF 举报

《神经网络入门》是一本由PatrickvanderSmagt和BenKrose共同编著的书籍，它属于第八版，首次出版于2021年11月，由荷兰阿姆斯特丹大学出版。该书专为对神经网络感兴趣的读者设计，特别是那些希望深入了解这一领域的初学者。书中内容涵盖了神经网络的基础理论、概念、结构以及其在实际应用中的各种技术。本书的核心知识点包括： 1. **神经元与神经网络基础**：介绍了神经元的工作原理，它们如何模仿生物大脑中的神经元进行信息处理。读者将学习到神经元的激活函数、权重和阈值等关键概念。 2. **神经网络架构**：涵盖了前馈神经网络（FFNN）、卷积神经网络（CNN）、循环神经网络（RNN）等不同类型的网络结构，以及深度学习的发展，如深度神经网络（DNN）和残差网络（ResNet）。 3. **训练与优化**：讲解了反向传播算法（Backpropagation），这是训练神经网络的关键方法，以及常见的优化算法如梯度下降、动量法和自适应学习率策略。 4. **激活函数**：阐述了诸如sigmoid、ReLU、Leaky ReLU等不同激活函数的特点和适用场景，它们在神经网络中的作用是引入非线性，增加模型表达能力。 5. **深度学习实践**：涉及如何设计和实施神经网络解决实际问题，如图像识别、自然语言处理、时间序列分析等领域的应用实例。 6. **神经网络的历史与未来**：书中还回顾了神经网络的发展历程，从Perceptron到现代深度学习的演变，并讨论了当前研究的前沿和未来可能的发展趋势。 7. **版权与许可**：明确指出，这本书可以非商业用途下免费分发，只要保持原貌且注明作者和荷兰阿姆斯特丹大学的名称。同时，课程使用需事先通知作者。为了获取更深入的理解，读者可以通过书中提供的联系方式联系作者获取进一步的资源或答疑解惑。此外，书中的网址http://www.fwi.uva.nl/research/neur指出了作者的研究团队网站，那里可能有更多的教育资源和研究更新。《神经网络入门》是一本全面且实用的教材，适合希望入门神经网络领域的学生、研究人员以及工程师阅读，为他们提供了一个系统化的学习框架和实践经验。

CHAPTER 2. FUNDAMENTALS



Figure 2.1: The basic comp onents of an articial neural network. The propagation rule used here is

the `standard' weighted summation.

an index

) which send data out of the neural network, and

hidden

units (indicated by an index

) whose input and output signals remain within the neural network.

During op eration, units can b e up dated either

synchronously

asynchronously

. With syn-

chronous up dating, all units up date their activation simultaneously; with asynchronous up dat-

ing, each unit has a (usually xed) probability of up dating its activation at a time

, and usually

only one unit will be able to do this at a time. In some cases the latter model has some

advantages.

2.1.2 Connections between units

In most cases we assume that each unit provides an additive contribution to the input of the

unit with which it is connected. The total input to unit

is simply the weighted sum of the

separate outputs from each of the connected units plus a

bias

oset

term



(

)

(



(

)

(2.1)

The contribution for positive

is considered as an

excitation

and for negative

inhibition

In some cases more complex rules for combining inputs are used, in which a distinction is made

between excitatory and inhibitory inputs. We call units with a propagation rule (2.1)

sigma

units

A dierent propagation rule, intro duced byFeldman and Ballard (Feldman & Ballard, 1982),

is known as the propagation rule for the

sigma-pi unit

(

)

(



(

)

(2.2)

Often, the

are weighted b efore multiplication. Although these units are not frequently used,

they have their value for gating of input, as well as implementation of lo okup tables (Mel, 1990).

2.1.3 Activation and output rules

We also need a rule whichgives the eect of the total input on the activation of the unit. We need

a function

whichtakes the total input

(

) and the currentactivation

(

) and pro duces a

new value of the activation of the unit

(

+1) =

(

)

(

))

(2.3)

2.2. NETWORK TOPOLOGIES

Often, the activation function is a nondecreasing function of the total input of the unit:

(

+1) =

(

)) =

(

)

(



(

)

;

(2.4)

although activation functions are not restricted to nondecreasing functions. Generally,somesort

of threshold function is used: a hard limiting threshold function (a sgn function), or a linear or

semi-linear function, or a smo othly limiting threshold (see gure 2.2). For this smo othly limiting

function often a sigmoid (S-shap ed) function like

(

;

(2.5)

is used. In some applications a hyp erb olic tangent is used, yielding output values in the range

[

;

+1].

sigmoid

sgn

semi-linear

iii

Figure 2.2: Various activation functions for a unit.

In some cases, the output of a unit can be a sto chastic function of the total input of the

unit. In that case the activation is not deterministically determined by the neuron input, but

the neuron input determines the probability

that a neuron get a high activation value:

(

1) =

;

(2.6)

in which

(cf. temp erature) is a parameter which determines the slope of the probability

function. This type of unit will b e discussed more extensively in chapter 5.

In all networks wedescribewe consider the output of a neuron to b e identical to its activation

level.

2.2 Network top ologies

In the previous section we discussed the prop erties of the basic pro cessing unit in an articial

neural network. This section fo cuses on the pattern of connections between the units and the

propagation of data.

As for this pattern of connections, the main distinction we can make is b etween:



Feed-forward

networks, where the data ow from input to output units is strictly feed-

forward. The data processing can extend over multiple (layers of ) units, but no feedback

connections are present, that is, connections extending from outputs of units to inputs of

units in the same layer or previous layers.



Recurrent

networks that do contain feedback connections. Contrary to feed-forward net-

works, the dynamical prop erties of the network are imp ortant. In some cases, the activa-

tion values of the units undergo a relaxation pro cess such that the network will evolve to

a stable state in which these activations do not change anymore. In other applications,

the change of the activation values of the output neurons are signicant, such that the

dynamical b ehaviour constitutes the output of the network (Pearlmutter, 1990).

CHAPTER 2. FUNDAMENTALS

Classical examples of feed-forward networks are the Perceptron and Adaline, which will be

discussed in the next chapter. Examples of recurrentnetworks have b een presented by Anderson

(Anderson, 1977), Kohonen (Kohonen, 1977), and Hopeld (Hopeld, 1982) and will b e discussed

in chapter 5.

2.3 Training of articial neural networks

A neural network has to be congured such that the application of a set of inputs pro duces

(either `direct' or via a relaxation pro cess) the desired set of outputs. Various metho ds to set

the strengths of the connections exist. One way is to set the weights explicitly, using

a priori

knowledge. Another way is to `train' the neural network by feeding it teaching patterns and

letting it change its weights according to some learning rule.

2.3.1 Paradigms of learning

We can categorise the learning situations in two distinct sorts. These are:



Sup ervised learning

Asso ciative learning

in whichthenetwork is trained by providing

it with input and matching output patterns. These input-output pairs can b e provided by

an external teacher, or by the system whichcontains the network (

self-sup ervised



Unsup ervised learning

Self-organisation

in which an (output) unit is trained to resp ond

to clusters of pattern within the input. In this paradigm the system is supp osed to dis-

cover statistically salient features of the input p opulation. Unlike the sup ervised learning

paradigm, there is no

a priori

set of categories into which the patterns are to b e classied;

rather the system must develop its own representation of the input stimuli.

2.3.2 Mo difying patterns of connectivity

Both learning paradigms discussed ab ove result in an adjustmentof theweights of the connec-

tions b etween units, according to some mo dication rule. Virtually all learning rules for mo dels

of this type can be considered as a variant of the

Hebbian

learning rule suggested by Hebb in

his classic book

Organization of Behaviour

(1949) (Hebb, 1949). The basic idea is that if two

units

and

are activesimultaneously, their interconnection must b e strengthened. If

receives

input from

, the simplest version of Hebbian learning prescrib es to mo dify the weight

with



y

;

(2.7)

where



is a p ositive constant of prop ortionality representing the

learning rate

. Another common

rule uses not the actual activation of unit

but the dierence between the actual and desired

activation for adjusting the weights:



y

(

;

)

;

(2.8)

in which

is the desired activation provided byateacher. This is often called the

Widrow-Ho

rule

or the

delta rule

, and will b e discussed in the next chapter.

Many variants (often very exotic ones) have b een published the last few years. In the next

chapters some of these up date rules will b e discussed.

2.4 Notation and terminology

Throughout the years researchers from dierent disciplines havecomeupwithavast number of

terms applicable in the eld of neural networks. Our computer scientist point-of-view enables

us to adhere to a subset of the terminology which is less biologically inspired, yet still conicts

arise. Our conventions are discussed b elow.

剩余134页未读，继续阅读

zymjxy

粉丝: 0
资源: 1

神经网络入门指南：理解与实践

An Introduction to Neural Networks

An Introduction to Neural Networks - Patrick van der Smagt.pdf

神经网络概论（Kevin Gurney）An Introduction to Neural Networks (Kevin Gurney)

Introduction to Neural Networks

An introduction to artificial neural networks(人工神经网络简介)

An-introduction-to-neural-networks-for-beginners

An introduction to RBM

An Introduction to Machine Learning

An Introduction to YOLOv8: The Evolutionary Journey of Convolutional Neural Networks

Information-Theoretic Aspects of Neural Networks

最新资源