FPGA技术驱动的神经网络实现进展

需积分: 9 201 浏览量更新于2023-06-01 收藏 4.49MB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

资源详情

资源推荐

4 FPGA Neurocomputers

represented by a set of weights, here denoted by w =(w

,...w

)

; and

(3) an activation function Φ that relates the total synaptic input to the output

(activation) of the neuron. The main components of an artiﬁcial neuron is

illustrated in Figure 1.

Figure 1: The basic components of an artiﬁcial neuron

The total synaptic input, u, to the neuron is given by the inner product of the

input and weight vectors:

u =



i=1

(1.1)

where we assume that the threshold of the activation is incorporated in the

weight vector. The output activation, y,isgivenby

y =Φ(u) (1.2)

where Φ denotes the activation function of the neuron. Consequently, the com-

putation of the inner-products is one of the most important arithmetic opera-

tions to be carried out for a hardware implementation of a neural network. This

means not just the individual multiplications and additions, but also the alterna-

tion of successive multiplications and additions — in other words, a sequence

of multiply-add (also commonly known as multiply-accumulate or MAC) op-

erations. We shall see that current FPGA devices are particularly well-suited

to such computations.

The total synaptic input is transformed to the output via the non-linear acti-

vation function. Commonly employed activation functions for neurons are

Review of neural-network basics 5

the threshold activation function (unit step function or hard limiter):

Φ(u)=



1.0, when u>0,

0.0, otherwise.

the ramp activation function:

Φ(u)=max{0.0, min{1.0,u+0.5}}

the sigmodal activation function, where the unipolar sigmoid function is

Φ(u)=

1+exp(−bu)

and the bipolar sigmoid is

Φ(u)=a



1 − exp(−bu)

1+exp(−bu)



where a and b represent, repectively, real constants the gain or amplitude

and the slope of the transfer function.

The second most important arithmetic operation required for neural networks

is the computation of such activation functions. We shall see below that the

structure of FPGAs limits the ways in which these operations can be carried

out at reasonable cost, but current FPGAs are also equipped to enable high-

speed implementations of these functions if the right choices are made.

A neuron with a threshold activation function is usually referred to as the

discrete perceptron, and with a continuous activation function, usually a sig-

moidal function, such a neuron is referred to as continuous perceptron. The

sigmoidal is the most pervasive and biologically plausible activation function.

Neural networks attain their operating characteristics through learning or

training. During training, the weights (or strengths) of connections are gradu-

ally adjusted in either supervised or unsupervised manner. In supervised learn-

ing, for each training input pattern, the network is presented with the desired

output (or a teacher), whereas in unsupervised learning, for each training input

pattern, the network adjusts the weights without knowing the correct target.

The network self-organizes to classify similar input patterns into clusters in

unsupervised learning. The learning of a continuous perceptron is by adjust-

ment (using a gradient-descent procedure) of the weight vector, through the

minimization of some error function, usually the square-error between the de-

sired output and the output of the neuron. The resultant learning is known as

In general, the slope of the ramp may be other than unity.

6 FPGA Neurocomputers

as delta learning: the new weight-vector, w

new

, after presentation of an input

x and a desired output d is given by

new

= w

old

+ αδx

where w

old

refers to the weight vector before the presentation of the input and

the error term, δ,is(d − y)Φ



(u), where y is as deﬁned in Equation 1.2 and



is the ﬁrst derivative of Φ. The constant α, where 0 <α≤ 1, denotes the

learning factor. Given a set of training data, Γ={(x

); i =1,...n}, the

complete procedure of training a continuous perceptron is as follows:

begin: /* training a continuous perceptron */

Initialize weights w

new

Repeat

For each pattern (x

) do

old

= w

new

= w

old

+ αδx

until convergence

end

The weights of the perceptron are initialized to random values, and the conver-

gence of the above algorithm is assumed to have been achieved when no more

signiﬁcant changes occur in the weight vector.

1.2.2 Multi-layer perceptron

The multi-layer perceptron (MLP) is a feedforward neural network consist-

ing of an input layer of nodes, followed by two or more layers of perceptrons,

the last of which is the output layer. The layers between the input layer and

output layer are referred to as hidden layers. MLPs have been applied success-

fully to many complex real-world problems consisting of non-linear decision

boundaries. Three-layer MLPs have been sufﬁcient for most of these applica-

tions. In what follows, we will brieﬂy describe the architecture and learning of

an L-layer MLP.

Let 0-layer and L-layer represent the input and output layers, respectively;

and let w

l+1

denote the synaptic weight connected to the k-th neuron of the

l +1layer from the j-th neuron of the l-th layer. If the number of perceptrons

in the l-th layer is N

, then we shall let W



= {w

}

l−1

denote the matrix

of weights connecting to l-th layer. The vector of synaptic inputs to the l-th

layer, u

=(u

,...u

)

is given by

= W

l−1

where y

l−1

=(y

l−1

,...y

l−1

)

denotes the vector of outputs at the l−1

layer. The generalized delta learning-rule for the layer l is, for perceptrons,

8 FPGA Neurocomputers

1.2.3 Self-organizing feature maps

Neurons in the cortex of the human brain are organized into layers of neu-

rons. These neurons not only have bottom-up and top-down connections, but

also have lateral connections. A neuron in a layer excites its closest neigh-

bors via lateral connections but inhibits the distant neighbors. Lateral inter-

actions allow neighbors to partially learn the information learned by a winner

(formally deﬁned below), which gives neighbors responding to similar pat-

terns after learning with the winner. This results in topological ordering of

formed clusters. The self-organizing feature map (SOFM) is a two-layer self-

organizing network which is capable of learning input patterns in a topolog-

ically ordered manner at the output layer. The most signiﬁcant concept in a

learning SOFM is that of learning within a neighbourhood around a winning

neuron. Therefore not only the weights of the winner but also those of the

neighbors of the winner change.

The winning neuron, m, for an input pattern x is chosen according to the

total synaptic input:

m = arg max

where w

denotes the weight-vector corresponding to the j-th output neuron.

x determines the neuron with the shortest Euclidean distance between its

weight vector and the input vector when the input patterns are normalized to

unity before training.

Let N

(t) denote a set of indices corresponding to the neighbourhood size

of the current winner m at the training time or iteration t. The radius of N

decreased as the training progresses; that is, N

) > N

) ...,

where t

.... The radius N

(t =0)can be very large at the

beginning of learning because it is needed for initial global ordering of weights,

but near the end of training, the neighbourhood may involve no neighbouring

neurons other than the winning one. The weights associated with the winner

and its neighbouring neurons are updated by

∆w

= α(j, t)(x − w

) for all j ∈N

(t),

where the positive learning factor depends on both the training time and the

size of the neighbourhood. For example, a commonly used neighbourhood

function is the Gaussian function

α(N

(t),t)=α(t)exp



−

r

− r



2σ

(t)



where r

and r

denote the positions of the winning neuron m and of the

winning neighbourhood neurons j, respectively. α(t) is usually reduced at a

剩余364页未读，继续阅读

yp2008gf

粉丝: 4
资源: 42

会员权益专享

FPGA技术驱动的神经网络实现进展

ZynqNet: An FPGA-Accelerated Embedded Convolutional Neural Network

Complex-Valued.Neural.Networks.Advances.and.Applications

AN5006-04设备详细资料

2 implementations of CachingConfigurer were found when only 1 was expected. Refactor the configuration such that CachingConfigurer is implemented only once or not at all.

FPGA FFT CORDIC

CNN+VisionTransformer python

PyTorch实现srresnet

PPO pytorch

怎么理解Rejection of inconsistent component implementations

Octal SPI fpga

tensorflow.contrib.rnn

some/ip有开源实现吗

MemoryError: Out of memory interning an attribute name

xilinx FPGA中DCI

In OOP, are Polymorphism and Inheritance the same?

Write a function link_to_list that takes in a linked list and returns the sequence as a Python list. You may assume that the input list is shallow; none of the elements is another linked list. Try to find both an iterative and recursive solution for this problem!

AQS tryAcquire

Describe these two approaches to polymorphism in Java: Overloading and Overriding. Give an example of each.

会员权益专享

最新资源