深度信念网络C++与CUDA实现：卷1

需积分: 9 35 浏览量更新于2024-07-18 收藏 3.13MB PDF 举报

"Deep Belief Nets in C++ and CUDA C_Volume 1-Apress(2018)" 是一本面向已经有一定神经网络基础的读者，旨在介绍、实验和编程深度信念网络（Deep Belief Nets, DBNs）的专业书籍。书中包含以下几个关键知识点： 1. **深度信念网络的动机**：作者提供了深入理解DBN背后的理论基础，帮助读者认识到这种网络架构的重要性。 2. **关键方程与组件**：书中详细阐述了最常见的DBN组件的关键方程，并适度地解释了它们的合理性。 3. **训练与执行算法**：书中给出了通用DBN模式的训练、执行和分析算法，这些算法是语言无关的，可以应用于不同的编程环境。 4. **DEEP程序用户手册**：本书还作为DEEP程序的详细用户指南，该程序可以从作者的网站免费下载。作者深入介绍了程序的内部运作机制。 5. **C++代码实现**：提供了许多核心DBN算法的C++代码实现，包括支持Windows多线程执行的版本，以及利用NVIDIA CUDA显卡超级计算机能力的CUDA C实现。 6. **不包含的内容**：尽管如此，书中的内容并不涉及详细的数学理论，而是更多地关注实践应用。此外，仅呈现了作者在实际工作中发现最有价值的模型，而并非涵盖所有模型。 7. **受限玻尔兹曼机（Restricted Boltzmann Machines, RBMs）和监督前馈网络**：RBMs是构成DBN的基础单元，而监督前馈网络则在DBN训练完成后用于进行分类或回归任务。书中详细介绍了这两类网络的工作原理和应用。 8. **误差反向传播**：这是监督学习中用于调整网络权重的关键算法，书中对此进行了讲解，并提供了计算梯度的代码示例。 9. **SoftMax输出**：在分类任务中，SoftMax函数用于将网络的连续输出转换为概率分布，便于多分类。 10. **权重惩罚**：如L1和L2正则化，用以防止过拟合，保持模型的泛化能力。 11. **多线程梯度计算**：为了加速训练过程，书中展示了如何利用多线程技术并行计算梯度。 12. **梯度计算**：书中可能包含对整个网络的梯度计算的详细步骤，这对于理解深度学习的优化过程至关重要。《Deep Belief Nets in C++ and CUDA C_Volume 1》是一本全面介绍深度信念网络实践应用的资源，它不仅涵盖了理论基础，还提供了丰富的代码实现和实际操作指导，对于希望深入理解和应用DBN的读者来说，是一本不可多得的参考书。

Suppose we have K output neurons numbered 1 through K. For a given training case,

let t

be the true value for this case, the value that we hope the network will produce, and

let o

be the output actually obtained. Then the mean squared error (MSE) for this single

case is given by Equation 2-1. To compute the MSE for the entire training set, sum this

quantity for all cases and divide by the number of cases.

()

(2-1)

Supervised training of a multiple-layer feedforward network amounts to finding

the weights and bias terms that minimize Equation 2-1. In any numerical minimization

algorithm, it is of great benefit to be able to efficiently compute the gradient, the partial

derivatives of the criterion being minimized with respect to each individual parameter.

Luckily, this is quite easy in this application. We just start at the output layer and work

backward, repeatedly invoking the chain rule of differentiation.

The activation of output neuron k is just a weighted sum of the activations of the

neurons in the prior layer. We’ll use the Greek letter delta to designate the derivative of

the error with respect to this weighted sum coming into output neuron k. Examination

of Equation 2-1 shows this to simply be as shown in Equation 2-2. In this equation, the

superscript O means that this signifies the delta of an output neuron.

ot=-

()

(2-2)

This neuron is receiving a weighted sum of activations from all neurons in the

prior layer. How can we compute the derivative of the error with respect to the weight

from neuron i in the prior layer? The simple chain rule says that this is the product of

the derivative in Equation 2-2 times the derivative of the net input (the weighted sum

coming into this output neuron) with respect to this weight.

This latter term is trivial. The contribution to the weighted sum from neuron i in

the prior layer is just the activation of that neuron times the weight connecting it to the

output neuron k. We shall designate this output weight as w

. So, the derivative of that

weighted sum with respect to w

is just the activation of neuron i. This leads us to the

formula for the partial derivatives of the error with respect to the weights connecting the

last hidden layer to the output layer. In Equation 2-3 we use the superscript M on a to

Chapter 2 SuperviSed FeedForward NetworkS

indicate that it is the activation of a neuron in hidden-layer M, where there are M hidden

layers numbered from 1 through M.

¶¶

(2-3)

There are two complications when we deal with the weights feeding hidden layers.

Let’s consider the weights leading from hidden-layer M−1 to hidden-layer M, the last

hidden layer. We ultimately want the partial derivatives of the error with respect to each

of these weights. As when dealing with the output layer, we’ll split this derivative into the

product of the derivative of the net input feeding this neuron with respect to the weight

times the derivative of the error with respect to this neuron’s net input. As before, the

former term here is trivial; it’s just the activation of the prior neuron feeding through this

weight. It’s the latter that’s messy.

The first complication is that the outputs are linear: the output activation is just the

net input to that output neuron. But the hidden neurons are nonlinear. In particular,

the function that maps the net input of a hidden neuron to its activation is the logistic

function shown in Equation 1-4. So, the chain rule states that the derivative of the error

with respect to the net input is the derivative of the error with respect to the output

times the derivative of the output with respect to the input. Luckily, the derivative of the

logistic function f (a) is simple, as shown in Equation 2-4.

()

fa fa fa1

(2-4)

The remaining term is more complicated because the output of a neuron in a hidden

layer feeds into every neuron in the next layer and thus impacts the error through

every one of those paths. Recall that δ

is the derivative of the error with respect to

the weighted sum coming into output neuron k. The contribution to this weighted

sum going into output neuron k from neuron i in the prior layer M is the activation of

hidden neuron i times the weight connecting it to output neuron k. So, the impact on

the derivative of the error from the activation of neuron i that goes through this path is

times the connecting weight. Since neuron i impacts the error through all output

neurons, we must sum these contributions, as shown in Equation 2-5.

¶¶

(2-5)

Chapter 2 SuperviSed FeedForward NetworkS

剩余221页未读，继续阅读

WindStand

粉丝: 35
资源: 367

深度信念网络C++与CUDA实现：卷1

Deep Belief Nets in C++ and CUDA C: Volume 3: Convolutional Nets

Deep Belief Nets in C++ and CUDA C Volume 2 Autoencoding in the Complex epub

Deep Belief Nets in C++ and CUDA C Volume 1 Restricted Boltzmann 无水印原版pdf

Deep Belief Nets in C++ and CUDA C Volume

Deep Belief Nets in C++ and CUDA C Volume 1 Restricted Boltzmann Machines epub

Deep Belief Nets in C++ and CUDA C Volume 2 Autoencoding in the Complex 无水印原版pdf

Deep Belief Nets in C and CUDA C Volume 1.pdf

Deep Belief Nets in C and CUDA C

Deep_Belief_nets.zip_Deep_Belief_nets_HINTON_complementary prior

RBM-DBN.rar_DBN matlab_Deep Belief Nets_MATLAB实现DBN_Matlab DBN

最新资源