深度信念网络C与CUDA C实现：卷1

需积分: 9 197 浏览量更新于2023-05-25 收藏 4.02MB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

"Deep Belief Nets in C and CUDA C Volume 1.pdf" 是一本面向已经有一定神经网络基础知识的读者的书籍，旨在介绍、探索和编程深度信念网络（Deep Belief Nets，DBNs）。作者Timothy Masters在书中： 1. 阐述了深度信念网络的动机，解释了为什么这种网络架构在机器学习中具有重要地位。 2. 推导并解释了最常见深度信念网络组件的关键方程，尽管没有深入到高级数学理论，但给出了足够的依据。 3. 提供了通用深度信念网络训练、执行和分析算法，这些算法不受特定编程语言限制。 4. 作为DEEP程序的详细用户手册，该程序可免费从作者网站下载，书中深入介绍了程序的内部工作原理。 5. 提供了C++代码实现，支持多线程在Windows系统上的执行，以及利用NVIDIA CUDA兼容GPU卡的超级计算机能力的CUDA C实现。书中的重点内容包括受限玻尔兹曼机（Restricted Boltzmann Machines, RBMs）和监督前馈网络（Supervised Feedforward Networks），这是构建深度信念网络的基础。RBMs是一种二分图模型，能用于特征学习和无监督预训练，而监督前馈网络则用于有监督的学习任务，如分类和回归。 CUDA是NVIDIA开发的一种并行计算平台和编程模型，用于加速计算密集型任务，如深度学习。通过CUDA C，开发者可以利用GPU的强大计算能力来提高深度信念网络的训练速度和效率。书中并未涉及详尽的数学理论，而是更注重实用性和实践价值。作者仅介绍了他在实际应用中发现最有价值的模型，但这并不意味着其他未提及的模型不重要或无效，只是它们可能未在作者的具体应用中表现出突出的实用性。这本书是深度学习爱好者和实践者的宝贵资源，特别是对于那些希望用C++和CUDA C进行深度信念网络实现的读者来说，它提供了从理论到实践的桥梁。通过阅读和实践，读者能够掌握如何构建、训练和优化深度信念网络，同时利用GPU的计算优势提升模型性能。

资源详情

资源推荐

2) For each training case, use the activations of the trained hidden

layer as inputs to another hidden layer. Teach this second hidden

layer to reectively reproduce its inputs.

3) Repeat this for as many hidden layers as desired. Each new hidden

layer encodes higher-level patterns. For example, the raw data

input may be a rectangular array of gray-level pixels. e rst

hidden layer may detect edges in various parts of the image. e

second hidden layer may detect patterns of connection of these

edges. A third hidden layer may assemble recognizable objects

out of the connected edges. Such progressive feature discovery is

impossible in a shallow network.

4) Use supervised training to map the activations of the hidden

neurons in the last (topmost) hidden layer to classes or

predictions. Alternatively, use the weights learned in the

unsupervised training as initial values for supervised training of a

traditional but very deep feedforward neural network.

Many variations of this general algorithm are possible, and we will explore several of

the most popular and useful variations in this book. But in a broad sense, the algorithm

described here covers much of the application of deep belief nets today.

One of the most fascinating properties of deep belief nets is their remarkable ability

to generalize beyond the universe of training examples. This is likely because the

output layer, rather than seeing the raw data, is seeing “universal” patterns in the raw

data—patterns that because of their universality are likely to reappear in the general

population.

A closely related property is that deep belief nets are shockingly robust against

overfitting. Every beginning statistics student learns the importance of using many, many

more training cases than optimizable parameters. The standard wisdom is that if one

uses 100 cases to train a model with 50 optimizable parameters, the resulting model will

learn as much about the noise in the training set as it learns about the legitimate patterns

and will hence be worthless. But consider the MNIST dataset of handwritten digits. Its

standard training set has 60,000 training cases and 10,000 independent test cases. One

can use these 60,000 cases to train a deep belief net that has several million optimizable

parameters and yet achieve on the order of 1 percent error rate on the test set!

Chapter 1 IntroduCtIon

Suppose we have K output neurons numbered 1 through K. For a given training case,

let t

be the true value for this case, the value that we hope the network will produce, and

let o

be the output actually obtained. Then the mean squared error (MSE) for this single

case is given by Equation 2-1. To compute the MSE for the entire training set, sum this

quantity for all cases and divide by the number of cases.

()

(2-1)

Supervised training of a multiple-layer feedforward network amounts to finding

the weights and bias terms that minimize Equation 2-1. In any numerical minimization

algorithm, it is of great benefit to be able to efficiently compute the gradient, the partial

derivatives of the criterion being minimized with respect to each individual parameter.

Luckily, this is quite easy in this application. We just start at the output layer and work

backward, repeatedly invoking the chain rule of differentiation.

The activation of output neuron k is just a weighted sum of the activations of the

neurons in the prior layer. We’ll use the Greek letter delta to designate the derivative of

the error with respect to this weighted sum coming into output neuron k. Examination

of Equation 2-1 shows this to simply be as shown in Equation 2-2. In this equation, the

superscript O means that this signifies the delta of an output neuron.

ot=-

()

(2-2)

This neuron is receiving a weighted sum of activations from all neurons in the

prior layer. How can we compute the derivative of the error with respect to the weight

from neuron i in the prior layer? The simple chain rule says that this is the product of

the derivative in Equation 2-2 times the derivative of the net input (the weighted sum

coming into this output neuron) with respect to this weight.

This latter term is trivial. The contribution to the weighted sum from neuron i in

the prior layer is just the activation of that neuron times the weight connecting it to the

output neuron k. We shall designate this output weight as w

. So, the derivative of that

weighted sum with respect to w

is just the activation of neuron i. This leads us to the

formula for the partial derivatives of the error with respect to the weights connecting the

last hidden layer to the output layer. In Equation 2-3 we use the superscript M on a to

Chapter 2 SuperviSed FeedForward NetworkS

indicate that it is the activation of a neuron in hidden-layer M, where there are M hidden

layers numbered from 1 through M.

¶¶

(2-3)

There are two complications when we deal with the weights feeding hidden layers.

Let’s consider the weights leading from hidden-layer M−1 to hidden-layer M, the last

hidden layer. We ultimately want the partial derivatives of the error with respect to each

of these weights. As when dealing with the output layer, we’ll split this derivative into the

product of the derivative of the net input feeding this neuron with respect to the weight

times the derivative of the error with respect to this neuron’s net input. As before, the

former term here is trivial; it’s just the activation of the prior neuron feeding through this

weight. It’s the latter that’s messy.

The first complication is that the outputs are linear: the output activation is just the

net input to that output neuron. But the hidden neurons are nonlinear. In particular,

the function that maps the net input of a hidden neuron to its activation is the logistic

function shown in Equation 1-4. So, the chain rule states that the derivative of the error

with respect to the net input is the derivative of the error with respect to the output

times the derivative of the output with respect to the input. Luckily, the derivative of the

logistic function f (a) is simple, as shown in Equation 2-4.

()

fa fa fa1

(2-4)

The remaining term is more complicated because the output of a neuron in a hidden

layer feeds into every neuron in the next layer and thus impacts the error through

every one of those paths. Recall that δ

is the derivative of the error with respect to

the weighted sum coming into output neuron k. The contribution to this weighted

sum going into output neuron k from neuron i in the prior layer M is the activation of

hidden neuron i times the weight connecting it to output neuron k. So, the impact on

the derivative of the error from the activation of neuron i that goes through this path is

times the connecting weight. Since neuron i impacts the error through all output

neurons, we must sum these contributions, as shown in Equation 2-5.

¶¶

(2-5)

Chapter 2 SuperviSed FeedForward NetworkS

剩余224页未读，继续阅读

WindStand

粉丝: 35
资源: 367

会员权益专享

深度信念网络C与CUDA C实现：卷1

Deep Belief Nets in C++ and CUDA C Volume 2 Autoencoding in the Complex epub

深度信念网络C++与CUDA实现：卷1

C++与CUDA C实现深度学习：卷1 - RBMs与监督网络

"深度学习中的受限玻尔兹曼机网络原理与实践解析

"神经网络与深度学习绪论及课程概要详解

"Step-by-Step 第三册答案及原文PDF下载

Hinton首次提出深度学习的概念是在Reduc the dimensionality of data with neural networks还是 a fast learning algorithm for deep belief nets中？

deep belief network的Matlab代码

全连接神经网络的相关参考文献

请给我找几篇有关神经网络的文献

python keras

deep belief network 用于时间序列分类的matlab代码

deep belief network

Deep Learning Toolbox

找几篇关于深度学习轴承监测诊断的案例、优缺点分析的近5年的文献

关于MATLAB的近期参考文献

Write a speech which can be read in 5 minutes, and the topic is"the power of self confidence"

Write an essay about 150 words. Statec learly what you stand for. Give convincing reasons for your arguments.

medeff中介效应模型参考文献

LDPC matlab

会员权益专享

最新资源