没有合适的资源?快使用搜索试试~ 我知道了~
首页深度学习基础教程:TensorFlow实战与历史背景
深度学习基础教程:TensorFlow实战与历史背景
需积分: 9 19 下载量 74 浏览量
更新于2024-07-19
收藏 5.08MB PDF 举报
《深度学习基础(2017) 斯坦福教科书》是一本面向具备基本微积分、矩阵知识和Python编程能力读者编写的教材。对于没有这些背景的读者,虽然可以自学,但可能面临较大的挑战。线性代数背景有助于理解书中部分数学阐述。本书的主要目标是培养读者对深度学习解决问题的直觉,了解现代深度学习方法的历史背景,以及掌握使用开源库TensorFlow实现深度学习算法的能力。 第1章"神经网络"从构建智能机器的角度出发,讨论了传统计算机程序的局限。章节中介绍了神经元的概念,特别是如何将线性感知器表示为神经元,展示了前馈神经网络的结构。接着探讨了线性、sigmoid、tanh和ReLU等不同类型的激活函数,以及它们在处理非线性问题上的作用。章节末尾提到softmax输出层,这是多分类任务中的常见选择。 第二部分"训练前馈神经网络"深入浅出地讲解了问题,比如“快餐问题”,即快速找到最优解的挑战。这里的重点是梯度下降法,包括最简单的梯度下降(即delta规则)以及与学习率相关的调整。章节进一步介绍了针对sigmoid神经元的梯度下降算法,并引入了反向传播算法,这是训练深层网络的关键技术。为了防止过拟合,作者还提到了随机梯度下降和小批量梯度下降这两种优化策略,以及测试集、验证集的使用。 总结来说,《深度学习基础》这本书为读者提供了一个坚实的理论基础,从神经元模型的构建到实际训练方法的实现,都通过实际案例和清晰的步骤进行解释。它不仅教授了深度学习的核心概念,还引导读者理解和应用这些技术,以解决实际问题并提升人工智能算法的性能。通过阅读这本书,读者将能够掌握深度学习的核心工具,为未来在这个领域的探索打下坚实的基础。
资源详情
资源推荐
![](https://csdnimg.cn/release/download_crawler_static/10267379/bg10.jpg)
4 Bubeck, Sébastien. “Convex optimization: Algorithms and complexity.” Foundations and Trends® in Machine
Learning. 8.3-4 (2015): 231-357.
Then it turns out, by selecting
θ =
−24 3 4
T
, our machine learning model makes
the correct prediction on every data point:
h
, θ =
−1 if 3x
1
+ 4x
2
− 24 < 0
1 if 3x
1
+ 4x
2
− 24 ≥ 0
An optimal parameter vector
θ
positions the classifier so that we make as many cor‐
rect predictions as possible. In most cases, there are many (or even infinitely
many) possible choices for
θ
that are optimal. Fortunately for us, most of the time
these alternatives are so close to one another that the difference is negligible. If this is
not the case, we may want to collect more data to narrow our choice of
θ
.
While the setup seems reasonable, there are still some pretty significant questions
that remain. First off, how do we even come up with an optimal value for the parame‐
ter vector
θ
in the first place? Solving this problem requires a technique commonly
known as optimization. An optimizer aims to maximize the performance of a
machine learning model by iteratively tweaking its parameters until the error is mini‐
mized. We’ll begin to tackle this question of learning parameter vectors in more detail
in
Chapter 2, when we describe the process of gradient descent.
4
In later chapters, we’ll
try to find ways to make this process even more efficient.
Second, it’s quite clear that this particular model (the linear perceptron model) is
quite limited in the relationships it can learn. For example, the distributions of data
shown in Figure 1-5 cannot be described well by a linear perceptron.
Figure 1-5. As our data takes on more complex forms, we need more complex models to
describe them
But these situations are only the tip of the iceberg. As we move on to much more
complex problems, such as object recognition and text analysis, our data becomes
extremely high dimensional, and the relationships we want to capture become highly
![](https://csdnimg.cn/release/download_crawler_static/10267379/bg11.jpg)
5 Restak, Richard M. and David Grubin. e Secret Life of the Brain. Joseph Henry Press, 2001.
nonlinear. To accommodate this complexity, recent research in machine learning has
attempted to build models that resemble the structures utilized by our brains. It’s
essentially this body of research, commonly referred to as deep learning, that has had
spectacular success in tackling problems in computer vision and natural language
processing. These algorithms not only far surpass other kinds of machine learning
algorithms, but also rival (or even exceed!) the accuracies achieved by humans.
The Neuron
The foundational unit of the human brain is the neuron. A tiny piece of the brain,
about the size of grain of rice, contains over 10,000 neurons, each of which forms an
average of 6,000 connections with other neurons.
5
It’s this massive biological network
that enables us to experience the world around us. Our goal in this section will be to
use this natural structure to build machine learning models that solve problems in an
analogous way.
At its core, the neuron is optimized to receive information from other neurons, pro‐
cess this information in a unique way, and send its result to other cells. This process is
summarized in Figure 1-6. The neuron receives its inputs along antennae-like struc‐
tures called dendrites. Each of these incoming connections is dynamically strength‐
ened or weakened based on how often it is used (this is how we learn new concepts!),
and it’s the strength of each connection that determines the contribution of the input
to the neuron’s output. After being weighted by the strength of their respective con‐
nections, the inputs are summed together in the cell body. This sum is then trans‐
formed into a new signal that’s propagated along the cell’s axon and sent off to other
neurons.
Figure 1-6. A functional description of a biological neuron’s structure
![](https://csdnimg.cn/release/download_crawler_static/10267379/bg12.jpg)
6 McCulloch, Warren S., and Walter Pitts. “A logical calculus of the ideas immanent in nervous activity.” e
Bulletin of Mathematical Biophysics. 5.4 (1943): 115-133.
We can translate this functional understanding of the neurons in our brain into an
artificial model that we can represent on our computer. Such a model is described
in
Figure 1-7, leveraging the approach first pioneered in 1943 by Warren S. McCul‐
loch and Walter H. Pitts.
6
Just as in biological neurons, our artificial neuron takes in
some number of inputs, x
1
, x
2
, . . . , x
n
, each of which is multiplied by a specific
weight,
w
1
, w
2
, . . . , w
n
. These weighted inputs are, as before, summed together to
produce the logit of the neuron,
z = ∑
i = 0
n
w
i
x
i
. In many cases, the logit also includes
a bias, which is a constant (not shown in the figure). The logit is then passed through
a function
f
to produce the output
y = f
z
. This output can be transmitted to other
neurons.
Figure 1-7. Schematic for a neuron in an articial neural net
We’ll conclude our mathematical discussion of the artificial neuron by re-expressing
its functionality in vector form. Let’s reformulate the inputs as a vector x = [x
1
x
2
... x
n
]
and the weights of the neuron as w = [w
1
w
2
... w
n
]. Then we can re-express the output
of the neuron as
y = f
· + b
, where
b
is the bias term. In other words, we can
compute the output by performing the dot product of the input and weight vectors,
adding in the bias term to produce the logit, and then applying the transformation
function. While this seems like a trivial reformulation, thinking about neurons as a
series of vector manipulations will be crucial to how we implement them in software
later in this book.
Expressing Linear Perceptrons as Neurons
In “The Mechanics of Machine Learning” on page 3, we talked about using machine
learning models to capture the relationship between success on exams and time spent
studying and sleeping. To tackle this problem, we constructed a linear perceptron
classifier that divided the Cartesian coordinate plane into two halves:
![](https://csdnimg.cn/release/download_crawler_static/10267379/bg13.jpg)
h
, θ =
−1 if 3x
1
+ 4x
2
− 24 < 0
1 if 3x
1
+ 4x
2
− 24 ≥ 0
As shown in
Figure 1-4, this is an optimal choice for
θ
because it correctly classifies
every sample in our dataset. Here, we show that our model h is easily using a neuron.
Consider the neuron depicted in Figure 1-8. The neuron has two inputs, a bias, and
uses the function:
f
z =
−1 if z < 0
1 if z ≥ 0
It’s very easy to show that our linear perceptron and the neuronal model are perfectly
equivalent. And in general, it’s quite simple to show that singular neurons are strictly
more expressive than linear perceptrons. In other words, every linear perceptron can
be expressed as a single neuron, but single neurons can also express models that can‐
not be expressed by any linear perceptron.
Figure 1-8. Expressing our exam performance perceptron as a neuron
Feed-Forward Neural Networks
Although single neurons are more powerful than linear perceptrons, they’re not
nearly expressive enough to solve complicated learning problems. There’s a reason
our brain is made of more than one neuron. For example, it is impossible for a single
neuron to differentiate handwritten digits. So to tackle much more complicated tasks,
we’ll have to take our machine learning model even further.
The neurons in the human brain are organized in layers. In fact, the human cerebral
cortex (the structure responsible for most of human intelligence) is made up of six
![](https://csdnimg.cn/release/download_crawler_static/10267379/bg14.jpg)
7 Mountcastle, Vernon B. “Modality and topographic properties of single neurons of cat’s somatic sensory cor‐
tex.” Journal of Neurophysiology 20.4 (1957): 408-434.
layers.
7
Information flows from one layer to another until sensory input is converted
into conceptual understanding. For example, the bottommost layer of the visual cor‐
tex receives raw visual data from the eyes. This information is processed by each layer
and passed on to the next until, in the sixth layer, we conclude whether we are look‐
ing at a cat, or a soda can, or an airplane. Figure 1-9 shows a more simplified version
of these layers.
Figure 1-9. A simple example of a feed-forward neural network with three layers (input,
one hidden, and output) and three neurons per layer
Borrowing from these concepts, we can construct an articial neural network. A neu‐
ral network comes about when we start hooking up neurons to each other, the input
data, and to the output nodes, which correspond to the network’s answer to a learn‐
ing problem. Figure 1-9 demonstrates a simple example of an artificial neural net‐
work, similar to the architecture described in McCulloch and Pitt’s work in 1943. The
剩余287页未读,继续阅读
![rar](https://img-home.csdnimg.cn/images/20210720083606.png)
![-](https://img-home.csdnimg.cn/images/20210720083327.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://profile-avatar.csdnimg.cn/default.jpg!1)
chaonan_shen
- 粉丝: 26
- 资源: 3
上传资源 快速赚钱
我的内容管理 展开
我的资源 快来上传第一个资源
我的收益
登录查看自己的收益我的积分 登录查看自己的积分
我的C币 登录后查看C币余额
我的收藏
我的下载
下载帮助
![](https://csdnimg.cn/release/wenkucmsfe/public/img/voice.245cc511.png)
最新资源
- Lombok 快速入门与注解详解
- SpringSecurity实战:声明式安全控制框架解析
- XML基础教程:从数据传输到存储解析
- Matlab实现图像空间平移与镜像变换示例
- Python流程控制与运算符详解
- Python基础:类型转换与循环语句
- 辰科CD-6024-4控制器说明书:LED亮度调节与触发功能解析
- AE particular插件全面解析:英汉对照与关键参数
- Shell脚本实践:创建tar包、字符串累加与简易运算器
- TMS320F28335:浮点处理器与ADC详解
- 互联网基础与结构解析:从ARPANET到多层次ISP
- Redhat系统中构建与Windows共享的Samba服务器实战
- microPython编程指南:从入门到实践
- 数据结构实验:顺序构建并遍历链表
- NVIDIA TX2系统安装与恢复指南
- C语言实现贪吃蛇游戏基础代码
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035111.png)
安全验证
文档复制为VIP权益,开通VIP直接复制
![](https://csdnimg.cn/release/wenkucmsfe/public/img/green-success.6a4acb44.png)