深度学习权威指南：Yoshua Bengio新书PDF精华版

5星 · 超过95%的资源需积分: 9 122 浏览量更新于2024-07-21 9 收藏 20.48MB PDF 举报

《Deep Learning》是由Ian Goodfellow、Aaron Courville和Yoshua Bengio三位知名专家合著的一本深度学习领域的经典著作。该书于2016年出版，随着技术的发展和更新，这份资料是基于2011年至2018年的版本整理而来，相较于之前的版本有所增补和修订。本书深入浅出地介绍了深度学习的基础概念和技术，适合那些希望在人工智能领域特别是神经网络方面有深入了解的学习者阅读。第一部分，"Introduction"，首先引导读者了解深度学习的历史背景和发展趋势，阐述了为何在这个时代深度学习变得如此重要，以及它如何在诸如计算机视觉、自然语言处理等众多领域取得突破性进展。第二部分，"Applied Math and Machine Learning Basics"，详细讲解了线性代数基础，涵盖了关键概念如标量、向量、矩阵和张量的定义与操作，以及矩阵乘法、单位矩阵与逆矩阵的性质。这部分内容强调了这些数学工具在神经网络中的核心作用，例如线性依赖和span的概念，以及 norms（范数）对于理解网络权重的重要性。作者还介绍了特征值分解（eigendecomposition）、奇异值分解（SVD）和Moore-Penrose伪逆（pseudoinverse），这些都是优化深层网络时不可或缺的技术。接下来的章节转向概率论和信息理论，解释了概率的基本概念，如随机变量、概率分布、边际概率和条件概率，以及它们在模型训练和决策过程中的运用。链式法则（chain rule）和独立性和条件独立性的概念在这里也得到了详尽阐述。作者还将探讨期望、方差和协方差，这些都是评估模型性能和不确定性的重要指标。最后，通过信息熵和交叉熵这样的信息论概念，读者能理解深度学习模型如何通过最小化损失函数来逼近数据的分布。《Deep Learning》这本书以清晰的逻辑结构和深入浅出的方式，为读者提供了深度学习坚实的数学基础和理论支撑，是每个想要进入或进一步研究深度学习领域的学生和专业人士不可或缺的参考资料。通过学习本书，读者将能够掌握深度学习的核心原理，并为实际应用打下坚实的基础。

CHAPTER 1. INTRODUCTION

Cartesian coordinates

Polar coordinates

Figure 1.1: Example of diﬀerent representations: suppose we want to separate two

categories of data by drawing a line between them in a scatterplot. In the plot on the left,

we represent some data using Cartesian coordinates, and the task is impossible. In the plot

on the right, we represent the data with polar coordinates and the task becomes simple to

solve with a vertical line. (Figure produced in collaboration with David Warde-Farley)

on the representation of the data they are given. For example, when logistic

regression is used to recommend cesarean delivery, the AI system does not examine

the patient directly. Instead, the doctor tells the system several pieces of relevant

information, such as the presence or absence of a uterine scar. Each piece of

information included in the representation of the patient is known as a feature.

Logistic regression learns how each of these features of the patient correlates with

various outcomes. However, it cannot inﬂuence the way that the features are

deﬁned in any way. If logistic regression was given a 3-D MRI image of the patient,

rather than the doctor’s formalized report, it would not be able to make useful

predictions. Individual voxels

in an MRI scan have negligible correlation with

any complications that might occur during delivery.

This dependence on representations is a general phenomenon that appears

throughout computer science and even daily life. In computer science, operations

such as searching a collection of data can proceed exponentially faster if the collec-

tion is structured and indexed intelligently. People can easily perform arithmetic

on Arabic numerals, but ﬁnd arithmetic on Roman numerals much more time

consuming. It is not surprising that the choice of representation has an enormous

eﬀect on the performance of machine learning algorithms. For a simple visual

example, see Fig. 1.1.

Many artiﬁcial intelligence tasks can be solved by designing the right set of

A voxel is the value at a single point in a 3-D scan, much as a pixel as the value at a single

point in an image.

CHAPTER 1. INTRODUCTION

features to extract for that task, then providing these features to a simple machine

learning algorithm. For example, a useful feature for speaker identiﬁcation from

sound is the pitch. The pitch can be formally speciﬁed—it is the lowest frequency

major peak of the spectrogram. It is useful for speaker identiﬁcation because it is

determined by the size of the vocal tract, and therefore gives a strong clue as to

whether the speaker is a man, woman, or child.

However, for many tasks, it is diﬃcult to know what features should be extracted.

For example, suppose that we would like to write a program to detect cars in

photographs. We know that cars have wheels, so we might like to use the presence

of a wheel as a feature. Unfortunately, it is diﬃcult to describe exactly what a

wheel looks like in terms of pixel values. A wheel has a simple geometric shape but

its image may be complicated by shadows falling on the wheel, the sun glaring oﬀ

the metal parts of the wheel, the fender of the car or an object in the foreground

obscuring part of the wheel, and so on.

One solution to this problem is to use machine learning to discover not only

the mapping from representation to output but also the representation itself.

This approach is known as representation learning. Learned representations often

result in much better performance than can be obtained with hand-designed

representations. They also allow AI systems to rapidly adapt to new tasks, with

minimal human intervention. A representation learning algorithm can discover a

good set of features for a simple task in minutes, or a complex task in hours to

months. Manually designing features for a complex task requires a great deal of

human time and eﬀort; it can take decades for an entire community of researchers.

The quintessential example of a representation learning algorithm is the au-

toencoder. An autoencoder is the combination of an encoder function that converts

the input data into a diﬀerent representation, and a decoder function that converts

the new representation back into the original format. Autoencoders are trained to

preserve as much information as possible when an input is run through the encoder

and then the decoder, but are also trained to make the new representation have

various nice properties. Diﬀerent kinds of autoencoders aim to achieve diﬀerent

kinds of properties.

When designing features or algorithms for learning features, our goal is usually

to separate the that explain the observed data. In this context,factors of variation

we use the word “factors” simply to refer to separate sources of inﬂuence; the factors

are usually not combined by multiplication. Such factors are often not quantities

that are directly observed but they may exist either as unobserved objects or forces

in the physical world that aﬀect observable quantities, or they are constructs in

the human mind that provide useful simplifying explanations or inferred causes

of the observed data. They can be thought of as concepts or abstractions that

help us make sense of the rich variability in the data. When analyzing a speech

CHAPTER 1. INTRODUCTION

recording, the factors of variation include the speaker’s age, their sex, their accent

and the words that they are speaking. When analyzing an image of a car, the

factors of variation include the position of the car, its color, and the angle and

brightness of the sun.

A major source of diﬃculty in many real-world artiﬁcial intelligence applications

is that many of the factors of variation inﬂuence every single piece of data we are

able to observe. The individual pixels in an image of a red car might be very close

to black at night. The shape of the car’s silhouette depends on the viewing angle.

Most applications require us to the factors of variation and discard thedisentangle

ones that we do not care about.

Of course, it can be very diﬃcult to extract such high-level, abstract features

from raw data. Many of these factors of variation, such as a speaker’s accent,

can only be identiﬁed using sophisticated, nearly human-level understanding of

the data. When it is nearly as diﬃcult to obtain a representation as to solve the

original problem, representation learning does not, at ﬁrst glance, seem to help us.

Deep learning solves this central problem in representation learning by introduc-

ing representations that are expressed in terms of other, simpler representations.

Deep learning allows the computer to build complex concepts out of simpler con-

cepts. Fig. 1.2 shows how a deep learning system can represent the concept of an

image of a person by combining simpler concepts, such as corners and contours,

which are in turn deﬁned in terms of edges.

The quintessential example of a deep learning model is the feedforward deep

network or multilayer perceptron (MLP). A multilayer perceptron is just a mathe-

matical function mapping some set of input values to output values. The function

is formed by composing many simpler functions. We can think of each application

of a diﬀerent mathematical function as providing a new representation of the input.

The idea of learning the right representation for the data provides one perspec-

tive on deep learning. Another perspective on deep learning is that it allows the

computer to learn a multi-step computer program. Each layer of the representation

can be thought of as the state of the computer’s memory after executing another

set of instructions in parallel. Networks with greater depth can execute more in-

structions in sequence. Being able to execute instructions sequentially oﬀers great

power because later instructions can refer back to the results of earlier instructions.

According to this view of deep learning, not all of the information in a layer’s

representation of the input necessarily encodes factors of variation that explain

the input. The representation is also used to store state information that helps to

execute a program that can make sense of the input. This state information could

be analogous to a counter or pointer in a traditional computer program. It has

nothing to do with the content of the input speciﬁcally, but it helps the model to

organize its processing.

CHAPTER 1. INTRODUCTION



⇥

Element

Set

⇥



Element

Set

Logistic

Regression

Logistic

Regression

Figure 1.3: Illustration of computational graphs mapping an input to an output where

each node performs an operation. Depth is the length of the longest path from input to

output but depends on the deﬁnition of what constitutes a possible computational step.

The computation depicted in these graphs is the output of a logistic regression model,

(

), where

is the logistic sigmoid function. If we use addition, multiplication and

logistic sigmoids as the elements of our computer language, then this model has depth

three. If we view logistic regression as an element itself, then this model has depth one.

There are two main ways of measuring the depth of a model.

The ﬁrst view is based on the number of sequential instructions that must

be executed to evaluate the architecture. We can think of this as the length of

the longest path through a ﬂow chart that describes how to compute each of the

model’s outputs given its inputs. Just as two equivalent computer programs will

have diﬀerent lengths depending on which language the program is written in, the

same function may be drawn as a ﬂow chart with diﬀerent depths depending on

which functions we allow to be used as individual steps in the ﬂow chart. Fig. 1.3

illustrates how this choice of language can give two diﬀerent measurements for the

same architecture.

Another approach, used by deep probabilistic models, illustrates not the depth

of the computational graph but the depth of the graph describing how concepts are

related to each other. In this case, the depth of the ﬂow-chart of the computations

needed to compute the representation of each concept may be much deeper than

the graph of the concepts themselves. This is because the system’s understanding

of the simpler concepts can be reﬁned given information about the more complex

concepts. For example, an AI system observing an image of a face with one eye in

shadow may initially only see one eye. After detecting that a face is present, it can

then infer that a second eye is probably present as well. In this case, the graph of

concepts only includes two layers—a layer for eyes and a layer for faces—but the

剩余688页未读，继续阅读

Aimar_yxj

粉丝: 1
资源: 5

深度学习权威指南：Yoshua Bengio新书PDF精华版

深度学习（中文版）（Ian Goodfellow, Yoshua Bengio, Aaron Courville）

【智能】【On Intelligence】Jeff Hawkins.pdf

卷积神经网络CNN学习笔记 pdf

Deep Learning Yoshua Bengio

Deep Learning Yoshua Bengio教授作品

Deep Learning - Yoshua Bengio

深度学习(Deep Learning) Yoshua Bengio & Ian GoodFellow中文版

深度学习(Deep Learning) Yoshua Bengio & Ian GoodFellow中文版1

中文版Deep Learning - Yoshua Bengio

Deep Learning（Yoshua Bengio 2015）

最新资源