深度学习入门经典： Ian Goodfellow等人的《花书》解析

需积分: 48 124 浏览量更新于2024-07-19 3 收藏 62.56MB PDF 举报

《花书：深度学习》是一本由深度学习领域的三位权威专家——Ian Goodfellow、Yoshua Bengio和Aaron Courville合著的经典教程，它在深度学习的学习者群体中享有极高的声誉。本书旨在为读者提供深入浅出的深度学习基础知识，包括但不限于线性代数、概率与信息理论等内容，帮助读者理解并掌握这一前沿技术。首先，章节1的"Introduction"介绍了书籍的目标读者群和深度学习的历史发展趋势。它强调了这本书对于想要入门或进一步提升深度学习技能的专业人士的价值，同时回顾了自神经网络兴起以来深度学习领域的重要里程碑。在"Applied Math and Machine Learning Basics"部分，第2章详细讲解了线性代数的基础概念。这包括： scalars、vectors、matrices和tensors的基本运算，如矩阵乘法、单位矩阵和逆矩阵的性质，以及线性依赖和span的概念。此外，还有关于向量和矩阵的范数、特征值分解（Eigen decomposition）、奇异值分解（Singular Value Decomposition）、 Moore-Penrose伪逆矩阵、迹（Trace）运算和行列式的计算。这些内容都是深度学习中不可或缺的数学工具，尤其是在神经网络的设计和优化过程中。第3章涵盖了概率和信息论的核心原理，解释了为什么概率在机器学习中的重要性。这里介绍了随机变量、概率分布、边际和条件概率，以及链式法则和独立性概念。这些知识有助于理解深度学习模型的训练过程，特别是关于梯度下降和反向传播算法背后的统计推导。 "Probability and Information Theory"部分还涉及期望、方差和协方差等概念，这些都是衡量数据集特性、评估模型性能的关键指标。通过这些概念，读者可以更好地理解和处理模型的不确定性以及模型参数的优化问题。《花书：深度学习》作为一本深度学习的经典教材，不仅提供了扎实的数学基础，还阐述了概率与信息理论在深度学习中的实际应用，是任何希望在这个领域取得成就的学者和从业者的重要参考资源。无论是对初学者还是经验丰富的研究人员，这本书都能提供系统且详尽的知识体系，帮助他们在这个不断发展的领域保持竞争力。

CONTENTS

Functions

f f: A B→ The function with domain and rangeA B

f g f g◦ Composition of the functions and

f( ; )x θ

A function of

parametrized by

. Sometimes

we just write

(

) and ignore the argument

lighten notation.

log x xNatural logarithm of

σ x( ) Logistic sigmoid,

1 + exp( )−x

ζ x x( ) log(1 + exp(Softplus, ))

|| ||x

norm of x

|| ||x L

norm of x

Positive part of , i.e.,x max(0 ), x

condition

is 1 if the condition is true, 0 otherwise

Sometimes we use a function

whose argument is a scalar, but apply it to a vector,

matrix, or tensor:

(

), or

(

). This means to apply

to the array

element-wise. For example, if

(

), then C

i,j,k

) for all valid values

of , and .i j k

Datasets and distributions

data

The data generating distribution

ˆp

data

The empirical distribution deﬁned by the training

set

X A set of training examples

( )i

The -th example (input) from a dataseti

( )i

or y

( )i

The target associated with

( )i

for supervised learn-

ing

The

m n×

matrix with input example

( )i

in row

i,:

xiv

CHAPTER 1. INTRODUCTION

concepts are built on top of each other, the graph is deep, with many layers. For

this reason, we call this approach to AI deep learning.

Many of the early successes of AI took place in relatively sterile and formal

environments and did not require computers to have much knowledge about

the world.For example, IBM’s Deep Blue chess-playing system defeated world

champion Garry Kasparov in 1997 ( , ). Chess is of course a very simpleHsu 2002

world, containing only sixty-four locations and thirty-two pieces that can move

in only rigidly circumscribed ways. Devising a successful chess strategy isa

tremendous accomplishment,but the challenge is not due to the diﬃculty of

describing the set of chess pieces and allowable moves to the computer. Chess

can be completely described by a very brief list of completely formal rules, easily

provided ahead of time by the programmer.

Ironically, abstract and formal tasks that are among the most diﬃcult mental

undertakings for a human being are among the easiest for a computer. Computers

have long been able to defeat even the best human chess player, but are only

recently matching some of the abilities of average human beings to recognize objects

or speech. A person’s everyday life requires an immense amount of knowledge

about the world. Much of this knowledge is subjective and intuitive, and therefore

diﬃcult to articulate in a formal way. Computers need to capture this same

knowledge in order to behave in an intelligent way. One of the key challenges in

artiﬁcial intelligence is how to get this informal knowledge into a computer.

Several artiﬁcial intelligence projects have sought to hard-code knowledge about

the world in formal languages. A computer can reason about statements in these

formal languages automatically using logical inference rules. This is known as the

knowledge base approach to artiﬁcial intelligence. None of these projects has led to

a major success. One of the most famous such projects is Cyc ( ,Lenat and Guha

1989). Cyc is an inference engine and a database of statements in a language

called CycL. These statements are entered by a staﬀ of human supervisors. It is an

unwieldy process. People struggle to devise formal rules with enough complexity

to accurately describe the world. For example, Cyc failed to understand a story

about a person named Fred shaving in the morning ( , ). Its inferenceLinde 1992

engine detected an inconsistency in the story:it knew that people do not have

electrical parts, but because Fred was holding an electric razor, it believed the

entity “FredWhileShaving” contained electrical parts. It therefore asked whether

Fred was still a person while he was shaving.

The diﬃculties faced by systems relying on hard-coded knowledge suggest that

AI systems need the ability to acquire their own knowledge, by extracting patterns

from raw data. This capability is known as machine learning. The introduction

CHAPTER 1. INTRODUCTION

of machine learning allowed computers to tackle problems involving knowledge

of the real world and make decisions that appear subjective. A simple machine

learning algorithm called logistic regression can determine whether to recommend

cesarean delivery (Mor-Yosef 1990et al., ). A simple machine learning algorithm

called can separate legitimate e-mail from spam e-mail.naive Bayes

The performance of these simple machine learning algorithms depends heavily

on the representation of the data they are given. For example, when logistic

regression is used to recommend cesarean delivery, the AI system does not examine

the patient directly. Instead, the doctor tells the system several pieces of relevant

information, such as the presence or absence of a uterine scar. Each piece of

information included in the representation of the patient is known as a feature.

Logistic regression learns how each of these features of the patient correlates with

various outcomes. However, it cannot inﬂuence the way that the features are

deﬁned in any way.If logistic regression was given an MRI scan of the patient,

rather than the doctor’s formalized report, it would not be able to make useful

predictions. Individual pixels in an MRI scan have negligible correlation with any

complications that might occur during delivery.

This dependence on representations is a general phenomenon that appears

throughout computer science and even daily life. In computer science, opera-

tions such as searching a collection of data can proceed exponentially faster if

the collection is structured and indexed intelligently.People can easily perform

arithmetic on Arabic numerals, but ﬁnd arithmetic on Roman numerals much

more time-consuming. It is not surprising that the choice of representation has an

enormous eﬀect on the performance of machine learning algorithms. For a simple

visual example, see Fig. .1.1

Many artiﬁcial intelligence tasks can be solved by designing the right set of

features to extract for that task, then providing these features to a simple machine

learning algorithm. For example, a useful feature for speaker identiﬁcation from

sound is an estimate of the size of speaker’s vocal tract. It therefore gives a strong

clue as to whether the speaker is a man, woman, or child.

However, for many tasks, it is diﬃcult to know what features should be extracted.

For example, suppose that we would like to write a program to detect cars in

photographs. We know that cars have wheels, so we might like to use the presence

of a wheel as a feature.Unfortunately, it is diﬃcult to describe exactly what a

wheel looks like in terms of pixel values. A wheel has a simple geometric shape but

its image may be complicated by shadows falling on the wheel, the sun glaring oﬀ

the metal parts of the wheel, the fender of the car or an object in the foreground

obscuring part of the wheel, and so on.

CHAPTER 1. INTRODUCTION

Cartesiancoordinates

Polarcoordinates

Figure 1.1: Example ofdiﬀerent representations: suppose we want to separatetwo

categories of data by drawing a line between them in a scatterplot. In the plot on the left,

we represent some data using Cartesian coordinates, and the task is impossible. In the plot

on the right, we represent the data with polar coordinates and the task becomes simple to

solve with a vertical line. (Figure produced in collaboration with David Warde-Farley)

One solution to this problem is to use machine learning to discover not only

the mapping from representation to output but also the representation itself.

This approach is known as representation learning. Learned representations often

result in much better performancethan can be obtainedwith hand-designed

representations. They also allow AI systems to rapidly adapt to new tasks, with

minimal human intervention. A representation learning algorithm can discover a

good set of features for a simple task in minutes, or a complex task in hours to

months. Manually designing features for a complex task requires a great deal of

human time and eﬀort; it can take decades for an entire community of researchers.

The quintessential example of a representation learning algorithm is the au-

toencoder. An autoencoder is the combination of an encoder function that converts

the input data into a diﬀerent representation, and a decoder function that converts

the new representation back into the original format. Autoencoders are trained to

preserve as much information as possible when an input is run through the encoder

and then the decoder, but are also trained to make the new representation have

various nice properties. Diﬀerent kinds of autoencoders aim to achieve diﬀerent

kinds of properties.

When designing features or algorithms for learning features, our goal is usually

to separate the that explain the observed data. In this context,factors of variation

we use the word “factors” simply to refer to separate sources of inﬂuence; the factors

are usually not combined by multiplication. Such factors are often not quantities

剩余802页未读，继续阅读

kerrity

粉丝: 2
资源: 2

深度学习入门经典： Ian Goodfellow等人的《花书》解析

深度学习 [deep learning] AI圣经 Deep Learning 英文版 (花书)

Deeplearning深度学习笔记v5.61.pdf

花皮书deep learning

deeplearning4j书籍

deep learning 英文版pdf

deep learning with python 中文版

neural networks and deep learning 这本书怎么样？

deep learning with python 2nd

deep learning ian goodfellowpdf

吴恩达deeplearning.aipdf

最新资源