深度学习基础：Ian Goodfellow著，线性代数与概率理论概览

5星 · 超过95%的资源需积分: 9 127 浏览量更新于2024-07-19 1 收藏 13.98MB PDF 举报

"《深度学习》- Ian Goodfellow (2017-10-31)，英文原版，带有完整书签并进行了大小优化。" 《深度学习》是由Ian Goodfellow、Yoshua Bengio和Aaron Courville合著的一本经典著作，这本书深入浅出地介绍了深度学习的理论基础和技术应用。书中涵盖了广泛的主题，是理解现代机器学习，特别是深度神经网络的基石。在介绍部分，作者强调了这本书面向的读者群体，包括那些希望深入理解深度学习的计算机科学家、工程师和数学家。同时，它也反映了深度学习自20世纪末以来的发展趋势，展示了这个领域如何从边缘走向主流，并且持续影响着人工智能的进步。第一部分“应用数学和机器学习基础”中，第一章“线性代数”是深度学习的数学基础。这部分从基本概念出发，如标量、向量、矩阵和张量，详细解释了它们在多维空间中的表示和运算。接着，书中讨论了矩阵与向量的乘法、单位矩阵与逆矩阵的概念，以及线性相关和线性独立的概念，这些都是理解线性变换的关键。此外，书中还涵盖了范数、特殊类型的矩阵和向量（如对角矩阵、正交矩阵等）、特征分解、奇异值分解、伪逆矩阵、迹操作和行列式，这些都是数据分析和优化中不可或缺的部分。以主成分分析为例，展示了这些概念在实际问题中的应用。第三章“概率和信息理论”阐述了概率论在深度学习中的核心地位。首先，作者探讨了为何需要概率，然后介绍了随机变量、概率分布和边缘概率的概念。条件概率、条件概率链式规则、独立性和条件独立性的讨论进一步深化了读者对概率的理解。期望、方差和协方差是衡量随机变量统计特性的重要工具，而常见概率分布（如均匀分布、高斯分布等）的介绍则为后续建模提供了基础。这本书详细讲解了深度学习所需的数学和统计基础，对于想要深入学习深度学习的读者来说，是一份宝贵的参考资料。通过这些基础知识的学习，读者将能够更好地理解深度神经网络的内部工作原理，以及如何有效地训练和优化这些网络。

CONTENTS

Functions

f f: A B→ The function with domain and rangeA B

f g f g◦ Composition of the functions and

f( ; )x θ

A function of

parametrized by

. (Sometimes

we write

(

) and omit the argument

to lighten

notation)

log x xNatural logarithm of

σ x( ) Logistic sigmoid,

1 + exp( )−x

ζ x x( ) log(1 + exp(Softplus, ))

|| ||x

norm of x

|| ||x L

norm of x

Positive part of , i.e.,x max(0 ), x

condition

is 1 if the condition is true, 0 otherwise

Sometimes we use a function

whose argument is a scalar but apply it to a

vector, matrix, or tensor:

(

), or

(

). This denotes the application of

to the array element-wise. For example, if

(

), then C

i,j,k

) for all

valid values of , and .i j k

Datasets and Distributions

data

The data generating distribution

ˆp

data

The empirical distribution deﬁned by the training

set

X A set of training examples

( )i

The -th example (input) from a dataseti

( )i

or y

( )i

The target associated with

( )i

for supervised learn-

ing

The

m n×

matrix with input example

( )i

in row

i,:

CHAPTER 1. INTRODUCTION

are built on top of each other, the graph is deep, with many layers. For this reason,

we call this approach to AI .deep learning

Many of the early successes of AI took place in relatively sterile and formal

environments and did not require computers to have much knowledge about

the world. For example, IBM’s Deep Blue chess-playing system defeated world

champion Garry Kasparov in 1997 ( , ). Chess is of course a very simpleHsu 2002

world, containing only sixty-four locations and thirty-two pieces that can move

in only rigidly circumscribed ways. Devising a successful chess strategy is a

tremendous accomplishment, but the challenge is not due to the diﬃculty of

describing the set of chess pieces and allowable moves to the computer. Chess

can be completely described by a very brief list of completely formal rules, easily

provided ahead of time by the programmer.

Ironically, abstract and formal tasks that are among the most diﬃcult mental

undertakings for a human being are among the easiest for a computer. Computers

have long been able to defeat even the best human chess player but only recently

have begun matching some of the abilities of average human beings to recognize

objects or speech. A person’s everyday life requires an immense amount of

knowledge about the world. Much of this knowledge is subjective and intuitive,

and therefore diﬃcult to articulate in a formal way. Computers need to capture

this same knowledge in order to behave in an intelligent way. One of the key

challenges in artiﬁcial intelligence is how to get this informal knowledge into a

computer.

Several artiﬁcial intelligence projects have sought to hard-code knowledge

about the world in formal languages. A computer can reason automatically about

statements in these formal languages using logical inference rules. This is known as

the

knowledge base

approach to artiﬁcial intelligence. None of these projects has

led to a major success. One of the most famous such projects is Cyc (Lenat and

Guha 1989, ). Cyc is an inference engine and a database of statements in a language

called CycL. These statements are entered by a staﬀ of human supervisors. It is an

unwieldy process. People struggle to devise formal rules with enough complexity

to accurately describe the world. For example, Cyc failed to understand a story

about a person named Fred shaving in the morning ( , ). Its inferenceLinde 1992

engine detected an inconsistency in the story: it knew that people do not have

electrical parts, but because Fred was holding an electric razor, it believed the

entity “FredWhileShaving” contained electrical parts. It therefore asked whether

Fred was still a person while he was shaving.

The diﬃculties faced by systems relying on hard-coded knowledge suggest

that AI systems need the ability to acquire their own knowledge, by extracting

CHAPTER 1. INTRODUCTION

patterns from raw data. This capability is known as

machine learning

. The

introduction of machine learning enabled computers to tackle problems involving

knowledge of the real world and make decisions that appear subjective. A simple

machine learning algorithm called

logistic regression

can determine whether to

recommend cesarean delivery (Mor-Yosef 1990et al., ). A simple machine learning

algorithm called naive Bayes can separate legitimate e-mail from spam e-mail.

The performance of these simple machine learning algorithms depends heavily

on the

representation

of the data they are given. For example, when logistic

regression is used to recommend cesarean delivery, the AI system does not examine

the patient directly. Instead, the doctor tells the system several pieces of relevant

information, such as the presence or absence of a uterine scar. Each piece of

information included in the representation of the patient is known as a

feature

Logistic regression learns how each of these features of the patient correlates with

various outcomes. However, it cannot inﬂuence how features are deﬁned in any

way. If logistic regression were given an MRI scan of the patient, rather than

the doctor’s formalized report, it would not be able to make useful predictions.

Individual pixels in an MRI scan have negligible correlation with any complications

that might occur during delivery.

This dependence on representations is a general phenomenon that appears

throughout computer science and even daily life. In computer science, operations

such as searching a collection of data can proceed exponentially faster if the collec-

tion is structured and indexed intelligently. People can easily perform arithmetic

on Arabic numerals but ﬁnd arithmetic on Roman numerals much more time

consuming. It is not surprising that the choice of representation has an enormous

eﬀect on the performance of machine learning algorithms. For a simple visual

example, see ﬁgure .1.1

Many artiﬁcial intelligence tasks can be solved by designing the right set of

features to extract for that task, then providing these features to a simple machine

learning algorithm. For example, a useful feature for speaker identiﬁcation from

sound is an estimate of the size of the speaker’s vocal tract. This feature gives a

strong clue as to whether the speaker is a man, woman, or child.

For many tasks, however, it is diﬃcult to know what features should be

extracted. For example, suppose that we would like to write a program to detect

cars in photographs. We know that cars have wheels, so we might like to use the

presence of a wheel as a feature. Unfortunately, it is diﬃcult to describe exactly

what a wheel looks like in terms of pixel values. A wheel has a simple geometric

shape, but its image may be complicated by shadows falling on the wheel, the sun

glaring oﬀ the metal parts of the wheel, the fender of the car or an object in the

CHAPTER 1. INTRODUCTION













Figure 1.1: Example of diﬀerent representations: suppose we want to separate two

categories of data by drawing a line between them in a scatterplot. In the plot on the left,

we represent some data using Cartesian coordinates, and the task is impossible. In the plot

on the right, we represent the data with polar coordinates and the task becomes simple to

solve with a vertical line. (Figure produced in collaboration with David Warde-Farley.)

foreground obscuring part of the wheel, and so on.

One solution to this problem is to use machine learning to discover not only

the mapping from representation to output but also the representation itself.

This approach is known as

representation learning

. Learned representations

often result in much better performance than can be obtained with hand-designed

representations. They also enable AI systems to rapidly adapt to new tasks, with

minimal human intervention. A representation learning algorithm can discover a

good set of features for a simple task in minutes, or for a complex task in hours to

months. Manually designing features for a complex task requires a great deal of

human time and eﬀort; it can take decades for an entire community of researchers.

The quintessential example of a representation learning algorithm is the

au-

toencoder

. An autoencoder is the combination of an

encoder

function, which

converts the input data into a diﬀerent representation, and a

decoder

function,

which converts the new representation back into the original format. Autoencoders

are trained to preserve as much information as possible when an input is run

through the encoder and then the decoder, but they are also trained to make the

new representation have various nice properties. Diﬀerent kinds of autoencoders

aim to achieve diﬀerent kinds of properties.

When designing features or algorithms for learning features, our goal is usually

to separate the

factors of variation

that explain the observed data. In this

剩余796页未读，继续阅读

Code4Quality

粉丝: 0
资源: 6

深度学习基础：Ian Goodfellow著，线性代数与概率理论概览

Deep Learning - Ian Goodfellow 完整高清英文版

《Deep Learning》 Ian Goodfellow Yoshua Bengio Aaron Courville.zip

deep learning ian goodfellowpdf

《deep learning》 ian 英文版 pdf

goodfellow的deep learning

近几年的算法书和文献

系统化学习人工智能需要看哪些书籍

深度学习电子书免费下载的地方

中文翻译的深度学习文献

学习神经网络的书籍推荐

最新资源