DeepLearningbyIanGoodfellow,YoshuaBengio,AaronCourville

深度学习

需积分: 9 54 浏览量更新于2023-03-16 评论 2 收藏 30.05MB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

资源详情

资源评论

资源推荐

2017/5/18 ӥ܌2)01

ᒫ 1 ᶭҁو 9 ᶭ҂http://www.deeplearningbook.org/contents/TOC.html

Deep

DeepDeep

Learning

LearningLearning

Ian

IanIan

GoGo

dfello

dfellodfello

osh

oshosh

uaua

Bengio

BengioBengio

Aaron

AaronAaron

Courville

CourvilleCourville

2017/5/18 ӥ܌2)01

ᒫ 2 ᶭҁو 9 ᶭ҂http://www.deeplearningbook.org/contents/TOC.html

Con

ConCon

ten

tenten

tsts

Website vii

Acknowledgments viii

Notation xi

1 Introduction 1

1.1 Who Should Read This Book? . . . . . . . . . . . . . . . . . . . . 8

1.2 Historical Trends in Deep Learning . . . . . . . . . . . . . . . . . 11

I Applied Math and Machine Learning Basics 29

2 Linear Algebra 31

2.1 Scalars, Vectors, Matrices and Tensors . . . . . . . . . . . . . . . 31

2.2 Multiplying Matrices and Vectors . . . . . . . . . . . . . . . . . . 34

2.3 Identity and Inverse Matrices . . . . . . . . . . . . . . . . . . . . 36

2.4 Linear Dependence and Span . . . . . . . . . . . . . . . . . . . . 37

2.5 Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2.6 Special Kinds of Matrices and Vectors . . . . . . . . . . . . . . . 40

2.7 Eigendecomposition . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.8 Singular Value Decomposition . . . . . . . . . . . . . . . . . . . . 44

2.9 The Moore-Penrose Pseudoinverse . . . . . . . . . . . . . . . . . . 45

2.10 The Trace Operator . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.11 The Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

2.12 Example: Principal Components Analysis . . . . . . . . . . . . . 48

2017/5/18 ӥ܌2)01

ᒫ 3 ᶭҁو 9 ᶭ҂http://www.deeplearningbook.org/contents/TOC.html

3 Probability and Information Theory 53

3.1 Why Probability? . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

CONTENTS

3.2 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . 56

3.3 Probability Distributions . . . . . . . . . . . . . . . . . . . . . . . 56

3.4 Marginal Probability . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.5 Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . 59

3.6 The Chain Rule of Conditional Probabilities . . . . . . . . . . . . 59

3.7 Independence and Conditional Independence . . . . . . . . . . . . 60

3.8 Expectation, Variance and Covariance . . . . . . . . . . . . . . . 60

3.9 Common Probability Distributions . . . . . . . . . . . . . . . . . 62

3.10 Useful Properties of Common Functions . . . . . . . . . . . . . . 67

3.11 Bayes’ Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

3.12 Technical Details of Continuous Variables . . . . . . . . . . . . . 71

3.13 Information Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3.14 Structured Probabilistic Models . . . . . . . . . . . . . . . . . . . 75

4 Numerical Computation 80

4.1 Overﬂow and Underﬂow . . . . . . . . . . . . . . . . . . . . . . . 80

4.2 Poor Conditioning . . . . . . . . . . . . . . . . . . . . . . . . . . 82

4.3 Gradient-Based Optimization . . . . . . . . . . . . . . . . . . . . 82

4.4 Constrained Optimization . . . . . . . . . . . . . . . . . . . . . . 93

4.5 Example: Linear Least Squares . . . . . . . . . . . . . . . . . . . 96

5 Machine Learning Basics 98

5.1 Learning Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 99

5.2 Capacity, Overﬁtting and Underﬁtting . . . . . . . . . . . . . . . 110

5.3 Hyperparameters and Validation Sets . . . . . . . . . . . . . . . . 120

5.4 Estimators, Bias and Variance . . . . . . . . . . . . . . . . . . . . 122

5.5 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . 131

5.6 Bayesian Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . 135

5.7 Supervised Learning Algorithms . . . . . . . . . . . . . . . . . . . 140

5.8 Unsupervised Learning Algorithms . . . . . . . . . . . . . . . . . 146

5.9 Stochastic Gradient Descent . . . . . . . . . . . . . . . . . . . . . 151

5.10 Building a Machine Learning Algorithm . . . . . . . . . . . . . . 153

5.11 Challenges Motivating Deep Learning . . . . . . . . . . . . . . . . 155

2017/5/18 ӥ܌2)01

ᒫ 4 ᶭҁو 9 ᶭ҂http://www.deeplearningbook.org/contents/TOC.html

5.11 Challenges Motivating Deep Learning . . . . . . . . . . . . . . . . 155

II Deep Networks: Modern Practices 166

6 Deep Feedforward Networks 168

6.1 Example: Learning XOR . . . . . . . . . . . . . . . . . . . . . . . 171

6.2 Gradient-Based Learning . . . . . . . . . . . . . . . . . . . . . . . 177

CONTENTS

6.3 Hidden Units . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

6.4 Architecture Design . . . . . . . . . . . . . . . . . . . . . . . . . . 197

6.5 Back-Propagation and Other Diﬀerentiation Algorithms . . . . . 204

6.6 Historical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

7 Regularization for Deep Learning 228

7.1 Parameter Norm Penalties . . . . . . . . . . . . . . . . . . . . . . 230

7.2 Norm Penalties as Constrained Optimization . . . . . . . . . . . . 237

7.3 Regularization and Under-Constrained Problems . . . . . . . . . 239

7.4 Dataset Augmentation . . . . . . . . . . . . . . . . . . . . . . . . 240

7.5 Noise Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

7.6 Semi-Supervised Learning . . . . . . . . . . . . . . . . . . . . . . 243

7.7 Multi-Task Learning . . . . . . . . . . . . . . . . . . . . . . . . . 244

7.8 Early Stopping . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

7.9 Parameter Tying and Parameter Sharing . . . . . . . . . . . . . . 253

7.10 Sparse Representations . . . . . . . . . . . . . . . . . . . . . . . . 254

7.11 Bagging and Other Ensemble Methods . . . . . . . . . . . . . . . 256

7.12 Dropout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

7.13 Adversarial Training . . . . . . . . . . . . . . . . . . . . . . . . . 268

7.14 Tangent Distance, Tangent Prop, and Manifold Tangent Classiﬁer 270

8 Optimization for Training Deep Models 274

8.1 How Learning Diﬀers from Pure Optimization . . . . . . . . . . . 275

8.2 Challenges in Neural Network Optimization . . . . . . . . . . . . 282

8.3 Basic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

8.4 Parameter Initialization Strategies . . . . . . . . . . . . . . . . . 301

8.5 Algorithms with Adaptive Learning Rates . . . . . . . . . . . . . 306

8.6 Approximate Second-Order Methods . . . . . . . . . . . . . . . . 310

8.7 Optimization Strategies and Meta-Algorithms . . . . . . . . . . . 317

2017/5/18 ӥ܌2)01

ᒫ 5 ᶭҁو 9 ᶭ҂http://www.deeplearningbook.org/contents/TOC.html

9 Convolutional Networks 330

9.1 The Convolution Operation . . . . . . . . . . . . . . . . . . . . . 331

9.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335

9.3 Pooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

9.4 Convolution and Pooling as an Inﬁnitely Strong Prior . . . . . . . 345

9.5 Variants of the Basic Convolution Function . . . . . . . . . . . . 347

9.6 Structured Outputs . . . . . . . . . . . . . . . . . . . . . . . . . . 358

9.7 Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360

9.8 Eﬃcient Convolution Algorithms . . . . . . . . . . . . . . . . . . 362

9.9 Random or Unsupervised Features . . . . . . . . . . . . . . . . . 363

iii

CONTENTS

9.10 The Neuroscientiﬁc Basis for Convolutional Networks . . . . . . . 364

9.11 Convolutional Networks and the History of Deep Learning . . . . 371

10 Sequence Modeling: Recurrent and Recursive Nets 373

10.1 Unfolding Computational Graphs . . . . . . . . . . . . . . . . . . 375

10.2 Recurrent Neural Networks . . . . . . . . . . . . . . . . . . . . . 378

10.3 Bidirectional RNNs . . . . . . . . . . . . . . . . . . . . . . . . . . 394

10.4 Encoder-Decoder Sequence-to-Sequence Architectures . . . . . . . 396

10.5 Deep Recurrent Networks . . . . . . . . . . . . . . . . . . . . . . 398

10.6 Recursive Neural Networks . . . . . . . . . . . . . . . . . . . . . . 400

10.7 The Challenge of Long-Term Dependencies . . . . . . . . . . . . . 401

10.8 Echo State Networks . . . . . . . . . . . . . . . . . . . . . . . . . 404

10.9 Leaky Units and Other Strategies for Multiple Time Scales . . . . 406

10.10 The Long Short-Term Memory and Other Gated RNNs . . . . . . 408

10.11 Optimization for Long-Term Dependencies . . . . . . . . . . . . . 413

10.12 Explicit Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . 416

11 Practical Methodology 421

11.1 Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . 422

11.2 Default Baseline Models . . . . . . . . . . . . . . . . . . . . . . . 425

11.3 Determining Whether to Gather More Data . . . . . . . . . . . . 426

11.4 Selecting Hyperparameters . . . . . . . . . . . . . . . . . . . . . . 427

11.5 Debugging Strategies . . . . . . . . . . . . . . . . . . . . . . . . . 436

11.6 Example: Multi-Digit Number Recognition . . . . . . . . . . . . . 440

12 Applications 443

剩余887页未读，继续阅读

bestcql

粉丝: 1
资源: 1

会员权益专享

Deep Learning by Ian Goodfellow, Yoshua Bengio, Aaron Courville

评论0

会员权益专享

最新资源

Deep Learning by Ian Goodfellow, Yoshua Bengio, Aaron Courville

评论0

Bengio写的MIT Press《Deep learning》PDF整理版

Deep Learning.pdf Ian Goodfellow Yoshua Bengio Aaron Courville

《Deep Learning》Yoshua Bengio即将出版的深度学习书PDF整理

《Deep Learning》 Ian Goodfellow Yoshua Bengio Aaron Courville.zip

Deep Learning ( Ian Goodfellow Yoshua Bengio Aaron Courville)

pdf:深度学习 ian goodfellow、yoshua bengio 和 aaron courville pdf

适合人工智能深度学习的参考文献

学习基于深度学习的图像特征提取可以参考那些文献或者资料

人工智能深度学习参考文献

计算机毕设英文参考文献

人工智能十大经典文献

请给出介绍神经网络发展历史的文献

图像处理与深度学习相关书籍与作者

推荐几本人工智能最前沿书籍

能详细说下训练chatgpt这样的机器人需要看那些书籍吗

括机器学习、深度学习、自然语言处理等方面推荐的书籍

学习人工智能可以看什么书籍

推荐一些有关AI的书籍

可以推荐一些课程或书籍吗

deep learning 英文版pdf

会员权益专享

最新资源