深度学习与神经网络基础教程

需积分: 13 196 浏览量更新于2024-07-18 收藏 11.49MB PDF 举报

"Neural Networks and Deep Learning" 是一本由 Charu C. Aggarwal 编著的专业书籍，由 Springer 出版社出版。这本书深入探讨了神经网络和深度学习这两个在人工智能领域至关重要的主题。书中的内容可能涵盖了神经网络的基本原理、架构、训练方法以及深度学习的应用，包括图像识别、自然语言处理等。这本书的ISBN编号为978-3-319-94462-3（纸质版）和978-3-319-94463-0（电子版），可以通过提供的DOI链接（https://doi.org/10.1007/978-3-319-94463-0）在线获取。此外，它还拥有Library of Congress Control Number: 2018947636，这表明该书已被美国国会图书馆收录。版权信息指出，本书受版权保护，所有权利归 Springer International Publishing AG 所有，禁止未经授权的翻译、重印、复制等活动。同时，书中可能引用的一般描述性名称、注册商标、服务标志等均受到相关法律法规的保护，并非免费供公众使用。作者Charu C. Aggarwal是IBM T.J. Watson Research Center的研究员，他在人工智能和数据挖掘领域具有深厚的专业知识。因此，这本书可以预期提供权威且准确的信息，适合对神经网络和深度学习感兴趣的学者、学生和从业者阅读。然而，尽管作者、出版者和编辑尽力确保书中内容的准确性和时效性，但仍然可能存在错误或遗漏，读者在应用书中信息时应谨慎对待。本书的内容可能包括神经网络的基础知识，如感知器、多层前馈网络、反向传播算法，以及深度学习的最新进展，如卷积神经网络（CNN）、循环神经网络（RNN）、长短期记忆网络（LSTM）、生成对抗网络（GAN）等。此外，可能会讨论优化方法（如梯度下降和Adam优化器）、损失函数、超参数调优策略，以及如何解决过拟合和欠拟合问题。书中可能还会涵盖数据预处理、模型评估和模型解释等实践环节。 "Neural Networks and Deep Learning" 是一本全面介绍神经网络和深度学习的教科书，对于希望深入了解这一领域的读者来说，是一份宝贵的资源。通过阅读这本书，读者可以系统地学习和掌握构建和应用神经网络与深度学习模型的关键技术和理论。

XVIII CONTENTS

6.5 Applications of Restricted Boltzmann Machines ................ 251

6.5.1 Dimensionality Reduction and Data Reconstruction ......... 252

6.5.2 RBMs for Collaborative Filtering .................... 254

6.5.3 Using RBMs for Classiﬁcation ...................... 257

6.5.4 Topic Models with RBMs ........................ 260

6.5.5 RBMs for Machine Learning with Multimodal Data ......... 262

6.6 Using RBMs Beyond Binary Data Types .................... 263

6.7 Stacking Restricted Boltzmann Machines ................... 264

6.7.1 Unsupervised Learning .......................... 266

6.7.2 Supervised Learning ........................... 267

6.7.3 Deep Boltzmann Machines and Deep Belief Networks ........ 267

6.8 Summary ..................................... 268

6.9 Bibliographic Notes ................................ 268

6.10 Exercises ..................................... 270

7 Recurrent Neural Networks 271

7.1 Introduction .................................... 271

7.1.1 Expressiveness of Recurrent Networks ................. 274

7.2 The Architecture of Recurrent Neural Networks ................ 274

7.2.1 Language Modeling Example of RNN ................. 277

7.2.1.1 Generating a Language Sample ................ 278

7.2.2 Backpropagation Through Time .................... 280

7.2.3 Bidirectional Recurrent Networks .................... 283

7.2.4 Multilayer Recurrent Networks ..................... 284

7.3 The Challenges of Training Recurrent Networks ................ 286

7.3.1 Layer Normalization ........................... 289

7.4 Echo-State Networks ............................... 290

7.5 Long Short-Term Memory (LSTM) ....................... 292

7.6 Gated Recurrent Units (GRUs) ......................... 295

7.7 Applications of Recurrent Neural Networks .................. 297

7.7.1 Application to Automatic Image Captioning .............. 298

7.7.2 Sequence-to-Sequence Learning and Machine Translation ...... 299

7.7.2.1 Question-Answering Systems ................. 301

7.7.3 Application to Sentence-Level Classiﬁcation .............. 303

7.7.4 Token-Level Classiﬁcation with Linguistic Features .......... 304

7.7.5 Time-Series Forecasting and Prediction ................ 305

7.7.6 Temporal Recommender Systems .................... 307

7.7.7 Secondary Protein Structure Prediction ................ 309

7.7.8 End-to-End Speech Recognition ..................... 309

7.7.9 Handwriting Recognition ........................ 309

7.8 Summary ..................................... 310

7.9 Bibliographic Notes ................................ 310

7.9.1 Software Resources ............................ 311

7.10 Exercises ..................................... 312

CONTENTS XIX

8 Convolutional Neural Networks 315

8.1 Introduction .................................... 315

8.1.1 Historical Perspective and Biological Inspiration ........... 316

8.1.2 Broader Observations About Convolutional Neural Networks .... 317

8.2 The Basic Structure of a Convolutional Network ............... 318

8.2.1 Padding .................................. 322

8.2.2 Strides ................................... 324

8.2.3 Typical Settings ............................. 324

8.2.4 The ReLU Layer ............................. 325

8.2.5 Pooling .................................. 326

8.2.6 Fully Connected Layers ......................... 327

8.2.7 The Interleaving Between Layers .................... 328

8.2.8 Local Response Normalization ..................... 330

8.2.9 Hierarchical Feature Engineering .................... 331

8.3 Training a Convolutional Network ....................... 332

8.3.1 Backpropagating Through Convolutions ................ 333

8.3.2 Backpropagation as Convolution with Inverted/Transposed Filter . . 334

8.3.3 Convolution/Backpropagation as Matrix Multiplications ....... 335

8.3.4 Data Augmentation ........................... 337

8.4 Case Studies of Convolutional Architectures .................. 338

8.4.1 AlexNet .................................. 339

8.4.2 ZFNet ................................... 341

8.4.3 VGG .................................... 342

8.4.4 GoogLeNet ................................ 345

8.4.5 ResNet .................................. 347

8.4.6 The Eﬀects of Depth ........................... 350

8.4.7 Pretrained Models ............................ 351

8.5 Visualization and Unsupervised Learning ................... 352

8.5.1 Visualizing the Features of a Trained Network ............ 353

8.5.2 Convolutional Autoencoders ....................... 357

8.6 Applications of Convolutional Networks .................... 363

8.6.1 Content-Based Image Retrieval ..................... 363

8.6.2 Object Localization ........................... 364

8.6.3 Object Detection ............................. 365

8.6.4 Natural Language and Sequence Learning ............... 366

8.6.5 Video Classiﬁcation ........................... 367

8.7 Summary ..................................... 368

8.8 Bibliographic Notes ................................ 368

8.8.1 Software Resources and Data Sets ................... 370

8.9 Exercises ..................................... 371

9 Deep Reinforcement Learning 373

9.1 Introduction .................................... 373

9.2 Stateless Algorithms: Multi-Armed Bandits .................. 375

9.2.1 Na¨ıve Algorithm ............................. 376

9.2.2 -Greedy Algorithm ........................... 376

9.2.3 Upper Bounding Methods ........................ 376

9.3 The Basic Framework of Reinforcement Learning ............... 377

9.3.1 Challenges of Reinforcement Learning ................. 379

XX CONTENTS

9.3.2 Simple Reinforcement Learning for Tic-Tac-Toe ............ 380

9.3.3 Role of Deep Learning and a Straw-Man Algorithm ......... 380

9.4 Bootstrapping for Value Function Learning .................. 383

9.4.1 Deep Learning Models as Function Approximators .......... 384

9.4.2 Example: Neural Network for Atari Setting .............. 386

9.4.3 On-Policy Versus Oﬀ-Policy Methods: SARSA ............ 387

9.4.4 Modeling States Versus State-Action Pairs ............... 389

9.5 Policy Gradient Methods ............................ 391

9.5.1 Finite Diﬀerence Methods ........................ 392

9.5.2 Likelihood Ratio Methods ........................ 393

9.5.3 Combining Supervised Learning with Policy Gradients ........ 395

9.5.4 Actor-Critic Methods .......................... 395

9.5.5 Continuous Action Spaces ........................ 397

9.5.6 Advantages and Disadvantages of Policy Gradients .......... 397

9.6 Monte Carlo Tree Search ............................. 398

9.7 Case Studies ................................... 399

9.7.1 AlphaGo: Championship Level Play at Go ............... 399

9.7.1.1 Alpha Zero: Enhancements to Zero Human Knowledge . . 402

9.7.2 Self-Learning Robots ........................... 404

9.7.2.1 Deep Learning of Locomotion Skills ............. 404

9.7.2.2 Deep Learning of Visuomotor Skills ............. 406

9.7.3 Building Conversational Systems: Deep Learning for Chatbots ... 407

9.7.4 Self-Driving Cars ............................. 410

9.7.5 Inferring Neural Architectures with Reinforcement Learning ..... 412

9.8 Practical Challenges Associated with Safety .................. 413

9.9 Summary ..................................... 414

9.10 Bibliographic Notes ................................ 414

9.10.1 Software Resources and Testbeds .................... 416

9.11 Exercises ..................................... 416

10 Advanced Topics in Deep Learning 419

10.1 Introduction .................................... 419

10.2 Attention Mechanisms .............................. 421

10.2.1 Recurrent Models of Visual Attention ................. 422

10.2.1.1 Application to Image Captioning ............... 424

10.2.2 Attention Mechanisms for Machine Translation ............ 425

10.3 Neural Networks with External Memory .................... 429

10.3.1 A Fantasy Video Game: Sorting by Example ............. 430

10.3.1.1 Implementing Swaps with Memory Operations ....... 431

10.3.2 Neural Turing Machines ......................... 432

10.3.3 Diﬀerentiable Neural Computer: A Brief Overview .......... 437

10.4 Generative Adversarial Networks (GANs) ................... 438

10.4.1 Training a Generative Adversarial Network .............. 439

10.4.2 Comparison with Variational Autoencoder ............... 442

10.4.3 Using GANs for Generating Image Data ................ 442

10.4.4 Conditional Generative Adversarial Networks ............. 444

10.5 Competitive Learning .............................. 449

10.5.1 Vector Quantization ........................... 450

10.5.2 Kohonen Self-Organizing Map ..................... 450

Author Biography

Charu C. Aggarwal is a Distinguished Research Staﬀ Member (DRSM) at the IBM

T. J. Watson Research Center in Yorktown Heights, New York. He completed his under-

graduate degree in Computer Science from the Indian Institute of Technology at Kan-

pur in 1993 and his Ph.D. from the Massachusetts Institute of Technology in 1996.

He has worked extensively in the ﬁeld of data mining. He has pub-

lished more than 350 papers in refereed conferences and journals

and authored over 80 patents. He is the author or editor of 18

books, including textbooks on data mining, recommender systems,

and outlier analysis. Because of the commercial value of his patents,

he has thrice been designated a Master Inventor at IBM. He is a

recipient of an IBM Corporate Award (2003) for his work on bio-

terrorist threat detection in data streams, a recipient of the IBM

Outstanding Innovation Award (2008) for his scientiﬁc contribu-

tions to privacy technology, and a recipient of two IBM Outstanding

Technical Achievement Awards (2009, 2015) for his work on data streams/high-dimensional

data. He received the EDBT 2014 Test of Time Award for his work on condensation-based

privacy-preserving data mining. He is also a recipient of the IEEE ICDM Research Con-

tributions Award (2015), which is one of the two highest awards for inﬂuential research

contributions in the ﬁeld of data mining.

He has served as the general co-chair of the IEEE Big Data Conference (2014) and as

the program co-chair of the ACM CIKM Conference (2015), the IEEE ICDM Conference

(2015), and the ACM KDD Conference (2016). He served as an associate editor of the IEEE

Transactions on Knowledge and Data Engineering from 2004 to 2008. He is an associate

editor of the IEEE Transactions on Big Data, an action editor of the Data Mining and

Knowledge Discovery Journal, and an associate editor of the Knowledge and Information

Systems Journal. He serves as the editor-in-chief of the ACM Transactions on Knowledge

Discovery from Data as well as the ACM SIGKDD Explorations. He serves on the advisory

board of the Lecture Notes on Social Networks, a publication by Springer. He has served as

the vice-president of the SIAM Activity Group on Data Mining and is a member of the SIAM

industry committee. He is a fellow of the SIAM, ACM, and the IEEE, for “contributions to

knowledge discovery and data mining algorithms.”

XXIII

剩余511页未读，继续阅读

Jesse-Jiang

粉丝: 1
资源: 30

深度学习与神经网络基础教程

《Neural Networks and Deep Learning》（美）Michael Nielsen 著 英文版.pdf

neural networks and deep learning

Neural networks and deep learning

Neural Networks and Deep Learning Deep Learning explained to your granny

Neural networks and deep learning 教程

Neural Networks and Deep Learning - Deep Learning explained to your granny

Neural Networks and Deep Learning.zip

Neural Networks and Deep Learning week 1

Neural Networks and Deep Learning中文版

neural networks and deep learning 电子书

最新资源

《Neural Networks and Deep Learning》（美）Michael Nielsen 著英文版.pdf