CONTENTS
6.3 Hidden Units ...... ........ ......... ...... 187
6.4 Architecture Design ......... ........ ........ . 193
6.5 Back-Propagation and Other Differentiation
Algorithms .... ......... ........ ......... . 200
6.6 Historical Notes ....... ........ ......... .... 220
7 Regularization for Deep Learning 224
7.1 Parameter Norm Penalties ..... ......... ........ 226
7.2 Norm Penalties as Constrained Optimization ........ .... 233
7.3 Regularization and Under-Constrained Problems .. ....... 235
7.4 Dataset Augmentation .......... ......... ..... 236
7.5 Noise Robustness ......... ........ ........ .. 238
7.6 Semi-Supervised Learning ................ ...... 240
7.7 Multitask Learning ............ ........ ...... 241
7.8 Early Stopping ......... ........ ........ ... 241
7.9 Parameter Tying and Parameter Sharing.............. 249
7.10 Sparse Representations ......... ........ ....... 251
7.11 Bagging and Other Ensemble Methods . ......... ..... 253
7.12 Dropout ........ ......... ........ ....... 255
7.13 Adversarial Training ........ ......... ........ 265
7.14 Tangent Distance, Tangent Prop and Manifold
Tangent Classifier ............ ......... ...... 267
8 Optimization for Training Deep Models 271
8.1 How Learning Differs from Pure Optimization ........... 272
8.2 Challenges in Neural Network Optimization ..... ....... 279
8.3 Basic Algorithms .......... ........ ......... 290
8.4 Parameter Initialization Strategies . ......... ....... 296
8.5 Algorithms with Adaptive Learning Rates ....... ...... 302
8.6 Approximate Second-Order Methods . ........ ....... 307
8.7 Optimization Strategies and Meta-Algorithms ..... ...... 313
9 Convolutional Networks 326
9.1 The Convolution Operation ................ ..... 327
9.2 Motivation .. ........ ......... ........ .... 329
9.3 Pooling ............. ........ ......... ... 335
9.4 Convolution and Pooling as an Infinitely Strong Prior .. ..... 339
9.5 Variants of the Basic Convolution Function ............ 342
9.6 Structured Outputs . ........ ......... ........ 352
9.7 Data Types ...... ........ ........ ........ 354
iii
我的内容管理
收起
我的收益 登录查看自己的收益
我的积分
登录查看自己的积分
我的C币
登录后查看C币余额
我的收藏 
我的下载 
下载帮助 
评论3