CONTENTS
6.3 Hidden Units ...... ........ ......... ...... 190
6.4 Architecture Design ......... ........ ........ . 196
6.5 Back-Propagation and Other Differentiation Algorithms ..... 203
6.6 Historical Notes ....... ........ ......... .... 224
7 Regularization for Deep Learning 228
7.1 Parameter Norm Penalties ..... ......... ........ 230
7.2 Norm Pena
lties as Constrained Optimization ........ .... 237
7.3 Regularization and Under-Constrained Problems .. ....... 239
7.4 Dataset Augmentation .......... ......... ..... 240
7.5 Noise Robustness ......... ........ ........ .. 242
7.6 Semi-Supervised Learning ................ ...... 244
7.7 Multi-Task Learning .............. ......... .. 2
45
7.8 Early Stopping ......... ........ ........ ... 246
7.9 Parameter Tying and Parameter Sharing.............. 251
7.10 Sparse Representations ......... ........ ....... 253
7.11 Bagging and Other Ensemble Methods . ......... ..... 255
7.12 Dropout ........ ......... ........ ....... 257
7.13 Adversarial Training ........ ......
... ........ 267
7.14 Tangent Distance, Tangent Prop, and Manifold Tangent Classifier 268
8 Optimization for Training Deep Models 274
8.1 How Learning Differs from Pure Optimization ........... 275
8.2 Challenges in Neural Network Optimization ..... ....... 282
8.3 Basic Algorithms ............. ........ ...... 294
8.4 Parameter Initialization Strategies . ......... ....... 301
8.5 Alg
orithms with Adaptive Learning Rates ....... ...... 306
8.6 Approximate Second-Order Methods .... ......... ... 310
8.7 Optimization Strategies and Meta-Algorithms ..... ...... 318
9 Convolutional Networks 331
9.1 The Convolution Operation ................ ..... 332
9.2 Motivation .. ........ ......... ........ .... 336
9.3 Pooling ............. ....
.... ......... ... 340
9.4 Convolution and Pooling as an Infinitely Strong Prior .. ..... 346
9.5 Variants of the Basic Convolution Function ............ 348
9.6 Structured Outputs . ........ ......... ........ 359
9.7 Data Types ...... ........ ........ ........ 361
9.8 Efficient Convolution Algorithms ........ ........ .. 363
9.9 Random or Unsupervised Feature
s ........ ........ . 364
iii
我的内容管理
收起
我的收益 登录查看自己的收益
我的积分
登录查看自己的积分
我的C币
登录后查看C币余额
我的收藏 
我的下载 
下载帮助 
评论0