CONTENTS 3
5 The Perceptron 64
5.1 The Perceptron neuro-computer . . . . . . . . . . . . . . . . . . . . . . 64
5.2 Perceptron learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.3 Perceptron in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.4 Limitations of the Perceptron . . . . . . . . . . . . . . . . . . . . . . . 76
6 Self-Organizing Maps 82
6.1 The SOM neural network model . . . . . . . . . . . . . . . . . . . . . 82
6.2 A SOM in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
6.3 SOM and the Cortex . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
7 Multi Layer Perceptrons 107
7.1 The goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
7.2 Basic idea is gradient descent . . . . . . . . . . . . . . . . . . . . . . . 108
7.3 Splitting the weight change formula into three parts . . . . . . . . . . 110
7.4 Computing the first part . . . . . . . . . . . . . . . . . . . . . . . . . . 111
7.5 Computing the second part . . . . . . . . . . . . . . . . . . . . . . . . 112
7.6 Computing the third part . . . . . . . . . . . . . . . . . . . . . . . . . 112
7.7 Backpropagation pseudo code . . . . . . . . . . . . . . . . . . . . . . . 116
7.8 MLP in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
7.9 Visualization of decision boundaries . . . . . . . . . . . . . . . . . . . 133
7.10 The need for non-linear transfer functions . . . . . . . . . . . . . . . . 137
8 TensorFlow 140
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
8.2 Training a linear model with TensorFlow . . . . . . . . . . . . . . . . . 149
8.3 A MLP with TensorFlow . . . . . . . . . . . . . . . . . . . . . . . . . . 151
9 Convolutional Neural Networks 159
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
9.2 Some history about the CNN model . . . . . . . . . . . . . . . . . . . 163
9.3 Convolutional and pooling layers in TensorFlow . . . . . . . . . . . . . 166
9.4 Parameters to be defined for a convolution layer . . . . . . . . . . . . 172
9.5 How to compute the dimension of an output tensor . . . . . . . . . . . 177
9.6 Parameters to be defined for a pooling layer . . . . . . . . . . . . . . . 178
9.7 A CNN in TensorFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
10 Deep Learning Tricks 194
10.1 Fighting against vanishing gradients . . . . . . . . . . . . . . . . . . . 194
10.2 Momentum optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 196
10.3 Nesterov Momentum Optimization . . . . . . . . . . . . . . . . . . . . 199
评论0