Fully Connected Neural Networks
History is witness that the field of Artificial Intelligence has gone through
multiple phases of boom-and-bust cycles. It is like stock markets, after every
bull run there is a correction and once in a while there is recession and
depression.
I would say that the field of deep learning started with emergence of Multi-
Layer Perceptrons(MLP). Simply put, it was a feed forward neural network.
These MLPs had an input layer, a hidden layer and an output layer. MLPs
could work well with data that is not linearly separable. The key thing about
MLP is that all of its layers were fully connected. It means that each neuron
of one layer was connected to all the neurons in the next layer.
In the year 1975, backpropagation became the practical way to train MLP.
However deep learning was still not a mainstream way to do machine
learning. That was because MLPs were usually trained on XOR datasets.
More than the lack of computational power there was also a lack of real life
datasets on which the practicality of MLP could be verified. Around 1995,
Yann LeCun released MNIST dataset. This dataset had collection of
handwritten digits. This for some time became benchmark for forthcoming
neural networks.
In today’s era we have a multitude of datasets available. Imagenets for
benchmarking object classification neural networks. MOT17, PASCAL VOC
etc for object tracking and detection. However, MNIST dataset is still
important today. It helps us in quickly prototyping and testing our models on
a light-weight dataset. If a model is performing well on MNIST dataset then
it is safe to pursue the design of the model one has thought of and then train
it with heavy datasets. There are some datasets which can be useful along
with MNIST i.e Fashion MNIST, Omniglot dataset, etc.
We have got a neural network which is fully connected. It has got one or
more than one hidden layers between input and output layers. We have
剩余22页未读，继续阅读
