References 9
22. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in Proceedings
of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778
23. K. He, G. Gkioxari, P. Dollár, R. Girshick, Mask R-CNN. arXiv:1703.06870 (2017, preprint)
24. G. Hinton, S. Osindero, Y. Teh, A fast learning algorithm for deep belief nets. Neural Comput.
18(7), 1527–1554 (2006)
25. S. Hochreiter, Untersuchungen zu dynamischen neuronalen netzen, Diploma, Technische
Universität München, 91, 1991
26. S. Hochreiter, J. Schmidhuber, Long short-term memory. Neural Comput. 9(8), 1735–1780
(1997)
27. S. Hochreiter, Y. Bengio, P. Frasconi, J. Schmidhuber, Gradient flow in recurrent nets: the
difficulty of learning long-term dependencies. In: A Field Guide to Dynamical Recurrent
Neural Networks. IEEE Press (2001)
28. K. Hornik, Approximation capabilities of multilayer feedforward networks. Neural Netw. 4(2),
251–257 (1991)
29. A. Ivakhnenko, V. Lapa, Cybernetic predicting devices, Technical report, DTIC Document,
1966
30. L. Kanal, Perceptron, in Encyclopedia of Computer Science (Wiley, Chichester, 2003)
31. Y. LeCun, D. Touresky, G. Hinton, T. Sejnowski, A theoretical framework for back-
propagation, in The Connectionist Models Summer School, vol. 1 (1988), pp. 21–28
32. Y. LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard, L. Jackel, Handwritten
digit recognition with a back-propagation network, in Advances in Neural Information
Processing Systems (1990), pp. 396–404
33. Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document
recognition. Proc. IEEE 86(11), 2278–2324 (1998)
34. Y. LeCun, C. Cortes, C. Burges, Mnist handwritten digit database. AT&T Labs [Online]. http://
yann.lecun.com/exdb/mnist, 2 (2010)
35. Y. LeCun, Y. Bengio, G. Hinton, Deep learning. Nature 521(7553), 436–444 (2015)
36. H. Lin, M. Tegmark, Why does deep and cheap learning work so well? arXiv:1608.08225
(2016, preprint)
37. H. Lutkepohl, Handbook of Matrices (Wiley, Hoboken, 1997)
38. A. Maas, A. Hannun, A. Ng, Rectifier nonlinearities improve neural network acoustic models,
in Proceedings of ICML, vol. 30 (2013)
39. W. McCulloch, W. Pitts, A logical calculus of the ideas immanent in nervous activity. Bull.
Math. Biol. 5(4), 115–133 (1943)
40. M. Minsky, S. Papert, Perceptrons (MIT press, Cambridge, 1969)
41. V. Mnih, K. Kavukcuoglu, D. Silver, A. Rusu, J. Veness, M. Bellemare, A. Graves, M. Ried-
miller et al., Human-level control through deep reinforcement learning. Nature 518(7540),
529–533 (2015)
42. G. Montufar, R. Pascanu, K. Cho, Y. Bengio, On the number of linear regions of deep neural
networks, in Advances in Neural Information Processing Systems (2014), pp. 2924–2932
43. V. Nair, G. Hinton, Rectified linear units improve restricted Boltzmann machines, in Pro-
ceedings of the 27th International Conference on Machine Learning (ICML-10) (2010), pp.
807–814
44. R. Pascanu, G. Montufar, Y. Bengio, On the number of response regions of deep feed forward
networks with piece-wise linear activations. arXiv:1312.6098 (2013, preprint)
45. A. Radford, L. Metz, S. Chintala, Unsupervised representation learning with deep convolu-
tional generative adversarial networks. arXiv:1511.06434 (2015, preprint)
46. F. Rosenblatt, The perceptron: a probabilistic model for information storage and organization
in the brain. Psychol. Rev. 65(6), 386 (1958)
47. D. Rumelhart, G. Hinton, R. Williams, Learning internal representations by error propagation,
Technical report, California Univ San Diego La Jolla Inst for Cognitive Science, 1985
48. T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, X. Chen, Improved
techniques for training GANs, in Advances in Neural Information Processing Systems (2016),
pp. 2226–2234