首页深度学习中的averaged stats

深度学习中的averaged stats

时间: 2024-04-27 22:22:22 浏览: 171

A Statistical View of deep learning

Deep learning and the use of deep neural networks are now established as a key tool for practical machine learning. Neural networks have an equivalence with many existing statistical and machine learning approaches and I would like to explore one of these views in this post. In particular, I'll look at the view of deep neural networks as recursive generalised linear models (RGLMs). Generalised linear models form one of the cornerstones of probabilistic modelling and are used in almost every field of experimental science, so this connection is an extremely useful one to have in mind. I'll focus here on what are called feedforward neural networks and leave a discussion of the statistical connections to recurrent networks to another post.

深度学习中的Averaged Stats通常指的是模型训练过程中的平均参数统计量。在一些优化算法中，比如SGD、Adam等，为了避免模型训练过程中出现过拟合现象，需要对参数进行正则化操作。其中，一种常见的正则化方法是L2正则化，它通过在损失函数中加入L2范数惩罚项，对模型参数进行约束。在L2正则化的过程中，会使用模型参数的平均值来计算惩罚项，从而对参数进行平滑化处理。这个平均值也被称为平均参数统计量。

阅读全文