首页batch normalization怎么用呢？

batch normalization怎么用呢？

时间: 2023-10-05 14:09:33 浏览: 84

Batch Normalization

Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of human raters.

Batch normalization 是一种常用的神经网络正则化方法，它用于在训练过程中减少内部协变量移位。一般而言，在每一个训练批次中，Batch normalization 会计算批次中的样本的均值和标准差，并将这些统计数据用于标准化每个样本的特征。这样做可以有效地防止由于特征值的变化而导致的梯度爆炸和消失问题，同时也有助于加速训练过程。在深度学习框架中，一般都会有现成的 batch normalization 函数可以直接调用。

阅读全文