batch normalization代码

Batch normalization 是一种常用的神经网络正则化方法，可以加速神经网络的训练过程。以下是一个简单的 batch normalization 的代码实现：

import numpy as np

class BatchNorm:
    def __init__(self, gamma, beta, eps=1e-5):
        self.gamma = gamma
        self.beta = beta
        self.eps = eps
        self.running_mean = None
        self.running_var = None
        self.batch_size = None
        self.xc = None
        self.std = None
        self.dgamma = None
        self.dbeta = None

    def forward(self, x, train=True):
        N, D = x.shape
        if self.running_mean is None:
            self.running_mean = np.zeros(D)
            self.running_var = np.zeros(D)

        if train:
            self.batch_size = N
            mu = np.mean(x, axis=0)
            xc = x - mu
            var = np.mean(xc ** 2, axis=0)
            std = np.sqrt(var + self.eps)
            xn = xc / std

            self.running_mean = 0.9 * self.running_mean + 0.1 * mu
            self.running_var = 0.9 * self.running_var + 0.1 * var

            self.xc = xc
            self.std = std
        else:
            xc = x - self.running_mean
            xn = xc / np.sqrt(self.running_var + self.eps)

        out = self.gamma * xn + self.beta
        return out

    def backward(self, dout):
        dxn = dout * self.gamma
        dxc = dxn / self.std
        dstd = -np.sum(dxn * self.xc, axis=0) / (self.std ** 2)
        dvar = 0.5 * dstd / self.std
        dxc += (2.0 / self.batch_size) * self.xc * dvar
        dmu = np.sum(dxc, axis=0)
        dx = dxc - dmu / self.batch_size

        self.dgamma = np.sum(dout * xn, axis=0)
        self.dbeta = np.sum(dout, axis=0)

        return dx

这段代码实现了一个 BatchNorm 类，可以在神经网络中使用。在 forward 函数中，首先计算出当前 batch 的均值和方差，然后对输入数据进行标准化，最后乘以 gamma 加上 beta 得到输出。在 backward 函数中，根据链式法则计算出梯度。