前馈神经网络反向传播手酸

### 前馈神经网络反向传播算法原理

前馈神经网络是一种多层感知器模型，在处理数据时信息仅向前流动，从输入层经过隐藏层最终到达输出层。对于非线性问题如异或运算，单个神经元无法模拟其特性，因此需要构建具有至少一层隐藏层的神经网络来逼近这种复杂函数[^2]。

#### 反向传播基本概念

反向传播（Backpropagation, BP）是用于训练人工神经网络的一种监督学习技术。该方法通过计算损失相对于权重的梯度，并利用这些梯度更新网络参数以最小化预测误差。具体来说：

- **正向传递**：给定一组输入样本及其对应的标签，先执行一次完整的前向遍历得到当前模型下的预测结果；
- **计算误差**：基于预测值与真实值之间的差异评估整体性能，通常采用均方差作为衡量指标；
- **反向求导**：依据链式法则逐层回溯至各节点处累积局部敏感度因子δ，进而获得每条连接上的偏微分贡献；
- **参数修正**：按照预设的学习率η以及可能存在的动量系数α调整相应链接强度w_ij，使得下一轮迭代能够更接近最优解路径[^4]。

import numpy as np

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

def sigmoid_derivative(x):
    return x * (1 - x)

class NeuralNetwork:
    def __init__(self, layers):
        self.layers = layers
        self.weights = []
        for i in range(1, len(layers)):
            w = np.random.randn(layers[i], layers[i-1])
            self.weights.append(w)
    
    def forward(self, X):
        activations = [X]
        z_values = []
        a = X
        
        for w in self.weights[:-1]:
            z = np.dot(a, w.T)
            a = sigmoid(z)
            activations.append(a)
            z_values.append(z)
        
        last_w = self.weights[-1]
        final_z = np.dot(a, last_w.T)
        output = sigmoid(final_z)
        activations.append(output)
        z_values.append(final_z)
        
        return activations, z_values
    
    def backward(self, X, y, learning_rate=0.5, momentum_factor=0.9):
        m = X.shape[0]
        delta_weights = [np.zeros_like(w) for w in self.weights]

        # Forward pass to get all activation and weighted sum values.
        activations, z_values = self.forward(X)
        prediction = activations[-1]
        error = -(y - prediction)

        deltas = []

        # Compute the gradient of cost function with respect to each weight matrix using backpropagation algorithm.
        for layer_index in reversed(range(len(self.weights))):
            if layer_index == len(self.weights)-1:
                delta = error * sigmoid_derivative(prediction)
            else:
                delta = np.dot(deltas[-1], self.weights[layer_index+1]) * sigmoid_derivative(activations[layer_index+1])

            deltas.insert(0, delta)

            grad_W = np.dot(delta.T, activations[layer_index])/m
            
            # Update weights considering both current gradient direction and previous updates' influence via momentum term.
            delta_weights[layer_index] = (-learning_rate*grad_W + momentum_factor*delta_weights[layer_index]).T

        for idx, dw in enumerate(delta_weights):
            self.weights[idx] += dw

nn = NeuralNetwork([2, 2, 1])
for epoch in range(10000):  
    nn.backward(np.array([[0,0],[0,1],[1,0],[1,1]]), np.array([[0],[1],[1],[0]]))
print(nn.forward(np.array([[0,0],[0,1],[1,0],[1,1]]))[0][-1])

此代码片段展示了如何创建一个简单的两层全连接型前馈神经网络并应用带有动量项的标准BP优化策略对其进行训练的过程。这里选择了经典的异或门分类任务作为测试案例，验证了所设计架构的有效性和泛化能力。