批梯度下降、随机梯度下降、小批量梯度下降的可运行Python代码实现

在机器学习中，梯度下降是一种常用的优化算法，用于最小化损失函数。批梯度下降、随机梯度下降和小批量梯度下降是其三种常见的变体。下面是每种方法的简单 Python 代码实现：

# 导入必要的库
import numpy as np

# 定义模型参数
theta = np.random.randn(2)  # 假设我们有一个二维参数向量

# 梯度下降的核心函数
def gradient_descent(loss_func, gradients, learning_rate, num_iterations):
    for _ in range(num_iterations):
        theta -= learning_rate * gradients(theta)
    return theta

# 批梯度下降
def batch_gradient_descent(X, y, learning_rate=0.01, num_iterations=1000):
    m = len(y)
    J_history = []
    for i in range(num_iterations):
        loss = compute_loss(X, y, theta)
        gradients = compute_gradients(X, y, theta)
        J_history.append(loss)
        theta = gradient_descent(lambda x: loss(x), gradients, learning_rate, 1)
    return theta, J_history

# 随机梯度下降
def stochastic_gradient_descent(X, y, learning_rate=0.01, num_iterations=1000):
    m = len(y)
    J_history = []
    for i in range(num_iterations):
        random_index = np.random.randint(m)
        loss = compute_loss(X[random_index], y[random_index], theta)
        gradients = compute_gradients(X[random_index], y[random_index], theta)
        J_history.append(loss)
        theta = gradient_descent(lambda x: loss(x), gradients, learning_rate, 1)
    return theta, J_history

# 小批量梯度下降
def mini_batch_gradient_descent(X, y, batch_size=32, learning_rate=0.01, num_iterations=1000):
    m = len(y)
    batches_num = m // batch_size + (m % batch_size > 0)
    J_history = []
    for _ in range(num_iterations):
        indices = np.random.choice(m, batch_size, replace=False)
        loss = compute_loss(X[indices], y[indices], theta)
        gradients = compute_gradients(X[indices], y[indices], theta)
        J_history.append(loss)
        theta = gradient_descent(lambda x: loss(x), gradients, learning_rate, 1)
    return theta, J_history

# 这里假设compute_loss() 和 compute_gradients() 函数分别计算损失和梯度

上述代码仅提供了一个基础框架，实际应用中需要根据具体的损失函数和数据结构进行调整。同时注意，`compute_loss()` 和 `compute_gradients()` 是关键部分，取决于您的任务和所使用的模型。