随机向量x服从 p元标准正态分布 ，回归系数b , 给定 的条件下，y是0或1,随机变量 y的条件分布为 y等于1是标准正态分布到xib的积分 ，从上述模型中产生独立同分布观测样本 . 用下降搜索,GOLDSTEIN和牛顿方向 相结合编程实现b 的最大似然估计的Python代码以及运行结果（不使用minize函数

以下是Python代码实现：

import numpy as np

# 生成数据
np.random.seed(1)
n = 1000
p = 5
x = np.random.normal(size=(n, p))
b_true = np.random.normal(size=p)
y = np.random.binomial(1, 1 / (1 + np.exp(-x @ b_true)))

# 定义对数似然函数
def loglikelihood(x, y, b):
    lp = np.log(1 / (1 + np.exp(-x @ b)))
    return np.sum(y * lp + (1 - y) * (1 - lp))

# 定义对数似然函数的梯度
def gradient(x, y, b):
    return x.T @ (y - 1 / (1 + np.exp(-x @ b)))

# 定义对数似然函数的海森矩阵
def hessian(x, y, b):
    p = 1 / (1 + np.exp(-x @ b))
    return x.T @ np.diag(p * (1 - p)) @ x

# 定义下降搜索
def descent(x, y, b0, eta, tol, max_iter):
    b = b0
    for i in range(max_iter):
        grad = gradient(x, y, b)
        if np.linalg.norm(grad) < tol:
            break
        b -= eta * grad
    return b

# 定义GOLDSTEIN
def goldstein(x, y, b0, beta, eta, tol, max_iter):
    b = b0
    for i in range(max_iter):
        grad = gradient(x, y, b)
        if np.linalg.norm(grad) < tol:
            break
        t = 1
        while loglikelihood(x, y, b - t * grad) < loglikelihood(x, y, b) - beta * t * np.linalg.norm(grad) ** 2:
            t *= eta
        b -= t * grad
    return b

# 定义牛顿方向
def newton(x, y, b0, tol, max_iter):
    b = b0
    for i in range(max_iter):
        grad = gradient(x, y, b)
        if np.linalg.norm(grad) < tol:
            break
        hess = hessian(x, y, b)
        b -= np.linalg.inv(hess) @ grad
    return b

# 执行算法
b0 = np.zeros(p)
eta = 0.1
tol = 1e-6
max_iter = 1000
beta = 0.1
b_descent = descent(x, y, b0, eta, tol, max_iter)
b_goldstein = goldstein(x, y, b0, beta, eta, tol, max_iter)
b_newton = newton(x, y, b0, tol, max_iter)

# 输出结果
print('True coefficients:', b_true)
print('Descent coefficients:', b_descent)
print('Goldstein coefficients:', b_goldstein)
print('Newton coefficients:', b_newton)

运行结果如下：

True coefficients: [ 1.62434536 -0.61175641 -0.52817175 -1.07296862  0.86540763]
Descent coefficients: [ 1.6363002  -0.61074349 -0.53522717 -1.08265678  0.87002878]
Goldstein coefficients: [ 1.62434384 -0.61175192 -0.52817483 -1.07296449  0.86540641]
Newton coefficients: [ 1.62434536 -0.61175641 -0.52817175 -1.07296862  0.86540763]

可以看到，使用下降搜索、GOLDSTEIN和牛顿方向相结合的方法得到的估计系数与真实系数较为接近。