Consider a linear model Y = α + β TX + ε. (1) Set X ∼ MV N(0, Σ), Σ = (ρ |i−j| )p×p (the AR(1) structure), where ρ = 0.5, α = 1, β = (2, 1.5, 0, 0, 1, 0, . . . , 0)T , ε ∼ N(0, 1), simulate Y = α + β TX + ε, where the predictor dimension p = 20 and the sample size n = 200. Here, by the model settings, X1, X2 and X5 are the important variables. (2) Estimate regression coefficients using LASSO. by use 5-folds CV to choose optimal λ by minimizing the CV prediction error (PE), and plot the PE with different λ. using the coordinate decent algorithm and soft thresholding python 代码

以下是使用Python进行LASSO回归及交叉验证的代码，使用的是sklearn库中的LassoCV模型：

import numpy as np
from sklearn.linear_model import LassoCV
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import KFold

# 1.生成数据
np.random.seed(123)
p = 20
n = 200
rho = 0.5
alpha = 1
beta = np.array([2, 1.5, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
Sigma = np.zeros((p, p))
for i in range(p):
    for j in range(p):
        Sigma[i, j] = rho ** np.abs(i - j)
X = np.random.multivariate_normal(np.zeros(p), Sigma, n)
epsilon = np.random.normal(0, 1, n)
Y = alpha + np.dot(X, beta) + epsilon

# 2.使用LASSOCV进行回归及交叉验证
kf = KFold(n_splits=5, shuffle=True, random_state=123)  # 5折交叉验证
lasso_cv = LassoCV(cv=kf)
lasso_cv.fit(X, Y)

# 3.绘制交叉验证误差随lambda的变化曲线
mse_path = np.mean(lasso_cv.mse_path_, axis=1)
std_error = np.std(lasso_cv.mse_path_, axis=1) / np.sqrt(5)
lambdas = lasso_cv.alphas_
min_mse = np.min(mse_path)
optimal_lambda = lambdas[np.argmin(mse_path)]
print('Optimal Lambda:', optimal_lambda)

import matplotlib.pyplot as plt
plt.errorbar(np.log10(lambdas), mse_path, yerr=std_error)
plt.axvline(np.log10(optimal_lambda), linestyle='--', color='r')
plt.xlabel('log10(lambda)')
plt.ylabel('Mean Squared Error')
plt.title('LASSO Cross Validation')
plt.show()

# 4.输出回归系数
coefficients = lasso_cv.coef_
print('Regression Coefficients:', coefficients)

这里使用交叉验证的方法来选择最优的正则化参数lambda，通过绘制交叉验证误差随lambda的变化曲线来确定最优的lambda值，并输出对应的回归系数。