gbdt回归预测的模型调优代码

GBDT（Gradient Boosting Decision Tree）是一种常用的回归预测模型，其调优可以通过以下步骤实现：

1. 导入需要的库和数据集，将数据集拆分为训练集和测试集。

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.metrics import mean_squared_error

data = pd.read_csv('data.csv')
X = data.drop('target', axis=1)
y = data['target']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

2. 使用默认参数训练模型，并计算测试集上的均方误差（MSE）。

gbdt = GradientBoostingRegressor(random_state=42)
gbdt.fit(X_train, y_train)
y_pred = gbdt.predict(X_test)
mse = mean_squared_error(y_test, y_pred)
print("MSE (default):", mse)

3. 调整树的数量（n_estimators）。

for n in range(50, 201, 50):
    gbdt = GradientBoostingRegressor(n_estimators=n, random_state=42)
    gbdt.fit(X_train, y_train)
    y_pred = gbdt.predict(X_test)
    mse = mean_squared_error(y_test, y_pred)
    print("n_estimators:", n, "MSE:", mse)

4. 调整每棵树的最大深度（max_depth）。

for d in range(3, 10):
    gbdt = GradientBoostingRegressor(max_depth=d, random_state=42)
    gbdt.fit(X_train, y_train)
    y_pred = gbdt.predict(X_test)
    mse = mean_squared_error(y_test, y_pred)
    print("max_depth:", d, "MSE:", mse)

5. 调整每棵树的叶子节点数量（min_samples_leaf）。

for l in range(1, 6):
    gbdt = GradientBoostingRegressor(min_samples_leaf=l, random_state=42)
    gbdt.fit(X_train, y_train)
    y_pred = gbdt.predict(X_test)
    mse = mean_squared_error(y_test, y_pred)
    print("min_samples_leaf:", l, "MSE:", mse)

6. 调整学习率（learning_rate）。

for r in [0.1, 0.05, 0.01]:
    gbdt = GradientBoostingRegressor(learning_rate=r, random_state=42)
    gbdt.fit(X_train, y_train)
    y_pred = gbdt.predict(X_test)
    mse = mean_squared_error(y_test, y_pred)
    print("learning_rate:", r, "MSE:", mse)

7. 根据调优结果选择最佳参数进行模型训练和预测。

gbdt = GradientBoostingRegressor(n_estimators=150, max_depth=5, min_samples_leaf=3, learning_rate=0.05, random_state=42)
gbdt.fit(X_train, y_train)
y_pred = gbdt.predict(X_test)
mse = mean_squared_error(y_test, y_pred)
print("MSE (best):", mse)