统计学习基石：集成学习详解
下载需积分: 8 | PDF格式 | 870KB |
更新于2024-08-02
| 41 浏览量 | 举报
《统计学习元素》(第二版)是Trevor Hastie等人于2008年编著的一本经典之作，它在大量计算和信息技术兴起的时代背景下，探讨了数据科学领域的新工具和技术发展。随着医学、生物学、金融和市场营销等领域数据的爆炸性增长，对这些数据的理解促使统计学领域诞生了新的概念，如数据挖掘、机器学习和生物信息学。本书的目标是把这些领域的核心思想置于一个共同的概念框架下，强调的是概念而非数学细节。
16章的主题是“集成学习”，这是一种构建预测模型的方法，通过集合简单基础模型的优势来提高整体性能。书中提及了几个例子，如第8.7节的自助法（bagging）和第15章的随机森林，它们都是用于分类的集成方法，每个弱学习者（如决策树）都为预测类别投出一票。第10章的提升算法起初也被视为委员会方法，与随机森林不同，弱学习者的组合会随着时间演变，权重也随之调整。
另外，第8.8节介绍的堆叠（stacking）是一种创新的方法，它将多个已训练模型的优势相结合，其中基函数可以被视为弱学习器。非参数回归中的贝叶斯方法也可以看作是集成方法，众多候选模型根据参数设置的后验分布进行平均。
集成学习主要分为两个任务：一是开发有效的基础模型或弱学习器，二是设计策略将这些模型组合起来形成强大的预测系统。这种方法的优势在于能够减少过拟合风险，提高泛化能力，并且在面对复杂问题时，通过集体智慧往往能提供更好的预测结果。
书中还涵盖了神经网络、支持向量机、分类树（CART）、MARS（多项式回归分析）以及投影寻踪等多种技术，这些都是首次在本书中得到全面的讨论。Trevor Hastie、Robert Tibshirani和Jerome Friedman三位作者作为斯坦福大学统计学教授，他们的研究成果和贡献使得本书成为统计学家和数据挖掘领域从业者的重要参考资源。无论是在科学还是工业界，对于数据理解与挖掘的探索者来说，《统计学习元素》都是不可或缺的指南。
608 16. Ensemble Learning
Algorithm 16.1 Forward Stagewise Linear Regression.
1. Initialize ˇα
k
=0,k=1,...,K. Set ε>0 to some small constant,
and M large.
2. For m =1toM:
(a) (β
∗
,k
∗
) = arg min
β,k
N
i=1
y
i
−
K
l=1
α
l
T
l
(x
i
) − βT
k
(x
i
)
2
.
(b) ˇα
k
∗
← ˇα
k
∗
+ ε · sign(β
∗
).
3. Output f
M
(x)=
K
k=1
ˇα
k
T
k
(x).
This seems reasonable since it is likely that only a small fraction of all pos-
sible trees will be relevant in approximating any particular target function.
However, the relevant subset will be different for different targets. Those
coefficients that are not set to zero are shrunk by the lasso in that their
absolute values are smaller than their corresponding least squares values
2
:
| ˆα
k
(λ) | < | ˆα
k
(0) |.Asλ increases, the coefficients all shrink, each one
ultimately becoming zero.
Owing to the very large number of basis functions T
k
, directly solving
(16.2) with the lasso penalty (16.4) is not possible. However, a feasible
forward stagewise strategy exists that closely approximates the effect of
the lasso, and is very similar to boosting and the forward stagewise Algo-
rithm 10.2. Algorithm 16.1 gives the details. Although phrased in terms
of tree basis functions T
k
, the algorithm can be used with any set of ba-
sis functions. Initially all coefficients are zero in line 1; this corresponds
to λ = ∞ in (16.2). At each successive step, the tree T
k
∗
is selected that
best fits the current residuals in line 2(a). Its corresponding coefficient ˇα
k
∗
is then incremented or decremented by an infinitesimal amount in 2(b),
while all other coefficientsˇα
k
,k= k
∗
are left unchanged. In principle, this
process could be iterated until either all the residuals are zero, or β
∗
=0.
The latter case can occur if K<N, and at that point the coefficient values
represent a least squares solution. This corresponds to λ = 0 in (16.2).
After applying Algorithm 16.1 with M<∞ iterations, many of the coef-
ficients will be zero, namely, those that have yet to be incremented. The oth-
ers will tend to have absolute values smaller than their corresponding least
squares solution values, | ˇα
k
(M) | < | ˆα
k
(0) |. Therefore this M -iteration
solution qualitatively resembles the lasso, with M inversely related to λ.
Figure 16.1 shows an example, using the prostate data studied in Chap-
ter 3. Here, instead of using trees T
k
(X) as basis functions, we use the origi-
2
If K>N, there is in general no unique “least squares value,” since infinitely many
solutions will exist that fit the data perfectly. We can pick the minimum L
1
-norm solution
amongst these, which is the unique lasso solution.
剩余19页未读，继续阅读
相关推荐
9 浏览量
普通网友
- 粉丝: 0
- 资源: 19
我的内容管理
展开
最新资源
- rsa-src.zip
- 煤矿采煤机自动化与智能化技术研究.rar
- Highlight to Google Calendar-crx插件
- 博通网卡管理软件(Management Applications) v17.0.5.1 官方版
- peep-object:检查对象的所有组件
- NetThief81_8582.7z
- 大亨游戏
- Enegy-Generation-Company-SunSolar-ForntEnd-
- Rapid BSR-crx插件
- autocert:Python Web应用程序的自动TLS证书发行和续签
- 网上书店模板（有demo设计文档和界面源码，界面很帅哟，）
- TinyLinqJs:Linq-to-Objects 的 JavaScript 实现，以便将其与标准 JavaScript 数组一起使用
- arya.adslab
- Zet-crx插件
- 人脸检测编程实验工具.rar
- 腾达W522U无线USB网卡驱动
