STATISTICAL VIEW OF CHEMOMETRICS REGRESSION TOOLS
with
s = ave(yx)
(15)
and I being the p x p identity matrix. The inverse
of the (possibly) ill-conditioned predictor-variable
covariance matrix V is thus stabilized by adding to
V a multiple of I. The degree of stabilization is reg-
ulated by the value of the "ridge" parameter A > 0.
A value of A = oo results in the model being the
response mean y = 0, whereas A = 0 gives rise to
the unregularized OLS estimates. A value for A in
any particular situation is generally obtained by con-
sidering it to be a meta parameter of the procedure
and estimating it through some model-selection pro-
cedure such as CV. Since here the response values
{Yi}l do enter linearly in the model estimates {.9i}j,
any of the competing model-selection criteria can
also be straightforwardly applied (see Golub, Heath,
and Wahba 1979).
3. A COMPARISON OF PCR, PLS, AND RR
From their preceding algorithmic descriptions, it
might appear that PCR, PLS, and RR are very dif-
ferent procedures leading to quite different model
estimates. In this section we provide a heuristic com-
parison that suggests that they are, in fact, quite
similar, in that they are all attempting to achieve the
same operational goal in slightly different ways. That
goal is to bias the solution coefficient vector a (5)
away from directions for which the projected sample
predictor variables have small spread; that is,
var(aTx/lal) = ave(aTx/lal)2 = small, (16)
where the average is over the training sample.
This comparison consists of regarding the regres-
sion procedure as a two-step process as in VSS (Stone
and Brooks 1990); first a K-dimensional subspace of
p-dimensional Euclidean space is defined, and then
the regression is performed under the restriction that
the coefficient vector a lies in that subspace:
K
a= akck, (17)
k=l
where the unit vectors {ck}f span the prescribed
subspace with ckck = 1. The regression procedures
can be compared by the way in which they define
the subspace {ck}f and the manner in which the
(constrained) regression is performed.
First, consider OLS in this setup. Here the sub-
space is defined by the (single) unit vector that max-
imizes the sample correlation (squared) between the
response and the corresponding linear combination
of the predictor variables
COLS = argmax corr2(y, CTX); (18)
CTc= 1
the OLS solution is then a simple least squares
regression of y on COLSX,
YOLS = [ave(ycOLsx)/aveO(coLs))2] OLSX. (19)
RR can also be cast into this framework. As in
OLS, the subspace is defined by a single unit vector,
but the criterion that defines that vector is somewhat
different:
var(cTx)
CRR = argmax corr2(y, cTx) var(cTx)
cTc=l var(crx) + A'
(20)
where A is the ridge parameter [(13)-(14)]. The ridge
solution is then taken to be a (shrinking) ridge regres-
sion of y on CRRX with the same value for the ridge
parameter
ave (yCRX) T
YRR ae( RRX)
_ave(cTRX)2 + R
(21)
(See Appendix.)
PCR defines a sequence of K-dimensional sub-
spaces each spanned by the first K eigenvectors (9)
of V (8). Thus each ck (1 < k < R) is the solution
to
c
ck(PCR) argmax var(cTx).
{TVC = 0} -1
TC= 1
(22)
The first constraint in (22) (V orthogonality) ensures
that the linear combinations associated with the dif-
ferent solution vectors are uncorrelated over the
training sample
corr(ckx, cfx) = 0,
k : 1.
(23)
As a consequence of this and the criterion (22), they
also turn out to be orthogonal cTc, = 0, k = 1. The
Kth PCR model is given by a least squares regression
of the response on the K linear combinations
{ckx}f. Since they are uncorrelated (23), this reduces
to the sum of univariate regressions on each one (10).
PLS regression also produces a sequence of K-
dimensional subspaces spanned by successive unit
vectors, and then the Kth PLS solution is obtained
by a least squares fit of the response onto the cor-
responding K-linear combinations in a strategy sim-
ilar to PCR. The only difference from PCR is in the
criterion used to define the vectors that span the K-
dimensional subspace and hence the corresponding
linear combinations. The criterion that gives rise to
PLS (Stone and Brooks 1990) is
ck(PLS) = argman corr2(y, cTx)var(cTx). (24)
{cTVc = 0}- 1
crc=1
As with PCR the vectors ck(PLS) are constrained to
be mutually V orthogonal so that the corresponding
linear combinations are uncorrelated over the train-
TECHNOMETRICS, MAY 1993, VOL. 35, NO. 2
113
This content downloaded from
106.37.205.239 on Mon, 26 Oct 2020 13:05:23 UTC
All use subject to https://about.jstor.org/terms
剩余27页未读，继续阅读
AcaData数据分析
- 粉丝: 0
- 资源: 1
我的内容管理
展开
最新资源
- 解决Eclipse配置与导入Java工程常见问题
- 真空发生器：工作原理与抽吸性能分析
- 爱立信RBS6201开站流程详解
- 电脑开机声音解析：故障诊断指南
- JAVA实现贪吃蛇游戏
- 模糊神经网络实现与自学习能力探索
- PID型模糊神经网络控制器设计与学习算法
- 模糊神经网络在自适应PID控制器中的应用
- C++实现的学生成绩管理系统设计
- 802.1D STP 实现与优化：二层交换机中的生成树协议
- 解决Windows无法完成SD卡格式化的九种方法
- 软件测试方法：Beta与Alpha测试详解
- 软件测试周期详解：从需求分析到维护测试
- CMMI模型详解：软件企业能力提升的关键
- 移动Web开发框架选择：jQueryMobile、jQTouch、SenchaTouch对比
- Java程序设计试题与复习指南
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
