统计学习导论：斯坦福STATS 202教材

需积分: 9 51 浏览量更新于2024-07-18 收藏 12.58MB PDF 举报

"Stanford STATS 202是斯坦福大学提供的一门关于统计学习的课程，这门课程关注的是理解和处理复杂数据集的方法。统计学习是统计学和计算机科学，特别是机器学习领域的一个新兴交叉学科，其中包括了各种方法，如套索回归（Lasso）、稀疏回归、分类与回归树以及提升算法（Boosting）和支持向量机（SVM）。" 《An Introduction to Statistical Learning》是一本与Stanford STATS 202课程相关的教科书，由Gareth James、Daniela Witten、Trevor Hastie和Robert Tibshirani四位作者撰写，他们均在统计学和相关领域有深厚的专业背景。这本书旨在介绍统计学习的基本概念，并结合R语言进行实际应用。本书涵盖了多个关键主题，包括： 1. **统计学习基础**：介绍统计学习的基本原理，解释如何利用这些工具来建立数据模型并理解数据的内在结构。 2. **线性模型**：讲解经典的线性回归模型，以及如何通过Lasso进行变量选择和正则化，以提高模型的预测能力和解释性。 3. **决策树和随机森林**：介绍分类和回归树（CART），以及随机森林等集成方法，这些方法在处理非线性关系和高维数据时特别有效。 4. **支持向量机（SVM）**：讨论支持向量机作为分类和回归工具的理论与应用，它能处理复杂的非线性问题。 5. ** Boosting**：详述Boosting算法，这是一种通过组合弱学习器形成强学习器的技术，常用于提高预测准确性和处理不平衡数据集。 6. **聚类分析**：探讨无监督学习中的聚类方法，帮助识别数据中的自然群体或模式。 7. **模型选择与评估**：介绍如何选择最佳模型，包括交叉验证、AIC和BIC等评估指标。 8. **R语言实践**：书中所有方法都与R语言的实际代码相结合，读者可以立即在R环境中实现和探索这些统计学习方法。该书作为 SpringerTexts in Statistics 系列的一部分，旨在为学生和研究人员提供一个易懂且实用的统计学习入门资源。书中的实例和练习题有助于加深对理论的理解，并培养解决实际问题的能力。 Stanford STATS 202和《An Introduction to Statistical Learning》提供了深入理解统计学习的宝贵资源，不仅适合初学者，也对有经验的数据科学家和统计学家有价值。通过学习这些内容，读者将能够掌握一系列统计学习工具，以应对现代大数据分析中的挑战。

1. Introduction 3

Yesterday

Today’s Direction

Percentage change in S&P

Two Days Previous

Percentage change in S&P

Down Up

Today’s Direction

Down Up

Today’s Direction

Down Up

−4 −2 0 2 4 6

Three Days Previous

Percentage change in S&P

FIGURE 1.2. Left: Boxplots of the previous day’s percentage change in the S&P

index for the days for which the market increased or decreased, obtained from the

Smarket data. Center and Right: Same as left panel, but the percentage changes

for 2 and 3 days previous are shown.

or qualitative output. For example, in Chapter 4 we examine a stock mar-

ket data set that contains the daily movements in the Standard & Poor’s

500 (S&P) stock index over a 5-year period between 2001 and 2005. We

refer to this as the

Smarket data. The goal is to predict whether the index

will increase or decrease on a given day using the past 5 days’ percentage

changes in the index. Here the statistical learning problem does not in-

volve predicting a numerical value. Instead it involves predicting whether

a given day’s stock market performance will fall into the

Up bucket or the

Down bucket. This is known as a classiﬁcation problem. A model that could

accurately predict the direction in which the market will move would be

very useful!

The left-hand panel of Figure 1.2 displays two boxplots of the previous

day’s percentage changes in the stock index: one for the 648 days for which

the market increased on the subsequent day, and one for the 602 days for

which the market decreased. The two plots look almost identical, suggest-

ing that there is no simple strategy for using yesterday’s movement in the

S&P to predict today’s returns. The remaining panels, which display box-

plots for the percentage changes 2 and 3 days previous to today, similarly

indicate little association between past and present returns. Of course, this

lack of pattern is to be expected: in the presence of strong correlations be-

tween successive days’ returns, one could adopt a simple trading strategy

to generate proﬁts from the market. Nevertheless, in Chapter 4,weexplore

these data using several diﬀerent statistical learning methods. Interestingly,

there are hints of some weak trends in the data that suggest that, at least

for this 5-year period, it is possible to correctly predict the direction of

movement in the market approximately 60% of the time (Figure 1.3).

4 1. Introduction

Down Up

0.46 0.48 0.50 0.52

Today’s Direction

Predicted Probability

FIGURE 1.3. We ﬁt a quadratic discriminant analysis model to the subset

of the Smarket data corresponding to the 2001–2004 time period, and predicted

the probability of a stock market decrease using the 2005 data. On average, the

predicted probability of decrease is higher for the days in which the market does

decrease. Based on these results, we are able to correctly predict the direction of

movement in the market 60% of the time.

Gene Expression Data

The previous two applications illustrate data sets with both input and

output variables. However, another important class of problems involves

situations in which we only observe input variables, with no corresponding

output. For example, in a marketing setting, we might have demographic

information for a number of current or potential customers. We may wish to

understand which types of customers are similar to each other by grouping

individuals according to their observed characteristics. This is known as a

clustering problem. Unlike in the previous examples, here we are not trying

to predict an output variable.

We devote Chapter 10 to a discussion of statistical learning methods

for problems in which no natural output variable is available. We consider

the

NCI60 data set, which consists of 6,830 gene expression measurements

for each of 64 cancer cell lines. Instead of predicting a particular output

variable, we are interested in determining whether there are groups, or

clusters, among the cell lines based on their gene expression measurements.

This is a diﬃcult question to address, in part because there are thousands

of gene expression measurements per cell line, making it hard to visualize

the data.

The left-hand panel of Figure 1.4 addresses this problem by represent-

ing each of the 64 cell lines using just two numbers, Z

and Z

.These

are the ﬁrst two principal components of the data, which summarize the

6, 830 expression measurements for each cell line down to two numbers or

dimensions. While it is likely that this dimension reduction has resulted in

1. Introduction 5

−40 −20 0 20 40 60

−60 −40 −20 0 20

−40 −20 0 20 40 60

FIGURE 1.4. Left: Representation of the NCI60 gene expression data set in

a two-dimensional space, Z

and Z

. Each point corresponds to one of the 64

cell lines. There appear to be four groups of cell lines, which we have represented

using diﬀerent colors. Right: Same as left panel except that we have represented

each of the 14 diﬀerent types of cancer using a diﬀerent colored symbol. Cell lines

corresponding to the same cancer type tend to be nearby in the two-dimensional

space.

some loss of information, it is now possible to visually examine the data for

evidence of clustering. Deciding on the number of clusters is often a diﬃ-

cult problem. But the left-hand panel of Figure 1.4 suggests at least four

groups of cell lines, which we have represented using separate colors. We

can now examine the cell lines within each cluster for similarities in their

types of cancer, in order to better understand the relationship between

gene expression levels and cancer.

In this particular data set, it turns out that the cell lines correspond

to 14 diﬀerent types of cancer. (However, this information was not used

to create the left-hand panel of Figure 1.4.) The right-hand panel of Fig-

ure 1.4 is identical to the left-hand panel, except that the 14 cancer types

are shown using distinct colored symbols. There is clear evidence that cell

lines with the same cancer type tend to be located near each other in this

two-dimensional representation. In addition, even though the cancer infor-

mation was not used to produce the left-hand panel, the clustering obtained

does bear some resemblance to some of the actual cancer types observed

in the right-hand panel. This provides some independent veriﬁcation of the

accuracy of our clustering analysis.

A Brief History of Statistical Learning

Though the term statistical learning is fairly new, many of the concepts

that underlie the ﬁeld were developed long ago. At the beginning of the

nineteenth century, Legendre and Gauss published papers on the method

6 1. Introduction

of least squares, which implemented the earliest form of what is now known

as linear regression. The approach was ﬁrst successfully applied to problems

in astronomy. Linear regression is used for predicting quantitative values,

such as an individual’s salary. In order to predict qualitative values, such as

whether a patient survives or dies, or whether the stock market increases

or decreases, Fisher proposed linear discriminant analysis in 1936. In the

1940s, various authors put forth an alternative approach, logistic regression.

In the early 1970s, Nelder and Wedderburn coined the term generalized

linear models for an entire class of statistical learning methods that include

both linear and logistic regression as special cases.

By the end of the 1970s, many more techniques for learning from data

were available. However, they were almost exclusively linear methods, be-

cause ﬁtting non-linear relationships was computationally infeasible at the

time. By the 1980s, computing technology had ﬁnally improved suﬃciently

that non-linear methods were no longer computationally prohibitive. In mid

1980s Breiman, Friedman, Olshen and Stone introduced classiﬁcation and

regression trees, and were among the ﬁrst to demonstrate the power of a

detailed practical implementation of a method, including cross-validation

for model selection. Hastie and Tibshirani coined the term generalized addi-

tive models in 1986 for a class of non-linear extensions to generalized linear

models, and also provided a practical software implementation.

Since that time, inspired by the advent of machine learning and other

disciplines, statistical learning has emerged as a new subﬁeld in statistics,

focused on supervised and unsupervised modeling and prediction. In recent

years, progress in statistical learning has been marked by the increasing

availability of powerful and relatively user-friendly software, such as the

popular and freely available

R system. This has the potential to continue

the transformation of the ﬁeld from a set of techniques used and developed

by statisticians and computer scientists to an essential toolkit for a much

broader community.

This Book

The Elements of Statistical Learning (ESL) by Hastie, Tibshirani, and

Friedman was ﬁrst published in 2001. Since that time, it has become an

important reference on the fundamentals of statistical machine learning.

Its success derives from its comprehensive and detailed treatment of many

important topics in statistical learning, as well as the fact that (relative to

many upper-level statistics textbooks) it is accessible to a wide audience.

However, the greatest factor behind the success of ESL has been its topical

nature. At the time of its publication, interest in the ﬁeld of statistical

1. Introduction 7

learning was starting to explode. ESL provided one of the ﬁrst accessible

and comprehensive introductions to the topic.

Since ESL was ﬁrst published, the ﬁeld of statistical learning has con-

tinued to ﬂourish. The ﬁeld’s expansion has taken two forms. The most

obvious growth has involved the development of new and improved statis-

tical learning approaches aimed at answering a range of scientiﬁc questions

across a number of ﬁelds. However, the ﬁeld of statistical learning has

also expanded its audience. In the 1990s, increases in computational power

generated a surge of interest in the ﬁeld from non-statisticians who were

eager to use cutting-edge statistical tools to analyze their data. Unfortu-

nately, the highly technical nature of these approaches meant that the user

community remained primarily restricted to experts in statistics, computer

science, and related ﬁelds with the training (and time) to understand and

implement them.

In recent years, new and improved software packages have signiﬁcantly

eased the implementation burden for many statistical learning methods.

At the same time, there has been growing recognition across a number of

ﬁelds, from business to health care to genetics to the social sciences and

beyond, that statistical learning is a powerful tool with important practical

applications. As a result, the ﬁeld has moved from one of primarily academic

interest to a mainstream discipline, with an enormous potential audience.

This trend will surely continue with the increasing availability of enormous

quantities of data and the software to analyze it.

The purpose of An Introduction to Statistical Learning (ISL) is to facili-

tate the transition of statistical learning from an academic to a mainstream

ﬁeld. ISL is not intended to replace ESL, which is a far more comprehen-

sive text both in terms of the number of approaches considered and the

depth to which they are explored. We consider ESL to be an important

companion for professionals (with graduate degrees in statistics, machine

learning, or related ﬁelds) who need to understand the technical details

behind statistical learning approaches. However, the community of users of

statistical learning techniques has expanded to include individuals with a

wider range of interests and backgrounds. Therefore, we believe that there

is now a place for a less technical and more accessible version of ESL.

In teaching these topics over the years, we have discovered that they are

of interest to master’s and PhD students in ﬁelds as disparate as business

administration, biology, and computer science, as well as to quantitatively-

oriented upper-division undergraduates. It is important for this diverse

group to be able to understand the models, intuitions, and strengths and

weaknesses of the various approaches. But for this audience, many of the

technical details behind statistical learning methods, such as optimiza-

tion algorithms and theoretical properties, are not of primary interest.

We believe that these students do not need a deep understanding of these

aspects in order to become informed users of the various methodologies, and

剩余433页未读，继续阅读

ddddddawn

粉丝: 0
资源: 1

统计学习导论：斯坦福STATS 202教材

Stanford CS41assignment1

C#下调用Stanford CoreNLP

StanfordPaRser

Stanford tregex

stanford parser

GPS stanford

stanford tagger

stanford bunny

globus.stanford.edu:globus.stanford.edu — Globus @ Stanford网站

Stanford.NLP.NET：.NET的Stanford NLP

最新资源