统计学习基础：第二版精华概述

需积分: 0 154 浏览量更新于2024-07-24 收藏 12.21MB PDF 举报

《统计学习要素》是机器学习领域的经典教材之一，由Hastie、Tibshirani和Friedman三位作者合著。该书自第一版以来深受读者喜爱，随着统计学习领域的快速发展，第二版应运而生，以适应研究的新进展和满足读者的需求。在第二版中，作者们做出了四章全新的内容添加和部分章节的更新，以保持教材的时效性和深度。尽管保留了第一版的框架，但为了尽量减少变化对读者的影响，他们在修订时保持谨慎。以下是主要的更新内容： 1. 新章节：新增的四章可能涵盖了诸如深度学习、非线性模型、高级特征工程和现代优化算法等前沿主题，反映了近年来统计学习方法的突破和创新。这些章节旨在帮助读者理解和掌握最新技术，提高在复杂数据集上的分析能力。 2. 更新内容：在原有的章节中，作者们可能会根据当前的研究成果更新理论基础、算法细节、实证案例和评估方法。他们可能讨论了更精确的模型解释、处理大规模数据的高效策略，以及对模型泛化性能的深入理解。 3. 引言与历史： Preface to the Second Edition 的前言部分引用了威廉·爱德华兹·戴明（William Edwards Deming）的名言，虽然这句话的真实来源存在争议，但反映了数据驱动决策的重要性，这也是统计学习的核心理念。 4. 在线资源与教学：由于网络的普及，书中可能包含了更多的在线资源链接，如代码示例、数据集和互动学习工具，以便读者可以在实践中应用所学知识。 5. 适应性与扩展性：第二版可能更加注重内容的可扩展性，允许读者根据自己的兴趣和需求选择阅读重点，同时保持整个体系的连贯性和完整性。《统计学习要素》第二版是对第一版的有益补充，它不仅巩固了基础理论，还紧跟时代的步伐，为读者提供了探索当今复杂数据世界的实用工具和理论支持。无论是初学者还是专业人士，都能从中受益匪浅。

xx Contents

14 Unsupervised Learning 485

14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 485

14.2 Association Rules . . . . . . . . . . . . . . . . . . . . . . 487

14.2.1 Market Basket Analysis . . . . . . . . . . . . . . 488

14.2.2 The Apriori Algorithm . . . . . . . . . . . . . . 489

14.2.3 Example: Market Basket Analysis . . . . . . . . 492

14.2.4 Unsupervised as Supervised Learning . . . . . . 495

14.2.5 Generalized Association Rules . . . . . . . . . . 497

14.2.6 Choice of Supervised Learning Method . . . . . 499

14.2.7 Example: Market Basket Analysis (Continued) . 499

14.3 Cluster Analysis . . . . . . . . . . . . . . . . . . . . . . . 501

14.3.1 Proximity Matrices . . . . . . . . . . . . . . . . 503

14.3.2 Dissimilarities Based on Attributes . . . . . . . 503

14.3.3 Object Dissimilarity . . . . . . . . . . . . . . . . 505

14.3.4 Clustering Algorithms . . . . . . . . . . . . . . . 507

14.3.5 Combinatorial Algorithms . . . . . . . . . . . . 507

14.3.6 K-means . . . . . . . . . . . . . . . . . . . . . . 509

14.3.7 Gaussian Mixtures as Soft K-means Clustering . 510

14.3.8 Example: Human Tumor Microarray Data . . . 512

14.3.9 Vector Quantization . . . . . . . . . . . . . . . . 514

14.3.10 K-medoids . . . . . . . . . . . . . . . . . . . . . 515

14.3.11 Practical Issues . . . . . . . . . . . . . . . . . . 518

14.3.12 Hierarchical Clustering . . . . . . . . . . . . . . 520

14.4 Self-Organizing Maps . . . . . . . . . . . . . . . . . . . . 528

14.5 Principal Components, Curves and Surfaces . . . . . . . . 534

14.5.1 Principal Components . . . . . . . . . . . . . . . 534

14.5.2 Principal Curves and Surfaces . . . . . . . . . . 541

14.5.3 Spectral Clustering . . . . . . . . . . . . . . . . 544

14.5.4 Kernel Principal Components . . . . . . . . . . . 547

14.5.5 Sparse Principal Components . . . . . . . . . . . 550

14.6 Non-negative Matrix Factorization . . . . . . . . . . . . . 553

14.6.1 Archetypal Analysis . . . . . . . . . . . . . . . . 554

14.7 Independent Component Analysis

and Exploratory Projection Pursuit . . . . . . . . . . . . 557

14.7.1 Latent Variables and Factor Analysis . . . . . . 558

14.7.2 Independent Component Analysis . . . . . . . . 560

14.7.3 Exploratory Projection Pursuit . . . . . . . . . . 565

14.7.4 A Direct Approach to ICA . . . . . . . . . . . . 565

14.8 Multidimensional Scaling . . . . . . . . . . . . . . . . . . 570

14.9 Nonlinear Dimension Reduction

and Local Multidimensional Scaling . . . . . . . . . . . . 572

14.10 The Google PageRank Algorithm . . . . . . . . . . . . . 576

Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 578

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 579

Contents xxi

15 Random Forests 587

15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 587

15.2 Deﬁnition of Random Forests . . . . . . . . . . . . . . . . 587

15.3 Details of Random Forests . . . . . . . . . . . . . . . . . 592

15.3.1 Out of Bag Samples . . . . . . . . . . . . . . . . 592

15.3.2 Variable Importance . . . . . . . . . . . . . . . . 593

15.3.3 Proximity Plots . . . . . . . . . . . . . . . . . . 595

15.3.4 Random Forests and Overﬁtting . . . . . . . . . 596

15.4 Analysis of Random Forests . . . . . . . . . . . . . . . . . 597

15.4.1 Variance and the De-Correlation Eﬀect . . . . . 597

15.4.2 Bias . . . . . . . . . . . . . . . . . . . . . . . . . 600

15.4.3 Adaptive Nearest Neighbors . . . . . . . . . . . 601

Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 602

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603

16 Ensemble Learning 605

16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 605

16.2 Boosting and Regularization Paths . . . . . . . . . . . . . 607

16.2.1 Penalized Regression . . . . . . . . . . . . . . . 607

16.2.2 The “Bet on Sparsity” Principle . . . . . . . . . 610

16.2.3 Regularization Paths, Over-ﬁtting and Margins . 613

16.3 Learning Ensembles . . . . . . . . . . . . . . . . . . . . . 616

16.3.1 Learning a Good Ensemble . . . . . . . . . . . . 617

16.3.2 Rule Ensembles . . . . . . . . . . . . . . . . . . 622

Bibliographic Notes . . . . . . . . . . . . . . . . . . . . . . . . . 623

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624

17 Undirected Graphical Models 625

17.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 625

17.2 Markov Graphs and Their Properties . . . . . . . . . . . 627

17.3 Undirected Graphical Models for Continuous Variables . 630

17.3.1 Estimation of the Parameters

when the Graph Structure is Known . . . . . . . 631

17.3.2 Estimation of the Graph Structure . . . . . . . . 635

17.4 Undirected Graphical Models for Discrete Variables . . . 638

17.4.1 Estimation of the Parameters

when the Graph Structure is Known . . . . . . . 639

17.4.2 Hidden Nodes . . . . . . . . . . . . . . . . . . . 641

17.4.3 Estimation of the Graph Structure . . . . . . . . 642

17.4.4 Restricted Boltzmann Machines . . . . . . . . . 643

Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645

18 High-Dimensional Problems: p ≫ N 649

18.1 When p is Much Bigger than N . . . . . . . . . . . . . . 649

2 1. Introduction

TABLE 1.1. Average percentage of words or characters in an email message

equal to the indicated word or character. We have chosen the words and characters

showing the largest diﬀerence between spam and email.

george you your hp free hpl ! our re edu remove

spam 0.00 2.26 1.38 0.02 0.52 0.01 0.51 0.51 0.13 0.01 0.28

1.27 1.27 0.44 0.90 0.07 0.43 0.11 0.18 0.42 0.29 0.01

measurements for a set of objects (such as people). Using this data we build

a prediction model, or learner, which will enable us to predict the outcome

for new unseen objects. A good learner is one that accurately predicts such

an outcome.

The examples above describe what is called the supervised learning prob-

lem. It is called “supervised” because of the presence of the outcome vari-

able to guide the learning process. In the unsupervised learning problem,

we observe only the features and have no measurements of the outcome.

Our task is rather to describe how the data are organized or clustered. We

devote most of this book to supervised learning; the unsupervised problem

is less developed in the literature, and is the focus of Chapter 14.

Here are some examples of real learning problems that are discussed in

this book.

Example 1: Email Spam

The data for this example consists of information from 4601 email mes-

sages, in a study to try to predict whether the email was junk email, or

“spam.” The objective was to design an automatic spam detector that

could ﬁlter out spam before clogging the users’ mailboxes. For all 4601

email messages, the true outcome (email type) email or spam is available,

along with the relative frequencies of 57 of the most commonly occurring

words and punctuation marks in the email message. This is a supervised

learning problem, with the outcome the class variable email/spam. It is also

called a classiﬁcation problem.

Table 1.1 lists the words and characters showing the largest average

diﬀerence between spam and email.

Our learning method has to decide which features to use and how: for

example, we might use a rule such as

if (%george < 0.6) & (%you > 1.5) then spam

else email.

Another form of a rule might be:

if (0.2 · %you − 0.3 · %george) > 0 then spam

else email.

剩余762页未读，继续阅读

Gavin_zju

粉丝: 0
资源: 1

统计学习基础：第二版精华概述

the elements of statistical learning

elements of statistical learning

The Elements of Statistical Learning

精选毕设项目-微笑话.zip

在线教育系统-springboot毕业项目，适合计算机毕-设、实训项目、大作业学习.zip

基于智能推荐的卫生健康系统-springboot毕业项目，适合计算机毕-设、实训项目、大作业学习.zip

精选毕设项目-课程预约.zip

同步机(VSG)三相并网仿真模型 有功功率从20k突变到10k再恢复至20k 系统始终稳定运行 该仿真主要用于基础原理的学习

南京理工大学毕业论文overleaf LaTex模板，微调版

智能物流管理系统-springboot毕业项目，适合计算机毕-设、实训项目、大作业学习.rar

最新资源

同步机(VSG)三相并网仿真模型有功功率从20k突变到10k再恢复至20k 系统始终稳定运行该仿真主要用于基础原理的学习