统计学习基础：Springer经典教程

需积分: 9 37 浏览量更新于2024-07-21 收藏 19.08MB PDF 举报

《统计学习元素》(Elements of Statistical Learning)是一本经典的国外教材，它将机器学习的基本原理从统计学的角度进行了深入剖析。该书隶属于Springer Series in Statistics系列，由三位知名学者Trevor Hastie、Robert Tibshirani和Jerome Friedman合著，他们在数据分析、推断与预测领域具有深厚的造诣。第二版的出版体现了作者们对统计学习理论的不断更新和完善。本书的内容涵盖了广泛的统计学习方法，包括但不限于回归分析、分类算法、模型选择、维度ality降维、特征选择以及模型评估等核心主题。作者们通过理论阐述和实际案例相结合的方式，帮助读者理解如何将统计思想应用于解决现实世界中的复杂问题，如预测市场趋势、医疗诊断、社交网络分析等。在版权方面，所有权利受到Springer Science+Business Media, LLC保护，未经书面许可，不得进行整本或部分翻译或复制，除非用于学术评论或研究分析。此外，任何形式的信息存储、检索、电子适应、计算机软件，以及相似或类似的方法，未经许可也不得使用。书中提及的商标、服务标志等商业标识的使用，并不意味着作者们对这些品牌的所有权有任何评价，只是作为学术讨论的一部分。印刷质量上，《统计学习元素》采用了酸性纸张，确保了长时间阅读的舒适度和书籍的耐久性。该书的网址可供读者查询更多该系列的其他出版物，进一步拓展学习资源。《统计学习元素》不仅是一本理论指导书籍，更是一本实践导向的工具书，对于希望深入理解机器学习统计基础的读者来说，它是一部不可或缺的参考文献。通过阅读和实践书中的内容，读者能够提升自己的数据分析技能，掌握现代统计学习技术，为职业生涯或科研工作提供强有力的支持。

Contents xvii

8.2 TheBootstrapandMaximumLikelihoodMethods.... 261

8.2.1 A Smoothing Example .............. 261

8.2.2 Maximum Likelihood Inference . . . ....... 265

8.2.3 Bootstrap versus Maximum Likelihood ..... 267

8.3 BayesianMethods ...................... 267

8.4 Relationship Between the Bootstrap

and Bayesian Inference ................... 271

8.5 The EM Algorithm ..................... 272

8.5.1 Two-Component Mixture Model . . ....... 272

8.5.2 The EM Algorithm in General . . . ....... 276

8.5.3 EM as a Maximization–Maximization Procedure 277

8.6 MCMCforSamplingfromthePosterior.......... 279

8.7 Bagging ............................ 282

8.7.1 Example: Trees with Simulated Data ...... 283

8.8 Model Averaging and Stacking ............... 288

8.9 StochasticSearch:Bumping................. 290

Bibliographic Notes ......................... 292

Exercises............................... 293

9 Additive Models, Trees, and Related Methods 295

9.1 Generalized Additive Models ................ 295

9.1.1 Fitting Additive Models .............. 297

9.1.2 Example: Additive Logistic Regression ..... 299

9.1.3 Summary ...................... 304

9.2 Tree-Based Methods ..................... 305

9.2.1 Background .................... 305

9.2.2 Regression Trees .................. 307

9.2.3 Classiﬁcation Trees ................ 308

9.2.4 Other Issues .................... 310

9.2.5 Spam Example (Continued) ........... 313

9.3 PRIM:BumpHunting.................... 317

9.3.1 Spam Example (Continued) ........... 320

9.4 MARS: Multivariate Adaptive Regression Splines ..... 321

9.4.1 Spam Example (Continued) ........... 326

9.4.2 Example (Simulated Data) ............ 327

9.4.3 Other Issues .................... 328

9.5 HierarchicalMixturesofExperts.............. 329

9.6 MissingData......................... 332

9.7 Computational Considerations ............... 334

Bibliographic Notes ......................... 334

Exercises............................... 335

10 Boosting and Additive Trees 337

10.1 Boosting Methods ...................... 337

10.1.1 Outline of This Chapter .............. 340

xviii Contents

10.2 Boosting Fits an Additive Model .............. 341

10.3 Forward Stagewise Additive Modeling ........... 342

10.4 Exponential Loss and AdaBoost .............. 343

10.5 Why Exponential Loss? ................... 345

10.6 Loss Functions and Robustness ............... 346

10.7 “Oﬀ-the-Shelf” Procedures for Data Mining . ....... 350

10.8 Example: Spam Data .................... 352

10.9 Boosting Trees ........................ 353

10.10 Numerical Optimization via Gradient Boosting ...... 358

10.10.1 Steepest Descent .................. 358

10.10.2 Gradient Boosting ................. 359

10.10.3 Implementations of Gradient Boosting ...... 360

10.11 Right-Sized Trees for Boosting ............... 361

10.12 Regularization ........................ 364

10.12.1 Shrinkage ...................... 364

10.12.2 Subsampling .................... 365

10.13 Interpretation ........................ 367

10.13.1 Relative Importance of Predictor Variables . . . 367

10.13.2 Partial Dependence Plots ............. 369

10.14 Illustrations .......................... 371

10.14.1 California Housing ................. 371

10.14.2 New Zealand Fish ................. 375

10.14.3 Demographics Data ................ 379

Bibliographic Notes ......................... 380

Exercises............................... 384

11 Neural Networks 389

11.1 Introduction ......................... 389

11.2 Projection Pursuit Regression ............... 389

11.3 Neural Networks ....................... 392

11.4 Fitting Neural Networks ................... 395

11.5 Some Issues in Training Neural Networks . . ....... 397

11.5.1 Starting Values ................... 397

11.5.2 Overﬁtting ..................... 398

11.5.3 Scaling of the Inputs ............... 398

11.5.4 Number of Hidden Units and Layers ....... 400

11.5.5 Multiple Minima .................. 400

11.6 Example: Simulated Data .................. 401

11.7 Example: ZIP Code Data .................. 404

11.8 Discussion .......................... 408

11.9 Bayesian Neural Nets and the NIPS 2003 Challenge . . . 409

11.9.1 Bayes, Boosting and Bagging ........... 410

11.9.2 Performance Comparisons ............ 412

11.10 Computational Considerations ............... 414

Bibliographic Notes ......................... 415

Contents xix

Exercises............................... 415

12 Support Vector Machines and

Flexible Discriminants 417

12.1 Introduction ......................... 417

12.2 The Support Vector Classiﬁer ................ 417

12.2.1 Computing the Support Vector Classiﬁer .... 420

12.2.2 Mixture Example (Continued) . . . ....... 421

12.3 Support Vector Machines and Kernels ........... 423

12.3.1 Computing the SVM for Classiﬁcation ...... 423

12.3.2 The SVM as a Penalization Method ....... 426

12.3.3 Function Estimation and Reproducing Kernels . 428

12.3.4 SVMs and the Curse of Dimensionality ..... 431

12.3.5 A Path Algorithm for the SVM Classiﬁer .... 432

12.3.6 Support Vector Machines for Regression ..... 434

12.3.7 Regression and Kernels .............. 436

12.3.8 Discussion ..................... 438

12.4 Generalizing Linear Discriminant Analysis . ....... 438

12.5 Flexible Discriminant Analysis ............... 440

12.5.1 Computing the FDA Estimates . . . ....... 444

12.6 Penalized Discriminant Analysis .............. 446

12.7 Mixture Discriminant Analysis ............... 449

12.7.1 Example: Waveform Data ............. 451

Bibliographic Notes ......................... 455

Exercises............................... 455

13 Prototype Methods and Nearest-Neighbors 459

13.1 Introduction ......................... 459

13.2 Prototype Methods ..................... 459

13.2.1 K-meansClustering................ 460

13.2.2 Learning Vector Quantization . . . ....... 462

13.2.3 Gaussian Mixtures ................. 463

13.3 k-Nearest-NeighborClassiﬁers ............... 463

13.3.1 Example: A Comparative Study . . ....... 468

13.3.2 Example: k-Nearest-Neighbors

and Image Scene Classiﬁcation . . . ....... 470

13.3.3 Invariant Metrics and Tangent Distance ..... 471

13.4 Adaptive Nearest-Neighbor Methods ............ 475

13.4.1 Example ...................... 478

13.4.2 Global Dimension Reduction

forNearest-Neighbors............... 479

13.5 Computational Considerations ............... 480

Bibliographic Notes ......................... 481

Exercises............................... 481

xx Contents

14 Unsupervised Learning 485

14.1 Introduction ......................... 485

14.2 Association Rules ...................... 487

14.2.1 Market Basket Analysis .............. 488

14.2.2 The Apriori Algorithm .............. 489

14.2.3 Example: Market Basket Analysis . ....... 492

14.2.4 Unsupervised as Supervised Learning ...... 495

14.2.5 Generalized Association Rules . . . ....... 497

14.2.6 Choice of Supervised Learning Method ..... 499

14.2.7 Example: Market Basket Analysis (Continued) . 499

14.3 Cluster Analysis ....................... 501

14.3.1 Proximity Matrices ................ 503

14.3.2 Dissimilarities Based on Attributes ....... 503

14.3.3 Object Dissimilarity ................ 505

14.3.4 Clustering Algorithms ............... 507

14.3.5 Combinatorial Algorithms ............ 507

14.3.6 K-means ...................... 509

14.3.7 Gaussian Mixtures as Soft K-means Clustering . 510

14.3.8 Example: Human Tumor Microarray Data . . . 512

14.3.9 Vector Quantization ................ 514

14.3.10 K-medoids ..................... 515

14.3.11 Practical Issues .................. 518

14.3.12 Hierarchical Clustering .............. 520

14.4 Self-Organizing Maps .................... 528

14.5 Principal Components, Curves and Surfaces . ....... 534

14.5.1 Principal Components ............... 534

14.5.2 Principal Curves and Surfaces . . . ....... 541

14.5.3 Spectral Clustering ................ 544

14.5.4 Kernel Principal Components ........... 547

14.5.5 Sparse Principal Components ........... 550

14.6 Non-negative Matrix Factorization ............. 553

14.6.1 Archetypal Analysis ................ 554

14.7 Independent Component Analysis

and Exploratory Projection Pursuit ............ 557

14.7.1 Latent Variables and Factor Analysis ...... 558

14.7.2 Independent Component Analysis . ....... 560

14.7.3 Exploratory Projection Pursuit . . . ....... 565

14.7.4 A Direct Approach to ICA ............ 565

14.8 Multidimensional Scaling .................. 570

14.9 Nonlinear Dimension Reduction

and Local Multidimensional Scaling ............ 572

14.10 The Google PageRank Algorithm ............. 576

Bibliographic Notes ......................... 578

Exercises............................... 579

剩余757页未读，继续阅读

zzy_foreverautumn

粉丝: 0
资源: 1

统计学习基础：Springer经典教程

The Elements of Statistical Learning (2nd edition).pdf

Elements of Statistical Learning高清pdf

The Elements of Statistical Learning.pdf

the elements of statistical learning 第四章答案

the elements of statistical learning

The Elements of Statistical Learning 引用

the elements of statistical learning资源

the elements of statistical learning, volume 1. springer series in statistic

监督学习算法与非监督学习算法的文献。

欧式距离学习参考文献

最新资源