高维数据分析：第四版新增变量选择章节

5星 · 超过95%的资源需积分: 27 118 浏览量更新于2024-07-21 1 收藏 11.83MB PDF 举报

"《应用多元统计分析》第四版是一本专为非数学家和实践者设计的书籍，重点介绍高维数据分析中的工具和概念。本书涵盖了描述性技术、多变量随机变量、多元技术等多个部分，新增了一章关于变量选择（Lasso、SCAD和Elastic Net）。书中通过实际练习探讨了不同领域的应用，如金融学、医学和定量营销，涉及大数据分析的关键领域。所有章节均提供R和MATLAB代码，并在附录中列出了符号和记号。" 本书是《应用多元统计分析》的第四版，作者Wolfgang Karl Härdle和Léopold Simar，旨在使非数学背景的读者也能理解和应用多元统计分析。全书分为四个部分： 1. **描述性技术**：第一章比较批次，探讨如何在高维数据中进行比较和分析。 2. **多变量随机变量**：第二章简述矩阵代数，第三章进一步深入高维，第四章介绍多变量分布，第五章讲解多元正态分布理论，第六章讨论估计理论，第七章涉及假设检验，为后续分析打下基础。 3. **多元技术**：第八章涵盖回归模型，第九章新增的变量选择章节详细介绍Lasso、SCAD和Elastic Net等方法，用于特征选择。第十章到第二十章涉及数据矩阵分解、主成分分析、因子分析、聚类分析、判别分析、对应分析、典型相关分析、多维尺度分析以及联合测量分析等多元统计方法。这些章节通过具体实例展示了在金融学、医学和营销研究中的应用。 4. **附录**：包含了符号和记号的参考，便于查阅。书中的每个章节都配有实用的练习，提供R和MATLAB代码，帮助读者实现和理解统计分析的过程。同时，解决方案可以在Härdle和Hlavka的《多元统计：练习与解答》中找到。本书特别适合于希望深入了解和应用多元统计分析的学者、研究人员和从业人员，特别是在大数据背景下处理高维数据的场景。通过学习，读者将能够掌握多种统计方法，进行有效的数据分析和决策。

Chapter 1

Comparison of Batches

Multivariate statistical analysis is concerned with analysing and understanding data

in high dimensions. We suppose that we are given a set fx

iD1

of n observations

of a variable vector X in R

. That is, we suppose that each observation x

has p

dimensions:

D .x

;:::;x

and that it is an observed value of a variable vector X 2 R

. Therefore, X is

composed of p random variables:

X D .X

;:::;X

where X

,forj D 1;:::;p, is a one-dimensional random variable. How do

we begin to analyse this kind of data? Before we investigate questions on what

inferences we can reach from the data, we should think about how to look at the data.

This involves descriptive techniques. Questions that we could answer by descriptive

techniques are:

• Are there components of X that are more spread out than others?

• Are there some elements of X that indicate sub-groups of the data?

• Are there outliers in the components of X?

• How “normal” is the distribution of the data?

• Are there “low-dimensional” linear combinations of X that show “non-normal”

behaviour?

One difﬁculty of descriptive methods for high-dimensional data is the human

perceptional system. Point clouds in two dimensions are easy to understand and to

interpret. With modern interactive computing techniques we have the possibility

to see real time 3D rotations and thus to perceive also three-dimensional data.

A “sliding technique” as described in Härdle and Scott (1992) may give insight

W.K. Härdle, L. Simar, Applied Multivariate Statistical Analysis,

DOI 10.1007/978-3-662-45171-7_1

4 1 Comparison of Batches

into four-dimensional structures by presenting dynamic 3D density contours as the

fourth variable is changed over its range.

A qualitative jump in presentation difﬁculties occurs for dimensions greater

than or equal to 5, unless the high-dimensional structure can be mapped into

lower-dimensional components (Klinke & Polzehl, 1995). Features like clustered

sub-groups or outliers, however, can be detected using a purely graphical analysis.

In this chapter, we investigate the basic descriptive and graphical techniques

allowing simple exploratory data analysis. We begin the exploration of a data

set using boxplots. A boxplot is a simple univariate device that detects outliers

component by component and that can compare distributions of the data among

different groups. Next, several multivariate techniques are introduced (Flury faces,

Andrews’ curves and parallel coordinates plots (PCPs)) which provide graphical

displays addressing the questions formulated above. The advantages and the

disadvantages of each of these techniques are stressed.

Two basic techniques for estimating densities are also presented: histograms and

kernel densities. A density estimate gives a quick insight into the shape of the

distribution of the data. We show that kernel density estimates (KDEs) overcome

some of the drawbacks of the histograms.

Finally, scatterplots are shown to be very useful for plotting bivariate or

trivariate variables against each other: they help to understand the nature of the

relationship among variables in a data set and allow for the detection of groups or

clusters of points. Draftman plots or matrix plots are the visualisation of several

bivariate scatterplots on the same display. They help detect structures in conditional

dependencies by brushing across the plots. Outliers and observations that need

special attention may be discovered with Andrews curves and PCPs. This chapter

ends with an explanatory analysis of the Boston Housing data.

1.1 Boxplots

Example 1.1 The Swiss bank data (see Chap. 22, Sect. 22.2) consists of 200

measurements on Swiss bank notes. The ﬁrst half of these measurements are from

genuine bank notes, the other half are from counterfeit bank notes.

The authorities measured, as indicated in Fig. 1.1,

D length of the bill

D height of the bill (left)

D height of the bill (right)

D distance of the inner frame to the lower border

D distance of the inner frame to the upper border

D length of the diagonal of the central picture.

剩余580页未读，继续阅读

ramissue

粉丝: 354
资源: 1487

高维数据分析：第四版新增变量选择章节

Applied Multivariate Statistical Analysis(6th Ed)

Applied Multivariate Statistical Analysis 6th Edition

Applied Multivariate Statistical Analysis

torch.distributions.multivariate_normal

python np.random.multivariate_normal

np.random.multivariate_normal()

npr.multivariate_normal

torch.distributions.multivariate_normal.log_prob

python中这个是什么意思：np.random.multivariate_normal(np.zeros(2), R1)

M=npr.multivariate_normal(mean=np.zeros((2,)),cov=corr_mat,size=50*50),r如何吧M拆成竖着的两列

最新资源