SAS Enterprise Miner 5：优化验证评估的简易模型

需积分: 12 45 浏览量更新于2024-07-17 1 收藏 20.08MB PDF 举报

"《应用SAS企业挖掘器5进行分析》课程笔记由Jim Georges开发，课程内容深入探讨了如何利用SAS Enterprise Miner 5这一强大的数据分析工具进行实际的业务应用。该课程强调在众多模型中，选择具有最高验证评估指标的最简单模型作为最佳实践。验证评估的关键因素包括目标测量尺度和预测类型。在课程中，参与者将学习如何根据目标变量的数据特性（如连续、分类或顺序数据）来选择合适的评估指标，如准确率、召回率、AUC-ROC曲线等。对于预测任务，如回归或分类问题，理解预测的置信区间和预测概率同样重要。此外，课程还将介绍模型选择的方法，如交叉验证和网格搜索，以确保模型的泛化能力。 Tom Bohannon、Mike Hardin、Dan Kelly、Bob Lucas和Sue Walsh也对课程内容做出了贡献，共同分享了他们在实际项目中的经验与案例。课程的制作和编辑工作由SAS学院的课程发展和支持部门负责。值得注意的是，SAS和所有提及的品牌名称均为SAS Institute Inc.在美国和其他国家注册的商标。版权信息显示，本书《应用SAS企业挖掘器5》版权归SAS Institute Inc.所有，未经许可，任何形式的复制、存储或传输都是违法的。本课程代码为AAEM_004，准备日期为2007年6月13日，出版编号为ISBN 978-1-59994-515-6。通过学习这门课程，学员将掌握如何有效地使用SAS Enterprise Miner 5进行数据挖掘、建模和解读结果，从而提升企业的决策支持能力和数据驱动策略的制定。"

1-8 Chapter 1 Introduction

SAS Enterprise Miner – Interface Tour

SEMMA tools palette

The SAS Enterprise Miner tools available to your analysis are contained in the tools palette. The tools

palette is arranged according to the SAS process for data mining, SEMMA.

SEMMA stands for

ample – You sample the data by creating one or more data tables. The samples should be large enough to

contain the significant information, yet small enough to process.

xplore – You explore the data by searching for anticipated relationships, unanticipated trends, and

anomalies in order to gain understanding and ideas.

odify – You modify the data by creating, selecting, and transforming the variables to focus the model

selection process.

odel – You model the data by using the analytical tools to search for a combination of the data that

reliably predicts a desired outcome.

ssess – You assess competing predictive models (build charts to evaluate the usefulness and reliability

of the findings from the data mining process).

Additional tools are available under the Utility group and, if licensed, the Credit Scoring group.

1.1 Introduction to SAS Enterprise Miner 1-9

SEMMA – Sample Tab

• Input Data

• Data Partition

• Sample

• Time Series

The Input Data tool represents the data source that you choose for your mining analysis and provides

details (metadata) about the variables in the data source that you want to use.

The Data Partition tool enables you to partition data sets into training, test, and validation data sets. The

training data set is used for preliminary model fitting. The validation data set is used to monitor and tune

the model during estimation and is also used for model assessment. The test data set is an additional

holdout data set that you can use for model assessment. This tool uses simple random sampling, stratified

random sampling, or cluster sampling to create partitioned data sets.

The Sample tool enables you to take simple random samples, n

observation samples, stratified random

samples, first-n samples, and cluster samples of data sets. For any type of sampling, you can specify

either a number of observations or a percentage of the population to select for the sample. If you are

working with rare events, the Sample tool can be configured for oversampling or stratified sampling.

Sampling is recommended for extremely large databases because it can significantly decrease model

training time. If the sample is sufficiently representative, relationships found in the sample can be

expected to generalize to the complete data set. The Sample tool writes the sampled observations to an

output data set and saves the seed values that are used to generate the random numbers for the samples so

that you may replicate the samples.

The Time Series tool converts transactional data to time series data. Transactional data is time-stamped

data that is collected over time at no particular frequency. By contrast, time series data is time-stamped

data that is summarized over time at a specific frequency. You might have many suppliers and many

customers, as well as transaction data that is associated with both. The size of each set of transactions can

be very large, which makes many traditional data mining tasks difficult. By condensing the information

into a time series, you can discover trends and seasonal variations in customer and supplier habits that

might not be visible in transactional data.

1-12 Chapter 1 Introduction

SEMMA – Modify Tab

•Drop

• Filter

• Impute

• Principal Components

• Replacement

• Transform Variables

The Drop tool is used to remove variables from scored data sets. You can remove all variables with the

role type that you specify, or you can manually specify individual variables to drop. For example, you

could remove all hidden, rejected, and residual variables from your exported data set, or you could

remove just a few variables that you identify yourself.

The Filter tool enables you to apply a filter to your training data set to exclude observations such as

outliers or other observations that you do not want to include in your data mining analysis. The tool does

not filter observations in the validation, test, or score data sets.

The Impute tool enables you to replace values for observations that have missing values. You can replace

missing values for interval variables with the mean, median, midrange, mid-minimum spacing, or

distribution-based replacement, or you can use a replacement M-estimator such as Tukey’s biweight,

Huber’s, or Andrew’s Wave. You can also estimate the replacement values for each interval input by using

a tree-based imputation method. Missing values for class variables can be replaced with the most

frequently occurring value, distribution-based replacement, tree-based imputation, or a constant.

The Principal Components tool calculates eigenvalues and eigenvectors from the uncorrected

covariance matrix, corrected covariance matrix, or the correlation matrix of input variables. Principal

components are calculated from the eigenvectors and are usually treated as the new set of input variables

for successor modeling tools. A principal components analysis is useful for data interpretation and data

dimension reduction.

The Replacement tool enables you to reassign and consolidate levels of categorical inputs. This can

improve the performance of predictive models.

The Transform Variables tool enables you to create new variables that are transformations of existing

variables in your data. Transformations are useful when you want to improve the fit of a model to the

data. For example, transformations can be used to stabilize variances, remove nonlinearity, improve

additivity, and correct nonnormality in variables. The Transform Variables tool supports various

transformation methods. The available methods depend on the type and the role of a variable.

剩余596页未读，继续阅读

weixin_44523404

粉丝: 1

SAS Enterprise Miner 5：优化验证评估的简易模型

SAS Enterprise Miner 4 数据挖掘实例教程

SAS Enterprise Miner 14.2高性能流程指南

SAS Enterprise Miner 4 数据挖掘教程

SAS Applied analytics Using SAS Enterprise Miner

Applied Analytics Using SAS Enterprise Miner

Data Mining Using SAS Enterprise Miner

Applying Data Mining Techniques Using SAS Enterprise Miner

Data Mining Using Sas Enterprise Miner 2Nd.pdf

Data Mining Using SAS Enterprise Miner.5.2.pdf

Applied_Analytics_Using_SAS_Enterprise_Miner_5.

最新资源