机器学习特征工程原理与数据科学家实用技术

需积分: 9 199 浏览量更新于2024-07-18 收藏 11.49MB PDF 举报

《机器学习特征工程：数据科学家的原则与技术》是由Alice Zheng和Amanda Casari合著的一本专著，于2018年出版。这本书是针对机器学习领域的关键实践者——数据科学家，深入探讨了特征工程在机器学习中的重要性和实施技巧。特征工程是机器学习流程中的基础步骤，它涉及到数据预处理、特征选择、特征提取和变换等，这些步骤直接影响模型的性能和预测能力。本书涵盖了以下几个核心知识点： 1. **理解特征工程概念**：特征工程不仅仅是数据清洗，更是创造新特征的过程，旨在将原始数据转化为对模型有用的输入形式，以便机器学习算法能够更好地理解和建模。 2. **数据预处理**：包括处理缺失值、异常值、重复值，以及数据类型转换、标准化或归一化等，确保数据质量并符合模型的输入要求。 3. **特征选择**：选择最具预测力的特征，可以减少模型复杂度和过拟合风险，同时提高计算效率。这可能涉及统计方法、相关性分析、特征重要性评估等。 4. **特征提取**：从原始数据中挖掘出更高级别的特征，如基于文本的词袋模型、TF-IDF表示，或者图像中的颜色直方图和纹理特征等。 5. **特征变换**：通过降维（如主成分分析PCA）、编码（如独热编码One-Hot Encoding）等方式，简化特征空间，有助于提升模型的解释性和性能。 6. **特征构建**：结合领域知识，创建新的合成特征，比如时间序列数据中的滑动窗口特征，或社交网络中的用户行为组合特征。 7. **评估与迭代**：特征工程是一个迭代过程，通过交叉验证和模型评估不断优化特征，直到达到最佳性能。 8. **实践经验分享**：书中提供了丰富的案例研究和实战指导，帮助读者理解和应用这些原则和技术。《机器学习特征工程：数据科学家的原则与技术》是一本实用的指南，适合那些希望深入理解特征工程在机器学习中作用的专业人士，无论他们是初学者还是经验丰富的数据科学家，都能从中获益匪浅。同时，作者还强调了版权信息，提醒读者尊重知识产权，并提供购买和获取更多资源的途径。

Model Evaluation

Features and models sit between raw data and the desired insights (see Figure 1-

2). In a machine learning workflow, we pick not only the model, but also the

features. This is a double-jointed lever, and the choice of one affects the other.

Good features make the subsequent modeling step easy and the resulting model

more capable of completing the desired task. Bad features may require a much

more complicated model to achieve the same level of performance. In the rest of

this book, we will cover different kinds of features and discuss their pros and

cons for different types of data and models. Without further ado, let’s get started!

Figure 1-2. The place of feature engineering in the machine learning workflow

Chapter 2. Fancy Tricks with

Simple Numbers

Before diving into complex data types such as text and image, let’s start with the

simplest: numeric data. This may come from a variety of sources: geolocation of

a place or a person, the price of a purchase, measurements from a sensor, traffic

counts, etc. Numeric data is already in a format that’s easily ingestible by

mathematical models. This doesn’t mean that feature engineering is no longer

necessary, though. Good features should not only represent salient aspects of the

data, but also conform to the assumptions of the model. Hence, transformations

are often necessary. Numeric feature engineering techniques are fundamental.

They can be applied whenever raw data is converted into numeric features.

The first sanity check for numeric data is whether the magnitude matters. Do we

just need to know whether it’s positive or negative? Or perhaps we only need to

know the magnitude at a very coarse granularity? This sanity check is

particularly important for automatically accrued numbers such as counts—the

number of daily visits to a website, the number of reviews garnered by a

restaurant, etc.

Next, consider the scale of the features. What are the largest and the smallest

values? Do they span several orders of magnitude? Models that are smooth

functions of input features are sensitive to the scale of the input. For example, 3x

+ 1 is a simple linear function of the input x, and the scale of its output depends

directly on the scale of the input. Other examples include k-means clustering,

nearest neighbors methods, radial basis function (RBF) kernels, and anything

that uses the Euclidean distance. For these models and modeling components, it

is often a good idea to normalize the features so that the output stays on an

expected scale.

Logical functions, on the other hand, are not sensitive to input feature scale.

Their output is binary no matter what the inputs are. For instance, the logical

AND takes any two variables and outputs 1 if and only if both of the inputs are

true. Another example of a logical function is the step function (e.g., is input x

greater than 5?). Decision tree models consist of step functions of input features.

Hence, models based on space-partitioning trees (decision trees, gradient boosted

machines, random forests) are not sensitive to scale. The only exception is if the

scale of the input grows over time, which is the case if the feature is an

accumulated count of some sort—eventually it will grow outside of the range

that the tree was trained on. If this might be the case, then it might be necessary

to rescale the inputs periodically. Another solution is the bin-counting method

discussed in Chapter 5.

It’s also important to consider the distribution of numeric features. Distribution

summarizes the probability of taking on a particular value. The distribution of

input features matters to some models more than others. For instance, the

training process of a linear regression model assumes that prediction errors are

distributed like a Gaussian. This is usually fine, except when the prediction

target spreads out over several orders of magnitude. In this case, the Gaussian

error assumption likely no longer holds. One way to deal with this is to

transform the output target in order to tame the magnitude of the growth.

(Strictly speaking this would be target engineering, not feature engineering.) Log

transforms, which are a type of power transform, take the distribution of the

variable closer to Gaussian.

In addition to features tailoring to the assumptions of the model or training

process, multiple features can be composed together into more complex features.

The hope is that complex features can more succinctly capture important

information in raw data. Making the input features more “eloquent” allows the

model itself to be simpler, easier to train and evaluate, and to make better

predictions. Taken to an extreme, complex features may themselves be the

output of statistical models. This is a concept known as model stacking, which

we discuss in much more detail in Chapters 7 and 8. In this chapter, we give the

simplest example of complex features: interaction features.

Interaction features are simple to formulate, but the combination of features

results in many more being input into the model. In order to reduce the

computational expense, it is usually necessary to prune the input features using

automatic feature selection.

We’ll begin with the basic concepts of scalars, vectors, and spaces, followed by

discussions of scale, distribution, interaction features, and feature selection.

剩余261页未读，继续阅读

PyQter

粉丝: 14
资源: 39

机器学习特征工程原理与数据科学家实用技术

Feature Engineering for Machine Learning and Data Analytics 无水印原版pdf

Feature_Engineering_for_Machine_Learning

Feature Engineering for Machine Learning_Principl

Feature Engineering for Machine Learning - Alice Zheng

Feature Engineering for Machine Learning_Principles and Techniques

Machine Learning and Feature Engineering for Computer Network Security

tell me about feature engineering in machine learning

这是我的毕业设计，是一个前端和后端分离的电子商务系统。使用Springboot+Myb

嵌入式-嵌入式产品级项目之洗衣机程序设计-STM32-优秀毕业设计.zip

卷积加速-基于TVM实现的用于CUDA+AMDGPU的winograd卷积加速-附项目源码+加速对比测试-优质HPC项目实现

最新资源