数据挖掘概念与技术：知识发现和数据库应用

需积分: 10 77 浏览量更新于2024-07-24 收藏 799KB PDF 举报

数据挖掘概念数据挖掘（英语：Data mining）是数据库知识发现（英语：Knowledge Discovery in Databases 简称：KDD 中的一个步骤。它是指从大型数据库中自动发现有价值的知识、模式、趋势和关系的过程。数据挖掘技术可以帮助组织或个人从大量数据中提取有价值的信息，提高决策效率和竞争力。数据挖掘的概念可以追溯到20世纪60年代，但直到20世纪90年代才开始普及。随着数据存储技术和计算机处理能力的改善，数据挖掘技术也得到了快速发展。在今天，数据挖掘已经广泛应用于商业、金融、医疗、教育等多个领域。数据挖掘的过程通常包括以下几个步骤： 1. 问题定义：确定挖掘的目标和范围。 2. 数据选择：选择相关的数据集。 3. 数据清洁：清洁和处理数据。 4. 数据变换：将数据转换为适合挖掘的格式。 5. 模式评估：评估挖掘结果的准确性和可靠性。 6. 模式表示：将挖掘结果表示出来。 7. 使用结果：将挖掘结果应用于实际中。数据挖掘技术可以分为两大类：描述性数据挖掘和预测性数据挖掘。描述性数据挖掘是指对数据进行总结、分析和描述，以了解数据的特点和趋势。预测性数据挖掘是指使用数据挖掘技术来预测未来事件或结果。数据挖掘技术有很多应用场景，例如： 1. 客户关系管理：数据挖掘可以帮助企业更好地理解客户的需求和行为，提高客户满意度和忠诚度。 2. 销售预测：数据挖掘可以帮助企业预测销售额和趋势，提高销售策略和执行。 3. 风险管理：数据挖掘可以帮助企业identify potential risks and opportunities，提高风险管理和决策能力。 4. 医疗保健：数据挖掘可以帮助医疗机构更好地理解疾病的病因和治疗方法，提高医疗服务质量。数据挖掘是指从大型数据库中自动发现有价值的知识、模式、趋势和关系的过程。它可以帮助组织或个人从大量数据中提取有价值的信息，提高决策效率和竞争力。

16 CHAPTER 2. DATA PREPROCESSING

• Median: To accurately calculate the median, we have to look at every value in the dataset. When we

add a new value or values, we have to sort the new set and then ﬁnd the median based on that new

sorted set. This is much harder and thus makes the incremental addition of new values diﬃcult.

2.6. In real-world data, tuples with missing values for some attributes are a common occurrence. Describe

various methods for handling this problem.

Answer:

The various methods for handling the problem of missing values in data tuples include:

(a) Ignoring the tuple: This is usually done when the class label is missing (assuming the mining task

involves classiﬁcation or description). This method is not very eﬀective unless the tuple contains several

attributes with missing values. It is especially poor when the percentage of missing values per attribute

varies considerably.

(b) Manually ﬁlling in the missing value: In general, this approach is time-consuming and may not

be a reasonable task for large data sets with many missing values, especially when the value to be ﬁlled

in is not easily determined.

the same constant, such as a label like “Unknown,” or −∞. If missing values are replaced by, say,

“Unknown,” then the mining program may mistakenly think that they form an interesting concept,

since they all have a value in common — that of “Unknown.” Hence, although this method is simple,

it is not recommended.

(d) Using the attribute mean for quantitative (numeric) values or attribute mode for categor-

ical (nominal) values: For example, suppose that the average income of AllElectronics customers is

$28,000. Use this value to replace any missing values for income.

(e) Using the attribute mean for quantitative (numeric) values or attribute mode for categor-

ical (nominal) values, for all samples belonging to the same class as the given tuple: For

example, if classifying customers according to credit risk, replace the missing value with the average

income value for customers in the same credit risk category as that of the given tuple.

(f) Using the most probable value to ﬁll in the missing value: This may be determined with

regression, inference-based tools using Bayesian formalism, or decision tree induction. For example,

using the other customer attributes in the data set, we can construct a decision tree to predict the

missing values for income.

2.7. Using the data for age given in Exercise 2.4, answer the following.

(a) Use smoothing by bin means to smooth the above data, using a bin depth of 3. Illustrate your steps.

Comment on the eﬀect of this technique for the given data.

(b) How might you determine outliers in the data?

Answer:

(a) Use smoothing by bin means to smooth the above data, using a bin depth of 3. Illustrate your steps.

Comment on the eﬀect of this technique for the given data.

The following steps are required to smooth the above data using smoothing by bin means with a bin

depth of 3.

• Step 1: Sort the data. (This step is not required here as the data are already sorted.)

• Step 2: Partition the data into equal-frequency bins of size 3.

2.8. EXERCISES 17

Bin 1: 13, 15, 16 Bin 2: 16, 19, 20 Bin 3: 20, 21, 22

Bin 4: 22, 25, 25 Bin 5: 25, 25, 30 Bin 6: 33, 33, 35

Bin 7: 35, 35, 35 Bin 8: 36, 40, 45 Bin 9: 46, 52, 70

• Step 3: Calculate the arithmetic mean of each bin.

• Step 4: Replace each of the values in each bin by the arithmetic mean calculated for the bin.

Bin 1: 142/3, 142/3, 142/3 Bin 2: 181/3, 181/3, 181/3 Bin 3: 21, 21, 21

Bin 4: 24, 24, 24 Bin 5: 262/3, 262/3, 262/3 Bin 6: 332/3, 332/3, 332/3

Bin 7: 35, 35, 35 Bin 8: 401/3, 401/3, 401/3 Bin 9: 56, 56, 56

(b) How might you determine outliers in the data?

Outliers in the data may be detected by clustering, where similar values are organized into groups, or

“clusters”. Values that fall outside of the set of clusters may be considered outliers. Alternatively, a

combination of computer and human inspection can be used where a predetermined data distribution

is implemented to allow the computer to identify possible outliers. These possible outliers can then be

veriﬁed by human inspection with much less eﬀort than would be required to verify the entire initial

data set.

Other methods that can b e used for data smoothing include alternate forms of binning such as smooth-

ing by bin medians or smoothing by bin boundaries. Alternatively, equal-width bins can be used to

implement any of the forms of binning, where the interval range of values in each bin is constant.

Methods other than binning include using regression techniques to smooth the data by ﬁtting it to a

function such as through linear or multiple regression. Classiﬁcation techniques can be used to imple-

ment concept hierarchies that can smooth the data by rolling-up lower level concepts to higher-level

concepts.

2.8. Discuss issues to consider during data integration.

Answer:

Data integration involves combining data from multiple sources into a coherent data store. Issues that must

be considered during such integration include:

• Schema integration: The metadata from the diﬀerent data sources must be integrated in order to

match up equivalent real-world entities. This is referred to as the entity identiﬁcation problem.

• Handling redundant data: Derived attributes may be redundant, and inconsistent attribute naming

may also lead to redundancies in the resulting data set. Duplications at the tuple level may occur and

thus need to be detected and resolved.

• Detection and resolution of data value conﬂicts: Diﬀerences in representation, scaling, or encod-

ing may cause the same real-world entity attribute values to diﬀer in the data sources being integrated.

2.9. Suppose a hospital tested the age and body fat data for 18 randomly selected adults with the following

result

age 23 23 27 27 39 41 47 49 50

%fat 9.5 26.5 7.8 17.8 31.4 25.9 27.4 27.2 31.2

age 52 54 54 56 57 58 58 60 61

%fat 34.6 42.5 28.8 33.4 30.2 34.1 32.9 41.2 35.7

(a) Calculate the mean, median and standard deviation of age and %fat.

(b) Draw the boxplots for age and %fat.

20 CHAPTER 2. DATA PREPROCESSING

(b) Use z-score normalization to transform the value 35 for age, where the standard deviation of age is

12.94 years.

(d) Comment on which method you would prefer to use for the given data, giving reasons as to why.

Answer:

(a) Use min-max normalization to transform the value 35 for age onto the range [0.0, 1.0].

For readability, let A be the attribute age. Using Equation (2.11) with min

= 13, max

= 70,

new min

= 0, new max

= 1.0, then v = 35 is transformed to v

= 0.39.

(b) Use z-score normalization to transform the value 35 for age, where the standard deviation of age is

12.94 years.

Using Equation (2.12) where A = 809/27 = 29.96 and σ

= 12.94, then v = 35 is transformed to

= 0.39.

Using Equation (2.13) where j = 2, v = 35 is transformed to v

= 0.35.

(d) Comment on which method you would prefer to use for the given data, giving reasons as to why.

Given the data, one may prefer decimal scaling for normalization because such a transformation would

maintain the data distribution and be intuitive to interpret, while still allowing mining on speciﬁc age

groups. Min-max normalization has the undesired eﬀect of not permitting any future values to fall

outside of the current minimum and maximum values without encountering an “out of bounds error”.

As such values may be present in future data, this method is less appropriate. [new Both min-max and

Z-score normalization change the original data quite a bit.] Z-score normalization transforms values

into measures that represent their distance from the mean, in terms of standard deviations. This type

of transformation may not b e as intuitive to the user in comparison with decimal scaling.

2.13. Use a ﬂow chart to summarize the following procedures for attribute subset selection:

(a) stepwise forward selection

(b) stepwise backward elimination

Answer:

(a) Stepwise forward selection

See Figure 2.3.

(b) Stepwise backward elimination

See Figure 2.4.

See Figure 2.5.

2.14. Suppose a group of 12 sales price records has been sorted as follows:

5, 10, 11, 13, 15, 35, 50, 55, 72, 92, 204, 215.

Partition them into three bins by each of the following methods.

(a) equal-frequency partitioning

(b) equal-width partitioning

剩余128页未读，继续阅读

cloudCx

粉丝: 0
资源: 1

数据挖掘概念与技术：知识发现和数据库应用

数据挖掘概念与技术第三版 范明译 中文ppt

数据挖掘概念与技术————PPT学习教案.pptx

数据挖掘基础教程，最基本的数据挖掘概念

数据挖掘概念与技术

数据挖掘经典：《数据挖掘概念与技术》韩家炜解析

python数据挖掘 概念

数据挖掘概念与技术ceb

数据挖掘概念与技术下载

数据挖掘概念综述.pdf

数据挖掘概念与技术 03

最新资源

数据挖掘概念与技术第三版范明译中文ppt

python数据挖掘概念