没有合适的资源?快使用搜索试试~ 我知道了~
首页《数据分析与数据挖掘入门指南》:理解与实践
《数据分析与数据挖掘:入门指南》是一本由Adelchi Azzalini和Bruno Scarpa合著的专业书籍,旨在为读者提供关于数据分析和数据挖掘领域的深入理解。本书适合那些希望在数据分析领域起步或对这两个主题有初步兴趣的人阅读,通过它,读者将能够获得一个全面且系统的知识框架。 数据分析是现代IT行业中至关重要的组成部分,它涵盖了收集、处理、解释和从海量数据中提取有价值信息的过程。这本书详细介绍了数据预处理、探索性数据分析(EDA)、统计推断、回归分析、聚类分析、关联规则学习以及深度学习等关键概念和技术。对于新手来说,它不仅提供了理论基础,还通过实例演示展示了如何实际应用这些方法。 数据挖掘则是在大数据时代背景下,对数据进行深入挖掘以发现隐藏模式、关联和预测的能力。作者在这本书中探讨了各种数据挖掘算法,如决策树、随机森林、支持向量机和神经网络,以及它们在商业智能、市场营销、客户行为分析等场景中的应用。此外,书中还会涉及如何评估模型性能,以及如何处理数据质量问题。 《数据分析与数据挖掘:An Introduction》不仅适合计算机科学专业的学生,也对数据科学家、分析师、业务管理人员以及其他任何希望通过掌握数据处理技术提升业务决策能力的人士有极大的帮助。它强调了理论与实践的结合,让读者能够在理论学习的同时,逐步掌握如何在实际工作中实施和优化数据驱动的决策过程。 在版权方面,本书由牛津大学出版社出版,强调了学术诚信和知识产权保护。阅读这本书不仅是获取知识的途径,也是尊重作者辛勤工作的体现。通过这本书,读者将开启通往复杂数据世界的大门,为未来在这个充满潜力的领域内取得成功打下坚实的基础。
资源详情
资源推荐
![](https://csdnimg.cn/release/download_crawler_static/10445336/bg10.jpg)
Introduction 5
data mining represents the work of processing, graphically or
numerically, large amounts or continuous streams of data, with the
aim of extracting information useful to those who possess them.
The expression “useful information” is deliberately general: in many cases, the
point of interest is not specified a priori at all and we often search for it by mining
the data. This aspect distinguishes between data mining and other searches related
to data analysis. In particular, the approach is diametrically opposed, for example,
to clinical studies, in which it is essential to specify very precisely a priori the aims
for which data are collected and analyzed.
What might constitute useful information varies considerably and depends on
the context in which we operate and on the objectives we set. This observation
is clearly also true in many other contexts, but in the area of data mining it has
additional value. We can make a distinction between two situations: (a) in one, the
interesting aspect is the global behavior of the phenomenon examined, and the aim
is the construction of its global model, taken from the available data; (b) in the other,
it is characterization of detail or the pattern structures of the data, as we are only
interested in cases outside standard behavior. In the example of telephone company
customers, we can examine phone traffic data to identify trends that allow us to
forecast customers’ behavior according to their price plans, geographical position,
and other known elements. However, we can also examine the data with the aim of
identifying behavioral anomalies in telephone usage with respect to the behavior of
the same customer in the past—perhaps to detect a fraudulent situation created by
a third party to a customer’s detriment.
Data mining is a recent discipline, lying at the intersection of various scientific
sectors, especially statistics, machine learning, and database management.
The connection with database management is implicit in that the operations
of data cleaning, the selection of portions of data, and so on, also drawn from
distributed databases, require competences and contributions from that sector.
The link with artificial intelligence reflects the intense activity in that field to
make machines “learn” how to calculate general rules originating from a series
of specific examples: this is very like the aim of extracting the laws that regulate
a phenomenon from sampled observations. This, among the methods that are
presented later, explains why some of them originate from the field of artificial
intelligence or similar ones.
In light of the foregoing, the statements of Hand et al. (2001) become clear:
Data mining is fundamentally an applied discipline … data mining
requires an understanding of both statistical and computational
issues. (p. xxviii)
The most fundamental difference between classical statistical appli-
cations and data mining is the size of the data. (p. 19)
The computational cost connected with large data sizes and dimensions obviously
has repercussions on the method of working with these data: as they increase,
![](https://csdnimg.cn/release/download_crawler_static/10445336/bg11.jpg)
6 DATA ANALYSIS AND DATA MINING
methods with high computational cost become less feasible. Clearly, in such cases,
we cannot identify an exact rule, because various factors other than those already
mentioned come into play, such as available resources for calculation and the time
needed for results. However, the effect unquestionably exists, and it prevents the
use of some tools, or at least renders them less practical, while favoring others of
lower computational cost.
It is also true that there are situations in which these aspects are of only marginal
importance, because the amount of data is not enough to influence the computing
element; this is partly thanks to the enormous increase in the power of computers.
We often see this situation with a large-scale problem, if it can be broken down into
subproblems, which make portions of the data more manageable. More traditional
methods of venerable age have not yet been put to rest. On the contrary, many of
them, which developed in a period of limited computing resources, are much less
demanding in terms of computational effort and are still valid if suitably applied.
1.1.3 SQL, OLTP, OLAP, DWH, and KDD
We have repeatedly mentioned the great availability of data, now collected in an
increasingly systematic and thorough way, as the starting point for processing.
However, the conversion of raw data to “clean” data is time-consuming and
sometimes very demanding.
We cannot presume that all the data of a complex organization can fit into a
single database on which we can simply draw and develop. In the business world,
even medium-sized companies are equipped with complex IT systems made up
of various databases designed for various aims (customers and their invoices,
employees’ careers and wages, suppliers, etc.). These databases are used by various
operators, both to insert data (e.g., from outlying sales offices) and to answer
queries about single entries, necessary for daily activities—for example, to know
whether and when customer X has paid invoice Y issued on day Z. The phrase
referring to methods of querying specific information in various databases, called
operational,isOnLine Transaction Processing (OLTP). Typically, these tools are
based on Structured Query Language (SQL), the standard tool for database queries.
For decision support, in particular analysis of data for CRM, these operational
databases are not the proper sources on which to work. They were all designed for
different goals, both in the sense that they were usually created for administrative
and accounting purposes and not for data analysis, and that those goals differ. This
means that their structures are heterogeneous and very often contain inconsistent
data, sometimes even structurally, because the definitions of the recorded variables
may be similar but are not identical. Nor is it appropriate for the strategic activities
of decision support to interfere with daily work on systems designed to work on
operational databases.
For these reasons, it is appropriate to develop focused databases and tools. We
thus construct a strategic database or Data WareHouse (DWH), in which data
from different database systems merge, are “cleaned” as much as possible, and are
organized round the postprocessing phase.
The development of a DWH is complex, and it must be carefully designed for
its future aims. From a functional point of view, the most common method of
![](https://csdnimg.cn/release/download_crawler_static/10445336/bg12.jpg)
Introduction 7
construction is progressive aggregation of various data marts—that is, of finalized
databases. For example, a data mart may contain all the relevant information
for a certain marketing division. After the DWH has been constructed, the later
aggregation must achieve a coherent, homogenous structure, and the DWH must
be periodically updated with new data from various operational databases.
After completing all these programming processes (which can then progress by
means of continual maintenance), a DWH can be used in at least two ways, which
are not mutually exclusive. The first recomposes data from the various original data
marts to create new ones: for example, if we have created a DWH by aggregating
data mart for several lines of products, we can create a new one for selling all those
products in a certain geographical area. A new data mart is therefore created for
every problem for which we want to develop quantitative analysis.
A second way of using a DWH, which flanks the first, directly generates
processing (albeit simplified) to extract certain information about the data
summary. This is called OnLine Analytical Processing (OLAP) and, as indicated
by its name, is made up of querying and processing designed in a certain way to be
a form of data analysis, although it is still raw and primarily descriptive.
For OLAP, the general support is a structure of intermediate processing,
called a hypercube. In statistical terms, this is a multiway table, in which every
dimension corresponds to a variable, and every cell at the intersection of different
levels contains a synthetic indicator, often a frequency. To give an example
of this, let us presume that the statistical units are university students. One
variable could be constructed by place of residence, another by department or
university membership, gender, and so on, and the individual cells of the cross-
table (hypercube) contain the frequencies for the various intersecting levels. This
table can be used for several forms of processing: marginalization or conditioning
with respect to one or more variables, level aggregation, and so on. They are
described in introductory statistical texts and need no mention here. Note that in
the field of computer science, the foregoing operations have different names.
As already noted, OLAP is an initial form of the extraction of information
from the data—relatively simple, at least from a conceptual point of view—
operating from a table with predefined variables and a scope of operations limited
to them. Therefore, strictly speaking, OLAP returns to data mining as defined
in section 1.1.2, but limited to a form that is conceptually a very simple way
of processing. Instead, “data mining” commonly refers to the inspection of a
strategic database and is characteristically more investigative in nature, typically
involving the identification of relations in certain significant ways among variables
or making specific and interesting patterns of the data. The distinction between
OLAP and data mining is therefore not completely clear, but essentially—as
already noted—the former involves inspecting a small number of prespecified
variables and has a limited number of operations, and the latter refers to a
more open and more clearly focused study on extracting knowledge from the
data. For the latter type of processing, much more computational than simple
management, it is not convenient to use SQL, because SQL does not provide
simple commands for intensive statistical processing. Alternatives are discussed
later.
![](https://csdnimg.cn/release/download_crawler_static/10445336/bg13.jpg)
8 DATA ANALYSIS AND DATA MINING
We can now think of a chain of phases, starting as follows:
• one or more operational databases to construct a strategic database
(DWH): this also involves an operation in which we homogenize the
definition of variables and data cleaning operations;
• we apply OLAP tools to this new database, to highlight points of interest
on variables singled out previously;
• data mining is the most specific phase of data analysis, and aims at finding
interesting elements in specific data marts extracted from the DWH.
The term Knowledge Discovery in Databases (KDD) is used to refer to this
complex chain, but this terminology is not unanimously accepted and data mining
is sometimes used as a synonym. In this work, data mining is intended in the more
restricted sense, which regards only the final phases of those described.
1.1.4 Complications
We have already touched on some aspects that differentiate data mining from other
areas of data analysis. We now elaborate this point.
In many cases, data were collected for reasons other than statistical analysis.
In particular, in the business sector, data are compiled primarily for accounting
purposes. This administrative requirement led to ways of organizing these data
becoming more complex; the realization that they could be used for other purposes,
that is, marketing analysis and CRM, came later.
Data, therefore, do not correspond to any sampling plan or experimental
design: they simply ‘exist’. The lack of canonical conditions for proper data
collection initially kept many statisticians away from the field of data mining,
whereas information technology (IT) experts were more prompt in exploiting this
challenge.
Even without these problems, we must also consider data collected in spurious
forms. This naturally entails greater difficulties and corresponding attention to
other applicative contexts.
The first extremely simple but useful observation in this sense has to do with
the validity of our conclusions. Because a company’s customer database does not
represent a random sample of the total population, the conclusions we may draw
from it cover at most already acquired customers, not prospective ones.
Another reason for the initial reluctance of statisticians to enter the field of
data mining was a second element, already mentioned in section 1.1.2—that is,
research sometimes focuses on an objective that was not declared a priori. When
we research ‘anything’, we end up finding ‘something’
...even if it is not there. To
illustrate this idea intuitively, assume that we are examining a sequence of random
numbers: ultimately, it seems that there is some regularity, at least if we examine a
sequence that is not too long. At this point, we must recall an aphorism coined by
an economist, which is very fashionable among applied statisticians: “If you torture
the data long enough, Nature will always confess” (Ronald H. Coase, 1991 Nobel
Prize for Economics).
![](https://csdnimg.cn/release/download_crawler_static/10445336/bg14.jpg)
Introduction 9
This practice of “looking for something” (when we do not know exactly what
it is) is therefore misleading, and thus the associated terms data snooping or
data dredging have negative connotations. When confronted with a considerable
amount of data, the danger of false findings decreases but is not eliminated
altogether. There are, however, techniques to counter this, as we shall see in
chapter 3.
One particularity, which seems trivial, regards the so-called leaker variables,
which are essentially surrogates of the variables of interest. For example, if the
variable of interest is the amount of money spent on telephone traffic by one
customer in one month, a leaker variable is given by the number of phone calls
made in that same month, as the first variable is recorded at the same moment
as the second variable. Conceptually, the situation is trivial, but when hundreds of
variables, often of different origin, are manipulated, this eventuality is not as remote
as it may appear. It at least signals the danger of using technology blindly, inserting
whole lists of variables without worrying about what they represent. We return to
this point in section 1.3.1.
Bibliographical notes
Hand et al. (2001) depict a broad picture of data mining, its connections with
other disciplines, and its general principles, although they do not enter into detailed
technical aspects. In particular, their chapter 12 contains a more highly developed
explanation of our section 1.1.3 about relationships between data management and
some techniques, like OLAP, closer to that context.
For descriptive statistics regarding tables of frequency and their handling, there
is a vast amount of literature, which started in the early stages of statistics and is still
developing. Some classical texts are Kendall & Stuart (1969, sections 1.30–1.34),
Bishop et al. (1975), and Agresti (2002).
For a more detailed description of the role of data mining in the corporate
context, in particular its connections with business promotion, see the first chapters
of Berry & Linoff (1997).
1.2 A
LL MODELS ARE WRONG
All models are wrong but some are useful.
—GEORGE E. P. BOX
1.2.1 What is a Model?
The term model is very fashionable in many contexts, mainly in the fields of science
and technology and also business management. Because the important attributes
of this term (which are often implicit) are so varied and often blurred, let us clarify
at once what we mean by it:
A model is a simplified representation of the phenomenon of interest,
functional for a specific objective.
www.allitebooks.com
剩余288页未读,继续阅读
![pdf](https://img-home.csdnimg.cn/images/20210720083512.png)
![pdf](https://img-home.csdnimg.cn/images/20210720083512.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://profile-avatar.csdnimg.cn/40c1dd2d38f443c4aed73e2c25f9cea7_cyydjt.jpg!1)
渔好学
- 粉丝: 80
- 资源: 1
上传资源 快速赚钱
我的内容管理 展开
我的资源 快来上传第一个资源
我的收益
登录查看自己的收益我的积分 登录查看自己的积分
我的C币 登录后查看C币余额
我的收藏
我的下载
下载帮助
![](https://csdnimg.cn/release/wenkucmsfe/public/img/voice.245cc511.png)
最新资源
- 岩石滑动与断层冲击地压:声发射特征分析
- 新时代煤炭工业八大战略新取向剖析
- 开采强度对华亭矿区冲击地压危险性的影响分析
- 新安煤矿探采对比:影响安全生产的关键因素与技术支撑
- 210MW循环流化床锅炉引风机变频改造优化与节能效果
- 浅埋煤层开采沉陷预测:方法应用与影响分析
- 优化巷道过陷落柱支护方案:数值模拟与现场试验
- 坚硬顶板定向断裂爆破技术:安全高效采煤新方案
- 北航2019算法课作业解析:投资策略与凸子多边形优化
- 断层破碎带垮冒下堆积体边界抗力分布关键影响因素
- 哈拉沟选煤厂优化末煤系统提高效率与处理能力
- 瓦斯涌出影响下采动煤体蠕变失稳研究
- 提升煤矿装载效率的可控箕斗定量装载设备设计
- 水分调控下烟煤瓦斯吸附特性对比研究
- 重介质悬浮液密度控制硬件设计与选型探讨
- 坚硬顶板巷道支护参数设计:现场测试与数值分析
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035111.png)
安全验证
文档复制为VIP权益,开通VIP直接复制
![](https://csdnimg.cn/release/wenkucmsfe/public/img/green-success.6a4acb44.png)