时空数据分析：探索性研究的系统方法

需积分: 16 158 浏览量更新于2024-07-17 收藏 15.61MB PDF 举报

"Exploratory Analysis of Spatial and Temporal Data - A Systematic Approach" 该书籍《时空数据分析的探索性分析——系统方法》由Natalia Andrienko和Gennady Andrienko撰写，专注于探讨如何有效地处理和分析时空数据。时空数据涉及到空间实体和空间现象在时间和空间维度上的复杂特性，这使得它们具有多维、语义丰富以及时空动态关联的特性。作者通过系统性的方法论，旨在解决时空大数据的挑战，包括如何形式化表达多维关联、动态构建关联关系模型以及进行多尺度关联分析。书中提到的"Exploratory Analysis"是指一种数据探索的方法，通常用于初步理解和揭示数据集中的模式、趋势和异常。在时空数据分析中，这可能涉及识别空间分布的模式、时间序列的变化规律，以及不同空间和时间点之间的相互作用。这种方法对于发现隐藏的关联、建立假设和驱动后续的统计或机器学习分析至关重要。作者们来自Fraunhofer Institute AIS，这是一家位于德国的研究机构，他们的工作可能涉及到高级的数据科学技术，如大数据处理、数据挖掘和可视化工具，这些都是有效分析时空数据所必需的。图书中的245个图表和34张表格可能展示了各种案例研究和可视化示例，帮助读者直观地理解这些概念和方法。图书的内容可能涵盖了以下关键知识点： 1. **时空数据的特性**：讨论时空数据的复杂性和多维性，包括空间、时间和属性维度的信息。 2. **形式化表达**：介绍如何将时空数据的多维关联进行形式化的数学描述，以便于计算机处理。 3. **关联关系动态建模**：讲述如何建立随时间变化的关联模型，捕捉空间实体之间的动态联系。 4. **多尺度关联分析**：解释如何在不同尺度上分析数据，以揭示不同粒度下的模式和关系。 5. **协同计算与重构**：探讨如何利用分布式计算技术处理大规模时空数据，并重构数据以满足特定任务的需求。 6. **面向任务的关联约束**：强调如何快速、准确地提取与特定分析任务相关的时空关联信息。 7. **数据可视化**：可能包含关于如何通过可视化技术来揭示和解释时空数据模式的章节。 8. **实例应用**：可能通过实际案例来展示上述理论和技术的应用，帮助读者理解和掌握这些方法。该书对于地理信息系统（GIS）专家、数据科学家、城市规划者、环境科学家以及其他需要处理和理解时空数据的专业人士来说，是一本重要的参考资料。它不仅提供了理论框架，还可能包含实用的算法和工具，帮助读者应对时空大数据的挑战。

2 1 Introduction

The difference between statistical analysis and GIS analysis seems to lie

only in the types of data that they deal with and in the methods used. In

both cases, data analysis appears to be driven by questions: the questions

motivate one to do analysis, determine the choice of data and methods, and

affect the interpretation of the results. Since the questions are so important,

what are they?

Neither statistical nor GIS handbooks provide any classification of pos-

sible questions but they give instead a few examples. Here are some exam-

ples from a GIS handbook (Mitchell 1999):

x Where were most of the burglaries last month?

x How much forest is in each watershed?

x Which parcels are within 500 feet of this liquor store?

For a comparison, here are some examples from a statistical handbook for

geographers (Burt and Barber 1996):

x What major explanatory variables account for the variation in individual

house prices in cities?

x Are locational variables more or less important than the characteristics

of the house itself or of the neighbourhood in which it is located?

x How do these results compare across cities?

It can be noticed that the example questions in the two groups have dis-

cernible flavours of the particular methods available in GIS and statistical

analysis, respectively, i.e. the questions have been formulated with certain

analysis methods in mind. This is natural for handbooks, which are in-

tended to teach their readers to use methods, but how does this match the

actual practice of data analysis?

We believe that questions oriented towards particular analysis methods

may indeed exist in many situations, for example, when somebody per-

forms routine analyses of data of the same type and structure. But what

happens when an analyst encounters new data that do not resemble any-

thing dealt with so far? It seems clear that the analyst needs to get ac-

quainted with the data before he/she can formulate questions like those

cited in the handbooks, i.e. questions that already imply what method to

use.

“Getting acquainted with data” is the topic pursued in exploratory data

analysis, or EDA. As has been said in an Internet course in statistics, “Of-

ten when working with statistics we wish to answer a specific question 

such as does smoking cigars lead to an increased risk of lung cancer? Or

does the number of keys carried by men exceed those carried by women?

... However sometimes we just wish to explore a data set to see what it

1.1 What Is Data Analysis? 3

might tell us. When we do this we are doing Exploratory Data Analysis”

(STAT 2005).

Although EDA emerged from statistics, this is not a set of specific tech-

niques, unlike statistics itself, but rather a philosophy of how data analysis

should be carried out. This philosophy was defined by John Tukey (Tukey

1977) as a counterbalance to a long-term bias in statistical research to-

wards developing mathematical methods for hypothesis testing. As Tukey

saw it, EDA was a return to the original goals of statistics, i.e. detecting

and describing patterns, trends, and relationships in data. Or, in other

words, EDA is about hypothesis generation rather than hypothesis testing.

The concept of EDA is strongly associated with the use of graphical rep-

resentations of data. As has been said in an electronic handbook on engi-

neering statistics, “Most EDA techniques are graphical in nature with a

few quantitative techniques. The reason for the heavy reliance on graphics

is that by its very nature the main role of EDA is to open-mindedly ex-

plore, and graphics gives the analysts unparalleled power to do so, enticing

the data to reveal its structural secrets, and being always ready to gain

some new, often unsuspected, insight into the data. In combination with

the natural pattern-recognition capabilities that we all possess, graphics

provides, of course, unparalleled power to carry this out”

(NIST/SEMATECH 2005).

Is the process of exploratory data analysis also question-driven, like tra-

ditional statistical analysis and GIS analysis? On the one hand, it is hardly

imaginable that someone would start exploring data without having any

question in mind; why then start at all? On the other hand, if any questions

are asked, they must be essentially different from the examples cited

above. They cannot be so specific and cannot imply what analysis method

will be used. Appropriate examples can be found in George Klir’s explana-

tion of what empirical investigation is (Klir 1985).

According to Klir, a meaningful empirical investigation implies an ob-

ject of investigation, a purpose of the investigation of the object, and con-

straints imposed upon the investigation. “The purpose of investigation can

be viewed as a set of questions regarding the object which the investigator

(or his client) wants to answer. For example, if the object of investigation

is New York City, the purpose of the investigation might be represented by

questions such as ‘How can crime be reduced in the city?’ or ‘How can

transportation be improved in the city?’; if the object of investigation is a

computer installation, the purpose of investigation might be to answer

questions ‘What are the bottlenecks in the installation?’, ‘What can be

done to improve performance?’, and the like; if a hospital is investigated,

the question might be ‘How can the ability to give immediate care to all

emergency cases be increased?’, ‘How can the average time spent by a

4 1 Introduction

patient in the hospital be reduced?’, or ‘What can be done to reduce the

cost while preserving the quality of services?’; if the object of interest of a

musicologist is a musical composer, say Igor Stravinsky, his question is

likely to be ‘What are the basic characteristics of Stravinsky’s composi-

tions which distinguish him from other composers?’ ” (Klir 1985, p. 83).

Although Klir does not use the term “exploratory data analysis”, it is clear

that exploratory analysis starts after collecting data about the object of in-

vestigation, and the questions representing the purpose of investigation

remain relevant.

According to the well-known “Information Seeking Mantra” introduced

by Ben Shneiderman (Shneiderman 1996), EDA can be generalised as a

three-step process: “Overview first, zoom and filter, and then details-on-

demand”. In the first step, an analyst needs to get an overview of the entire

data collection. In this overview, the analyst identifies “items of interest”.

In the second step, the analyst zooms in on the items of interest and filters

out uninteresting items. In the third step, the analyst selects an item or

group of items for “drilling down” and accessing more details. Again, the

process is iterative, with many returns to the previous steps. Although

Shneiderman does not explicitly state this, it seems natural that it is the

general goal of investigation that determines what items will be found “in-

teresting” and deserving of further examination.

On this basis, we adopt the following view of EDA. The analyst has a

certain purpose of investigation, which motivates the analysis. The purpose

is specified as a general question or a set of general questions. The analyst

starts the analysis with looking what is interesting in the data, where “in-

terestingness” is understood as relevance to the purpose of investigation.

When something interesting is detected, new, more specific questions ap-

pear, which motivate the analyst to look for details. These questions affect

what details will be viewed and in what ways. Hence, questions play an

important role in EDA and can determine the choice of analysis methods.

There are a few distinctions in comparison with the example questions

given in textbooks on statistics and GIS:

x EDA essentially involves many different questions;

x the questions vary in their level of generality;

x most of the questions arise in the course of analysis rather than being

formulated in advance.

These peculiarities make it rather difficult to formulate any guidelines for

successful data exploration, any instructions concerning what methods to

use in what situation. Still, we want to try.

There is an implication of the multitude and diversity of questions in-

volved in exploratory data analysis: this kind of analysis requires multiple

1.2 Objectives of the Book 5

tools and techniques to be used in combination, since no single tool can

provide answers to all the questions. Ideally, a software system intended to

support EDA must contain a set of tools that could help an analyst to an-

swer any possible question (of course, only if the necessary information is

available in the data). This ideal will, probably, never be achieved, but a

designer conceiving a system or tool kit for data analysis needs to antici-

pate the potential questions and at least make a rational choice concerning

which of them to support.

1.2 Objectives of the Book

This is a book about exploratory data analysis and, in particular, explora-

tory analysis of spatial and temporal data. The originator of EDA, John

Tukey, begins his seminal book with comparing exploratory data analysis

to detective work, and dwells further upon this analogy: “A detective in-

vestigating a crime needs both tools and understanding. If he has no fin-

gerprint powder, he will fail to find fingerprints on most surfaces. If he

does not understand where the criminal is likely to have put his fingers, he

will not look in the right places. Equally, the analyst of data needs both

tools and understanding” (Tukey 1977, p. 1).

Like Tukey, we also want to talk about tools and understanding. We

want to consider current computer-based tools suitable for exploratory

analysis of spatial and spatio-temporal data. By “tools”, we do not mean

primarily ready-to-use software executables; we also mean approaches,

techniques, and methods that have, for example, been demonstrated on

pilot prototypes but have not yet come to real implementation.

Unlike Tukey, we have not set ourselves the goal of describing each tool

in detail and explaining how to use it. Instead, we aim to systemise the

tools (which are quite numerous) into a sort of catalogue and thereby lead

readers to an understanding of the principles of choosing appropriate tools.

The ultimate goal is that an analyst can easily determine what tools would

be useful in any particular case of data exploration.

The most important factors for tool selection are the data to be analysed

and the question(s) to be answered by means of analysis. Hence, these two

factors must form part of the basis of our systemisation, in spite of the fact

that every dataset is different and the number of possible questions is infi-

nite. To cope with this multitude, it is necessary to think about data and

questions in a general, domain-independent manner. First, we need to de-

termine what general characteristics of data are essential to the problem of

choosing the right exploratory tools. We want not only to be domain-

6 1 Introduction

independent but also to put aside any specifics of data collection, organisa-

tion, storage, and representation formats. Second, we need to abstract a

reasonable number of general question types, or data analysis tasks, from

the myriad particular questions. While any particular question is formu-

lated in terms of a specific domain that the data under analysis are relevant

to, a general task is defined in terms of structural components of the data

and relations between them.

Accordingly, we start by developing a general view of data structure and

characteristics and then, on this basis, build a general task typology. After

that, we try to extend the generality attained to the consideration of exist-

ing methods and techniques for exploratory data analysis. We abstract

from the particular tools and functions available in current software pack-

ages to types of tools and general approaches. The general tool typology

uses the major concepts of the data framework and of the task typology.

Throughout all this general discussion, we give many concrete examples,

which should help in understanding the abstract concepts.

Although each subsequent element in the chain “datataskstools” re-

fers to the major concepts of the previous element(s), this sort of linkage

does not provide explicit guidelines for choosing tools and approaches in

the course of data exploration. Therefore, we complete the chain by reveal-

ing the general principles of exploratory data analysis, which include rec-

ommendations for choosing tools and methods but extend beyond this by

suggesting a kit of generic procedures for data exploration and by encour-

aging a certain amount of discipline in dealing with data.

In this way, we hope to accomplish our goal: to enumerate the tools and

to give understanding of how to choose and use them. In parallel, we hope

to give some useful guidelines for tool designers. We expect that the gen-

eral typology of data and tasks will help them to anticipate the typical

questions that may arise in data exploration. In the catalogue of techniques,

designers may find good solutions that could be reused. If this is not the

case (we expect that our cataloguing work will expose some gaps in the

datatask space which are not covered by the existing tools), the general

principles and approaches should be helpful in designing new tools.

1.3 Outline of the Book

1.3.1 Data

As we said earlier, we begin with introducing a general view of the struc-

ture and properties of the data; this is done in the next chapter, entitled

剩余711页未读，继续阅读

小太皮

粉丝: 1
资源: 3

时空数据分析：探索性研究的系统方法

spatial_gan, 空间生成对抗性网络.zip

data_analysis_4_social_science:具有“社会科学家数据分析”类的作业的存储库

temporal_network_viz

帮我写出python算法项目的非常完善的目录结构规范，并说明各个目录的作用

找不到或无法加载主类 @C:\Users\mooc\AppData\Local\Temp\idea_arg_file959826638

think stats: exploratory data analysis

exploratory data analysis

post-hoc analysis

在数据分析项目中，如何有效地应用EDA（Exploratory Data Analysis）技术来提取数据洞察？

最新资源