物联网中的传感器数据管理和挖掘

需积分: 10 70 浏览量更新于2024-07-22 收藏 7.53MB PDF 举报

"Managing and Mining Sensor Data" 本书详细探讨了物联网系统中的传感器数据管理主题，重点关注如何有效管理和挖掘这些海量数据。书中的内容涵盖了数据挖掘、数据过滤、事件处理以及分布式数据处理等多个关键领域。在物联网(IoT)环境中，传感器数据的管理是一项核心任务。这些数据通常具有实时性、海量性和异构性，需要高效的数据存储和检索策略来应对。数据管理部分可能包括数据的收集、整合、存储和查询优化，确保在复杂网络环境下能够快速响应并处理来自不同类型的传感器数据。数据挖掘是书中另一个重点，它涉及到从传感器数据中发现有价值的信息和模式。这可能涵盖预处理（如清洗和转换）、特征提取、模式识别、异常检测和预测分析等步骤。通过这些方法，可以深入理解物联网系统的运行状况，预测未来趋势，或者识别潜在的问题和机会。数据过滤则是处理实时和大规模数据流的关键技术。例如，使用流计算和近似算法可以在不牺牲太多精度的情况下减少数据处理的复杂性和延迟。这些技术对于处理物联网环境中的连续、高速数据流至关重要。事件处理是物联网系统中的另一个重要组成部分，它涉及识别和响应特定的系统或环境事件。这可能包括基于规则的事件引擎，用于检测预定义的条件，触发相应的行动，如警报或自动控制操作。分布式数据处理，特别是在大数据背景下，是处理传感器数据不可或缺的工具。例如，Apache Hadoop和Spark等框架提供了处理和分析大规模数据集的能力，适合处理来自全球各地的传感器网络所产生的数据。此外，书中可能还会讨论到隐私保护、数据安全和合规性问题，这些都是在管理和挖掘传感器数据时必须考虑的因素。这些技术与策略的结合，使得物联网系统的数据不仅能够被有效地管理和利用，还能确保用户隐私和系统安全。 "Managing and Mining Sensor Data"这本书提供了一个全面的视角，帮助读者理解和掌握在物联网环境中如何处理和利用传感器数据，从而实现更智能、更高效的系统操作和决策支持。

2 MANAGING AND MINING SENSOR DATA

data is produced in the context of a wide variety of applications such as

the following:

A wide variety of mobile devices are now GPS-enabled. This has

lead to unprecedented opportunities in the context of several ap-

plications such as social sensing [4]. GPS data is also available in

the context of many location-aware devices and applications.

The decreasing cost of RFID tags has lead to tremendous volumes

of RFID data. The cost of an RFID tag is now in the range

of under 5 cents. This has allowed cost-eﬀective deployment of

RFID tags on products of even modest price. RFID data poses

numerous challenges because of the tremendous amounts of noise

in the collected data [5].

Numerous military applications use a wide variety of sensors in

order to track for unusual events or activity. This could include

visual or audio cameras, or seismometers for tracking movements

of large objects [9].

Sensors are also deployed in the context of a wide variety of en-

vironmental applications, such as detecting weather and climate

trends [7], and tracking pollution levels in water networks [11].

Sensor data brings numerous challenges with it in the context of data

collection, storage and processing. This is because sensor data processing

often requires eﬃcient and real-time processing from massive volumes of

possibly uncertain data. Some of these challenges may be enumerated

as follows:

Data collection is a huge challenge in the context of sensor pro-

cessing because of the natural errors and incompleteness in the

collection process. Sensors often have limited battery life, because

of which many of the sensors in a network may not be able to col-

lect or transmit their data over large periods of time. The errors

in the underlying data may lead to uncertainty of the data repre-

sentation [8]. Therefore, methods need to be designed to process

the data in the presence of uncertainty.

Sensors are often designed for applications which require real-time

processing. This requires the design of eﬃcient methods for stream

processing [1]. Such algorithms need to be executed in one pass of

the data, since it is typically not often possible to store the entire

data set because of storage and other constraints.

Licensed to marie zheng<marieprince@hotmail.com>

An Introduction to Sensor Data Analytics 3

The large volumes of data lead to huge challenges in terms of

storage and processing of the data. It has been estimated that since

2008, the number of internet-connected devices has exceeded the

number of people on the planet. Thus, it is clear that the amount

of machine generated data today greatly exceeds the amount of

human generated data, and this gap is only likely to increase in

the forseeable future. This is widely known as the big data problem

in the context of analytical applications [10], or the information

overload problem in stream processing.

In many cases, it is critical to perform in-network processing,wherein

the data is processed within the network itself, rather than at a

centralized service. This needs eﬀective design of distributed pro-

cessing algorithms, wherein queries and other mining algorithm

can be processed within the network in real time [12].

In this book, we will provide an overview of the key areas of research

in sensor processing, as they related to these challenges. We will also

study a number of new applications of sensor data such as social sensing,

mobile data processing, RFID processing, and the internet of things.

This chapter is organized as follows. In the next section, we will dis-

cuss the key areas of research in sensor processing, as they relate to the

afore-mentioned challenges. We will also relate the diﬀerent research

areas to these challenges. Section 3 discusses the conclusions and sum-

mary.

2. Research in Sensor Processing

The research issues in the area of sensor processing arise along all

stages of the pipeline, beginning from data collection, cleaning, data

management, and knowledge discovery and mining. Furthermore, many

research issues arise in the context of in-network processing, which are

speciﬁc to the particular application domain. The speciﬁcity to the

application domain may arise in the context of other parts of the pipeline

as well. Therefore, we summarize the key research issues which arise in

the context of sensor data processing as follows:

Data Collection and Cleaning Issues: Numerous issues arise

in the context of collection of sensor data. Sensor data is inher-

ently noisy and uncertain, and may either have missed readings or

redundant readings depending upon the application domain. For

example, in the context of RFID data, almost 30% of the readings

are dropped, and multiple sensors may track the same RFID ob-

ject. In the context of battery-driven sensors, numerous errors may

Licensed to marie zheng<marieprince@hotmail.com>

4 MANAGING AND MINING SENSOR DATA

arise during data transmission, and there may also be signiﬁcant

incompleteness because of limited battery life.

Data Management Issues: The large volumes of collected data

poses signiﬁcant challenges for the collected data. Sometimes, the

volume of the data is so large, that it may be impractical to store

the entire raw data, and it may be desirable to either compress

or drop portions of the data. What parts of the data should be

dropped or compressed? The errors and uncertainty in sensor data,

have spurred the development of algorithms for uncertain database

management [2].

Sensor Data Mining and Processing: The large volumes of

sensor data necessitate the design of eﬃcient one-pass algorithms

which require at most one scan of the data. These are traditionally

referred to as data stream mining algorithms. Furthermore, it

may sometimes be advantageous to perform in-network processing,

which can perform partial processing of the data in the network

before sending these results on to a higher level of storage.

Application-Speciﬁc Issues: Sensor data can arise in many do-

mains such as retail data (RFID), military sensor networks, astron-

omy, the environment, and mobile data. Diﬀerent domains may

lead to diﬀerent issues in the context of storage and processing.

For example, RFID data may have larger levels of redundancy and

uncertainty, whereas mobile data mining applications may require

spatio-temporal mining techniques.

The diﬀerent chapters of this book will study these diﬀerent aspects of

sensor stream processing. Therefore, the book will be organized so as

to comprehensively study these diﬀerent aspects. The diﬀerent topics

covered by the chapters of this book are as follows:

Data Collection and Management Issues The key data collec-

tion and management issues are discussed in Chapter 2. This chapter

discusses some of the key database management aspects, which have

recently been designed in the context of sensor data. Issues involving

data uncertainty and query processing are discussed in this chapter, es-

pecially in the context of sensor data. The area of indexing and query

processing is very important in the context of sensor data, and therefore

we have also designed chapters speciﬁcally for this topic.

Query Processing of Sensor Data Sensor data poses numerous

challenges from the perspective of indexing and query processing, be-

Licensed to marie zheng<marieprince@hotmail.com>

An Introduction to Sensor Data Analytics 5

cause of the massive volume of the data which is received over time. A

special case of query processing in sensor data is that of event detection,

wherein continuous queries are posed on the sensor data in order to de-

tect the underlying events. The main challenge in event processing is

that the high level semantic events are often a complex function of the

underlying raw sensor data. In some cases, the event-query cannot be

posed exactly, since the event detection process is ambiguously related

to the underlying data. Methods for query processing of sensor data

are discussed in Chapter 3. Specialized methods for event processing of

sensor data are discussed in Chapter 4.

Mining Sensor Data A variety of data mining methods such as clus-

tering, classiﬁcation, frequent pattern mining, and outlier detection are

oftenappliedtosensordatainordertoextractactionableinsights. This

data usually needs to be compressed and ﬁltered for more eﬀective min-

ing and analysis. The main challenge is that conventional mining al-

gorithms are often not designed for real time processing of the data.

Therefore, new algorithms for sensor data stream processing need to

perform the analytics in a single pass in real time. In addition, the sen-

sor scenario may often require in-network processing, wherein the data is

processed to higher level representations before further processing. This

reduces the transmission costs, and the data overload from a storage

perspective. The problems of stream compression [3] and stream mining

are therefore tightly integrated together from an eﬃciency perspective.

For example, compression and hidden variable modeling provides sum-

marized representations which can be leveraged for applications such as

forecasting and outlier analysis. A survey of methods for dimensional-

ity reduction, compression and ﬁltering of sensor streams is provided in

Chapter 5. This chapter studies the issue of stream correlation analysis,

compression across streams in terms of hidden variables, and compres-

sion across time in a given stream. The application of these concepts to

a few stream mining problems is also studied in the same chapter. A

number of methods for real-time sensor stream mining, processing and

analytics are discussed in Chapters 6 and 7. Speciﬁc methods for mining

sensor streams in the distributed setting are presented in Chapter 8.

Social Sensing Applications and Mobile Data The popularity of

mobile phones and other sensor-enabled devices has lead to a plethora

of “socially-aware data” which can be mined in the context of a wide

variety of applications. This trend has lead to the integration of sensors

and dynamic social networks. A number of architectural, privacy and

trust issues arise in the collection of socially aware sensor data. These

Licensed to marie zheng<marieprince@hotmail.com>

6 MANAGING AND MINING SENSOR DATA

issues are discussed in detail in Chapter 9. The chapter also discusses

the issues of mining the diﬀerent kinds of GPS- and content-based data

generated in such applications.

Much of the data in social sensing applications often contains GPS

trajectory data. Mobile data has a number of characteristics, which can

be exploited in order to create more eﬃcient methods for clustering, clas-

siﬁcation, anomaly detection, and pattern mining. Therefore, we have

included a chapter which discusses algorithms for mobile data analysis

in detail. Chapter 10 provides a detailed discussion of a wide variety of

indexing and mining algorithms in the context of mobile data.

RFID Data and the Internet of Things The trend towards ubiq-

uitous and embedded sensing has lead to a natural focus on machine-

to-machine (M2M) paradigms in sensor processing. These paradigms

use small RFID sensors to collect data about many smart objects. The

data generated from such applications can be shared by diﬀerent devices

for heterogeneous fusion and inference, especially if the devices are con-

nected to the internet. A number of issues also arise about how such

devices can be eﬀectively discovered and used by diﬀerent network par-

ticipants. Chapter 11 provides an overview on RFID applications for

collecting such data. Issues about how such data can be used in the

context of the internet of things are discussed in Chapter 12.

Software Bug Tracing in Sensor Networks Most of the afore-

mentioned chapters provide application-speciﬁc insights on the basis of

the collected data. Sensors also produce diagnostic data, which can be

used in order to determine diagnostic bugs within the sensor software.

Thus, this kind of mining process can be used in order to improve the

performance of the underlying sensor network. A survey of methods

and algorithms for software bug tracing in sensor networks is provided

in Chapter 13.

Healthcare Applications Sensor data has found increasing applica-

tion in the health care domain. A wide variety of Intensive Care Unit

(ICU) applications use sensors such as ECG, EEG, blood pressure mon-

itors, respiratory monitors, and a wide variety of other sensors in order

to track the condition of the patient. The volume of such data is ex-

tremely large and the inferences from such data need to be performed

in a time-critical fashion. Chapter 14 provides an overview of sensor

mining applications in the context of health-care data.

Licensed to marie zheng<marieprince@hotmail.com>

剩余544页未读，继续阅读

irisdeer

粉丝: 0
资源: 1

物联网中的传感器数据管理和挖掘

"大数据管理与数据库系统概论：DT时代的动力和应用

"学生成绩分析中的数据挖掘应用研究

"HBase技术在新能源汽车监控系统中的应用与挑战分析"。

MANAGING AND MINING GRAPH DATA

Managing and Mining Uncertain Data

Managing and Mining Graph Data

managing data mining advice

Managing Metadata with Oracle Data Integrator

MS.Azure.Planning.Deploying.and.Managing.Your.Data.Center.in.the.Clou

managing-hierarchical-data-in-mysql.rar

最新资源