异常检测：技术与应用 - 数据挖掘视角

需积分: 15 28 浏览量更新于2024-07-17 收藏 3.7MB PDF 举报

"《异常检测：技术与应用》(N._N._R._Ranga Suri, Narasimha Murty M, G._Athithan著)是智能系统参考库第155卷中的一本关于数据挖掘视角下的异常检测重要图书。作者深入探讨了在数据挖掘背景下异常检测的技术和应用。" 这本书主要关注的是数据科学中的一个关键领域——异常检测，也称为异常或离群值检测。异常检测是识别数据集中不寻常或不一致观测值的过程，这些观测值可能表示错误、欺诈、故障或其他值得关注的现象。在各种领域如金融、网络安全、健康监测和工业生产中，异常检测都是数据分析的核心部分。书中内容可能涵盖了以下几个方面： 1. 异常检测的定义和重要性：解释为什么检测数据集中的异常值对于理解数据的模式和做出准确决策至关重要。 2. 异常检测的理论基础：介绍统计学和机器学习的基础概念，包括概率分布、聚类分析、回归分析等，这些都是异常检测方法的基础。 3. 异常检测算法：讨论多种异常检测技术，如基于统计的方法（如Z-Score、IQR方法）、基于距离的方法（如LOF - 局部离群因子）、基于密度的方法（如DBSCAN）以及深度学习方法等。 4. 应用案例：提供实际应用场景，如网络入侵检测、信用卡欺诈检测、医疗诊断中的异常识别等，展示如何将异常检测技术应用于解决实际问题。 5. 评估与性能度量：解释如何评估异常检测模型的效果，包括真阳性、假阳性、真阴性和假阴性的计算，以及ROC曲线和AUC等指标。 6. 数据预处理和特征工程：讨论如何通过数据清洗、归一化、降维等手段改善异常检测效果。 7. 面临的挑战与未来趋势：分析当前异常检测领域的挑战，如高维数据、实时检测和大规模数据集的处理，以及新兴的深度学习和人工智能技术如何推动这一领域的进步。通过这本全面的参考书籍，读者能够掌握异常检测的基本原理和实践技巧，从而在自己的研究或工作中有效地应用这些方法来发现数据中的异常模式。对于希望深入理解和应用异常检测技术的学者和从业者来说，这本书是一个宝贵的资源。

Acronyms

AAAI Association for the Advancement of Artiﬁcial Intelligence

ANMI Average Normalized Mutual Information

ANN Approximate Nearest Neighbor

AS Autonomous Systems

AUC Area Under Curve

AVF Attribute Value Frequency

BGP Border Gateway Protocol

BIRCH Balanced Iterative Reducing and Clustering using Hierarchies

CBLOF Cluster-Based Local Outlier Factor

DBLP DataBase and Logic Programming (Website at University of Trier)

DBSCAN Density-Based Spatial Clustering of Applications with Noise

DDoS Distributed Denial of Service

DSD Dense Subgraph Discovery

EDPL Egonet Density Power Law

EER Equal Error Rate

FPR False Positive Rate

FSD Frequent Subgraph Discovery

IG Information Gain

IGFS Information Gain-based Feature Selection

IQR Inter-Quartile Range

IS Information System

KDD Knowledge Discovery in Databases

KD-tree K-Dimensional tree

LBSN Location-Based Social Network

LOCI LOcal Correlation Integral

LOF Local Outlier Factor

LRD Local Reachability Density

LS Likely Set

MDL Minimum Description Length

MI Mutual Information

xvii

List of Figures

Fig. 1.1 A single outlier in a data set............................ 4

Fig. 1.2 Multiple outliers in a data set

........................... 5

Fig. 1.3 A sample outlier community

............................ 5

Fig. 1.4 Transformation from Outliers to Events of Interest

........... 9

Fig. 2.1 A sample box plot indicating a possible outlier

............. 14

Fig. 3.1 Outliers in data: two sample scenarios

.................... 30

Fig. 3.2 Research issues in outlier detection

...................... 32

Fig. 3.3 A taxonomy of the outlier detection methods

............... 33

Fig. 3.4 Differences among various distance-based outlier deﬁnitions

... 34

Fig. 3.5 Illustration of density-based outliers

...................... 35

Fig. 4.1 Steps in data preparation for outlier detection............... 62

Fig. 4.2 Confusion matrix associated with a classiﬁer

............... 63

Fig. 4.3 A sample 2-class data distribution with large class skew

...... 64

Fig. 4.4 A sample plot with various ROC curves

................... 65

Fig. 5.1 Scenarios of outlier occurrence in categorical space

.......... 79

Fig. 5.2 Illustration of cluster distance computation

................. 80

Fig. 5.3 Determining the likely set of outliers in data

............... 81

Fig. 5.4 Computational ﬂow of ROAD algorithm

.................. 83

Fig. 5.5 Performance comparison using bench mark data sets

.......... 87

Fig. 5.6 Sensitivity analysis on Tic-Tac-Toe data set

................ 88

Fig. 6.1 Various views of outliers with different selected feature

subsets

............................................ 96

Fig. 6.2 Outlier detection through unsupervised feature selection

....... 101

Fig. 6.3 Relevance based ranking of features with their selection

status

............................................. 105

Fig. 6.4 Effect of feature selection on outlier detection

.............. 107

Fig. 6.5 Effect of the number of features selected

.................. 108

Fig. 7.1 Uncertainty in clustering of data involving outliers

........... 114

Fig. 7.2 Lower and upper approximations of a rough set

............. 115

Fig. 7.3 Determining the nearest cluster centroid

................... 119

Fig. 7.4 Flow of computation with RKModes algorithm

............. 121

xix

List of Tables

Table 1.1 A sample 2-dimensional data set ....................... 6

Table 3.1 Sample experimentation on numerical data using Orca

method

........................................... 43

Table 3.2 Observations on categorical data for outlier detection

....... 44

Table 4.1 Performance meas ures for a binary classiﬁer

.............. 64

Table 5.1 Details on various scales of measure

.................... 70

Table 5.2 Example categorical attributes with sample values

.......... 71

Table 5.3 Frequency distribution of a sample 2-D car data

(for illustration)

.................................... 71

Table 5.4 Details of benchmark catego rical data sets

................ 85

Table 5.5 Categorical attributes describing Breast Cancer data ........ 85

Table 5.6 Performance compa rison on benchmark data sets

........... 86

Table 6.1 Details of high dimensional categorical data sets

........... 104

Table 6.2 Details of normal and outlier objects in data sets

........... 104

Table 6.3 Redundancy threshold values assigned for various data sets

.... 106

Table 6.4 Details of lung cancer data set

......................... 108

Table 7.1 Symbolic notation used in RKMod es algorithm

............ 122

Table 7.2 Details of benchmark catego rical data sets

................ 124

Table 7.3 Clustering results on Mushroom data (with k ¼ 20)

......... 125

Table 7.4 Clustering results on breast cancer data (with k ¼ 2)

........ 125

Table 7.5 Performance compa rison of clustering algorithms

.......... 125

Table 7.6 Outlier detection results using Rough ROAD framework

..... 126

Table 8.1 Experimental setting for anomaly detection in graph

data sets

.......................................... 145

Table 8.2 Egonet type anomalous subgraphs detected in AS-2000

graph

............................................ 146

Table 8.3 Arbitrary anomalous subgraphs found with benchmark

data

............................................. 147

Table 9.1 A subset of the global graph metrics

.................... 168

Table 10.1 Details of dynamic graph data sets used for anomaly

detection

.......................................... 183

xxi

剩余226页未读，继续阅读

zhlei12345

粉丝: 48
资源: 3

异常检测：技术与应用 - 数据挖掘视角

Grokking the System Design Interview by Design Gurus (z-lib.org).pdf

Linear Algebra Done Right Solutions Manual by Sheldon Axler (z-lib.org).pdf

[Dick_Grune,_Ceriel_J._H._Jacobs]_Parsing_Techniqu(z-lib.org).pdf

Mastering Spring 5.0 - Ranga Rao Karanam

Mastering Spring 5.0 Ranga RaoKaranam

class_assig_rangakuttamatlab_Solving2ndorder_matlab_

CRUD_Operations_ByUsing_JDBC

Apress.Beginning.C.Sharp.2005.Databases.From.Novice.to.Professional.Dec.2006

Beginning C# 2008 Databases.pdf

spark-hive-udf:Spark Hive UDF示例

最新资源