实时自适应异常检测：保障关键网络安全的新策略

需积分: 9 14 浏览量更新于2024-07-18 收藏 986KB PDF 举报

身份认证购VIP最低享 7 折!

领优惠券(最高得80元）

本研究论文《Adaptive real-time anomaly detection for safeguarding critical networks》由Kalle Burbeck撰写，主要关注于在保护关键网络方面应用自适应实时异常检测技术。论文的英文版收录于Linköping Studies in Science and Technology系列，Thesis No.1231，共167页，ISBN号码为9185497231，ID为493543。作者在计算机与信息科学系，Linköping大学，瑞典提交了这份学位论文，旨在完成获取工程学硕士（Licentiate of Engineering）的部分学业要求。论文的核心内容围绕着深度防御策略在保障关键网络安全中的重要性，特别强调了异常检测方法。在异常检测领域，一种关键方法是通过模型化用户在受保护系统中的正常行为，通常利用机器学习或数据挖掘技术来实现。这种方法的工作原理是在实时监控过程中，将新数据与预设的正常模式进行比较，如果发现数据与预期行为不符，即认为存在潜在的威胁或异常情况。在论文的摘要部分，作者强调了实时性的重要性，这意味着异常检测系统必须能够在短时间内对网络流量进行分析，以便迅速响应并阻止可能的攻击。此外，自适应性意味着该系统能够随着网络环境的变化和新的威胁模式动态调整其检测策略，以提高识别能力和准确性。论文详细探讨了如何设计和实施这样的自适应实时异常检测系统，包括数据收集、特征选择、模型训练和性能评估等步骤。同时，它还可能涵盖了对于现有技术挑战的讨论，如处理大量实时数据的效率问题，以及如何在保持低误报率的同时提高检测率。这篇论文对于网络安全专业人员、研究人员和实际网络管理员具有重要的参考价值，因为它提供了一种前沿的方法论，有助于提高关键网络的安全防护水平。读者可以从中学到如何将先进的数据分析技术与网络安全实践相结合，以应对不断演变的威胁环境。

资源详情

资源推荐

2 1.1. MOTIVATION

100

150

200

250

300

350

1998 1999 2000 2001 2002 2003 2004 2005

Year

Number of hosts

(millions)

Figure 1.1: Number of hosts on the Internet advertised in domain name servers

• A malicious (or just curious) person/organisation obtains information on

the vulnerability

• An attack tool is developed (requiring a considerable amount of technical

insight)

• The attack tool is released and used by many (requiring little actual knowl-

edge)

• The software vendor obtains information on the vulnerability

• A patch for the software is developed by the software vendor

• The patch is released and applied to a subset of systems removing the vul-

nerability from those systems.

The order of these events is very signiﬁcant. The listed order is the most

unfortunate since the attack tool is released before the patch is applied to end user

systems resulting in a potentially very large number of compromised systems.

Unfortunately the time from public discovery of a new vulnerability to the release

of an attack tool is decreasing and is currently in the order of days. This means that

the time window for developing and applying patches is becoming very short. One

example is the Zotob-A worm [47] and its variants. On Tuesday 9th August 2005

Microsoft released a patch and less than three days [39] later exploit code was

publicly available on the Internet. In four days (Saturday 13th) worms exploiting

the vulnerability were spreading.

INTRODUCTION 3

Rapid patching is important [76] but not sufﬁcient. For a production system

continuous patching may not be viable due to system complexity and diversity

as well as compatibility requirements. For important systems, defence in depth

is needed incorporating many different security technologies [112]. This may in-

clude ﬁrewalls at network boundaries and on individual hosts, removal of unused

software and services, virus scanners and so on. To further harden the defence,

intrusion detection systems may be applied.

Intrusion detection systems look for traces of computer misuse by examining

data sources such as program or user behaviour, network trafﬁc or logs. When

traces of misuse are detected, alerts are produced and manual or automatic re-

sponse may be initiated. Speciﬁc attacks may not be visible in every type of data

source and diverse approaches may provide complementing information. It there-

fore makes sense to use multiple intrusion detection sensors either in isolation or

preferably also combining their output by correlating the alerts.

The main detection scheme of most commercial intrusion detection systems is

called misuse detection, where known bad behaviours (attacks) are encoded into

signatures. Misuse detection is only able to detect attacks that are well known and

for which signatures have been written.

An alternative approach is anomaly detection where good (normal) behaviour

of users or the protected system is modelled, often using machine learning or data

mining techniques. During detection new data is matched against the normality

model, and deviations are marked as anomalies. Since no knowledge of attacks

is needed to train the normality model, anomaly detection may detect previously

unknown attacks. If an attack tool is published before a patch is applied and before

attack signatures are developed or installed, the anomaly detection system may be

the only remaining defence. Some attack types, including a subset of denial of

service and scanning attacks, alter the statistical distribution of system data when

present. This implies that anomaly detection may be a general and perhaps the

most viable approach to detect such attacks.

1.2 Research challenges

A fundamental problem of intrusion detection research is the limited availability

of appropriate data to be used for evaluation. Producing intrusion detection data

is a labour intensive and complex task involving generation of normal system

data as well as attacks, and labelling the data to make evaluation possible. If a

real network is used, the problem of producing good normal data is reduced, but

then the data may be too sensitive to be released to other researchers publicly.

4 1.2. RESEARCH CHALLENGES

Learning-based methods require data not only for testing and comparison but also

for training, resulting in even higher data requirements. The data used for training

needs to be representative for the network to which the learning-based method

will be applied, possibly requiring generation of new data for each deployment.

Classiﬁcation-based methods [40,83] require training data that contains nor-

mal data as well as good representatives of those attacks that should be detected,

to be able to separate attacks from normality. Producing a good coverage of the

very large attack space (including unknown attacks) is not practical for any net-

work. Also the data needs to be labelled and attacks to be marked. One advantage

of clustering-based methods [57,84,90,101] is that they require no labelled train-

ing data set containing attacks, signiﬁcantly reducing the data requirement. There

exist at least two approaches.

When doing unsupervised anomaly detection [57, 90, 101] a model based on

clusters of data is trained using unlabelled data, normal as well as attacks.If

the underlying assumption holds (i.e. attacks are sparse in data) attacks may be

detected based on cluster sizes, where small clusters correspond to attack data.

Unsupervised anomaly detection is a very attractive idea, but unfortunately the

experiences so far indicate that acceptable accuracy is very hard to obtain. Also,

the assumption of unsupervised anomaly detection is not always fulﬁlled making

the approach unsuitable for attacks such as denial of service (DoS) and scanning.

In the second approach, which we simply denote (pure) anomaly detection in

this thesis, training data is assumed to consist only of normal data. Munson and

Wimer [84] used a cluster-based model (Watcher) to protect a real web server,

proving anomaly detection based on clustering to be useful in real life. The anom-

aly detection algorithm presented here uses pure anomaly detection to reduce the

training data requirement of classiﬁcation-based methods and to avoid the attack

volume assumption of unsupervised anomaly detection. By including only normal

data in the detection model the low accuracy of unsupervised anomaly detection

can be signiﬁcantly improved.

In a real live network with connection to the Internet, data can never be as-

sumed to be free of attacks. Pure anomaly detection also works when some attacks

are included in the training data, but those attacks will be considered normal dur-

ing detection and therefore not detected. To increase detection coverage, attacks

should be removed from the training data to as large an extent as possible, with

a trade-off between coverage and data cleaning effort. Attack data can be ﬁltered

away from training data using updated misuse detectors, or multiple anomaly de-

tection models may be combined by voting to reduce costly human effort.

An intrusion detection system in a real-time environment needs to be fast

INTRODUCTION 5

enough to cope with the information ﬂow, to have explicit limits on resource us-

age, and adapt to changes in the protected network in real-time. Many proposed

clustering techniques require quadratic time for training [69], making real-time

adaptation of a cluster-based model hard. They may also not be scalable, requir-

ing all training data to be kept in main memory during training, limiting the size

of the trained model. We argue that it is important to consider scalability and

performance in parallel to detection quality when evaluating algorithms for intru-

sion detection. Most work on applications of data mining to intrusion detection

considers those issues to a very limited degree or not at all.

One fundamental problem of anomaly detection in general is the false posi-

tives rate. In most realistic settings normality is hard to capture and even worse,

is changing over time. This implies that in addition to facilitate modelling the

normality of a very complex system, an anomaly detection scheme needs to adapt

over time.

1.3 Contribution

Many different anomaly detection schemes have been evaluated by other authors,

but not all aspects of anomaly detection is getting the attention it deserves. Two

such aspects are adaptability and performance. The primary contribution of this

thesis is the design and implementation of the ADWICE (Anomaly Detection

With fast Incremental Clustering) algorithm with the following properties:

• Adaptation - Rather than making use of extensive periodical retraining ses-

sions on stored off-line data to handle changes, ADWICE is fully incre-

mental making very ﬂexible on-line training of the model possible without

destroying what is already learnt. When subsets of the model are not useful

anymore, those clusters can be forgotten.

• Performance - ADWICE is linear in the number of input data thereby heav-

ily reducing training time compared to alternative clustering algorithms.

Training time as well as detection time is further reduced by the use of an

integrated search-index.

• Scalability - Rather than keeping all data in memory, only compact cluster

summaries are used. The linear time complexity also improves scalability

of training.

When performing anomaly detection and improving performance by using a

search-index, detection accuracy can be inﬂuenced by the index. In this thesis we

剩余166页未读，继续阅读

zztq

粉丝: 7
资源: 53

实时自适应异常检测：保障关键网络安全的新策略

ABCNet - Real-time Scene Text Spotting with Adaptive Bezier-curve Network.mp4

Adaptive Color Attributes for Real-Time Visual Tracking算法流程

Levi D. McClenny∗ Ulisses Braga-Neto写的Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism中提到的最大化损失函数什么意思

Adaptive Normalized Risk-Averting Training for Deep Neural Networks

分析Adaptive as-natural-as-possible image stitching算法流程

domain adaptive faster r-cnn for object detection in the wild

Visual Tracking via Adaptive Spatially-Regularized Correlation Filters(**CVPR2019 Oral**).

基于preevision的autosar adaptive设计-上篇

adaptive fringe-pattern projection for image saturation avoidance in 3d surf

space-time adaptive processing for airborne radar j.ward

基于yolo的低照度目标检测英文文献

SAM-DETR算法参考文献

stm32f373c8t6

transformer的目标检测

引用Distributed adaptive coverage control algorithm for mobile sensor networks

视频流打包的方式有什么？

adaptive autosar standards-21-11

3rd order Adaptive Torque Filter

RL-Adaptive-DE算法的伪代码

auto bandwidth

最新资源

Visual Tracking via Adaptive Spatially-Regularized Correlation Filters(CVPR2019 Oral).