人工智能与网络安全：Cyber Threat Intelligence探索

需积分: 10 67 浏览量更新于2024-07-16 收藏 8.45MB PDF 举报

"《Cyber Threat Intelligence》是由Ali Dehghantanha、Mauro Conti和Tooska Dargahi合编的一本书，属于 Advances in Information Security 系列的第70卷。这本书关注的是网络安全威胁情报的领域，探讨了在日益增多的网络攻击背景下，如何利用人工智能、机器学习和高级数据挖掘技术来收集、分析和解释网络攻击的证据，以及时防御和应对这些威胁。" 在当前的网络环境中，网络攻击的数量不断攀升，对网络安全和取证专家提出了更高的要求，他们需要近乎实时地检测、分析并抵御网络威胁。这使得对攻击特征的深入理解和采取相应的智能防御行动变得至关重要，这就是所谓的网络威胁情报的核心概念。然而，没有人工智能、机器学习和先进的数据挖掘技术，处理如此大量的攻击是不现实的。该书的引言部分首先阐述了网络威胁情报的概念，讨论了它所面临的挑战，如海量攻击数据的处理、快速响应的需求以及情报的准确性等，并指出了其中蕴含的机会。接着，它简要介绍了书中各个章节，这些章节将针对识别出的挑战提出解决方案，或提供机会性的策略来提供威胁情报。书中的章节可能涵盖了以下主题： 1. 网络威胁情报的定义和框架：详细介绍了威胁情报的定义、其构成元素以及如何构建一个有效的威胁情报系统。 2. 收集威胁情报的方法：探讨了通过公开源情报、网络监控、蜜罐系统等多种手段获取威胁情报的策略。 3. 数据挖掘与机器学习应用：阐述如何运用这些技术从大量数据中提取有用信息，识别潜在威胁模式。 4. 智能防御系统：介绍基于威胁情报的自动化防御系统，如入侵检测系统和预防措施的改进。 5. 威胁情报共享与标准化：讨论威胁情报的共享机制和标准，以提高整个行业的响应速度和协作能力。 6. 实时分析与预测：研究如何实现实时分析和预测网络威胁，以提前防范未来攻击。 7. 法证分析与法律问题：涉及网络威胁情报在法证调查中的角色，以及相关的隐私和法规问题。 8. 教育与培训：提出培养网络威胁情报专业人员的必要性，以及教育和培训的最佳实践。通过阅读这本书，读者可以深入了解网络威胁情报的理论和实践，提升网络安全防护能力，为应对不断演进的网络威胁做好准备。

Machine Learning Aided Static Malware Analysis: A Survey and Tutorial 9

function calls, header sections, etc. [83]. Such characteristics may reveal indicators

of malicious activity that are going to be used in CTI [57]. However, static analysis

is quite a slow process and requires a lot of human interpretation and hence [8].

Static analysis of PE32 is a many-sided challenge that was studied by different

authors. Static malware analysis also was used before for discovering interconnec-

tions in malware species for improved Cyber Threat Intelligence [42, 66]. As 32-bit

malware are still capable of infecting 64-bit platforms and considering there are

still many 32-bit Windows OS it is not surprising that still majority of Windows

malware are 32-bit Portable Executable ﬁles [8]. To authors knowledge there has not

been a comparative study of ML-based static malware using a single dataset which

produces comparable results. We believe that utilization of ML-aided automated

analysis can speed up intelligent malware analysis process and reduce human

interaction required for binaries processing. Therefore, there is a need for thorough

review of the relevant scientiﬁc contributions and offer a taxonomy for automated

static malware analysis.

The remainder of this paper is organized as follows. We ﬁrst offer a com-

prehensive review of existing literature in machine learning aided static malware

analysis. We believe this survey paves the way for further research in application

of machine learning in static malware analysis and calls for further development

in this ﬁeld. Then, taxonomy of feature construction methods for variety of static

characteristics and corresponding ML classiﬁcation methods is offered. Afterwards,

we offer a tutorial that applies variety of set of machine learning techniques and

compares their performance. The tutorial ﬁndings provide a clear picture of pros and

cons of ML-aided static malware analysis techniques. To equally compare all the

methods we used one benign and two malware datasets to evaluate all of the studied

methods. This important part complements the paper due to the fact that most of the

surveyed works used own collections, sometimes not available for public access or

not published at all. Therefore, experimental study showed performance comparison

and other practical aspects of ML-aided malware analysis. Section 4 gives an insight

into a practical routine that we used to establish our experimental setup. Analysis of

results and ﬁndings are given in Sect. 5. Finally, the paper is concluded and several

future works are suggested in Sect. 6.

2 An Overview of Machine Learning-Aided Static

Malware Detection

This section provides an analysis of detectable static properties of 32 bit PE malware

followed by detailed description of different machine learning techniques to develop

a taxonomy of machine learning techniques for static malware analysis.

10 A. Shalaginov et al.

2.1 Static Characteristics of PE Files

PE ﬁle format was introduced in Windows 3.1 as PE32 and further developed as

PE32+ format for 64 bit Windows Operating Systems. PE ﬁles contain a Common

Object File Format (COFF) header, standard COFF ﬁelds such as header, section

table, data directories and Import Address Table (IAT). Beside the PE header ﬁelds

a number of other static features can be extracted from a binary executable such as

strings, entropy and size of various sections.

To be able to apply Machine Learning PE32 ﬁles static characteristics should

be converted into a machine-understandable features. There exist different types of

features depending on the nature of their values such that numerical that describes a

quantitative measure (can be integer, real or binary value) or nominal that describes

ﬁnite set of categories or labels. An example of the numerical feature is CPU (in %)

or RAM (in Megabytes) usage, while nominal can be a ﬁle type (like ∗.dll or ∗.exe)

or Application Program Interface (API) function call (like write() or read()).

1. n-grams of byte sequences is a well-known method of feature construction

utilizing sequences of bytes from binary ﬁles to create features. Many tools

have been developed for this purpose such as hexdump [39] created 4-grams

from byte sequences of PE32 ﬁles. The features are collected by sliding window

of n bytes. This resulted in 200 millions of features using 10-grams for about

two thousands ﬁles in overall. Moreover, feature selection (FS) was applied to

select 500 most valuable features based on Information Gain metric. Achieved

accuracy on malware detection was up to 97% using such features. Another work

on byte n-grams [51] described usage of 100–500 selected n-grams yet on a

set of 250 malicious and 250 benign samples. Similar approach [31] was used

with 10,...,10, 000 best n-grams for n = 1,...,10. Additionally, ML methods

such that Naive Bayes, C4.5, k-NN and others were investigated to evaluate their

applicability and accuracy. Finally, a range of 1–8 n-grams [27] can result in 500

best selected n-grams that are used later to train AdaBoost and Random Forests

in addition to previously mentioned works.

2. Opcode sequences or operation codes are set of consecutive low level machine

abstractions used to perform various CPU operations. As it was shown [62]

such features can be used to train Machine Learning methods for successful

classiﬁcation of the malware samples. However, there should be a balance

between the size of the feature set and the length of n-gram opcode sequence.

n-Grams with the size of 4 and 5 result in highest classiﬁcation accuracy as

unknown malware samples could be unveiled on a collection of 17,000 malware

and 1000 benign ﬁles with a classiﬁcation accuracy up to 94% [58]. Bragen [6]

explored reliability of malware analysis using sequences of opcodes based on

the 992 PE-ﬁles malware and benign samples. During the experiments, about 50

millions of opcodes were extracted. 1-gram- and 2-gram-based features showed

good computational results and accuracy. Wang et al. [79] presented how the 2-

tuple opcode sequences can be used in combination with density clustering to

detect malicious or benign ﬁles.

Machine Learning Aided Static Malware Analysis: A Survey and Tutorial 11

3. API calls are the function calls used by a program to execute speciﬁc functional-

ity. We have to distinguish between System API calls that are available through

standard system DLLs and User API calls provided by user installed software.

These are designed to perform a pre-deﬁned task during invocation. Suspicious

API calls, anti-VM and anti-debugger hooks and calls can be extracted by PE

analysers such as PEframe [4]. Zabidi et al. [83] studied 23 malware samples

and found that some of the API calls are present only in malwares rather than

benign software. Function calls may compose in graphs to represent PE32 header

features as nodes, edges and subgraphs [84]. This work shows that ML methods

achieve accuracy of 96% on 24 features extracted after analysis of 1037 malware

and 2072 benign executables. Further, in [71] 20,682 API calls were extracted

using PE parser for 1593 malicious and benign samples. Such large number of

extracted features can help to create linearly separable model that is crucial for

many ML methods as Support Vector Machines (SVM) or single-layer Neural

Networks. Another work by [55] described how API sequences can be analysed

in analogy with byte n-grams and opcode n-grams to extract corresponding

features to classify malware and benign ﬁles. Also in this work, an array of

API calls from IAT (PE32 header ﬁled) was processed by Fisher score to select

relevant features after analysis of more than 34 k samples.

4. PE header represents a collection of meta data related to a Portable Executable

ﬁle. Basic features that can be extracted from PE32 header are Size of Header,

Size of Uninitialized Data, Size of Stack Reserve, which may indicate if a

binary ﬁle is malicious or benign [15]. The work [76] utilized Decision Trees

to analyse PE header structural information for describing malicious and benign

ﬁles. Ugarte-Pedrero et al. [77] used 125 raw header characteristics, 31 section

characteristics, 29 section characteristics to detect unknown malware in a semi-

supervised approach. Another work [80] used a dataset containing 7863 malware

samples from Vx Heaven web site in addition to 1908 benign ﬁles to develop

a SVM based malware detection model with accuracy of 98%. Markel and

Bilzor [38] used F-score as a performance metric to analyse PE32 header features

of 164,802 malicious and benign samples. Also [29] presented research of two

novel methods related to PE32 string-based classier that do not require additional

extraction of structural or meta-data information from the binary ﬁles. Moreover,

[84] described application of 24 features along with API calls for classiﬁcation of

malware and benign samples from VxHeaven and Windows XP SP3 respectively.

Further, ensemble of features was explored [59], where authors used in total 209

features including structural and raw data from PE32 ﬁle header. Further, Le-

Khac et al. [35] focused on Control Flow Change over ﬁrst 256 addresses to

construct n-gram features.

In addition to study of speciﬁc features used for malware detection, we analyzed

articles devoted to application of ML for static malware analysis published between

2000 and 2016, which covers the timeline of Windows NT family that are still in

use as depicted in Fig. 1. We can see that the number of papers that are relevant

Machine Learning Aided Static Malware Analysis: A Survey and Tutorial 13

where P(C

) is a prior probability of class C

,k = 1,...,m

which is calcu-

lated from collected statistics according to description of variables provided by

Kononenko et al. [32]. This method is considered to tackle just binary classiﬁcation

problem (benign against malicious) since it was originally designed as multinomial

classiﬁer. V =v

,...,v

 is the vector of attributes values that belongs to a

sample. In case of Naive Bayes input values should be symbolical, for example

strings, opcodes, instruction n-grams etc. P(V) is the prior probability of a sample

described with vector V . Having training data set and given vector V we count

how many samples contain equal values of attributes (e.g. based on the number of

sections or given opcode sequence). It is important to mention that V have not to be

of length of full attribute vector and can contain only one attribute value. P(V|C

)

is the conditional probability of a sample described with V given the class C

.And

P(C

|V) conditional probability of class C

with V . Based on simple probability

theory we can describe conditional independence of attribute values v

given the

class C

P(V|C

) = P(v

∧ ...∧ v

) =



i=1

P(v

) (2)

Dropping the mathematical operations we get ﬁnal version of Eq. (1):

P(C

|V)= P(C

)



i=1

P(C

)

P(C

)

(3)

So, the task of this machine learning algorithm is to calculate conditional and

unconditional probabilities as described in Eq. (3) using a training dataset. To be

more speciﬁc, the Algorithm 1 pseudo code shows the calculation of the conditional

probability.

So, we can see from Eq. (1) that given output is as a probability that a questioned

software sample belongs to one or another class. Therefore, the classiﬁcation

decision will be made by ﬁnding a maximal value from set of corresponding class

likelihoods. Equation (4) provides formula that assigns class label to the output:

ˆy = argmax

k∈{1,...,K}

P(C

)P (V |C

). (4)

Bayesian Networks is a probabilistic directed acyclic graphical model (some-

times also named as Bayesian Belief Networks), which shows conditional depen-

dencies using directed acyclic graph. Network can be used to detect “update

knowledge of the state of a subset of variables when other variables (the evidence

variables) are observed” [32]. Bayesian Networks are used in many cases of

classiﬁcation and information retrieval (such as semantic search). The method’s

routine can be described as following. If edge goes from vertex A to vertex B,

then A is a parent of B, and B is an ancestor of A. If from A there is oriented path

剩余333页未读，继续阅读

minorsnow

粉丝: 0
资源: 3

人工智能与网络安全：Cyber Threat Intelligence探索

Collaborative Cyber Threat Intelligence azw3

网络安全cyber threat

Collaborative Cyber Threat Intelligence 无水印pdf

threat intelligence

Wos、DBLP、IEEExplorer、Springer、Elservier、Wiley有哪些关于人工智能和机器学习用于威胁检测和响应的文献

hands-on artificial intelligence for cybersecurity

我把-I/path/to/cyber/includes更改为 cyber/cyber.h 时终端显示cyber/cyber.h: No such file or directory怎么办？

为什么在cyber的docker容器内使用source cyber/setup.bash命令终端没有反应？

Cyber RT 非docker环境移植

为什么ubuntu中在cyber的docker容器内使用source cyber/setup.bash命令终端没有反应？

最新资源