机器学习生产系统的网络安全风险评估

版权申诉

57 浏览量更新于2024-07-06 收藏 2.09MB PDF 举报

"这篇文档是关于评估真实世界中机器学习生产系统的网络安全风险的研究。作者来自以色列本古里安大学和日本NEC公司，探讨了针对机器学习系统的威胁分析，包括资产识别、威胁模型、攻击类型以及已知的对抗性机器学习（Adversarial Machine Learning, AML）攻击实例。" 在当前数字化的时代，机器学习已经成为许多关键业务的核心部分，其生产系统处理大量敏感数据，并在决策过程中起着至关重要的作用。然而，随着技术的进步，网络安全威胁也日益严重，尤其是针对机器学习系统的攻击。这篇文章深入研究了这些潜在的风险。首先，文章列举了典型机器学习生产系统中的关键资产，这包括训练和测试数据、模型本身、基础设施（如服务器和网络设备）以及依赖的软件和服务。这些资产的价值在于它们能直接影响到系统的性能和安全性。其次，作者构建了一个威胁模型，描述了可能的对手（例如黑客、竞争对手或内部威胁者）、他们的能力（如数据篡改、模型注入或网络渗透）以及他们的目标（如窃取信息、破坏服务或影响决策结果）。这个模型对于理解攻击者的动机和行为模式至关重要。接着，文章详细指出了多种威胁机器学习系统的方式，包括数据中毒（Data Poisoning）、模型盗窃（Model Stealing）和对抗性样本（Adversarial Examples）等。这些威胁可能导致模型失效、数据泄露或者被用于恶意目的。为了量化对抗性机器学习威胁的风险，作者引入了一种新的方法来评估这些风险。这种方法考虑了攻击的可行性、影响程度以及防御的难易程度，为安全从业者提供了一种衡量和管理风险的工具。此外，文章回顾了大量的实际攻击案例，这些案例展示了理论威胁如何在现实中得以实现。通过这些案例，读者可以更好地理解攻击的复杂性和多样性，以及如何预防和应对这些威胁。最后，文章讨论了现有的防护策略和未来的研究方向，强调了在设计和实施机器学习系统时应集成安全措施的重要性。整体而言，这篇论文为理解和应对机器学习生产系统的网络安全风险提供了全面的框架和指导，对从事相关领域工作的人员具有很高的参考价值。

Evaluating the Cybersecurity Risk of Real World, Machine Learning Production Systems 7

this process is commonly raw data, and predeﬁned labeling logic, and the output consists

of the labels of each data sample.

[A7] Feature Selection.

This component is responsible for selecting the subset of features

that will be used for training the model.

[A8] Model Training.

This component is responsible for training ML algorithms with

prepared data. The output of this step is a trained model.

[A9] Model.

The trained model ﬁle, which is produced by feeding training data to the

learning algorithm.

[A10] Model Repository.

This component is a database responsible for storing models,

their performance, versioning, and other conﬁguration information (e.g., features used,

hyperparameter values).

[A11] Model Evaluation (ofﬂine).

This component is responsible for evaluating, using

various measurements, the performance of a trained model on an unseen ofﬂine dataset

(known as a test set).

[A12] Model Validation (ofﬂine).

This component is responsible for validating that the

new model is adequate for deployment (i.e., is better than a certain baseline).

[A13] Model Validation (online).

This component (also known as A/B testing) is responsi-

ble for evaluating, the performance of a trained model, on a small portion of unseen online

data. The performance of the existing and new models are then compared, and the superior

model is selected for serving.

[A14] Model Serving.

This component is responsible for deploying the model to provide

predictions (e.g., using a REST API).

[A15] Model Performance Monitoring.

This component is responsible for actively moni-

toring model performance to detect performance degradation.

3.2 Threat Model

In this section, we deﬁne the threat model, which is based on the attacker’s access capabili-

ties and the knowledge regarding the target system.

3.2.1 Aacker’s access capabilities. The set of capabilities accessible to the attacker regarding

the target system (see Figure 3).

[AC1] Pipeline Access.

In this threat model, we assume an adversary with access to all

of the components included in the target pipeline, including the training data, algorithm,

hyperparameters, model, and classiﬁcation.

[AC2] Model Access.

In this threat model, we assume an adversary that has access to the

exact model used in the target pipeline.

[AC3] Prediction Access.

In this threat model, we assume an adversary that has access to

the outputs (predictions) of the model used in the target pipeline.

[AC4] Raw Data Access.

In this threat model, we assume an adversary with access to

the raw data (this data will be used for training the model after applying some data

transformations).

[AC5] Training Data Access.

In this threat model, we assume an adversary with access to

the dataset used for training the target model.

[AC6] Labeled Data Access.

In this threat model, we assume an adversary with access to

the labels of the dataset used for training the target model.

[AC7] Validation Data Access.

In this threat model, we assume an adversary with access

to the dataset used for the external validation of the target model.

8 Bion, et al.

Data Lake

Sample

0.1

0.2

0.1

0.8

No Entrey

Input

Layer

Hidden Layers

Output

Layer

Descision-Based

Query Access [AC9]

(read only)

Prediction Access [AC3]

(read/modify)

[AC4] Raw Data

Access

(read/write)

DescisionInferenceSensor Data

[AC1] Pipeline Access

(complete access)

Model Access [AC2]

Score-Based Query Access [AC9]

(read only)

Raw Data Processed Data

Labled Dataset

Data

Transformation

Camera

Object

[AC1] Pipeline Access

(limited access)

[AC5] Training

Data Access

(read/write)

[AC6] Labeled

Data Access

(read/write)

Sensor Data Access [AC10]

Training

Validation

AI-Based Application

Fig. 3. An illustration of the various access capabilities available to the adversary for dierent threat models.

[AC8] Surrogate Data Access.

In this threat model, we assume an adversary that has access

to a reference dataset with similar characteristics and data distribution to the dataset used

to train the model.

[AC9] Score-Based Query Access.

In this threat model, we assume an adversary with the

ability to query the trained model. We further assume that given a query, the attacker has

access to the probability vector (i.e., score), which describes the conﬁdence for each class.

However, we do not assume the adversary has access to the training data.

[AC10] Decision-Based Query Access.

In this threat model, we assume an adversary with

the ability to query the trained model. We further assume that given a query, the attacker

has access only to the decision. However, we do not assume the adversary has access to the

probability vector or training data.

[AC11] Sensor Data Access.

In this threat model, we assume an adversary with the ability

to manipulate the data captured/measured by the sensors (e.g., an adversary that can

manipulate an object that resides within the viewpoint of a camera).

3.2.2 Aacker’s knowledge. Information regarding the target system that is available to the

attacker and can be used to more effectively generate the attacks.

[AK1] Perfect Knowledge.

In this threat model, we assume an adversary with complete

knowledge on the target pipeline, including the training data, algorithm, hyperparameters,

model, and classiﬁcation.

[AK2] Model Knowledge.

In this threat model, we assume an adversary that knows the

exact model used in the target pipeline (e.g., the target pipeline uses a public model).

[AK3] Hyperparameter Knowledge.

In this threat model, we assume an adversary that

knows the exact algorithm and hyperparameters used to train the algorithm. For exam-

ple, for artiﬁcial neural networks, the hyperparameters include the network architecture,

number of epochs used to train the model, selected learning rate, etc.

[AK4] Algorithm Knowledge.

In this threat model, we assume an adversary that knows

the algorithm used to train the model, but does not know the exact hyperparameters.

剩余35页未读，继续阅读

易小侠

粉丝: 6547
资源: 9万+

机器学习生产系统的网络安全风险评估

基于AHP的洪水风险评估模型中权重敏感性的评估_Evaluating the weight sensitivity in AHP

Risk Management

GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection

error evaluating 'initfcn' callback of block_diagram

网上售房管理系统相关外国文献

There was an error in evaluating the Pre-request Script:TypeError: Cannot read properties of undefined (reading 'headers')

from sklearn.model_selection import train_test_split

Do you agree or disagree with the following statement? The most important aspect of a job is the money a person earns.Use specific reasons and examples to support your opinion. 请以不同意的观点写写英文120字

from sklearn.metrics import confusion_matrix

最新资源