深度学习中的局部差分隐私技术

需积分: 0 96 浏览量更新于2024-07-01 收藏 1.4MB PDF 举报

"局部差分隐私在深度学习中的应用-谢荣1" 文章“Local Differential Privacy for Deep Learning”探讨了在物联网（IoT）环境中，如何利用局部差分隐私（Local Differential Privacy, LDP）技术来保护用户数据隐私，同时进行有效的深度学习（Deep Learning, DL）训练。物联网的发展正在各个行业中掀起变革，包括医疗、农业、金融、能源和交通等，而这些领域的数据往往涉及到大量的敏感信息。随着软件定义网络（Software-Defined Networking, SDN）和网络功能虚拟化（Network Function Virtualization, NFV）等技术在边缘计算与云计算交互中的应用，数据的收集和处理变得更加便捷，但同时也带来了隐私保护的挑战。深度学习因其在大数据集上表现出的高精度而受到广泛关注，但这种依赖大量数据的特性也意味着可能侵犯到个人隐私。局部差分隐私提供了一种方法，允许数据在被收集时添加随机噪声，使得单个个体的数据贡献变得难以识别，从而保护了个人隐私。LDP的核心思想是在不牺牲模型性能的前提下，最大化数据匿名性。文章可能详细介绍了LDP的原理，包括它如何通过调整噪声水平来平衡隐私保护和学习效率，以及如何在分布式环境下应用LDP。作者可能会讨论LDP算法的具体实现，如差分隐私随机梯度下降（Differentially Private Stochastic Gradient Descent, DPSGD），这种算法在更新模型参数时注入噪声，以确保训练过程中的隐私。此外，文章可能会探讨LDP在实际IoT应用中的挑战，比如如何在有限的计算资源和通信带宽下实现高效的LDP机制，以及如何处理非独立同分布（non-i.i.d.）的数据。还可能涉及LDP与其他隐私保护技术（如联邦学习）的比较，分析各自的优缺点和适用场景。最后，论文可能会提出一些未来的研究方向，如改进LDP算法以提高在复杂深度学习模型上的性能，或者开发新的隐私增强技术，以适应不断变化的物联网环境和更严格的隐私法规。这篇文章对于理解如何在保护用户隐私的同时利用深度学习进行数据分析和模型训练具有重要的理论和实践意义，对于物联网行业的隐私保护策略制定者和技术开发者来说是一份宝贵的参考资料。

2327-4662 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2019.2952146, IEEE Internet of

Things Journal

IEEE INTERNET OF THINGS JOURNAL, VOL. XX, NO. XX 4

privacy provided by the data/results released after perturbation.

Composition is the degradation of privacy when multiple

differentially private algorithms are performed on the same or

overlapping datasets [18]. According to DP deﬁnitions, when

two DP algorithms ε

-DP and ε

-DP are applied to the same

or overlapping datasets, the union of the results is equal to

(ε

+ ε

)-DP [18]. The more DP algorithms are applied to the

same data, the more privacy loss is accumulated. Depending

on the process of synthesis, DP algorithms can be categorized

into the two types; basic algorithms or derived algorithms [19].

Differential privacy is self-contained in basic algorithms while

the derived algorithms are derived from existing methods

by applying the theories of composition and postprocessing

invariance.

v_1

v_2

v_n

Binary

sequences

Flattened

layer

Merged

binary

sequence

Randomized

Response

Randomized

binary

sequence

ANN

DP output

Input Image

Convolution

Pooling

Flattening

Convolutional Module ANN ModuleLATENT

Inputs

convolution + pooling layers

Feature Mapping and Extraction

Flattening

Classiﬁcation

Fully Connected

Layers

Output

Fig. 2: Generic architecture of a convolutional neural network

F. Deep Learning Using Convolutional Neural Networks

A CNN is commonly trained to recognize essential features

of images. As shown in Fig. 2, a CNN uses a collection of

layers named convolution layers with large receptive ﬁelds.

A sequence of steps through this stack of convolution layers

is followed by an intermediate functionality called pooling to

reduce the dimensions from the previous layer to the next

layer. The ﬁnal pooled output which is produced from the

last convolution layer is ﬂattened to produce a sizeable 1-D

vector [20]. Then a fully connected artiﬁcial neural network

(ANN) is trained using these input vectors to generate pre-

dictions (classiﬁcations) on the inputs (images). An ANN is

more or less a connected network of processing modules called

neurons, each producing a sequence of real-valued activations.

Overﬁtting happens when the training accuracy is signiﬁ-

cantly higher than the testing accuracy [20], [21]. Good quality

models avoid overﬁtting. Regularization, image augmentation,

and hyperparameter tuning are three of the commonly used

concepts to avoid overﬁtting and improve the performance

and robustness of neural networks [20], [21]. Regularization

is the process of applying any modiﬁcation to a learning

algorithm to reduce the generalization error. Regularization

can be achieved using dropouts, where a certain percentage of

neurons are randomly dropped in each epoch (training cycle)

to avoid overﬁtting. Image augmentation is a data preparation

technique, which uses the existing input images in the training

dataset and manipulates them to create many altered versions

of the same input using different transformation methods

such as reﬂection, sheer, and rotation. This technique allows

an ANN to learn a wider variety of inputs to make the

trained model more generalizable with high robustness [22]. In

hyperparameter tuning the inputs to hyperparameters such as

percentage dropouts, batch size, activation functions, number

of neurons, number of epochs, and optimizer are changed

under different training phases to identify the best case study

that returns the best results [20], [21]. Batch size is the

number of training examples that are going to be propagated in

one forward/backward pass [21]. Activation functions deﬁne

the output of a particular neuron given a set of inputs that,

with corresponding weights, introduce non-linear properties

to the network [20]. A neuron (also called a node) is the

primary component of an artiﬁcial neural network. A single

pass in which the entire dataset is introduced forward and

backward through the neural network is called an epoch [20].

An optimizer (or an optimization algorithm) is used to update

the model parameters such as weights and bias values [20]

G. Amalgamation of SDN and NFV in Edge-Cloud Interplay

SDN and NFV are two types of programmable infrastruc-

tures that improve the versatility of networking. Both technolo-

gies work based on the concept of creating virtual instances

for network functionalities where SDN virtualizes controlling

aspects, and NFV virtualizes essential network functions such

as encrypting channels. The amalgamation between SDN and

NFV can bring forward many advanced capabilities in terms

of ﬂexibility, efﬁciency, and quality of service with increased

reconﬁgurability. SDN-controlled NFV can introduce a series

of virtualizations that can beneﬁt the edge-cloud interplay,

which can improve the security and quality of service of

communication between local devices and cloud servers [23],

[24].

III. OUR APPROACH: LATENT

This section discusses the differentially private mechanism

employed in LATENT for deep learning. LATENT can be

classiﬁed as a derived differentially private algorithm which

is based on the randomized response technique. LATENT

uses two properties of differential privacy: postprocessing

invariance and composition when applying differential privacy

to a CNN. LATENT uses regularization, image augmenta-

tion, and hyperparameter tuning to optimize its performance

under noisy input conditions in the randomization process.

We implemented and tested LATENT on convolutional neural

networks using the Python Keras neural networks API, which

runs on top of the TensorFlow dataﬂow engine developed by

Google [3], [25]. Keras provides a high-level neural-network

API designed primarily for fast experimentation.

A. Introduction of the Intermediate Layer (LATENT) to Inject

Differential Privacy into the CNN Architecture

As shown in Fig. 3, we divide the structure of a convolu-

tional neural network into two main modules, and introduce an

intermediate module of randomization into the CNN structure.

Recall (described in Section II-F) that in CNN, the input

features are initially subjected to dimensionality reduction,

using a collection of convolutional layers and pooling layers.

The output of the ﬁnal pooling layers is ﬂattened into a single

1-D array before feeding it to a fully connected artiﬁcial neural

剩余15页未读，继续阅读

yxldr

粉丝: 22
资源: 327

深度学习中的局部差分隐私技术

manipulation attacks in local differential privacy

使用差分隐私技术对图像进行保护的参考文献

a deep learning library for solving differential equations

写一篇轮式机器人轨迹跟踪文献综述

总结一下DeepXDE: A Deep Learning Library for Solving Differential Equations文章

用神经网络代码实现 Monte Carlo fPINNs: Deep learning method for forward and inverse problems involving high dimensional fractional partial differential equations

如果我想写一篇关于AI算法安全的技术调研，你推荐我使用那些文献

载波相位差分法高质量论文推荐

Diffusion PDE (Partial Differential Equation) Solver using Split-Operator Method也叫dpm吗

最新资源