![](https://csdnimg.cn/release/download_crawler_static/86288216/bg4.jpg)
2327-4662 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2019.2952146, IEEE Internet of
Things Journal
IEEE INTERNET OF THINGS JOURNAL, VOL. XX, NO. XX 4
privacy provided by the data/results released after perturbation.
Composition is the degradation of privacy when multiple
differentially private algorithms are performed on the same or
overlapping datasets [18]. According to DP definitions, when
two DP algorithms ε
1
-DP and ε
2
-DP are applied to the same
or overlapping datasets, the union of the results is equal to
(ε
1
+ ε
2
)-DP [18]. The more DP algorithms are applied to the
same data, the more privacy loss is accumulated. Depending
on the process of synthesis, DP algorithms can be categorized
into the two types; basic algorithms or derived algorithms [19].
Differential privacy is self-contained in basic algorithms while
the derived algorithms are derived from existing methods
by applying the theories of composition and postprocessing
invariance.
v_1
v_2
v_n
Binary
sequences
Flattened
layer
Merged
binary
sequence
1
0
1
1
0
1
1
0
Randomized
Response
Randomized
binary
sequence
0
0
1
0
0
0
1
1
ANN
DP output
Input Image
Convolution
+
Pooling
Flattening
Convolutional Module ANN ModuleLATENT
Inputs
convolution + pooling layers
Feature Mapping and Extraction
Flattening
Classification
Fully Connected
Layers
Output
Fig. 2: Generic architecture of a convolutional neural network
F. Deep Learning Using Convolutional Neural Networks
A CNN is commonly trained to recognize essential features
of images. As shown in Fig. 2, a CNN uses a collection of
layers named convolution layers with large receptive fields.
A sequence of steps through this stack of convolution layers
is followed by an intermediate functionality called pooling to
reduce the dimensions from the previous layer to the next
layer. The final pooled output which is produced from the
last convolution layer is flattened to produce a sizeable 1-D
vector [20]. Then a fully connected artificial neural network
(ANN) is trained using these input vectors to generate pre-
dictions (classifications) on the inputs (images). An ANN is
more or less a connected network of processing modules called
neurons, each producing a sequence of real-valued activations.
Overfitting happens when the training accuracy is signifi-
cantly higher than the testing accuracy [20], [21]. Good quality
models avoid overfitting. Regularization, image augmentation,
and hyperparameter tuning are three of the commonly used
concepts to avoid overfitting and improve the performance
and robustness of neural networks [20], [21]. Regularization
is the process of applying any modification to a learning
algorithm to reduce the generalization error. Regularization
can be achieved using dropouts, where a certain percentage of
neurons are randomly dropped in each epoch (training cycle)
to avoid overfitting. Image augmentation is a data preparation
technique, which uses the existing input images in the training
dataset and manipulates them to create many altered versions
of the same input using different transformation methods
such as reflection, sheer, and rotation. This technique allows
an ANN to learn a wider variety of inputs to make the
trained model more generalizable with high robustness [22]. In
hyperparameter tuning the inputs to hyperparameters such as
percentage dropouts, batch size, activation functions, number
of neurons, number of epochs, and optimizer are changed
under different training phases to identify the best case study
that returns the best results [20], [21]. Batch size is the
number of training examples that are going to be propagated in
one forward/backward pass [21]. Activation functions define
the output of a particular neuron given a set of inputs that,
with corresponding weights, introduce non-linear properties
to the network [20]. A neuron (also called a node) is the
primary component of an artificial neural network. A single
pass in which the entire dataset is introduced forward and
backward through the neural network is called an epoch [20].
An optimizer (or an optimization algorithm) is used to update
the model parameters such as weights and bias values [20]
G. Amalgamation of SDN and NFV in Edge-Cloud Interplay
SDN and NFV are two types of programmable infrastruc-
tures that improve the versatility of networking. Both technolo-
gies work based on the concept of creating virtual instances
for network functionalities where SDN virtualizes controlling
aspects, and NFV virtualizes essential network functions such
as encrypting channels. The amalgamation between SDN and
NFV can bring forward many advanced capabilities in terms
of flexibility, efficiency, and quality of service with increased
reconfigurability. SDN-controlled NFV can introduce a series
of virtualizations that can benefit the edge-cloud interplay,
which can improve the security and quality of service of
communication between local devices and cloud servers [23],
[24].
III. OUR APPROACH: LATENT
This section discusses the differentially private mechanism
employed in LATENT for deep learning. LATENT can be
classified as a derived differentially private algorithm which
is based on the randomized response technique. LATENT
uses two properties of differential privacy: postprocessing
invariance and composition when applying differential privacy
to a CNN. LATENT uses regularization, image augmenta-
tion, and hyperparameter tuning to optimize its performance
under noisy input conditions in the randomization process.
We implemented and tested LATENT on convolutional neural
networks using the Python Keras neural networks API, which
runs on top of the TensorFlow dataflow engine developed by
Google [3], [25]. Keras provides a high-level neural-network
API designed primarily for fast experimentation.
A. Introduction of the Intermediate Layer (LATENT) to Inject
Differential Privacy into the CNN Architecture
As shown in Fig. 3, we divide the structure of a convolu-
tional neural network into two main modules, and introduce an
intermediate module of randomization into the CNN structure.
Recall (described in Section II-F) that in CNN, the input
features are initially subjected to dimensionality reduction,
using a collection of convolutional layers and pooling layers.
The output of the final pooling layers is flattened into a single
1-D array before feeding it to a fully connected artificial neural