A Neural Network Architecture Combining Gated Recurrent
Unit (GRU) and Support Vector Machine (SVM) for Intrusion
Detection in Network Traic Data
Abien Fred M. Agarap
abienfred.agarap@gmail.com
ABSTRACT
Gated Recurrent Unit (GRU) is a recently-developed variation of the
long short-term memory (LSTM) unit, both of which are variants
of recurrent neural network (RNN). Through empirical evidence,
both models have been proven to be eective in a wide variety of
machine learning tasks such as natural language processing[
23
],
speech recognition[
4
], and text classication[
24
]. Conventionally,
like most neural networks, both of the aforementioned RNN vari-
ants employ the Softmax function as its nal output layer for its
prediction, and the cross-entropy function for computing its loss.
In this paper, we present an amendment to this norm by introduc-
ing linear support vector machine (SVM) as the replacement for
Softmax in the nal output layer of a GRU model. Furthermore,
the cross-entropy function shall be replaced with a margin-based
function. While there have been similar studies[
2
,
22
], this proposal
is primarily intended for binary classication on intrusion detec-
tion using the 2013 network trac data from the honeypot systems
of Kyoto University. Results show that the GRU-SVM model per-
forms relatively higher than the conventional GRU-Softmax model.
The proposed model reached a training accuracy of
≈
81.54% and
a testing accuracy of
≈
84.15%, while the latter was able to reach a
training accuracy of
≈
63.07% and a testing accuracy of
≈
70.75%. In
addition, the juxtaposition of these two nal output layers indicate
that the SVM would outperform Softmax in prediction time - a
theoretical implication which was supported by the actual training
and testing time in the study.
CCS CONCEPTS
• Computing methodologies → Supervised learning by clas-
sication
;
Support vector machines
;
Neural networks
;
• Se-
curity and privacy → Intrusion detection systems;
KEYWORDS
articial intelligence; articial neural networks; gated recurrent
units; intrusion detection; machine learning; recurrent neural net-
works; support vector machine
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from permissions@acm.org.
ICMLC 2018, February 26–28, 2018, Macau, China
© 2018 Association for Computing Machinery.
ACM ISBN 978-1-4503-6353-2/18/02.. . $15.00
https://doi.org/10.1145/3195106.3195117
ACM Reference Format:
Abien Fred M. Agarap. 2018. A Neural Network Architecture Combin-
ing Gated Recurrent Unit (GRU) and Support Vector Machine (SVM) for
Intrusion Detection in Network Trac Data. In ICMLC 2018: 2018 10th
International Conference on Machine Learning and Computing, February
26–28, 2018, Macau, China. ACM, New York, NY, USA, 5 pages. https:
//doi.org/10.1145/3195106.3195117
1 INTRODUCTION
By 2019, the cost to the global economy due to cybercrime is pro-
jected to reach $2 trillion[
10
]. Among the contributory felonies to
cybercrime is intrusions, which is dened as illegal or unauthorized
use of a network or a system by attackers[
7
]. An intrusion detec-
tion system (IDS) is used to identify the said malicious activity[
7
].
The most common method used for uncovering intrusions is the
analysis of user activities[
7
,
13
,
17
]. However, the aforementioned
method is laborious when done manually, since the data of user
activities is massive in nature[
6
,
14
]. To simplify the problem, au-
tomation through machine learning must be done.
A study by Mukkamala, Janoski, & Sung (2002)[
17
] shows how
support vector machine (SVM) and articial neural network (ANN)
can be used to accomplish the said task. In machine learning, SVM
separates two classes of data points using a hyperplane[
5
]. On the
other hand, an ANN is a computational model that represents the
human brain, and shows information is passed from a neuron to
another[18].
An approach combining ANN and SVM was proposed by Alal-
shekmubarak & Smith[
2
], for time-series classication. Specically,
they combined echo state network (ESN, a variant of recurrent neu-
ral network or RNN) and SVM. This research presents a modied
version of the aforementioned proposal, and use it for intrusion
detection. The proposed model will use recurrent neural network
(RNNs) with gated recurrent units (GRUs) in place of ESN. RNNs
are used for analyzing and/or predicting sequential data, making it
a viable candidate for intrusion detection[
18
], since network trac
data is sequential in nature.
2 METHODOLOGY
2.1 Machine Intelligence Library
Google TensorFlow[
1
] was used to implement the neural network
models in this study – both the proposed and its comparator.
2.2 The Dataset
The 2013 Kyoto University honeypot systems’ network trac data[
20
]
was used in this study. It has 24 statistical features[
20
]; (1) 14 fea-
tures from the KDD Cup 1999 dataset[
21
], and (2) 10 additional
features, which according to Song, Takakura, & Okabe (2006)[
20
],
arXiv:1709.03082v8 [cs.NE] 7 Feb 2019