Deep Learning in Automatic Text Summarization
Som Gupta and S.K Gupta
somi.11ce@gmail.com, guptask_biet@rediffmail.com
Research Scholar AKTU Lucknow, Computer Science Department BIET Jhansi
F
Abstract—Exponential increase of amount of data has led to the need
of automatic text summarization approaches to reduce the manual effort
and save time of searching the relevant information. Machine Learning,
Natural Language Processing and Unsupervised approaches have been
widely applied successfully for automatic summarization problem. Deep
Learning is an emerging data-driven approach which has outperformed
all the above mentioned traditional approaches and in combination with
these traditional approaches, gives good results in terms of redundancy
and coverage. Seq2Seq encoder-decoder models are the most widely
used deep learning models for the purpose of summarization. But the
need of huge training dataset is one of the big challenge in this field.
Lot of researchers are working on this issue. The aim of the paper is
to give a brief overview of the recent works done using deep learning
in the field of text summarization, give a brief introduction to various
techniques being used while using deep learning, list down the various
challenges and various datasets being used to perform this task.
Keywords—Recurrent Neural Networks; Convolutional Neural Net-
works, deep learning; Neural Networks; Attention Models;
I INTRODUCTION
Automatic Text summarization, is to shorten the amount of
text in the document without losing the essence, is one of
the most challenging and a time-consuming activity because
of the complexity of natural language processing involved
with different kind of data. There are number of techniques
available to perform automatic text summarization and are
classified broadly into extractive and abstractive. Extractive
approaches are where the summarization is performed by
extracting the important sentences and it is mostly done
by using the features from the text and processing them
with soft computing techniques like fuzzy logic, genetic pro-
gramming, machine learning or neural networks. Whereas,
abstractive approaches are where the new sentences are
generated for consideration into the summary and is mostly
performed by considering the semantic information present
in the sentences of the text.
With the increasing amount of data available due to the
emergence of online platforms, the challenge is to produce
the automatic summaries which are grammatically correct,
less redundant, coherent and good in coverage. Deep Learn-
ing is one of the emerging machine learning approach which
uses neural networks inside it with multiple-layers of hid-
den layers to perform summarization which helps solve the
above mentioned challenges. Deep Learning involves the
use of neural networks, where the input is fed to the system
and then the input goes to various hidden layers for fine-
tuning and then finally the output is obtained. Mostly non-
linear processing is performed at the hidden layers to obtain
the output. Deep learning models help reduce the semantic
space of the text. Researchers have proved that the deep
learning models outperform than the existing approaches in
terms of both coherency and linguistic quality.
The paper is organized in this way: Section 2 describes
the various deep learning techniques which have been used
to perform text summarization. Section 3 describes the
various works done using deep learning techniques and
are being classified into extractive and abstractive. Section
4 describes the various evaluation measures which have
been used to perform summarization. Section 5 lists down
the various challenges and future areas for researchers. And
finally the conclusion.
II TECHNIQUES USED
Mostly the Restricted Boltzmann Machine, Se-
quence2Sequence Models using Encoder-Decoder approach
and Unsupervised approaches have been used for
summarization purpose and are described as below:
II.1 Restricted Boltzmann Machine, RBM
It consists of three layers input, hidden and output layer.
It helps remove the redundant information. Verma et al.
[1] used the feature-based extraction along with the Re-
stricted Boltzmann Machine(RBM) where they used one
hidden layer and 9 perceptrons in each layer, to create the
extractive summaries. Liu et al. [2] used Support Vector
Machines(SVM) and Deep Belief networks to perform multi-
document query-specific text summarization. They have
used Restricted Boltzmann Machines(RBM) in the hidden
layers. Yao et al. [3] have also used the Restricted Boltzmann
Machine to perform summarization using 1 visible and 4
hidden layers.
II.2 Sequence-to-Sequence Models
When the input and output are both the sequence of words,
it is called as Sequence to Sequence problem. Like in text
summarization, the input is in the form of string which is a
sequence of words and the output is also a text summary
which is again the sequence of words. They consist of
two parts namely encoder and decoder. Input is fed to the
International Journal of Computer Science and Information Security (IJCSIS),
Vol. 16, No. 11, November 2018
https://sites.google.com/site/ijcsis/
ISSN 1947-5500