Chinese Journal of Electronics
Vol.28, No.1, Jan. 2019
A Text Sentiment Classification Modeling
Method Based on Coordinated
CNN-LSTM-Attention Model
∗
ZHANG Yangsen
1,2
, ZHENG Jia
1
, JIANG Yuru
1,2
, HUANG Gaijuan
1,2
and CHEN Ruoyu
1,2
(1. Institute of Intelligent Information Processing, Beijing Information Science and Technology University,
Beijing 100192, China)
(2. Beijing Laboratory of National Economic Security Early-Warning Engineering,Beijing 100192,China)
Abstract — The major challenge that text sentiment
classification modeling faces is how to capture the
intrinsic semantic, emotional dependence information and
the key part of the emotional expression of text. To
solve this problem, we proposed a Coordinated CNN-
LSTM-Attention(CCLA) model. We learned the vector
representations of sentence with CCLA unit. Semantic and
emotional information of sentences and their relations are
adaptively encoded to vector representations of document.
We used softmax regression classifier to identify the
sentiment tendencies in the text. Compared with other
methods, the CCLA model can well capture the local
and long distance semantic and emotional information.
Experimental results demonstrated the effectiveness of
CCLA model. It shows superior performances over several
state-of-the-art baseline methods.
Key words — Coordinated CNN-LSTM-Attention,
Sentiment analysis, Text modeling, Semantic information.
I. Introduction
Text sentiment classification modeling is a funda-
mental problem in the field of Nature language processing
(NLP) and is a crux to understand user intention
in product reviews or social networks
[1,2]
. The core
of text sentiment classification modeling is to capture
semantic features from variable-length text units. As a
traditional method, the bag-of-words model
[3]
is the most
common and popular vector representations method for
texts because of its efficiency, simplicity and surprising
accuracy. But the bag-of-words model treats sentence or
document as an unordered collection of words. Lacking
word order, different sentences can have the exactly same
representation, given that the same words are used.
Until now, some machine learning algorithms have
achieved good results on text sentiment classification
modeling
[4]
, but with the deep learning models have
achieved remarkable effects in the field of speech
recognition and computer vision in recent years, order-
sensitive models based on the neural networks model such
as Recursive neural networks (RNNs), Recurrent neural
networks (RNN), Convolutional neural networks (CNN),
Long short-term memory (LSTM) and attention model
are becoming increasingly popular due to their ability
to capture word order information and further learn
the semantic and emotional information from text. Deep
learning comes from traditional neural network models.
It is not just a multi-layer network but emphasizes the
extraction of hidden features and higher-level abstract
features.
II. Related Work
1. Deep learning model
RNNs have been proved effective in modeling text
semantics
[5−7]
. However, it need to construct semantic
tree and its performance depends on the accuracy of the
semantic tree. But, the semantic relationship between
two sentences may not be able to form a tree structure.
RNN do not need to build the semantic tree
[8]
and it
can capture the context information over long distances.
However, RNN is a bias model, or to be more specific, a
positive model, in which the relatively backward words
in the text occupy a more dominant position. At the
same time, RNN also have the problem of exploding and
vanishing gradient.
In order to solve the semantic bias problem of RNN,
it is proposed to use CNN for text semantic modeling.
∗
Manuscript Received Aug. 4, 2017; Accepted May 29, 2018. This work is supported by the National Natural Science
Foundation of China (No.61772081, No.61602044) and the Science and Technology Development Project of Beijing Municipal Education
Commission(No.KM201711232014).
© 2019 Chinese Institute of Electronics. DOI:10.1049/cje.2018.11.004