Deep Multi-Task Learning for Aspect Term Extraction
with Memory Interaction
∗
Xin Li and Wai Lam
Key Laboratory on High Confidence Software Technologies (Sub-Lab, CUHK),
Ministry of Education, and
Department of Systems Engineering and Engineering Management
The Chinese University of Hong Kong, Hong Kong
{lixin, wlam}@se.cuhk.edu.hk
Abstract
We propose a novel LSTM-based deep
multi-task learning framework for aspect
term extraction from user review sen-
tences. Two LSTMs equipped with ex-
tended memories and neural memory op-
erations are designed for jointly handling
the extraction tasks of aspects and opin-
ions via memory interactions. Sentimental
sentence constraint is also added for more
accurate prediction via another LSTM.
Experiment results over two benchmark
datasets demonstrate the effectiveness of
our framework.
1 Introduction
The aspect-based sentiment analysis (ABSA) task
is to identify opinions expressed towards specific
entities such as laptop or attributes of entities such
as price (Liu, 2012a). This task involves three sub-
tasks: Aspect Term Extraction (ATE), Aspect Po-
larity Detection and Aspect Category Detection.
As a fundamental subtask in ABSA, the goal of
the ATE task is to identify opinionated aspect ex-
pressions. One of most important characteristics
is that opinion words can provide indicative clues
for aspect detection since opinion words should
co-occur with aspect words. Most publicly avail-
able datasets contain the gold standard annotations
for opinionated aspects, but the ground truth of
the corresponding opinion words is not commonly
provided. Some works tackling the ATE task ig-
nore the consideration of opinion words and just
focus on aspect term modeling and learning (Jin
∗
The work described in this paper is substantially sup-
ported by a grant from the Research Grant Council of the
Hong Kong Special Administrative Region, China (Project
Code: 14203414). We thank Lidong Bing and Piji Li for their
helpful comments on this draft and the anonymous reviewers
for their valuable feedback.
et al., 2009; Jakob and Gurevych, 2010; Toh and
Wang, 2014; Chernyshevich, 2014; Manek et al.,
2017; San Vicente et al., 2015; Liu et al., 2015;
Poria et al., 2016; Toh and Su, 2016; Yin et al.,
2016). They fail to leverage opinion information
which is supposed to be useful clues.
Some works tackling the ATE task con-
sider opinion information (Hu and Liu, 2004a,b;
Popescu and Etzioni, 2005; Zhuang et al., 2006;
Qiu et al., 2011; Liu et al., 2012b, 2013a,b, 2014)
in an unsupervised or partially supervised manner.
Qiu et al. (2011) proposed Double Propagation
(DP) to collectively extract aspect terms and opin-
ion words based on information propagation over
a dependency graph. One drawback is that it heav-
ily relies on the dependency parser, which is prone
to generate mistakes when applying on informal
online reviews. Liu et al. (2014) modeled relation
between aspects and opinions by constructing a bi-
partite heterogenous graph. It cannot perform well
without a high-quality phrase chunker and POS
tagger reducing its flexibility. As unsupervised or
partially supervised frameworks cannot take the
full advantages of aspect annotations commonly
found in the training data, the above methods lead
to deficiency in leveraging the data. Recently,
Wang et al. (2016) considered relation between
opinion words and aspect words in a supervised
model named RNCRF. However, RNCRF tends to
suffer from parsing errors since the structure of the
recursive network hinges on the dependency parse
tree. CMLA (Wang et al., 2017a) used a multi-
layer neural model where each layer consists of
aspect attention and opinion attention. However
CMLA merely employs standard GRU without ex-
tended memories.
We propose MIN (Memory Interaction Net-
work), a novel LSTM-based deep multi-task learn-
ing framework for the ATE task. Two LSTMs
with extended memory are designed for handling