A Context-aware Aention Network for Interactive
estion Answering
∗
Huayu Li
1
, Martin Renqiang Min
2
, Yong Ge
3
, Asim Kadav
2
1
Department of Computer Science, UNC Charloe
2
Machine Learning Group, NEC Laboratories America
3
Management Information Systems, University of Arizona
hli38@uncc.edu,{renqiang,asim}@nec-labs.com,yongge@email.arizona.edu.
ABSTRACT
Neural network based sequence-to-sequence models in an encoder-
decoder framework have been successfully applied to solve es-
tion Answering (QA) problems, predicting answers from statements
and questions. However, almost all previous models have failed to
consider detailed context information and unknown states under
which systems do not have enough information to answer given
questions. ese scenarios with incomplete or ambiguous infor-
mation are very common in the seing of Interactive estion
Answering (IQA). To address this challenge, we develop a novel
model, employing context-dependent word-level aention for more
accurate statement representations and question-guided sentence-
level aention for beer context modeling. We also generate unique
IQA datasets to test our model, which will be made publicly avail-
able. Employing these aention mechanisms, our model accurately
understands when it can output an answer or when it requires gen-
erating a supplementary question for additional input depending
on dierent contexts. When available, user’s feedback is encoded
and directly applied to update sentence-level aention to infer an
answer. Extensive experiments on QA and IQA datasets quantita-
tively demonstrate the eectiveness of our model with signicant
improvement over state-of-the-art conventional QA models.
KEYWORDS
estion Answering; Interactive estion Answering; Aention;
Recurrent Neural Network
1 INTRODUCTION
With the availability of large-scale QA datasets, high-capacity ma-
chine learning/data mining models, and powerful computational
devices, research on QA has become active and fruitful. Commer-
cial QA products such as Google Assistant, Apple Siri, Amazon
Alexa, Facebook M, Microso Cortana, Xiaobing in Chinese, Rinna
in Japanese, and MedWhat have been released in the past several
years. e ultimate goal of QA research is to build intelligent sys-
tems capable of naturally communicating with humans, which
∗
Most of this work was done when the rst author was an intern at NEC Labs America.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permied. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from permissions@acm.org.
KDD’17, August 13–17, 2017, Halifax, NS, Canada.
© 2017 ACM. 978-1-4503-4887-4/17/08. . . $15.00
DOI: hp://dx.doi.org/10.1145/3097983.3098115
poses a major challenge for natural language processing and ma-
chine learning. Inspired by recent success of sequence-to-sequence
models with an encoder-decoder framework [
5
,
21
], researchers
have aempted to apply variants of such models with explicit mem-
ory and aention to QA tasks, aiming to move a step further from
machine learning to machine reasoning [
12
,
17
,
26
]. Similarly, all
these models employ encoders to map statements and questions
to xed-length feature vectors, and a decoder to generate outputs.
Empowered by the adoption of memory and aention, they have
achieved remarkable success on several challenging public datasets,
including the recently acclaimed Facebook bAbI dataset [24].
However, previous models suer from the following impor-
tant limitations [
12
,
17
,
25
,
26
]. First, they fail to model context-
dependent meaning of words. Dierent words may have dierent
meanings in dierent contexts, which increases the diculty of
extracting the essential semantic logic ow of each sentence in
dierent paragraphs. Second, many existing models only work
in ideal QA seings and fail to address the uncertain situations
under which models require additional user input to gather com-
plete information to answer a given question. As shown in Table 1,
the example on the top is an ideal QA problem. We can clearly
understand what the question is and then locate the relevant in-
put sentences to generate the answer. But it is hard to answer the
question in the boom example, because there are two types of bed-
rooms mentioned in all input sentences (i.e., the story) and we do
not know which bedroom the user refers to. ese scenarios with
incomplete information naturally appear in human conversations,
and thus, eectively handling them is a key capability of intelligent
QA models.
To address the challenges presented above, we propose a Context-
aware Aention Network (CAN) to learn ne-grained represen-
tations for input sentences, and develop a mechanism to interact
with user to comprehensively understand a given question. Specif-
ically, we employ two-level aention applied at word level and
sentence level to compute representations of all input sentences.
e context information extracted from an input story is allowed
to inuence the aention over each word, and governs the word
semantic meaning contributing to a sentence representation. In
addition, an interactive mechanism is created to generate a supple-
mentary question for the user when the model feels that it does not
have enough information to answer a given question. User’s feed-
back for the supplementary question is then encoded and exploited
to aend over all input sentences to infer an answer. Our proposed
model CAN can be viewed as an encoder-decoder approach aug-
mented with two-level aention and an interactive mechanism,
rendering our model self-adaptive, as illustrated in Figure 1.