different templates for 2782 predicates. The large amount guaran-
tees the wide coverage of template-based QA.
The procedure of learning the predicate of a template is as fol-
lows. First, for each QA pair in Yahoo! Answer, we extract the
entity in question and the corresponding value. Then, we find the
predicate from the knowledge base by looking up the direct predi-
cate connecting the entity and the value. Our basic idea is, if most
instances of a template share the same predicate, we map the tem-
plate to this predicate. For example, suppose questions derived
by template how many people are there in $city? al-
ways map to the predicate population, no matter what specific
$city it is. We can conclude that for certain probability the tem-
plate maps to population. Learning templates that map to a com-
plex knowledge base structure employs a similar process. The only
difference is that we find “expanded predicates” that correspond to
a path consisting of multiple edges which lead from an entity to a
certain value (e.g., marriage → person → name).
1.4 Paper Organization
The rest of the paper is organized as follows. In Sec 2, we give an
overview of KBQA. The major contribution of this paper is learn-
ing templates from QA corpora. All technique parts are close-
ly related to it. Sec 3 shows the online question answering with
templates. Sec 4 elaborates the predicates inference for templates,
which is the key step to use templates. Sec 5 extends our solution to
answer a complex question. Sec 6 extends the ability of templates
to infer complex predicates. We present experimental studies in
Sec 7, discuss more related works in Sec 8, and conclude in Sec 9.
2. SYSTEM OVERVIEW
In this section, we introduce some background knowledge and
give an overview of KBQA. In Table 2, we list the notations used
in this paper.
Table 2: Notations
Notation Description Notation Description
q question s subject
a answer p predicate
QA QA corpus o object
e entity K knowledge base
v value c category
t template p
+
expanded predicate
V (e, p) {v|(e, p, v) ∈ K} s
2
⊂ s
1
s
2
is a substring of s
1
t(q, e, c) template of q by θ
(s)
estimation of θ
conceptualizing e to c at iteration s
Binary factoid QA We focus on binary factoid questions
(BFQs), that is, questions asking about a specific property of an
entity. For example, all questions except
f
in Table 1 are BFQs.
RDF knowledge base Given a question, we find its answer in
an RDF knowledge base. An RDF knowledge base K is a set of
triples in the form of (s, p, o), where s, p, and o denote subjec-
t, predicate, and object respectively. Figure 1 shows a toy RDF
knowledge base via an edge-labeled directed graph. Each (s, p, o)
is represented by a directed edge from s to o labeled with predicate
p. For example, the edge from a to 1961 with label dob represents
an RDF triple (a, dob, 1961), which represents the knowledge of
Barack Obama’s birthday.
Table 3: Sample QA Pairs from a QA Corpus
Id Question Answer
(q
1
, a
1
) When was Barack Obama
born?
The politician was born in
1961.
(q
2
, a
2
) When was Barack Obama
born?
He was born in 1961.
(q
3
, a
3
) How many people are
there in Honolulu?
It’s 390K.
QA corpora We learn question templates from Yahoo! Answer,
which consists of 41 million QA pairs. The QA corpora is denoted
by QA = {(q
1
, a
1
), (q
2
, a
2
), ..., (q
n
, a
n
)}, where q
i
is a ques-
tion and a
i
is the reply to q
i
. Each reply a
i
consists of several
sentences, and the exact factoid answer is contained in the reply.
Table 3 shows a sample from a QA corpus.
Templates. We derive a template t from a question q by replacing
each entity e with one of e’s categories c. We denote this template
as t = t(q, e, c). A question may contain multiple entities, and
an entity may belong to multiple categories. We obtain concept
distribution of e through context-aware conceptualization [32]. For
example, question q
1
in Table 3 contains entity a in Figure 1. Since
a belongs to two categories: $Person, $Politician, we can derive
two templates from the question: When was $Person born?
and When was $Politician born?.
Figure 3: System Overview
System Architecture. Figure 3 shows the pipeline of our QA sys-
tem, which consists of two major procedures:
• Online procedure: When a question comes in, we first parse
and decompose it into a series of binary factoid questions. The
decomposition process is described in Sec 5. For each binary
factoid question, we use a probabilistic inference approach to
find its value, shown in Sec 3. The inference is based on the
predicate distribution of given templates, i.e. P (p|t). Such dis-
tribution is learned offline.
• Offline procedure: The goal of offline procedure is to learn the
mapping from templates to predicates. This is represented by
P (p|t), which is estimated in Sec 4. And we expand predicates
in the knowledge base in Sec 6, so that we can learn more com-
plex predicate forms (e.g., marriage → person → name in
Figure 1).
3. OUR APPROACH: KBQA
In this section, we first formalize our problem in a probabilistic
framework in Sec 3.1. We present the details for most probability
estimations in Sec 3.2, leaving only the estimation of P(p|t) in
Sec 4. We elaborate the online procedure in Sec 3.3.
3.1 Problem Model
KBQA learns question answering by using a QA corpus and a
knowledge base. Due to issues such as uncertainty (e.g. some ques-
tions’ intents are vague), incompleteness (e.g. the knowledge base
is almost always incomplete), and noise (e.g. answers in the QA
567