Classifying Relations via Long Short Term Memory Networks
along Shortest Dependency Paths
Yan Xu,
†
Lili Mou,
†
Ge Li,
†∗
Yunchuan Chen,
‡
Hao Peng,
†
Zhi Jin
†∗
†
Software Institute, Peking University, 100871, P. R. China
{xuyan14,lige,zhijin}@sei.pku.edu.cn,{doublepower.mou,penghao.pku}@gmail.com
‡
University of Chinese Academy of Sciences, chenyunchuan11@mails.ucas.ac.cn
Abstract
Relation classification is an important re-
search arena in the field of natural lan-
guage processing (NLP). In this paper, we
present SDP-LSTM, a novel neural net-
work to classify the relation of two enti-
ties in a sentence. Our neural architecture
leverages the shortest dependency path
(SDP) between two entities; multichan-
nel recurrent neural networks, with long
short term memory (LSTM) units, pick
up heterogeneous information along the
SDP. Our proposed model has several dis-
tinct features: (1) The shortest dependency
paths retain most relevant information (to
relation classification), while eliminating
irrelevant words in the sentence. (2) The
multichannel LSTM networks allow ef-
fective information integration from het-
erogeneous sources over the dependency
paths. (3) A customized dropout strategy
regularizes the neural network to allevi-
ate overfitting. We test our model on the
SemEval 2010 relation classification task,
and achieve an F
1
-score of 83.7%, higher
than competing methods in the literature.
1 Introduction
Relation classification is an important NLP task.
It plays a key role in various scenarios, e.g., in-
formation extraction (Wu and Weld, 2010), ques-
tion answering (Yao and Van Durme, 2014), med-
ical informatics (Wang and Fan, 2014), ontol-
ogy learning (Xu et al., 2014), etc. The aim
of relation classification is to categorize into pre-
defined classes the relations between pairs of
marked entities in given texts. For instance, in
the sentence “A trillion gallons of [water]
e
1
have
been poured into an empty [region]
e
2
of outer
∗
Corresponding authors.
space,” the entities water and region are of rela-
tion Entity-Destination(e
1
, e
2
).
Traditional relation classification approaches
rely largely on feature representation (Kambhatla,
2004), or kernel design (Zelenko et al., 2003;
Bunescu and Mooney, 2005). The former method
usually incorporates a large set of features; it is
difficult to improve the model performance if the
feature set is not very well chosen. The latter ap-
proach, on the other hand, depends largely on the
designed kernel, which summarizes all data infor-
mation. Deep neural networks, emerging recently,
provide a way of highly automatic feature learning
(Bengio et al., 2013), and have exhibited consid-
erable potential (Zeng et al., 2014; dos Santos et
al., 2015). However, human engineering—that is,
incorporating human knowledge to the network’s
architecture—is still important and beneficial.
This paper proposes a new neural network,
SDP-LSTM, for relation classification. Our model
utilizes the shortest dependency path (SDP) be-
tween two entities in a sentence; we also design a
long short term memory (LSTM)-based recurrent
neural network for information processing. The
neural architecture is mainly inspired by the fol-
lowing observations.
• Shortest dependency paths are informative
(Fundel et al., 2007; Chen et al., 2014). To
determine the two entities’ relation, we find it
mostly sufficient to use only the words along
the SDP: they concentrate on most relevant
information while diminishing less relevant
noise. Figure 1 depicts the dependency parse
tree of the aforementioned sentence. Words
along the SDP form a trimmed phrase (gal-
lons of water poured into region) of the orig-
inal sentence, which conveys much informa-
tion about the target relation. Other words,
such as a, trillion, outer space, are less infor-
mative and may bring noise if not dealt with
properly.