Graph-based reasoning model for multiple relation extraction
Heyan Huang
a
, Ming Lei
a,
⇑
, Chong Feng
a
a
5 South Zhongguancun Street, Haidian District, Beijing, China
article info
Article history:
Received 7 November 2019
Revised 19 June 2020
Accepted 8 September 2020
Available online 28 September 2020
Communicated by Wu Jia
Keywords:
Relation extraction
Information extraction
Neural networks
Natural language processing
abstract
Linguistic knowledge is useful for various NLP tasks, but the difficul ty lies in the representation and appli-
cation. We consider that linguistic knowledge is implied in a large-scale corpus, while classification
knowledge, the knowledge related to the definitions of entity and relation types, is implied in the labeled
training data. Therefore, a corpus subgraph is proposed to mine more linguistic knowledge from the
easily accessible unlabeled data, and sentence subgraphs are used to acquire classification knowledge.
They jointly constitute a relation knowledge graph (RKG) to extract relations from sentences in this
paper. On RKG, entity recognition can be regarded as a property value filling problem and relation clas-
sification can be regarded as a link prediction problem. Thus, the multiple relation extraction can be trea-
ted as a reasoning process for knowledge completion. We combine statistical reasoning and neural
network reasoning to segment sentences into entity chunks and non-entity chunks, then propose a novel
Chunk Graph LSTM network to learn the representations of entity chunks and infer the relations among
them. The experiments on two standard datasets demonstrate our model outperforms the previous mod-
els for multiple relation extraction.
Ó 2020 Elsevier B.V. All rights reserved.
1. Introduction
Relation extraction (RE) is a task of assigning appropriate rela-
tion types to the entity pairs from sentences. It is helpful for web
mining, information retrieval, question answering, machine trans-
lation and other natural language processing (NLP) tasks [1,2].In
addition, it is also an essential step for constructing knowledge
bases automatically [3,4]. So, RE is an important research topic in
information extraction. Generally, a triplet, (entity 1, relation type,
entity 2), is used as the format of the structured representation of a
relation. Sometimes a sentence could contain multiple entities and
relation triplets, and an entity may belong to multiple different tri-
plets. Thus there are Cn; 2ðÞcandidate relations to be classified in a
sentence with nentities in the multiple relation extraction task. As
shown in Fig. 1, 7 entities and 6 relation triplets are labeled in the
example sentence. PER (person), WEA (weapon) and GPE (geo-
graphical/political) are entity types. PHYS (physical), ART (agent-
artifact), ORG-AFF (organization-affiliation) and GEN-AFF (general
affiliation) are relation types.
We can see that according to the positions of the entity pairs,
the triplets have overlapping, nested, intersected structures and
so on. These complex structures are difficult for the sequence-
based models [5–8] to handle. Recently some effective
graph-based models have been proposed to solve the multiple rela-
tion extraction. The work [9] presented a model for multiple rela-
tions extraction, which linked the newly identified entities to the
previous ones and used a feature matrix to learn graph structures.
The graph model for n-ary relation extraction [10], which is a spe-
cial case of overlapping relation extraction, partitioned sentences
into two directed acyclic graphs and classified relations by a Graph
LSTM. The model [11] defined a series of entity and relation tran-
sition actions and treated the extraction task as a dynamic gener-
ative process of a graph. The work [12] first adopted a bi-RNN
and a GCN (graph convolutional network) to extract both sequen-
tial and regional dependency word features and then applied a
relation-weighted GCN to extract implicit features among all word
pairs.
These graph-based models have achieved great success in mul-
tiple relation extraction. However, they mainly exploit the labeled
training data to learn the classification knowledge but neglect the
easily accessible unlabeled corpus. They leverage the semantic
information from word embeddings produced by generative pre-
training on unlabeled corpus [13–16] or gain linguistic knowledge
from auxiliary tool kits, such as dependency parser, POS, NER tools
and so forth. We argue that a large-scale corpus contains abundant
linguistic knowledge. So a corpus subgraph is proposed to mine
task-related linguistic knowledge from the unlabeled corpus. It is
combined with sentence subgraphs to constitute a relation knowl-
edge graph (RKG) for multiple relation extraction. On RKG, entity
https://doi.org/10.1016/j.neucom.2020.09.025
0925-2312/Ó 2020 Elsevier B.V. All rights reserved.
⇑
Corresponding author.
E-mail address: 66529158@qq.com (M. Lei).
Neurocomputing 420 (2021) 162–170
Contents lists available at ScienceDirect
Neurocomputing
journal homepage: www.elsevier.com/locate/neucom