![](https://csdnimg.cn/release/download_crawler_static/19714593/bg13.jpg)
GRAPH NEURAL NETWORKS FOR NLP: A SURVEY
which e
1
is the source entity, e
2
is the target entity, and rel is the relation type. Then one adds two
nodes v
1
(for the source element e
1
) and v
2
(for the target element e
2
) in the KG and add a directed
edge from node v
1
to node v
2
with edge type rel. An example of such KG is shown in Fig. 5.
It is worth noting that the KG plays different roles in various applications. In some applications
(e.g. knowledge graph completion and knowledge base question answering), KG is always treated
as part of the inputs. In this scenario (Ye et al., 2019; Zhang et al., 2020g; Li et al., 2019; Wu
et al., 2020a), researchers typically use the whole KG G as the learning object. But for some other
applications (e.g. natural language translation), the KG can be treated as the data augmentation
method. In this case, the whole KG such as ConceptNet (Speer et al., 2017) is usually too large and
noisy for some domain-specific applications (Kapanipathi et al., 2020; Lin et al., 2019a), and thus
it is not suitable to use the whole graph as inputs. In contrast, as shown in Figure 5, one instead
usually constructs subgraphs from the given query (it is often the text-based inputs like the queries
in the reading comprehension task) (Xu et al., 2019b; Teru et al., 2020; Kapanipathi et al., 2020).
The construction methods could may vary dramatically in the literature. Here, we only present
one representative method for illustration purpose (Teru et al., 2020). The first thing for constructing
KG is to fetch the term instances in the given query. Then, they could link the term instances to the
concepts in the KG by some matching algorithms such as max-substring matching. The concepts
are regarded as the initial nodes in the extracted subgraph. Next step is to fetch the 1-hop neighbors
of the initial nodes in the KG. Additionally, one may calculate the relevance of the neighbors with
the initial nodes by applying some graph node relevance model such as the Personalized PageRank
(PPR) algorithm (Page et al., 1999). Then based on the results, one may further prune out the edges
with relevance score that is below the confidence threshold and remove the isolated neighbors. The
remaining final subgraph is then used to feed any graph representation learning module later.
Coreference Graph Construction In linguistics, coreference (or co-reference) occurs when two
or more terms in a given paragraph refer to the same object. Many works have demonstrated that
such phenomenon is helpful for better understanding the complex structure and logic of the corpus
and resolve the ambiguities (Xu et al., 2020c; De Cao et al., 2018; Sahu et al., 2019). To effec-
tively leverage the coreference information, the coreference graph is constructed to explicitly model
the implicit coreference relations. Given a set of phrases, a coreference graph can link the nodes
(phrases) which refer to the same entity in the text corpus. In the following subsection, we focus on
the coreference graph construction for a paragraph para consisting of m sentences. We will briefly
discuss the coreference relation and then discuss the approaches for building the coreference graph
in various NLP tasks (De Cao et al., 2018; Sahu et al., 2019; Qian et al., 2019; Xu et al., 2020c,a;
Luan et al., 2019). It is worth noting that although it is similar to the IE graph’s first step, the coref-
erence graph will explicitly model the coreference relationship by graph instead of collapse into one
node.
Step 1: Coreference Relation. The coreference relations can be obtained easily by the coreference
resolution system, as discussed in IE graph construction. Similarly, we can obtain the coreference
clusters C given a specific paragraph. All phrases in a cluster C
i
∈ C refer to the same object.
Step 2: Coreference Graph. The coreference graph are built on the coreference relation set R
coref
.
It can be generally divided into two main category depending on the node type: 1) phrases (or
mentions) (Koncel-Kedziorski et al., 2019; Luan et al., 2019; De Cao et al., 2018), 2) words (Sahu
et al., 2019). For the first class, the coreference graph G consists of all mentions in relation set
R
coref
. For each phrase pair p
i
, p
j
in cluster C
k
∈ C, one may add an undirected edge between
19