统一序列到序列框架解决NER子任务
版权申诉
165 浏览量
更新于2024-07-06
收藏 666KB PDF 举报
本文档探讨了命名实体识别（Named Entity Recognition，NER）这一自然语言处理领域的核心任务，它旨在从句子中定位表示实体的跨度。NER通常分为三个子任务：平坦命名实体识别（flat NER）、嵌套命名实体识别（nested NER）以及不连续命名实体识别（discontinuous NER）。传统上，这些问题分别通过词级别序列标注或跨度级别分类方法来解决。然而，这些方法往往难以同时处理这三种不同类型的NER。
作者提出了一种统一的生成框架，将NER子任务转化为实体跨度序列生成任务。这个框架基于序列到序列（Sequence-to-Sequence，Seq2Seq）模型，能够在一个单一的架构中处理所有三种NER类型。通过这种方式，模型可以利用预训练的Seq2Seq技术，如Transformer或其他递归神经网络结构，学习捕捉文本中的上下文信息，以便更有效地识别和生成实体的正确位置和类别。
该论文的核心贡献包括：
1. **方法创新**：提出了将NER任务统一为生成式模型，而非传统的分类或标注方式，这使得模型能够灵活应对不同类型的实体关系，包括是否嵌套以及实体跨度的断裂情况。
2. **Seq2Seq模型应用**：展示了如何将预训练的Seq2Seq模型应用于NER，通过端到端的学习，简化了模型设计，提高了识别性能。
3. **跨任务处理能力**：通过统一的框架，模型能够在不牺牲准确性的前提下，同时处理多种NER子任务，提高了整体的效率和通用性。
4. **实验评估**：论文还可能包含详细的实验设置和结果分析，展示了在基准数据集上的性能对比，证明了新框架的有效性和优越性。
5. **潜在应用**：这种统一的生成框架对于那些需要处理复杂实体结构的领域，如信息提取、问答系统和知识图谱构建，具有显著的应用前景。
本文的研究成果提供了一个强大的工具，有望推动命名实体识别领域的研究进一步向前发展，提高系统的灵活性和泛化能力。
tried to predict a tagging sequence. Therefore, they
still need to design tagging schemas for different
NER subtasks.
Span-level classification
When applying the se-
quence labelling method to the nested NER and
discontinous NER subtasks, the tagging will be
complex (Strakov
´
a et al., 2019; Metke-Jimenez and
Karimi, 2016) or multi-level (Ju et al., 2018; Fisher
and Vlachos, 2019; Shibuya and Hovy, 2020).
Therefore, the second line of work directly con-
ducted the span-level classification. The main dif-
ference between publications in this line of work is
how to get the spans. Finkel and Manning (2009)
regarded the parsing nodes as a span. Xu et al.
(2017); Luan et al. (2019); Yamada et al. (2020); Li
et al. (2020b); Yu et al. (2020); Wang et al. (2020a)
tried to enumerate all spans. Following Lu and
Roth (2015), hypergraph methods which can effec-
tively represent exponentially many possible nested
mentions in a sentence have been extensively stud-
ied in the NER tasks (Katiyar and Cardie, 2018;
Wang and Lu, 2018; Muis and Lu, 2016).
Combined token-level and span-level classifi-
cation
To avoid enumerating all possible spans
and incorporate the entity boundary information
into the model, Wang and Lu (2019); Zheng et al.
(2019); Lin et al. (2019); Wang et al. (2020b); Luo
and Zhao (2020) proposed combining the token-
level classification and span-level classification.
2.2 Sequence-to-Sequence Models
The Seq2Seq framework has been long studied and
adopted in NLP (Sutskever et al., 2014; Cho et al.,
2014; Luong et al., 2015; Vaswani et al., 2017;
Vinyals et al., 2015). Gillick et al. (2016) pro-
posed a Seq2Seq model to predict the entity’s start,
span length and label for the NER task. Recently,
the amazing performance gain achieved by PTMs
(pre-trained models) (Qiu et al., 2020; Peters et al.,
2018; Devlin et al., 2019; Dai et al., 2021; Yan
et al., 2020) has attracted several attempts to pre-
train a Seq2Seq model (Song et al., 2019; Lewis
et al., 2020; Raffel et al., 2020). We mainly focus
on the newly proposed BART (Lewis et al., 2020)
model because it can achieve better performance
than MASS (Song et al., 2019). And the sentence-
piece tokenization used in T5 (Raffel et al., 2020)
will cause different tokenizations for the same to-
ken, making it hard to generate pointer indexes to
conduct the entity extraction.
BART is formed by several transformer encoder
and decoder layers, like the transformer model used
in the machine translation (Vaswani et al., 2017).
BART’s pre-training task is to recover corrupted
text into the original text. BART uses the encoder
to input the corrupted sentence and the decoder
to recover the original sentence. BART has base
and large versions. The base version has 6 encoder
layers and 6 decoder layers, while the large version
has 12. Therefore, the number of parameters is
similar to its equivalently sized BERT
5
.
3 Proposed Method
In this part, we first introduce the task formulation,
then we describe how we use the Seq2Seq model
with the pointer mechanism to generate the entity
index sequences. After that, we present the detailed
formulation of our model with BART.
3.1 NER Task
The three kinds of NER tasks can all be formulated
as follows, given an input sentence of
n
tokens
X = [x
1
, x
2
, ..., x
n
]
, the target sequence is
Y =
[s
11
, e
11
, ..., s
1j
, e
1j
, t
1
, ..., s
i1
, e
i1
, ..., s
ik
, e
ik
, t
i
]
,
where
s, e
are the start and end index of a span,
since an entity may contain one (for flat and
nested NER) or more than one (for discontinu-
ous NER) spans, each entity is represented as
[s
i1
, e
i1
, ..., s
ij
, e
ij
, t
i
]
, where
t
i
is the entity tag
index. We use
G = [g
1
, ..., g
l
]
to denote the entity
tag tokens (such as “Person”, “Location”, etc.),
where
l
is the number of entity tags. We make
t
i
∈ (n, n + l]
, the
n
shift is to make sure
t
i
is not
confusing with pointer indexes (pointer indexes
will be in range [1, n]).
3.2 Seq2Seq for Unified Decoding
Since we formulate the NER task in a generative
way, we can view the NER task as the following
equation:
P (Y |X) =
m
Y
t=1
P (y
t
|X, Y
<t
) (1)
where
y
0
is the special “start of sentence” control
token.
We use the Seq2Seq framework with the pointer
mechanism to tackle this task. Therefore, our
model consists of two components:
5
Because of the cross-attention between encoder and de-
coder, the number of parameters of BART is about 10% larger
than its equivalently sized of BERT (Lewis et al., 2020).
剩余14页未读，继续阅读
2021-10-01 上传
2024-10-16 上传
2022-09-23 上传
2021-10-01 上传
2022-07-14 上传
2021-10-04 上传
2021-10-03 上传
2021-10-04 上传
