Adversarial Multitask Learning for
Technology Entity Recognition
Hui Gao
1,2,*
, Ting Wang
1
, Wei Luo
2
, Lin Gui
1
1.
School of Computer, National University of Defense Technology, Changsha, China
2. Information Research Center of Military Science, Beijing, China
*corresponding author
gaohui_baixiang@163.com
Abstract—When reading scholar papers, the first thing
researchers want to know is which tasks and processes the
papers describe, which materials they use, etc.In this paper,
these concepts are referred as technology entities. Technology
entity recognition (TER) is the basis for carrying out
subsequent high-level technology analysis works, such as
technical foresight, technology roadmap, and technological
innovation. However, the challenges TER faces are much
greater than that of normal named entity recognition (NER).
Those challenges include the difficulty in feature extraction,
the lack of annotation data, and the differences between
different domains. To deal with the first challenge, we use a
deep neural network to extract features from text. For the
other two challenges, we propose an adversarial multitask
learning method. The existing knowledge from a big dataset on
a source domain is transferred to implement TER on a target
domain with only a small number of labeled samples. The
experiments show that the proposed method significantly
outperforms comparison systems.
Keywords—technology entity recognition, multitask learning,
domain adaptation, transfer learning
Empirical research requires gaining and maintaining an
understanding of the body of work in specific area. For
example, typical questions researchers face are which papers
describe which tasks and processes, use which materials and
how those relate to one another[1]. The key and basic task
tackled here is mention-level identification and classification
of technology entity, i.e. Technology entity Recognition
(TER).
In research area of information extraction, there are many
similar expressions to a technical entity, such as a key phrase,
a technical concept and a technical term. In recent years,
TER based higher-level analysis has received great interest,
such as the tracking or prediction for influence [2,3],
technology forecasting [4] and research communities study
[5].
However, the TER are much more challenging to the
normal NER(Named entity recognition) e.g. person names
recognition. The TER faces several difficult problems: 1)
The technology entity lacks regularity, and the number of
new words and unregistered words increase frequently. It is
difficult for traditional feature engineering to extract high-
quality features. 2) The definition of “technology” in
information extraction is not clear. Related researches
generally have different interpretations and inconsistencies in
technology according to different research backgrounds and
purposes, resulting in the lack of authoritative labeled corpus
and evaluation criteria, which makes it difficult to use
supervised machine learning for TER. 3) Even if several
labeled datasets had been released, such as Semeval-2017
released test corpus on the domains of computer, physics,
etc., the domains involved were limited. Since the
technology entities vary significantly in terms of composition,
indicators and context between different domains, it is
difficult training a TER model for specific domain by using
corpus in other domains.
In this paper, we present an adversarial multitask neural
network for the TER. By translating the data into compact
intermediate representations akin to principal components,
deep neural network can learn features directly from the data
without manual feature extraction. The proposed method
uses multitask transfer learning to implement domain
adaptation ion. By studying the distribution of samples
between the source and target domains, the existing
knowledge is transferred to implement the TER on the target
domain with only a small number of labeled data or even
without labeled data. Furthermore, an adversarial task is
employed to discriminate whether the features of the source
and target domains conform to the same distribution in the
shared space, which can ensure a better transfer effect.
The main contributions of this paper are:
We focus on technology entity recognition from
scientific literature, which is a challenging
information extraction task. We analyzed the
difficulties faced by TER and possible solutions, and
constructed a comprehensive architecture for the task
of TER.
We presented an adversarial multitask learning model
to implement TER with only limited labeled samples.
We conducted extensive experiments on Semeval-
2017 task10 corpus, and evaluated the results with
different baselines, in which the proposed method
showed an outstanding performance.
The American Heritage Science Dictionary defines
technology as
1
: a. The application of science, especially to
industrial or commercial objectives. b. The scientific method
and material used to achieve a commercial or industrial
objective. It can be seen that the definition of ‘technology’ is
very extensive, and there are many different expressions for
‘technology’ in previous researches, such as a technology
concept [6], a technology entity[7], a technical terms[8,9],
and a key phrase[6], etc. Eytan[6] thought that the technical
1
https://www.ahdictionary.com/word/search.html?q=Technology