神经网络方法：自然语言处理的新进展

需积分: 9 61 浏览量更新于2024-07-19 2 收藏 2.02MB PDF 举报

《神经网络方法在自然语言处理中的应用》是Yoav Goldberg所著的一本综述性书籍，收录于Synthesis Lectures on Human Language Technologies系列，由Graeme Hirst编辑，该系列专注于探讨与自然语言处理、计算语言学、信息检索以及口语理解相关的主题。这些讲座论文集强调了新颖技术、新应用和跨领域研究的重要性。本书主要关注神经网络方法在自然语言处理领域的创新和发展。随着人类生产文本量的逐年增长，计算机与用户进行语言交互的需求日益迫切，这就推动了自然语言处理技术的进步。然而，人类语言天然具有模糊性、变化性和定义不明确的特点，为处理带来了挑战。神经网络作为一种强大的工具，被应用于解决这些问题，它能够模仿人脑的结构和功能，学习复杂的语言模式，并在诸如机器翻译（如Syntax-based Statistical Machine Translation by Philip Williams等人）、语义标注（如Domain-Sensitive Temporal Tagging by Jannik Strötgen和Michael Gertz）、知识图谱构建（Linked Lexical Knowledge Bases: Foundations and Applications by Iryna Gurevych等人）和语言理解的深层次分析（如Metaphor: A Computational Perspective by Tony Veale等人）等方面发挥关键作用。例如，神经网络通过深度学习算法，能够进行端到端的翻译，减少了传统统计机器翻译中规则和模板的依赖，提高了翻译的质量和流畅度。在语义标注任务中，神经网络模型可以捕捉到词语之间的复杂关系，帮助识别时间序列数据或特定领域的标记，增强文本理解和上下文关联。此外，神经网络也被用于构建和扩展知识图谱，将语言数据与外部知识源相结合，提供更丰富的语义支持。在处理语言中的比喻这种隐喻现象时，神经网络通过模拟人类的抽象思维能力，解析并解释隐喻表达背后的深层意义，促进了对语言的理解和生成的深化。《神经网络方法在自然语言处理》一书深入剖析了神经网络如何革新自然语言处理技术，使之在诸多应用场景中更加智能、灵活和准确，从而为提高人机交互体验和人工智能的整体性能做出了重要贡献。同时，它也揭示了未来在处理自然语言复杂性方面的研究趋势和挑战。

xvi

20.4.3 Preposition Sense Disambiguation and Preposition Translation

Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

20.4.4 Conditioned Generation: Multilingual Machine Translation,

Parsing, and Image Captioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

20.5 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

21.1 What Have We Seen? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

21.2 e Challenges Ahead . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

Author’s Biography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

xvii

Preface

Natural language processing (NLP) is a collective term referring to automatic computational pro-

cessing of human languages. is includes both algorithms that take human-produced text as

input, and algorithms that produce natural looking text as outputs. e need for such algorithms

is ever increasing: human produce ever increasing amounts of text each year, and expect com-

puter interfaces to communicate with them in their own language. Natural language processing

is also very challenging, as human language is inherently ambiguous, ever changing, and not well

deﬁned.

Natural language is symbolic in nature, and the ﬁrst attempts at processing language were

symbolic: based on logic, rules, and ontologies. However, natural language is also highly ambigu-

ous and highly variable, calling for a more statistical algorithmic approach. Indeed, the current-

day dominant approaches to language processing are all based on statistical machine learning. For

over a decade, core NLP techniques were dominated by linear modeling approaches to supervised

learning, centered around algorithms such as Perceptrons, linear Support Vector Machines, and

Logistic Regression, trained over very high dimensional yet very sparse feature vectors.

Around 2014, the ﬁeld has started to see some success in switching from such linear models

over sparse inputs to nonlinear neural network models over dense inputs. Some of the neural-

network techniques are simple generalizations of the linear models and can be used as almost

drop-in replacements for the linear classiﬁers. Others are more advanced, require a change of

mindset, and provide new modeling opportunities. In particular, a family of approaches based

on recurrent neural networks (RNNs) alleviates the reliance on the Markov Assumption that was

prevalent in sequence models, allowing to condition on arbitrarily long sequences and produce

eﬀective feature extractors. ese advances led to breakthroughs in language modeling, automatic

machine translation, and various other applications.

While powerful, the neural network methods exhibit a rather strong barrier of entry, for

various reasons. In this book, I attempt to provide NLP practitioners as well as newcomers with

the basic background, jargon, tools, and methodologies that will allow them to understand the

principles behind neural network models for language, and apply them in their own work. I also

hope to provide machine learning and neural network practitioners with the background, jargon,

tools, and mindset that will allow them to eﬀectively work with language data.

Finally, I hope this book can also serve a relatively gentle (if somewhat incomplete) intro-

duction to both NLP and machine learning for people who are newcomers to both ﬁelds.

xviii PREFACE

INTENDED READERSHIP

is book is aimed at readers with a technical background in computer science or a related ﬁeld,

who want to get up to speed with neural network techniques for natural language processing.

While the primary audience of the book is graduate students in language processing and machine

learning, I made an eﬀort to make it useful also to established researchers in either NLP or ma-

chine learning (by including some advanced material), and to people without prior exposure to

either machine learning or NLP (by covering the basics from the grounds up). is last group of

people will, obviously, need to work harder.

While the book is self contained, I do assume knowledge of mathematics, in particular

undergraduate level of probability, algebra, and calculus, as well as basic knowledge of algorithms

and data structures. Prior exposure to machine learning is very helpful, but not required.

is book evolved out of a survey paper [Goldberg, 2016], which was greatly expanded and

somewhat re-organized to provide a more comprehensive exposition, and more in-depth coverage

of some topics that were left out of the survey for various reasons. is book also contains many

more concrete examples of applications of neural networks to language data that do not exist in

the survey. While this book is intended to be useful also for people without NLP or machine

learning backgrounds, the survey paper assumes knowledge in the ﬁeld. Indeed, readers who are

familiar with natural language processing as practiced between roughly 2006 and 2014, with heavy

reliance on machine learning and linear models, may ﬁnd the journal version quicker to read and

better organized for their needs. However, such readers may also appreciate reading the chapters

on word embeddings (10 and 11), the chapter on conditioned generation with RNNs (17), and

the chapters on structured prediction and multi-task learning (MTL) (19 and 20).

FOCUS OF THIS BOOK

is book is intended to be self-contained, while presenting the diﬀerent approaches under a uni-

ﬁed notation and framework. However, the main purpose of the book is in introducing the neural-

networks (deep-learning) machinery and its application to language data, and not in providing

an in-depth coverage of the basics of machine learning theory and natural language technology.

I refer the reader to external sources when these are needed.

Likewise, the book is not intended as a comprehensive resource for those who will go on

and develop the next advances in neural network machinery (although it may serve as a good

entry point). Rather, it is aimed at those readers who are interested in taking the existing, use-

ful technology and applying it in useful and creative ways to their favorite language-processing

problems.

剩余308页未读，继续阅读

yuyongpinghuo

粉丝: 0
资源: 3

神经网络方法：自然语言处理的新进展

Neural Network Methods for Natural Language Process

Neural Network Methods for Natural Language Processing

(Yoav Goldberg)Neural Network Methods for Natural Language Processing适合nlp进阶

how can i trained a neural network for identify the cat and dog

帮我解释一下下面的代码：class NeuralNetwork(nn.Module): def __init__(self): super(NeuralNetwork, sel...

super(BPNeuralNetwork4, self).__init__()

subsasgn Assign fields of a neural network.

class NeuralNet(nn.Module): ''' A simple fully-connected deep neural network '''

卷积神经网络有哪些衍生卷积神经网络

深度学习神经网络模型有哪些

最新资源

帮我解释一下下面的代码：class NeuralNetwork(nn.Module): def init(self): super(NeuralNetwork, sel...

super(BPNeuralNetwork4, self).init()