《Speech and Language Processing》第三版重大更新：自然语言处理的权威教材

3星 · 超过75%的资源需积分: 28 87 浏览量更新于2024-07-18 2 收藏 18.21MB PDF 举报

"《Speech and Language Processing》是斯坦福大学教授 Dan Jurafsky 和科罗拉多大学波德分校的 James H. Martin 教授合作撰写的一本自然语言处理领域的经典教材，已翻译成60多种语言。这本书适用于高等教育中的自然语言处理和计算语言学课程，同时也是研究人员和技术人员的重要参考书籍。" 《Speech and Language Processing》第三版的更新涵盖了广泛的主题，旨在介绍自然语言处理、计算语言学以及语音识别的基础和进阶概念。以下是对书中部分章节的概述： 1. 引言: 本章介绍了自然语言处理的基本概念，以及该领域的重要性和发展历程。 2. 正则表达式、文本规范化和编辑距离: 这一章讲解了如何使用正则表达式匹配文本模式，文本规范化在处理语言数据时的作用，以及编辑距离在比较文本相似性方面的应用。 3. N-gram语言模型: N-gram模型是统计语言建模的基础，用于预测序列中的下一个元素，这在语音识别和机器翻译中至关重要。 4.朴素贝叶斯和情感分类: 朴素贝叶斯方法是一种简单但有效的分类技术，本章探讨了如何将其应用于文本情感分析。 5.逻辑回归: 逻辑回归用于二分类问题，如文本分类和情感极性判断。 6. 向量语义: 讨论了词向量表示法，如Word2Vec，它们能够捕捉词汇之间的语义关系。 7. 神经网络和神经语言模型: 介绍了深度学习在自然语言处理中的应用，尤其是循环神经网络（RNN）在语言建模中的作用。 8. 词性标注: 词性标注是识别句子中单词语法角色的过程，这对于句法分析和理解句子结构至关重要。 9. 序列处理与循环神经网络: 本章深入讨论了如何利用RNN处理序列数据，例如在语音识别或机器翻译中的应用。 10. 英语的正规形式语法: 解释了形式语法在理解和生成自然语言句子中的作用。 11. 句法解析: 探讨了如何分析句子结构以提取句法树，这是理解语义的关键步骤。 12. 统计句法解析: 介绍基于统计的方法来自动解析句子结构，如隐马尔可夫模型（HMM）和条件随机场（CRF）。 13. 依存关系解析: 依存语法是另一种句法分析方法，强调词语之间的依赖关系而非树状结构。 14. 句子意义的表示: 如何将句子转化为形式化的结构，以便计算机可以理解其含义。 15. 计算语义学: 研究如何用计算方法来处理语言的语义问题。 16. 语义解析: 将自然语言表达转化为形式逻辑表示，以实现更精确的语义理解。 17. 信息抽取: 从大量文本中自动提取结构化信息，如实体和关系。 18. 语义角色标注: 分析句子中动词和其他词的功能，以揭示事件的参与者和属性。 19. 词汇表、情感、影响和暗示: 讨论如何构建和使用词汇表来识别情感、情绪和上下文含义。 20. 核心ference解决和实体链接: 解决文本中代词和名称的指代问题，并将实体链接到知识库中的相应条目。 21. 会话连贯性: 研究如何理解对话和文本的连贯性，以提升人机交互体验。 22. 机器翻译: 介绍了将一种语言自动翻译成另一种语言的技术，包括统计机器翻译和神经机器翻译。 23. 问题回答: 研究如何设计系统来回答自然语言问题，涉及信息检索、推理和对话管理等多个方面。《Speech and Language Processing》第三版是一本全面而深入的教材，涵盖了自然语言处理的各个核心领域，对理论和实践都有深入的探讨，为学习者提供了坚实的理论基础和实践经验。

16 CHAPTER 2 • REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE

errors comes up again and again in implementing speech and language processing

systems. Reducing the overall error rate for an application thus involves two antag-

onistic efforts:

• Increasing precision (minimizing false positives)

• Increasing recall (minimizing false negatives)

2.1.4 A More Complex Example

Let’s try out a more signiﬁcant example of the power of REs. Suppose we want to

build an application to help a user buy a computer on the Web. The user might want

“any machine with at least 6 GHz and 500 GB of disk space for less than $1000”.

To do this kind of retrieval, we ﬁrst need to be able to look for expressions like 6

GHz or 500 GB or Mac or $999.99. In the rest of this section we’ll work out some

simple regular expressions for this task.

First, let’s complete our regular expression for prices. Here’s a regular expres-

sion for a dollar sign followed by a string of digits:

/$[0-9]+/

Note that the $ character has a different function here than the end-of-line function

we discussed earlier. Most regular expression parsers are smart enough to realize

that $ here doesn’t mean end-of-line. (As a thought experiment, think about how

regex parsers might ﬁgure out the function of $ from the context.)

Now we just need to deal with fractions of dollars. We’ll add a decimal point

and two digits afterwards:

/$[0-9]+\.[0-9][0-9]/

This pattern only allows $199.99 but not $199. We need to make the cents

optional and to make sure we’re at a word boundary:

/(ˆ|\W)$[0-9]+(\.[0-9][0-9])?\b/

One last catch! This pattern allows prices like $199999.99 which would be far

too expensive! We need to limit the dollar

/(ˆ|\W)$[0-9]{0,3}(\.[0-9][0-9])?\b/

How about speciﬁcations for > 6GHz processor speed? Here’s a pattern for that:

/\b[6-9]+ *(GHz|[Gg]igahertz)\b/

Note that we use / */ to mean “zero or more spaces” since there might always

be extra spaces lying around. For disk space, we’ll need to allow for optional frac-

tions again (5.5 GB); note the use of ? for making the ﬁnal s optional:

/\b[0-9]+(\.[0-9]+)? *(GB|[Gg]igabytes?)\b/

Modifying this regular expression so that it only matches more than 500 GB is

left as an exercise for the reader.

2.1.5 More Operators

Figure 2.7 shows some aliases for common ranges, which can be used mainly to

save typing. Besides the Kleene * and Kleene + we can also use explicit numbers as

2.1 • REGULAR EXPRESSIONS 17

counters, by enclosing them in curly brackets. The regular expression /{3}/ means

“exactly 3 occurrences of the previous character or expression”. So /a\.{24}z/

will match a followed by 24 dots followed by z (but not a followed by 23 or 25 dots

followed by a z).

RE Expansion Match First Matches

\d [0-9] any digit Party of 5

\D [ˆ0-9] any non-digit Blue moon

\w [a-zA-Z0-9_] any alphanumeric/underscore Daiyu

\W [ˆ\w] a non-alphanumeric !!!!

\s [ \r\t\n\f] whitespace (space, tab)

\S [ˆ\s] Non-whitespace in Concord

Figure 2.7 Aliases for common sets of characters.

A range of numbers can also be speciﬁed. So /{n,m}/ speciﬁes from n to m

occurrences of the previous char or expression, and /{n,}/ means at least n occur-

rences of the previous expression. REs for counting are summarized in Fig. 2.8.

RE Match

* zero or more occurrences of the previous char or expression

+ one or more occurrences of the previous char or expression

? exactly zero or one occurrence of the previous char or expression

{n} n occurrences of the previous char or expression

{n,m} from n to m occurrences of the previous char or expression

{n,} at least n occurrences of the previous char or expression

{,m} up to m occurrences of the previous char or expression

Figure 2.8 Regular expression operators for counting.

Finally, certain special characters are referred to by special notation based on the

backslash (\) (see Fig. 2.9). The most common of these are the newline character

Newline

\n and the tab character \t. To refer to characters that are special themselves (like

., *, [, and \), precede them with a backslash, (i.e., /\./, /\*/, /\[/, and /\\/).

RE Match First Patterns Matched

\* an asterisk “*” “K*A*P*L*A*N”

\. a period “.” “Dr. Livingston, I presume”

\? a question mark “Why don’t they come and lend a hand?”

\n a newline

\t a tab

Figure 2.9 Some characters that need to be backslashed.

2.1.6 Regular Expression Substitution, Capture Groups, and ELIZA

An important use of regular expressions is in substitutions. For example, the substi-

substitution

tution operator s/regexp1/pattern/ used in Python and in Unix commands like

vim or sed allows a string characterized by a regular expression to be replaced by

another string:

s/colour/color/

It is often useful to be able to refer to a particular subpart of the string matching

the ﬁrst pattern. For example, suppose we wanted to put angle brackets around all

18 CHAPTER 2 • REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE

integers in a text, for example, changing the 35 boxes to the <35> boxes. We’d

like a way to refer to the integer we’ve found so that we can easily add the brackets.

To do this, we put parentheses ( and ) around the ﬁrst pattern and use the number

operator \1 in the second pattern to refer back. Here’s how it looks:

s/([0-9]+)/<\1>/

The parenthesis and number operators can also specify that a certain string or

expression must occur twice in the text. For example, suppose we are looking for

the pattern “the Xer they were, the Xer they will be”, where we want to constrain

the two X’s to be the same string. We do this by surrounding the ﬁrst X with the

parenthesis operator, and replacing the second X with the number operator \1, as

follows:

/the (.*)er they were, the \1er they will be/

Here the \1 will be replaced by whatever string matched the ﬁrst item in paren-

theses. So this will match the bigger they were, the bigger they will be but not the

bigger they were, the faster they will be.

This use of parentheses to store a pattern in memory is called a capture group.

capture group

Every time a capture group is used (i.e., parentheses surround a pattern), the re-

sulting match is stored in a numbered register. If you match two different sets of

parentheses, \2 means whatever matched the second capture group. Thus

/the (.*)er they (.*), the \1er we \2/

will match the faster they ran, the faster we ran but not the faster they ran, the faster

we ate. Similarly, the third capture group is stored in \3, the fourth is \4, and so on.

Parentheses thus have a double function in regular expressions; they are used to

group terms for specifying the order in which operators should apply, and they are

used to capture something in a register. Occasionally we might want to use parenthe-

ses for grouping, but don’t want to capture the resulting pattern in a register. In that

case we use a non-capturing group, which is speciﬁed by putting the commands

non-capturing

group

?: after the open paren, in the form (?: pattern ).

/(?:some|a few) (people|cats) like some \1/

will match some cats like some cats but not some cats like some a few.

Substitutions and capture groups are very useful in implementing simple chat-

bots like ELIZA (Weizenbaum, 1966). Recall that ELIZA simulates a Rogerian

psychologist by carrying on conversations like the following:

User

: Men are all alike.

ELIZA

: IN WHAT WAY

User

: They’re always bugging us about something or other.

ELIZA

: CAN YOU THINK OF A SPECIFIC EXAMPLE

User

: Well, my boyfriend made me come here.

ELIZA

: YOUR BOYFRIEND MADE YOU COME HERE

User

: He says I’m depressed much of the time.

ELIZA

: I AM SORRY TO HEAR YOU ARE DEPRESSED

ELIZA works by having a series or cascade of regular expression substitutions

each of which matches and changes some part of the input lines. Input lines are

ﬁrst uppercased. The ﬁrst substitutions then change all instances of MY to YOUR,

and I’M to YOU ARE, and so on. The next set of substitutions matches and replaces

other patterns in the input. Here are some examples:

2.2 • WORDS 19

s/.* I’M (depressed|sad) .*/I AM SORRY TO HEAR YOU ARE \1/

s/.* I AM (depressed|sad) .*/WHY DO YOU THINK YOU ARE \1/

s/.* all .*/IN WHAT WAY/

s/.* always .*/CAN YOU THINK OF A SPECIFIC EXAMPLE/

Since multiple substitutions can apply to a given input, substitutions are assigned

a rank and applied in order. Creating patterns is the topic of Exercise 2.3, and we

return to the details of the ELIZA architecture in Chapter 24.

2.1.7 Lookahead assertions

Finally, there will be times when we need to predict the future: look ahead in the

text to see if some pattern matches, but not advance the match cursor, so that we can

then deal with the pattern if it occurs.

These lookahead assertions make use of the (? syntax that we saw in the previ-

lookahead

ous section for non-capture groups. The operator (?= pattern) is true if pattern

occurs, but is zero-width, i.e. the match pointer doesn’t advance. The operator

zero-width

(?! pattern) only returns true if a pattern does not match, but again is zero-width

and doesn’t advance the cursor. Negative lookahead is commonly used when we

are parsing some complex pattern but want to rule out a special case. For example

suppose we want to match, at the beginning of a line, any single word that doesn’t

start with “Volcano”. We can use negative lookahead to do this:

/ˆ(?!Volcano)[A-Za-z]+/

2.2 Words

Before we talk about processing words, we need to decide what counts as a word.

Let’s start by looking at one particular corpus (plural corpora), a computer-readable

corpus

corpora

collection of text or speech. For example the Brown corpus is a million-word col-

lection of samples from 500 written English texts from different genres (newspa-

per, ﬁction, non-ﬁction, academic, etc.), assembled at Brown University in 1963–64

(Ku

cera and Francis, 1967). How many words are in the following Brown sentence?

He stepped out into the hall, was delighted to encounter a water brother.

This sentence has 13 words if we don’t count punctuation marks as words, 15

if we count punctuation. Whether we treat period (“.”), comma (“,”), and so on as

words depends on the task. Punctuation is critical for ﬁnding boundaries of things

(commas, periods, colons) and for identifying some aspects of meaning (question

marks, exclamation marks, quotation marks). For some tasks, like part-of-speech

tagging or parsing or speech synthesis, we sometimes treat punctuation marks as if

they were separate words.

The Switchboard corpus of American English telephone conversations between

strangers was collected in the early 1990s; it contains 2430 conversations averaging

6 minutes each, totaling 240 hours of speech and about 3 million words (Godfrey

et al., 1992). Such corpora of spoken language don’t have punctuation but do intro-

duce other complications with regard to deﬁning words. Let’s look at one utterance

from Switchboard; an utterance is the spoken correlate of a sentence:

utterance

I do uh main- mainly business data processing

20 CHAPTER 2 • REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE

This utterance has two kinds of disﬂuencies. The broken-off word main- is

disﬂuency

called a fragment. Words like uh and um are called ﬁllers or ﬁlled pauses. Should

fragment

ﬁlled pause

we consider these to be words? Again, it depends on the application. If we are

building a speech transcription system, we might want to eventually strip out the

disﬂuencies.

But we also sometimes keep disﬂuencies around. Disﬂuencies like uh or um

are actually helpful in speech recognition in predicting the upcoming word, because

they may signal that the speaker is restarting the clause or idea, and so for speech

recognition they are treated as regular words. Because people use different disﬂu-

encies they can also be a cue to speaker identiﬁcation. In fact Clark and Fox Tree

(2002) showed that uh and um have different meanings. What do you think they are?

Are capitalized tokens like They and uncapitalized tokens like they the same

word? These are lumped together in some tasks (speech recognition), while for part-

of-speech or named-entity tagging, capitalization is a useful feature and is retained.

How about inﬂected forms like cats versus cat? These two words have the same

lemma cat but are different wordforms. A lemma is a set of lexical forms having

lemma

the same stem, the same major part-of-speech, and the same word sense. The word-

form is the full inﬂected or derived form of the word. For morphologically complex

wordform

languages like Arabic, we often need to deal with lemmatization. For many tasks in

English, however, wordforms are sufﬁcient.

How many words are there in English? To answer this question we need to

distinguish two ways of talking about words. Types are the number of distinct words

word type

in a corpus; if the set of words in the vocabulary is V, the number of types is the

vocabulary size |V|. Tokens are the total number N of running words. If we ignore

word token

punctuation, the following Brown sentence has 16 tokens and 14 types:

They picnicked by the pool, then lay back on the grass and looked at the stars.

When we speak about the number of words in the language, we are generally

referring to word types.

Corpus Tokens = N Types = |V|

Shakespeare 884 thousand 31 thousand

Brown corpus 1 million 38 thousand

Switchboard telephone conversations 2.4 million 20 thousand

COCA 440 million 2 million

Google N-grams 1 trillion 13 million

Figure 2.10 Rough numbers of types and tokens for some English language corpora. The

largest, the Google N-grams corpus, contains 13 million types, but this count only includes

types appearing 40 or more times, so the true number would be much larger.

Fig. 2.10 shows the rough numbers of types and tokens computed from some

popular English corpora. The larger the corpora we look at, the more word types

we ﬁnd, and in fact this relationship between the number of types |V| and number

of tokens N is called Herdan’s Law (Herdan, 1960) or Heaps’ Law (Heaps, 1978)

Herdan’s Law

Heaps’ Law

after its discoverers (in linguistics and information retrieval respectively). It is shown

in Eq. 2.1, where k and β are positive constants, and 0 < β < 1.

|V| = kN

(2.1)

The value of β depends on the corpus size and the genre, but at least for the

large corpora in Fig. 2.10, β ranges from .67 to .75. Roughly then we can say that

剩余557页未读，继续阅读

sangwq

粉丝: 1
资源: 19

《Speech and Language Processing》第三版重大更新：自然语言处理的权威教材

Speech and Language Processing，自然语言处理综论

speech and language processing 2nd

Speech.and.Language.Processing.pdf

与自然语言处理有关的参考文献

自然语言处理教材有哪些

自然语言处理的学习路径，列出一些学习资源

自然语言 java 混合

python自然语言处理的库

自然语言处理命名实体识别课程数据集

机器人学导论第三版pdf

最新资源