《Speech and Language Processing》：自然语言处理经典教材

需积分: 11 96 浏览量更新于2024-07-18 收藏 16.33MB PDF 举报

《自然语言处理：语音、语言与计算语言学及语音识别入门》第三版是一本经典且权威的英文原著，由斯坦福大学的Daniel Jurafsky教授和科罗拉多大学博尔德分校的James H. Martin教授合著。该书于2018年版权修订，初稿日期为2018年8月15日，欢迎读者提出宝贵意见和纠正错漏。本书全面覆盖了自然语言处理（NLP）领域的关键概念和技术，旨在为学习者提供一个深入理解自然语言处理理论和实践的坚实基础。全书共分为23章，从介绍自然语言处理的基本概念开始，逐步深入到各个子领域： 1. **引言**：概述自然语言处理的目标和历史，介绍研究背景和应用前景。 2. **正则表达式与文本规范化**：介绍文本处理中的基础工具，包括字符串匹配和格式转换。 3. **n-gram语言模型**：讲解如何通过统计方法预测文本序列的概率，是基本的语言模型基础。 4. **朴素贝叶斯分类与情感分析**：介绍基于概率的简单分类算法及其在情绪判断中的应用。 5. **逻辑回归**：一种常用的统计学习方法，用于预测任务中的变量关系。 6. **向量语义**：探讨词向量表示和它们在理解语义关系中的作用。 7. **神经网络和神经语言模型**：涉及深度学习在NLP中的应用，如循环神经网络(RNN)和更复杂的架构。 8. **词性标注**：识别文本中单词的语法类别，是语言处理中的基础任务。 9. **递归神经网络**：深度学习在处理序列数据中的另一种形式，如句法结构的建模。 10. **英语的句法规则**：介绍英语句子的结构规则和分析方法。 11. **句法解析**：识别句子成分和短语结构，如依存句法分析。 12. **统计句法分析**：使用概率方法进行自动解析的探讨。 13. **依存关系解析**：关注词汇之间的依赖关系，以捕捉句子的深层结构。 14. **句子意义的表示**：探讨如何量化和表达文本的抽象意义。 15. **计算语义**：利用计算机处理语言中的意义和概念。 16. **语义解析**：将自然语言转化为机器可理解的形式，如逻辑表达式或知识图谱查询。 17. **信息抽取**：从文本中提取结构化的信息，如命名实体和事件。 18. **语义角色标注**：确定句子中词汇在句子结构中的具体作用。 19. **词义计算**：处理多义词和同义词，提高语言理解的精确度。 20. **情感和情绪词典**：利用词典资源支持情感和情绪分析。 21. **核心ference解析和实体链接**：解决文本中提及同一事物的不同表达问题。 22. **篇章连贯性**：研究如何理解文本的上下文关联。 23. **机器翻译**：介绍自然语言处理在跨语言交流中的应用，如将文本从一种语言自动翻译成另一种语言。这本书不仅适合研究生和专业研究人员，也适合对自然语言处理感兴趣的开发者、工程师和语言技术爱好者。它不仅提供了理论知识，还包含了许多实用的技术和案例，有助于读者在实际项目中应用所学知识。

16 CHAPTER 2 • REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE

errors comes up again and again in implementing speech and language processing

systems. Reducing the overall error rate for an application thus involves two antag-

onistic efforts:

• Increasing precision (minimizing false positives)

• Increasing recall (minimizing false negatives)

2.1.4 A More Complex Example

Let’s try out a more signiﬁcant example of the power of REs. Suppose we want to

build an application to help a user buy a computer on the Web. The user might want

“any machine with at least 6 GHz and 500 GB of disk space for less than $1000”.

To do this kind of retrieval, we ﬁrst need to be able to look for expressions like 6

GHz or 500 GB or Mac or $999.99. In the rest of this section we’ll work out some

simple regular expressions for this task.

First, let’s complete our regular expression for prices. Here’s a regular expres-

sion for a dollar sign followed by a string of digits:

/$[0-9]+/

Note that the $ character has a different function here than the end-of-line function

we discussed earlier. Most regular expression parsers are smart enough to realize

that $ here doesn’t mean end-of-line. (As a thought experiment, think about how

regex parsers might ﬁgure out the function of $ from the context.)

Now we just need to deal with fractions of dollars. We’ll add a decimal point

and two digits afterwards:

/$[0-9]+\.[0-9][0-9]/

This pattern only allows $199.99 but not $199. We need to make the cents

optional and to make sure we’re at a word boundary:

/(ˆ|\W)$[0-9]+(\.[0-9][0-9])?\b/

One last catch! This pattern allows prices like $199999.99 which would be far

too expensive! We need to limit the dollar

/(ˆ|\W)$[0-9]{0,3}(\.[0-9][0-9])?\b/

How about speciﬁcations for > 6GHz processor speed? Here’s a pattern for that:

/\b[6-9]+ *(GHz|[Gg]igahertz)\b/

Note that we use / */ to mean “zero or more spaces” since there might always

be extra spaces lying around. For disk space, we’ll need to allow for optional frac-

tions again (5.5 GB); note the use of ? for making the ﬁnal s optional:

/\b[0-9]+(\.[0-9]+)? *(GB|[Gg]igabytes?)\b/

Modifying this regular expression so that it only matches more than 500 GB is

left as an exercise for the reader.

2.1.5 More Operators

Figure 2.7 shows some aliases for common ranges, which can be used mainly to

save typing. Besides the Kleene * and Kleene + we can also use explicit numbers as

2.1 • REGULAR EXPRESSIONS 17

counters, by enclosing them in curly brackets. The regular expression /{3}/ means

“exactly 3 occurrences of the previous character or expression”. So /a\.{24}z/

will match a followed by 24 dots followed by z (but not a followed by 23 or 25 dots

followed by a z).

RE Expansion Match First Matches

\d [0-9] any digit Party of 5

\D [ˆ0-9] any non-digit Blue moon

\w [a-zA-Z0-9_] any alphanumeric/underscore Daiyu

\W [ˆ\w] a non-alphanumeric !!!!

\s [ \r\t\n\f] whitespace (space, tab)

\S [ˆ\s] Non-whitespace in Concord

Figure 2.7 Aliases for common sets of characters.

A range of numbers can also be speciﬁed. So /{n,m}/ speciﬁes from n to m

occurrences of the previous char or expression, and /{n,}/ means at least n occur-

rences of the previous expression. REs for counting are summarized in Fig. 2.8.

RE Match

* zero or more occurrences of the previous char or expression

+ one or more occurrences of the previous char or expression

? exactly zero or one occurrence of the previous char or expression

{n} n occurrences of the previous char or expression

{n,m} from n to m occurrences of the previous char or expression

{n,} at least n occurrences of the previous char or expression

{,m} up to m occurrences of the previous char or expression

Figure 2.8 Regular expression operators for counting.

Finally, certain special characters are referred to by special notation based on the

backslash (\) (see Fig. 2.9). The most common of these are the newline character

Newline

\n and the tab character \t. To refer to characters that are special themselves (like

., *, [, and \), precede them with a backslash, (i.e., /\./, /\*/, /\[/, and /\\/).

RE Match First Patterns Matched

\* an asterisk “*” “K*A*P*L*A*N”

\. a period “.” “Dr. Livingston, I presume”

\? a question mark “Why don’t they come and lend a hand?”

\n a newline

\t a tab

Figure 2.9 Some characters that need to be backslashed.

2.1.6 Regular Expression Substitution, Capture Groups, and ELIZA

An important use of regular expressions is in substitutions. For example, the substi-

substitution

tution operator s/regexp1/pattern/ used in Python and in Unix commands like

vim or sed allows a string characterized by a regular expression to be replaced by

another string:

s/colour/color/

It is often useful to be able to refer to a particular subpart of the string matching

the ﬁrst pattern. For example, suppose we wanted to put angle brackets around all

18 CHAPTER 2 • REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE

integers in a text, for example, changing the 35 boxes to the <35> boxes. We’d

like a way to refer to the integer we’ve found so that we can easily add the brackets.

To do this, we put parentheses ( and ) around the ﬁrst pattern and use the number

operator \1 in the second pattern to refer back. Here’s how it looks:

s/([0-9]+)/<\1>/

The parenthesis and number operators can also specify that a certain string or

expression must occur twice in the text. For example, suppose we are looking for

the pattern “the Xer they were, the Xer they will be”, where we want to constrain

the two X’s to be the same string. We do this by surrounding the ﬁrst X with the

parenthesis operator, and replacing the second X with the number operator \1, as

follows:

/the (.*)er they were, the \1er they will be/

Here the \1 will be replaced by whatever string matched the ﬁrst item in paren-

theses. So this will match the bigger they were, the bigger they will be but not the

bigger they were, the faster they will be.

This use of parentheses to store a pattern in memory is called a capture group.

capture group

Every time a capture group is used (i.e., parentheses surround a pattern), the re-

sulting match is stored in a numbered register. If you match two different sets of

parentheses, \2 means whatever matched the second capture group. Thus

/the (.*)er they (.*), the \1er we \2/

will match the faster they ran, the faster we ran but not the faster they ran, the faster

we ate. Similarly, the third capture group is stored in \3, the fourth is \4, and so on.

Parentheses thus have a double function in regular expressions; they are used to

group terms for specifying the order in which operators should apply, and they are

used to capture something in a register. Occasionally we might want to use parenthe-

ses for grouping, but don’t want to capture the resulting pattern in a register. In that

case we use a non-capturing group, which is speciﬁed by putting the commands

non-capturing

group

?: after the open paren, in the form (?: pattern ).

/(?:some|a few) (people|cats) like some \1/

will match some cats like some cats but not some cats like some a few.

Substitutions and capture groups are very useful in implementing simple chat-

bots like ELIZA (Weizenbaum, 1966). Recall that ELIZA simulates a Rogerian

psychologist by carrying on conversations like the following:

User

: Men are all alike.

ELIZA

: IN WHAT WAY

User

: They’re always bugging us about something or other.

ELIZA

: CAN YOU THINK OF A SPECIFIC EXAMPLE

User

: Well, my boyfriend made me come here.

ELIZA

: YOUR BOYFRIEND MADE YOU COME HERE

User

: He says I’m depressed much of the time.

ELIZA

: I AM SORRY TO HEAR YOU ARE DEPRESSED

ELIZA works by having a series or cascade of regular expression substitutions

each of which matches and changes some part of the input lines. Input lines are

ﬁrst uppercased. The ﬁrst substitutions then change all instances of MY to YOUR,

and I’M to YOU ARE, and so on. The next set of substitutions matches and replaces

other patterns in the input. Here are some examples:

2.2 • WORDS 19

s/.* I’M (depressed|sad) .*/I AM SORRY TO HEAR YOU ARE \1/

s/.* I AM (depressed|sad) .*/WHY DO YOU THINK YOU ARE \1/

s/.* all .*/IN WHAT WAY/

s/.* always .*/CAN YOU THINK OF A SPECIFIC EXAMPLE/

Since multiple substitutions can apply to a given input, substitutions are assigned

a rank and applied in order. Creating patterns is the topic of Exercise 2.3, and we

return to the details of the ELIZA architecture in Chapter 25.

2.1.7 Lookahead assertions

Finally, there will be times when we need to predict the future: look ahead in the

text to see if some pattern matches, but not advance the match cursor, so that we can

then deal with the pattern if it occurs.

These lookahead assertions make use of the (? syntax that we saw in the previ-

lookahead

ous section for non-capture groups. The operator (?= pattern) is true if pattern

occurs, but is zero-width, i.e. the match pointer doesn’t advance. The operator

zero-width

(?! pattern) only returns true if a pattern does not match, but again is zero-width

and doesn’t advance the cursor. Negative lookahead is commonly used when we

are parsing some complex pattern but want to rule out a special case. For example

suppose we want to match, at the beginning of a line, any single word that doesn’t

start with “Volcano”. We can use negative lookahead to do this:

/ˆ(?!Volcano)[A-Za-z]+/

2.2 Words

Before we talk about processing words, we need to decide what counts as a word.

Let’s start by looking at one particular corpus (plural corpora), a computer-readable

corpus

corpora

collection of text or speech. For example the Brown corpus is a million-word col-

lection of samples from 500 written English texts from different genres (newspa-

per, ﬁction, non-ﬁction, academic, etc.), assembled at Brown University in 1963–64

(Ku

cera and Francis, 1967). How many words are in the following Brown sentence?

He stepped out into the hall, was delighted to encounter a water brother.

This sentence has 13 words if we don’t count punctuation marks as words, 15

if we count punctuation. Whether we treat period (“.”), comma (“,”), and so on as

words depends on the task. Punctuation is critical for ﬁnding boundaries of things

(commas, periods, colons) and for identifying some aspects of meaning (question

marks, exclamation marks, quotation marks). For some tasks, like part-of-speech

tagging or parsing or speech synthesis, we sometimes treat punctuation marks as if

they were separate words.

The Switchboard corpus of American English telephone conversations between

strangers was collected in the early 1990s; it contains 2430 conversations averaging

6 minutes each, totaling 240 hours of speech and about 3 million words (Godfrey

et al., 1992). Such corpora of spoken language don’t have punctuation but do intro-

duce other complications with regard to deﬁning words. Let’s look at one utterance

from Switchboard; an utterance is the spoken correlate of a sentence:

utterance

I do uh main- mainly business data processing

20 CHAPTER 2 • REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE

This utterance has two kinds of disﬂuencies. The broken-off word main- is

disﬂuency

called a fragment. Words like uh and um are called ﬁllers or ﬁlled pauses. Should

fragment

ﬁlled pause

we consider these to be words? Again, it depends on the application. If we are

building a speech transcription system, we might want to eventually strip out the

disﬂuencies.

But we also sometimes keep disﬂuencies around. Disﬂuencies like uh or um

are actually helpful in speech recognition in predicting the upcoming word, because

they may signal that the speaker is restarting the clause or idea, and so for speech

recognition they are treated as regular words. Because people use different disﬂu-

encies they can also be a cue to speaker identiﬁcation. In fact Clark and Fox Tree

(2002) showed that uh and um have different meanings. What do you think they are?

Are capitalized tokens like They and uncapitalized tokens like they the same

word? These are lumped together in some tasks (speech recognition), while for part-

of-speech or named-entity tagging, capitalization is a useful feature and is retained.

How about inﬂected forms like cats versus cat? These two words have the same

lemma cat but are different wordforms. A lemma is a set of lexical forms having

lemma

the same stem, the same major part-of-speech, and the same word sense. The word-

form is the full inﬂected or derived form of the word. For morphologically complex

wordform

languages like Arabic, we often need to deal with lemmatization. For many tasks in

English, however, wordforms are sufﬁcient.

How many words are there in English? To answer this question we need to

distinguish two ways of talking about words. Types are the number of distinct words

word type

in a corpus; if the set of words in the vocabulary is V, the number of types is the

vocabulary size |V|. Tokens are the total number N of running words. If we ignore

word token

punctuation, the following Brown sentence has 16 tokens and 14 types:

They picnicked by the pool, then lay back on the grass and looked at the stars.

When we speak about the number of words in the language, we are generally

referring to word types.

Corpus Tokens = N Types = |V|

Shakespeare 884 thousand 31 thousand

Brown corpus 1 million 38 thousand

Switchboard telephone conversations 2.4 million 20 thousand

COCA 440 million 2 million

Google N-grams 1 trillion 13 million

Figure 2.10 Rough numbers of types and tokens for some English language corpora. The

largest, the Google N-grams corpus, contains 13 million types, but this count only includes

types appearing 40 or more times, so the true number would be much larger.

Fig. 2.10 shows the rough numbers of types and tokens computed from some

popular English corpora. The larger the corpora we look at, the more word types

we ﬁnd, and in fact this relationship between the number of types |V| and number

of tokens N is called Herdan’s Law (Herdan, 1960) or Heaps’ Law (Heaps, 1978)

Herdan’s Law

Heaps’ Law

after its discoverers (in linguistics and information retrieval respectively). It is shown

in Eq. 2.1, where k and β are positive constants, and 0 < β < 1.

|V|= kN

(2.1)

The value of β depends on the corpus size and the genre, but at least for the

large corpora in Fig. 2.10, β ranges from .67 to .75. Roughly then we can say that

剩余472页未读，继续阅读

msen_yang

粉丝: 0
资源: 1

《Speech and Language Processing》：自然语言处理经典教材

程序员必读的十篇论文

图书：使用Python进行自然语言处理

自然语言处理

中文图书数据集数据挖掘自然语言处理中国图书分类法图书情报学数据挖掘文本分类.zip

北大语言学 自然语言处理课程 NLP系列课程 1_自然语言处理概论 共48页.pptx

Python自然语言处理入门经典教程

余弦相似性在自然语言处理图书推荐中的应用

统计自然语言处理基石：经典教材

Python与自然语言处理：经典入门指南

自然语言处理入门书籍

最新资源

北大语言学自然语言处理课程 NLP系列课程 1_自然语言处理概论共48页.pptx