《语音与语言处理》第三版：自然语言处理入门

需积分: 8 197 浏览量更新于2024-07-19 收藏 15.21MB PDF 举报

"Speech and Language Processing-3th" 是一本关于自然语言处理的入门经典书籍，由Daniel Jurafsky和James H. Martin合著，适用于英文基础较好的读者。这本书涵盖了广泛的NLP（自然语言处理）主题，包括计算语言学和语音识别。在书中，作者首先介绍了自然语言处理的基本概念，引导读者进入这一领域。接着，他们详细讨论了正则表达式和文本规范化，这是处理和理解自然语言数据的关键步骤。编辑距离的概念也被介绍，这对于比较和校对文本非常有用。第三章深入到有限状态转换器（Finite State Transducers），这是一种用于建模简单语言结构的工具，常见于语音识别和文本处理系统。第四章则转向语言建模，特别是n-gram模型，这是预测文本序列概率的基础，对语音识别和自动文本完成等任务至关重要。第五章涉及拼写纠正和嘈杂通道模型，这些技术可以改进输入错误或不准确的数据处理。第六章和第七章分别讲解朴素贝叶斯分类器和逻辑回归，它们在情感分析和其他文本分类任务中扮演重要角色。接下来，书中涵盖了神经网络和神经语言模型，这是深度学习在NLP领域的应用。隐藏马尔可夫模型（HMMs）在第九章被讲解，广泛应用于词性标注。第十章专门讨论了词性标注，这是理解句子结构的关键步骤。第十一章至第十四章探讨了英语的形式语法、句法解析、统计解析和依存关系解析，这些都是理解和生成自然语言句法结构的重要技术。第十五章和第十六章介绍了向量语义学和密集向量表示，这些方法使计算机能理解词语之间的语义关系。第十七章讨论了词义计算，第十八章则关注情感和影响提取的词汇资源。第十九章和第二十章涉及句子意义的表示和计算语义学，这是理解文本深层含义的高级话题。信息抽取、语义角色标注和共指消解在后面的章节中被介绍，这些都是从大量文本中提取结构化信息的核心技术。 "Speech and Language Processing-3th" 是一本全面的教材，适合对NLP感兴趣的初学者和专业人士，它覆盖了从基础概念到先进方法的广泛内容，是深入理解自然语言处理不可或缺的资源。

16 CHAPTER 2 • REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE

2.1.4 A More Complex Example

Let’s try out a more signiﬁcant example of the power of REs. Suppose we want to

build an application to help a user buy a computer on the Web. The user might want

“any machine with more than 6 GHz and 500 GB of disk space for less than $1000”.

To do this kind of retrieval, we ﬁrst need to be able to look for expressions like 6

GHz or 500 GB or Mac or $999.99. In the rest of this section we’ll work out some

simple regular expressions for this task.

First, let’s complete our regular expression for prices. Here’s a regular expres-

sion for a dollar sign followed by a string of digits:

/$[0-9]+/

Note that the $ character has a different function here than the end-of-line function

we discussed earlier. Regular expression parsers are in fact smart enough to realize

that $ here doesn’t mean end-of-line. (As a thought experiment, think about how

regex parsers might ﬁgure out the function of $ from the context.)

Now we just need to deal with fractions of dollars. We’ll add a decimal point

and two digits afterwards:

/$[0-9]+\.[0-9][0-9]/

This pattern only allows $199.99 but not $199. We need to make the cents

optional and to make sure we’re at a word boundary:

/\b$[0-9]+(\.[0-9][0-9])?\b/

How about speciﬁcations for processor speed? Here’s a pattern for that:

/\b[0-9]+ *(GHz|[Gg]igahertz)\b/

Note that we use / */ to mean “zero or more spaces” since there might always

be extra spaces lying around. We also need to allow for optional fractions again (5.5

GB); note the use of ? for making the ﬁnal s optional:

/\b[0-9]+(\.[0-9]+)? *(GB|[Gg]igabytes?)\b/

2.1.5 More Operators

Figure 2.7 shows some aliases for common ranges, which can be used mainly to

save typing. Besides the Kleene * and Kleene + we can also use explicit numbers as

counters, by enclosing them in curly brackets. The regular expression /{3}/ means

“exactly 3 occurrences of the previous character or expression”. So /a\.{24}z/

will match a followed by 24 dots followed by z (but not a followed by 23 or 25 dots

followed by a z).

RE Expansion Match First Matches

\d [0-9] any digit Party of 5

\D [ˆ0-9] any non-digit Blue moon

\w [a-zA-Z0-9_] any alphanumeric/underscore Daiyu

\W [ˆ\w] a non-alphanumeric !!!!

\s [ \r\t\n\f] whitespace (space, tab)

\S [ˆ\s] Non-whitespace in Concord

Figure 2.7 Aliases for common sets of characters.

2.1 • REGULAR EXPRESSIONS 17

A range of numbers can also be speciﬁed. So /{n,m}/ speciﬁes from n to m

occurrences of the previous char or expression, and /{n,}/ means at least n occur-

rences of the previous expression. REs for counting are summarized in Fig. 2.8.

RE Match

* zero or more occurrences of the previous char or expression

+ one or more occurrences of the previous char or expression

? exactly zero or one occurrence of the previous char or expression

{n} n occurrences of the previous char or expression

{n,m} from n to m occurrences of the previous char or expression

{n,} at least n occurrences of the previous char or expression

Figure 2.8 Regular expression operators for counting.

Finally, certain special characters are referred to by special notation based on the

backslash (\) (see Fig. 2.9). The most common of these are the newline character

Newline

\n and the tab character \t. To refer to characters that are special themselves (like

., *, [, and \), precede them with a backslash, (i.e., /\./, /\*/, /\[/, and /\\/).

RE Match First Patterns Matched

\* an asterisk “*” “K*A*P*L*A*N”

\. a period “.” “Dr. Livingston, I presume”

\? a question mark “Why don’t they come and lend a hand?”

\n a newline

\t a tab

Figure 2.9 Some characters that need to be backslashed.

2.1.6 Regular Expression Substitution, Capture Groups, and ELIZA

An important use of regular expressions is in substitutions. For example, the substi-

substitution

tution operator s/regexp1/pattern/ used in Python and in Unix commands like

vim or sed allows a string characterized by a regular expression to be replaced by

another string:

s/colour/color/

It is often useful to be able to refer to a particular subpart of the string matching

the ﬁrst pattern. For example, suppose we wanted to put angle brackets around all

integers in a text, for example, changing the 35 boxes to the <35> boxes. We’d

like a way to refer to the integer we’ve found so that we can easily add the brackets.

To do this, we put parentheses ( and ) around the ﬁrst pattern and use the number

operator \1 in the second pattern to refer back. Here’s how it looks:

s/([0-9]+)/<\1>/

The parenthesis and number operators can also specify that a certain string or

expression must occur twice in the text. For example, suppose we are looking for

the pattern “the Xer they were, the Xer they will be”, where we want to constrain

the two X’s to be the same string. We do this by surrounding the ﬁrst X with the

parenthesis operator, and replacing the second X with the number operator \1, as

follows:

/the (.*)er they were, the \1er they will be/

18 CHAPTER 2 • REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE

Here the \1 will be replaced by whatever string matched the ﬁrst item in paren-

theses. So this will match The bigger they were, the bigger they will be but not The

bigger they were, the faster they will be.

This use of parentheses to store a pattern in memory is called a capture group.

capture group

Every time a capture group is used (i.e., parentheses surround a pattern), the re-

sulting match is stored in a numbered register. If you match two different sets of

parentheses, \2 means whatever matched the second capture group. Thus

/the (.*)er they (.*), the \1er we \2/

will match The faster they ran, the faster we ran but not The faster they ran, the

faster we ate. Similarly, the third capture group is stored in \3, the fourth is \4, and

so on.

Parentheses thus have a double function in regular expressions; they are used to

group terms for specifying the order in which operators should apply, and they are

used to capture something in a register. Occasionally we might want to use parenthe-

ses for grouping, but don’t want to capture the resulting pattern in a register. In that

case we use a non-capturing group, which is speciﬁed by putting the commands

non-capturing

group

?: after the open paren, in the form (?: pattern ).

/(?:some|a few) (people|cats) like some \1/

will match some cats like some people but not some people like some a few.

Substitutions and capture groups are very useful in implementing simple chat-

bots like ELIZA (Weizenbaum, 1966). Recall that ELIZA simulates a Rogerian

psychologist by carrying on conversations like the following:

User

: Men are all alike.

ELIZA

: IN WHAT WAY

User

: They’re always bugging us about something or other.

ELIZA

: CAN YOU THINK OF A SPECIFIC EXAMPLE

User

: Well, my boyfriend made me come here.

ELIZA

: YOUR BOYFRIEND MADE YOU COME HERE

User

: He says I’m depressed much of the time.

ELIZA

: I AM SORRY TO HEAR YOU ARE DEPRESSED

ELIZA works by having a series or cascade of regular expression substitutions

each of which matches and changes some part of the input lines. The ﬁrst substitu-

tions change all instances of my to YOUR, and I’m to YOU ARE, and so on. The next

set of substitutions matches and replaces other patterns in the input. Here are some

examples:

s/.* I’M (depressed|sad) .*/I AM SORRY TO HEAR YOU ARE \1/

s/.* I AM (depressed|sad) .*/WHY DO YOU THINK YOU ARE \1/

s/.* all .*/IN WHAT WAY/

s/.* always .*/CAN YOU THINK OF A SPECIFIC EXAMPLE/

Since multiple substitutions can apply to a given input, substitutions are assigned

a rank and applied in order. Creating patterns is the topic of Exercise 2.3, and we

return to the details of the ELIZA architecture in Chapter 29.

2.1.7 Lookahead assertions

Finally, there will be times when we need to predict the future: look ahead in the

text to see if some pattern matches, but not advance the match cursor, so that we can

then deal with the pattern if it occurs.

2.2 • WORDS AND CORPORA 19

These lookahead assertions make use of the (? syntax that we saw in the previ-

lookahead

ous section for non-capture groups. The operator (?= pattern) is true if pattern

occurs, but is zero-width, i.e. the match pointer doesn’t advance. The operator

zero-width

(?! pattern) only returns true if a pattern does not match, but again is zero-width

and doesn’t advance the cursor. Negative lookahead is commonly used when we

are parsing some complex pattern but want to rule out a special case. For example

suppose we want to match, at the beginning of a line, any single word that doesn’t

start with ”Volcano”. We can use negative lookahead to do this:

/(ˆ?!Volcano)[A-Za-z]+/

2.2 Words and Corpora

Before we talk about processing words, we need to decide what counts as a word.

Let’s start by looking at a corpus (plural corpora), a computer-readable collection

corpus

corpora

of text or speech. For example the Brown corpus is a million-word collection of sam-

ples from 500 written texts from different genres (newspaper, ﬁction, non-ﬁction,

academic, etc.), assembled at Brown University in 1963–64 (Ku

cera and Francis,

1967). How many words are in the following Brown sentence?

He stepped out into the hall, was delighted to encounter a water brother.

This sentence has 13 words if we don’t count punctuation marks as words, 15

if we count punctuation. Whether we treat period (“.”), comma (“,”), and so on as

words depends on the task. Punctuation is critical for ﬁnding boundaries of things

(commas, periods, colons) and for identifying some aspects of meaning (question

marks, exclamation marks, quotation marks). For some tasks, like part-of-speech

tagging or parsing or speech synthesis, we sometimes treat punctuation marks as if

they were separate words.

The Switchboard corpus of telephone conversations between strangers was col-

lected in the early 1990s; it contains 2430 conversations averaging 6 minutes each,

totaling 240 hours of speech and about 3 million words (Godfrey et al., 1992). Such

corpora of spoken language don’t have punctuation but do introduce other compli-

cations with regard to deﬁning words. Let’s look at one utterance from Switchboard;

an utterance is the spoken correlate of a sentence:

utterance

I do uh main- mainly business data processing

This utterance has two kinds of disﬂuencies. The broken-off word main- is

disﬂuency

called a fragment. Words like uh and um are called ﬁllers or ﬁlled pauses. Should

fragment

ﬁlled pause

we consider these to be words? Again, it depends on the application. If we are

building a speech transcription system, we might want to eventually strip out the

disﬂuencies.

But we also sometimes keep disﬂuencies around. Disﬂuencies like uh or um

are actually helpful in speech recognition in predicting the upcoming word, because

they may signal that the speaker is restarting the clause or idea, and so for speech

recognition they are treated as regular words. Because people use different disﬂu-

encies they can also be a cue to speaker identiﬁcation. In fact Clark and Fox Tree

(2002) showed that uh and um have different meanings. What do you think they are?

Are capitalized tokens like They and uncapitalized tokens like they the same

word? These are lumped together in some tasks (speech recognition), while for part-

of-speech or named-entity tagging, capitalization is a useful feature and is retained.

20 CHAPTER 2 • REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE

How about inﬂected forms like cats versus cat? These two words have the same

lemma cat but are different wordforms. A lemma is a set of lexical forms having

lemma

the same stem, the same major part-of-speech, and the same word sense. The word-

form is the full inﬂected or derived form of the word. For morphologically complex

wordform

languages like Arabic, we often need to deal with lemmatization. For many tasks in

English, however, wordforms are sufﬁcient.

How many words are there in English? To answer this question we need to

distinguish two ways of talking about words. Types are the number of distinct words

word type

in a corpus; if the set of words in the vocabulary is V, the number of types is the

vocabulary size |V|. Tokens are the total number N of running words. If we ignore

word token

punctuation, the following Brown sentence has 16 tokens and 14 types:

They picnicked by the pool, then lay back on the grass and looked at the stars.

When we speak about the number of words in the language, we are generally

referring to word types.

Corpus Tokens = N Types = |V|

Shakespeare 884 thousand 31 thousand

Brown corpus 1 million 38 thousand

Switchboard telephone conversations 2.4 million 20 thousand

COCA 440 million 2 million

Google N-grams 1 trillion 13 million

Figure 2.10 Rough numbers of types and tokens for some corpora. The largest, the Google

N-grams corpus, contains 13 million types, but this count only includes types appearing 40 or

more times, so the true number would be much larger.

Fig. 2.10 shows the rough numbers of types and tokens computed from some

popular English corpora. The larger the corpora we look at, the more word types

we ﬁnd, and in fact this relationship between the number of types |V | and number

of tokens N is called Herdan’s Law (Herdan, 1960) or Heaps’ Law (Heaps, 1978)

Herdan’s Law

Heaps’ Law

after its discoverers (in linguistics and information retrieval respectively). It is shown

in Eq. 2.1, where k and β are positive constants, and 0 < β < 1.

|V| = kN

(2.1)

The value of β depends on the corpus size and the genre, but at least for the

large corpora in Fig. 2.10, β ranges from .67 to .75. Roughly then we can say that

the vocabulary size for a text goes up signiﬁcantly faster than the square root of its

length in words.

Another measure of the number of words in the language is the number of lem-

mas instead of wordform types. Dictionaries can help in giving lemma counts; dic-

tionary entries or boldface forms are a very rough upper bound on the number of

lemmas (since some lemmas have multiple boldface forms). The 1989 edition of the

Oxford English Dictionary had 615,000 entries.

2.3 Text Normalization

Before almost any natural language processing of a text, the text has to be normal-

ized. At least three tasks are commonly applied as part of any normalization process:

1. Segmenting/tokenizing words from running text

剩余498页未读，继续阅读

CCWinner

粉丝: 1
资源: 14

《语音与语言处理》第三版：自然语言处理入门

UNIX Network Programming Volume.1-3th

java7hashmap源码-Effective-Java-3th:Effective-Java-3th

Software-Engineering-Theory-and-Practice 3th

digital image processing 3th 中文

digital signal processing s.k. mitra 3th edition

pass a context object connect 3th paramter

openDocument:fail not found 3th activity

uniapp打开文档时openDocument:fail not found 3th activity

game engine architecture 3th pdf

"errMsg": "openDocument:fail not found 3th activity",

最新资源