自然语言处理与语音识别入门：核心技术概览

需积分: 8 185 浏览量更新于2024-07-17 收藏 15.93MB PDF 举报

《语音与语言处理》是一本介绍自然语言处理、计算语言学以及语音识别领域的入门教材，第三版草案由丹尼尔·朱拉夫斯基（Daniel Jurafsky）和詹姆斯·H·马丁（James H. Martin）合著，两位作者分别来自斯坦福大学和科罗拉多大学博尔德分校。该书版权日期为2018年，最新版本更新至2018年8月12日，欢迎读者提出宝贵意见和修正错漏。本书的主要内容涵盖了多个核心主题，从基础知识到前沿技术： 1. **介绍**：首先引导读者了解自然语言处理的背景和重要性，为后续章节打下基础。 2. **正则表达式与文本规范化**：讲解如何处理文本数据，包括常用的字符串操作和规范化技术，如编辑距离。 3. **n-gram语言模型**：介绍基于统计的词序列模型，用于预测文本的概率分布。 4. **朴素贝叶斯与情感分类**：通过实例演示简单概率模型在文本分类任务中的应用，特别是情感分析。 5. **逻辑回归**：探讨线性回归在NLP中的应用，特别是在预测任务中的作用。 6. **向量语义**：介绍词汇表征的演变，如何利用词向量表示文本的语义关系。 7. **神经网络与神经语言模型**：深入讲解深度学习在语言处理中的应用，如循环神经网络（RNN）和卷积神经网络（CNN）。 8. **词性标注**：介绍如何识别句子中每个单词的词性，是构建语法模型的基础。 9. **递归神经网络**：探讨递归结构在处理语法结构时的优势。 10. **英语形式语法**：介绍英语句子的句法规则，为后续的句法分析提供理论框架。 11. **句法解析**：区分句法分析的多种方法，包括统计语法分析和依存句法分析。 12. **统计句法分析**：侧重于基于概率的句法树构造算法。 13. **依存句法分析**：详细解释词语之间的依存关系及其在语义理解中的作用。 14. **句子意义的表示**：讨论如何将文本转化为形式化的语义表示，如语义角色标注。 15. **计算语义学**：探讨词汇和短语的意义表示，以及如何量化它们的相似度。 16. **语义解析**：将自然语言表达转化为机器可理解的形式，以便执行复杂的查询或推理。 17. **信息抽取**：提取文本中的关键信息，如命名实体、事件等。 18. **语义角色标注**：识别动作的发起者、承受者等语义角色。 19. **词义计算**：研究如何处理同义词和多义词，提高理解和生成的准确性。 20. **情感和情绪提取词典**：利用情感词典支持情感分析和情绪识别。 21. **核心ference分辨率和实体链接**：解决指称消解问题，即同一实体在文本中的不同称呼如何关联。 22. **篇章连贯性**：研究文本段落间的逻辑关系和上下文一致性。 23. **机器翻译**：探讨跨语言的自然语言处理技术，包括翻译模型的构建和评估。《语音与语言处理》不仅涵盖了基础理论，还涉及了当前最热门的技术，对于希望进入这个领域学习或从事NLP工作的读者来说，是一本不可或缺的参考书籍。

16 CHAPTER 2 • REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE

errors comes up again and again in implementing speech and language processing

systems. Reducing the overall error rate for an application thus involves two antag-

onistic efforts:

• Increasing precision (minimizing false positives)

• Increasing recall (minimizing false negatives)

2.1.4 A More Complex Example

Let’s try out a more signiﬁcant example of the power of REs. Suppose we want to

build an application to help a user buy a computer on the Web. The user might want

“any machine with at least 6 GHz and 500 GB of disk space for less than $1000”.

To do this kind of retrieval, we ﬁrst need to be able to look for expressions like 6

GHz or 500 GB or Mac or $999.99. In the rest of this section we’ll work out some

simple regular expressions for this task.

First, let’s complete our regular expression for prices. Here’s a regular expres-

sion for a dollar sign followed by a string of digits:

/$[0-9]+/

Note that the $ character has a different function here than the end-of-line function

we discussed earlier. Most regular expression parsers are smart enough to realize

that $ here doesn’t mean end-of-line. (As a thought experiment, think about how

regex parsers might ﬁgure out the function of $ from the context.)

Now we just need to deal with fractions of dollars. We’ll add a decimal point

and two digits afterwards:

/$[0-9]+\.[0-9][0-9]/

This pattern only allows $199.99 but not $199. We need to make the cents

optional and to make sure we’re at a word boundary:

/(ˆ|\W)$[0-9]+(\.[0-9][0-9])?\b/

One last catch! This pattern allows prices like $199999.99 which would be far

too expensive! We need to limit the dollar

/(ˆ|\W)$[0-9]{0,3}(\.[0-9][0-9])?\b/

How about speciﬁcations for > 6GHz processor speed? Here’s a pattern for that:

/\b[6-9]+ *(GHz|[Gg]igahertz)\b/

Note that we use / */ to mean “zero or more spaces” since there might always

be extra spaces lying around. For disk space, we’ll need to allow for optional frac-

tions again (5.5 GB); note the use of ? for making the ﬁnal s optional:

/\b[0-9]+(\.[0-9]+)? *(GB|[Gg]igabytes?)\b/

Modifying this regular expression so that it only matches more than 500 GB is

left as an exercise for the reader.

2.1.5 More Operators

Figure 2.7 shows some aliases for common ranges, which can be used mainly to

save typing. Besides the Kleene * and Kleene + we can also use explicit numbers as

2.1 • REGULAR EXPRESSIONS 17

counters, by enclosing them in curly brackets. The regular expression /{3}/ means

“exactly 3 occurrences of the previous character or expression”. So /a\.{24}z/

will match a followed by 24 dots followed by z (but not a followed by 23 or 25 dots

followed by a z).

RE Expansion Match First Matches

\d [0-9] any digit Party of 5

\D [ˆ0-9] any non-digit Blue moon

\w [a-zA-Z0-9_] any alphanumeric/underscore Daiyu

\W [ˆ\w] a non-alphanumeric !!!!

\s [ \r\t\n\f] whitespace (space, tab)

\S [ˆ\s] Non-whitespace in Concord

Figure 2.7 Aliases for common sets of characters.

A range of numbers can also be speciﬁed. So /{n,m}/ speciﬁes from n to m

occurrences of the previous char or expression, and /{n,}/ means at least n occur-

rences of the previous expression. REs for counting are summarized in Fig. 2.8.

RE Match

* zero or more occurrences of the previous char or expression

+ one or more occurrences of the previous char or expression

? exactly zero or one occurrence of the previous char or expression

{n} n occurrences of the previous char or expression

{n,m} from n to m occurrences of the previous char or expression

{n,} at least n occurrences of the previous char or expression

{,m} up to m occurrences of the previous char or expression

Figure 2.8 Regular expression operators for counting.

Finally, certain special characters are referred to by special notation based on the

backslash (\) (see Fig. 2.9). The most common of these are the newline character

Newline

\n and the tab character \t. To refer to characters that are special themselves (like

., *, [, and \), precede them with a backslash, (i.e., /\./, /\*/, /\[/, and /\\/).

RE Match First Patterns Matched

\* an asterisk “*” “K*A*P*L*A*N”

\. a period “.” “Dr. Livingston, I presume”

\? a question mark “Why don’t they come and lend a hand?”

\n a newline

\t a tab

Figure 2.9 Some characters that need to be backslashed.

2.1.6 Regular Expression Substitution, Capture Groups, and ELIZA

An important use of regular expressions is in substitutions. For example, the substi-

substitution

tution operator s/regexp1/pattern/ used in Python and in Unix commands like

vim or sed allows a string characterized by a regular expression to be replaced by

another string:

s/colour/color/

It is often useful to be able to refer to a particular subpart of the string matching

the ﬁrst pattern. For example, suppose we wanted to put angle brackets around all

18 CHAPTER 2 • REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE

integers in a text, for example, changing the 35 boxes to the <35> boxes. We’d

like a way to refer to the integer we’ve found so that we can easily add the brackets.

To do this, we put parentheses ( and ) around the ﬁrst pattern and use the number

operator \1 in the second pattern to refer back. Here’s how it looks:

s/([0-9]+)/<\1>/

The parenthesis and number operators can also specify that a certain string or

expression must occur twice in the text. For example, suppose we are looking for

the pattern “the Xer they were, the Xer they will be”, where we want to constrain

the two X’s to be the same string. We do this by surrounding the ﬁrst X with the

parenthesis operator, and replacing the second X with the number operator \1, as

follows:

/the (.*)er they were, the \1er they will be/

Here the \1 will be replaced by whatever string matched the ﬁrst item in paren-

theses. So this will match the bigger they were, the bigger they will be but not the

bigger they were, the faster they will be.

This use of parentheses to store a pattern in memory is called a capture group.

capture group

Every time a capture group is used (i.e., parentheses surround a pattern), the re-

sulting match is stored in a numbered register. If you match two different sets of

parentheses, \2 means whatever matched the second capture group. Thus

/the (.*)er they (.*), the \1er we \2/

will match the faster they ran, the faster we ran but not the faster they ran, the faster

we ate. Similarly, the third capture group is stored in \3, the fourth is \4, and so on.

Parentheses thus have a double function in regular expressions; they are used to

group terms for specifying the order in which operators should apply, and they are

used to capture something in a register. Occasionally we might want to use parenthe-

ses for grouping, but don’t want to capture the resulting pattern in a register. In that

case we use a non-capturing group, which is speciﬁed by putting the commands

non-capturing

group

?: after the open paren, in the form (?: pattern ).

/(?:some|a few) (people|cats) like some \1/

will match some cats like some cats but not some cats like some a few.

Substitutions and capture groups are very useful in implementing simple chat-

bots like ELIZA (Weizenbaum, 1966). Recall that ELIZA simulates a Rogerian

psychologist by carrying on conversations like the following:

User

: Men are all alike.

ELIZA

: IN WHAT WAY

User

: They’re always bugging us about something or other.

ELIZA

: CAN YOU THINK OF A SPECIFIC EXAMPLE

User

: Well, my boyfriend made me come here.

ELIZA

: YOUR BOYFRIEND MADE YOU COME HERE

User

: He says I’m depressed much of the time.

ELIZA

: I AM SORRY TO HEAR YOU ARE DEPRESSED

ELIZA works by having a series or cascade of regular expression substitutions

each of which matches and changes some part of the input lines. Input lines are

ﬁrst uppercased. The ﬁrst substitutions then change all instances of MY to YOUR,

and I’M to YOU ARE, and so on. The next set of substitutions matches and replaces

other patterns in the input. Here are some examples:

2.2 • WORDS 19

s/.* I’M (depressed|sad) .*/I AM SORRY TO HEAR YOU ARE \1/

s/.* I AM (depressed|sad) .*/WHY DO YOU THINK YOU ARE \1/

s/.* all .*/IN WHAT WAY/

s/.* always .*/CAN YOU THINK OF A SPECIFIC EXAMPLE/

Since multiple substitutions can apply to a given input, substitutions are assigned

a rank and applied in order. Creating patterns is the topic of Exercise 2.3, and we

return to the details of the ELIZA architecture in Chapter 25.

2.1.7 Lookahead assertions

Finally, there will be times when we need to predict the future: look ahead in the

text to see if some pattern matches, but not advance the match cursor, so that we can

then deal with the pattern if it occurs.

These lookahead assertions make use of the (? syntax that we saw in the previ-

lookahead

ous section for non-capture groups. The operator (?= pattern) is true if pattern

occurs, but is zero-width, i.e. the match pointer doesn’t advance. The operator

zero-width

(?! pattern) only returns true if a pattern does not match, but again is zero-width

and doesn’t advance the cursor. Negative lookahead is commonly used when we

are parsing some complex pattern but want to rule out a special case. For example

suppose we want to match, at the beginning of a line, any single word that doesn’t

start with “Volcano”. We can use negative lookahead to do this:

/ˆ(?!Volcano)[A-Za-z]+/

2.2 Words

Before we talk about processing words, we need to decide what counts as a word.

Let’s start by looking at one particular corpus (plural corpora), a computer-readable

corpus

corpora

collection of text or speech. For example the Brown corpus is a million-word col-

lection of samples from 500 written English texts from different genres (newspa-

per, ﬁction, non-ﬁction, academic, etc.), assembled at Brown University in 1963–64

(Ku

cera and Francis, 1967). How many words are in the following Brown sentence?

He stepped out into the hall, was delighted to encounter a water brother.

This sentence has 13 words if we don’t count punctuation marks as words, 15

if we count punctuation. Whether we treat period (“.”), comma (“,”), and so on as

words depends on the task. Punctuation is critical for ﬁnding boundaries of things

(commas, periods, colons) and for identifying some aspects of meaning (question

marks, exclamation marks, quotation marks). For some tasks, like part-of-speech

tagging or parsing or speech synthesis, we sometimes treat punctuation marks as if

they were separate words.

The Switchboard corpus of American English telephone conversations between

strangers was collected in the early 1990s; it contains 2430 conversations averaging

6 minutes each, totaling 240 hours of speech and about 3 million words (Godfrey

et al., 1992). Such corpora of spoken language don’t have punctuation but do intro-

duce other complications with regard to deﬁning words. Let’s look at one utterance

from Switchboard; an utterance is the spoken correlate of a sentence:

utterance

I do uh main- mainly business data processing

20 CHAPTER 2 • REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE

This utterance has two kinds of disﬂuencies. The broken-off word main- is

disﬂuency

called a fragment. Words like uh and um are called ﬁllers or ﬁlled pauses. Should

fragment

ﬁlled pause

we consider these to be words? Again, it depends on the application. If we are

building a speech transcription system, we might want to eventually strip out the

disﬂuencies.

But we also sometimes keep disﬂuencies around. Disﬂuencies like uh or um

are actually helpful in speech recognition in predicting the upcoming word, because

they may signal that the speaker is restarting the clause or idea, and so for speech

recognition they are treated as regular words. Because people use different disﬂu-

encies they can also be a cue to speaker identiﬁcation. In fact Clark and Fox Tree

(2002) showed that uh and um have different meanings. What do you think they are?

Are capitalized tokens like They and uncapitalized tokens like they the same

word? These are lumped together in some tasks (speech recognition), while for part-

of-speech or named-entity tagging, capitalization is a useful feature and is retained.

How about inﬂected forms like cats versus cat? These two words have the same

lemma cat but are different wordforms. A lemma is a set of lexical forms having

lemma

the same stem, the same major part-of-speech, and the same word sense. The word-

form is the full inﬂected or derived form of the word. For morphologically complex

wordform

languages like Arabic, we often need to deal with lemmatization. For many tasks in

English, however, wordforms are sufﬁcient.

How many words are there in English? To answer this question we need to

distinguish two ways of talking about words. Types are the number of distinct words

word type

in a corpus; if the set of words in the vocabulary is V, the number of types is the

vocabulary size |V|. Tokens are the total number N of running words. If we ignore

word token

punctuation, the following Brown sentence has 16 tokens and 14 types:

They picnicked by the pool, then lay back on the grass and looked at the stars.

When we speak about the number of words in the language, we are generally

referring to word types.

Corpus Tokens = N Types = |V|

Shakespeare 884 thousand 31 thousand

Brown corpus 1 million 38 thousand

Switchboard telephone conversations 2.4 million 20 thousand

COCA 440 million 2 million

Google N-grams 1 trillion 13 million

Figure 2.10 Rough numbers of types and tokens for some English language corpora. The

largest, the Google N-grams corpus, contains 13 million types, but this count only includes

types appearing 40 or more times, so the true number would be much larger.

Fig. 2.10 shows the rough numbers of types and tokens computed from some

popular English corpora. The larger the corpora we look at, the more word types

we ﬁnd, and in fact this relationship between the number of types |V| and number

of tokens N is called Herdan’s Law (Herdan, 1960) or Heaps’ Law (Heaps, 1978)

Herdan’s Law

Heaps’ Law

after its discoverers (in linguistics and information retrieval respectively). It is shown

in Eq. 2.1, where k and β are positive constants, and 0 < β < 1.

|V|= kN

(2.1)

The value of β depends on the corpus size and the genre, but at least for the

large corpora in Fig. 2.10, β ranges from .67 to .75. Roughly then we can say that

剩余470页未读，继续阅读

eefelix

粉丝: 0
资源: 2

自然语言处理与语音识别入门：核心技术概览

SPEECH and LANGUAGE PROCESSING

Speech and Language Processing, 2nd Edition.pdf

推荐下自然语言处理的教材

自然语言处理技术的书推荐几本

我想学习自然语言处理，帮我推荐几本书籍

Spark和NLP技术参考文献

partial frequency trajectory

能详细说下训练chatgpt这样的机器人需要看那些书籍吗

国外文本挖掘研究现状和参考文献

国外文本挖掘应用参考文献

最新资源