模式识别：理论与实践

5星 · 超过95%的资源需积分: 19 142 浏览量更新于2024-07-30 1 收藏 12.32MB PDF 举报

"Pattern Recognition 4th Edition.pdf" 是一本由Elsevier出版的关于模式识别的学术书籍，适合电气与电子工程、计算机工程、计算机科学、信息学等不同背景的学生，以及参与自动化项目的跨学科研究生阅读。这本书由作者在近20年教学高级本科和研究生课程的经验基础上编写，旨在尽可能自成一体，满足不同背景读者的需求。读者只需具备基本微积分、线性代数和概率理论基础知识即可。书中附有四个附录，涵盖了概率统计和约束优化等数学工具，适用于一学期或两学期的课程，并可用作自我学习和研究参考书。本书的内容可能涵盖模式识别的基本概念、理论和应用，可能包括特征提取、分类算法（如支持向量机、决策树、神经网络等）、图像处理、信号处理、机器学习以及数据挖掘等领域。此外，还可能讨论了概率模型（如高斯混合模型）、贝叶斯决策理论、统计假设检验等相关主题。书中通过实例和练习题帮助读者理解和应用这些理论，以解决实际问题。在版权方面，这本书明确指出未经许可，不得以任何形式复制或传播，包括电子和机械方式，如影印、录音或信息存储检索系统。若需获得复制或使用权限，应直接联系出版社获取许可。此外，该书已申请美国国会图书馆和英国图书馆的编目，具有正式的国际标准书号（ISBN:978-1-59749-272-0）。 "Pattern Recognition 4th Edition.pdf" 是一本深入浅出的模式识别教材，不仅适用于学生学习，也适用于专业人士作为研究和实践的参考。它将理论知识与实际应用相结合，是理解和掌握模式识别技术的重要资源。

“03-Ch01-SA272” 17/9/2008 page 12

12 CHAPTER 1 Introduction

the computer. The sections related to estimation of the number of clusters and

neural network implementations are bypassed.

Chapter 13 deals with hierarchical clustering algorithms. In a ﬁrst course, only

the general a gglomerative scheme is considered with an emphasis on single link

and complete link algorithms, based on matrix theory. Agglomerative algorithms

based on graph theory concepts as well as the divisive schemes are bypassed.

Chapter 14 deals with clustering algorithms based on cost function optimization,

using tools from differential calculus. Hard clustering and fuzzy and possibilistic

schemes are considered,based on various types of cluster representatives,including

point representatives,hyperplane representatives,and shell-shaped representatives.

In a ﬁrst course, most of these algorithms are b ypassed, and emphasis is given to

the isodata algorithm.

Chapter 15 features a high degree of modularity. It deals with clustering algo-

rithms based on different ideas,which cannot be grouped under a single philosophy.

Spectral clustering, competitive lear ning, branch and bound, simulated annealing,

and genetic algorithms are some of the schemes treated in this chapter. These are

bypassed in a ﬁrst course.

Chapter 16 deals with the clustering validity stage of a clustering procedure. It

contains rather advanced concepts and is omitted in a ﬁrst course. Emphasis is given

to the deﬁnitions of internal,external,and relative cr iteria and the random hypothe-

ses used in each case. Indices, adopted in the framework of external and internal

criteria,are presented,and examples are provided showing the use of these indices.

Syntactic pattern recognition methods are not treated in this book. Syntactic

pattern recognition methods differ in philosophy from the methods discussed in

this book and, in general, are applicable to different types of problems. In syntactic

pattern recognition, the structure of the patterns is of paramount importance, and

pattern recognition is performed on the basis of a set of pattern primitives, a set

of rules in the form of a grammar, and a recognizer called automaton. Thus, we

were faced with a dilemma: either to increase the size of the book substantially, or

to provide a short overview (which, however, exists in a number of other books),

or to omit it. The last option seemed to be the most sensible choice.

“04-Ch02-SA272” 18/9/2008 page 13

CHAPTER

Classiﬁers Based on

Bayes Decision Theory

2.1 INTRODUCTION

This is the ﬁrst chapter, out of three, dealing with the design of the classiﬁer in a

pattern recognition system. The approach to be followed builds upon probabilistic

arguments stemming from the statistical nature of the generated features. As has

already been pointed out in the introductory chapter, this is due to the statistical

variation of the patterns as well as to the noise in the measuring sensors. Adopting

this reasoning as our kickoff point,we will design classiﬁers that classify an unknown

pattern in the most probable of the classes. Thus, our task now becomes that of

deﬁning what “most probable”means.

Given a classiﬁcation task of M classes,␻

, ␻

, ..., ␻

,and an unknown pattern,

which is represented by a feature vector x,we form the M conditional probabilities

P(␻

|x), i ⫽ 1, 2, ..., M. Sometimes, these are also referred to as a posteriori

probabilities. In words, each of them represents the probability that the unknown

pattern belongs to the respective class ␻

, given that the corresponding feature

vector takes the value x. Who could then argue that these conditional probabilities

are not sensible choices to quantify the term most probable? Indeed, the classiﬁers

to be considered in this chapter compute either the maximum of these M values

or, equivalently, the maximum of an appropriately deﬁned function of them. The

unknown pattern is then assigned to the class corresponding to this maximum.

The ﬁrst task we are faced with is the computation of the conditional proba-

bilities. The Bayes rule will once more prove its usefulness! A major effort in this

chapter will be devoted to techniques for estimating probability density functions

(pdf ), based on the available experimental evidence, that is, the feature vectors

corresponding to the patterns of the training set.

2.2 BAYES DECISION THEORY

We will initially focus on the two-class case. Let ␻

, ␻

be the two classes in which

our patterns belong. In the sequel, we assume that the a priori probabilities

“04-Ch02-SA272” 18/9/2008 page 14

14 CHAPTER 2 Classiﬁers Based on Bayes Decision Theory

P(␻

), P(␻

) are known. This is a very reasonable assumption, because even if

they are not known, they can easily be estimated from the available training feature

vectors. Indeed, if N is the total number of available training patterns, and N

, N

of them belong to ␻

and ␻

, respectively, then P(␻

) ≈ N

/N and P(␻

) ≈ N

/N .

The other statistical quantities assumed to be known are the class-conditional

probability density functions p(x|␻

), i ⫽ 1, 2, describing the distribution of the

feature vectors in each of the classes. If these are not known, they can also be

estimated from the available training data, as we will discuss later on in this chapter.

The pdf p(x|␻

) is sometimes referred to as the likelihood function of ␻

with

respect to x. Here we should stress the fact that an implicit assumption has been

made. That is, the feature vectors can take any value in the l-dimensional feature

space. In the case that feature vectors can take only discrete values,density functions

p(x|␻

) become probabilities and will be denoted by P(x|␻

We now have all the ingredients to compute our conditional probabilities, as

stated in the introduction. To this end, let us recall from our probability course

basics the Bayes rule (Appendix A)

P(␻

|x) ⫽

p(x|␻

)P(␻

)

p(x)

(2.1)

where p(x) is the pdf of x and for which we have (Appendix A)

p(x) ⫽



i⫽1

p(x|␻

)P(␻

) (2.2)

The Bayes c lassiﬁcation rule can now be stated as

If P(␻

|x) ⬎ P(␻

|x), x is classiﬁed to ␻

If P(␻

|x) ⬍ P(␻

|x), x is classiﬁed to ␻

(2.3)

The case of equality is detrimental and the pattern can be assigned to either of the

two classes. Using (2.1), the decision can equivalently be based on the inequalities

p(x|␻

)P(␻

) ≷ p(x|␻

)P(␻

) (2.4)

p(x) is not taken into account, because it is the same for all classes and it does

not affect the decision. Furthermore, if the a priori probabilities are equal, that is,

P(␻

) ⫽ P(␻

) ⫽ 1/2, Eq. (2.4) becomes

p(x|␻

) ≷ p(x|␻

) (2.5)

Thus, the search for the maximum now rests on the values of the conditional pdfs

evaluated at x. Figure 2.1 presents an example of two equiprobable classes and

shows the variations of p(x|␻

), i ⫽ 1, 2, as functions of x for the simple case of a

single feature (l ⫽ 1). The dotted line at x

is a threshold partitioning the feature

space into two regions,R

and R

. According to the Bayes decision rule,for all values

of x in R

the classiﬁer decides ␻

and for all values in R

it decides ␻

. However,

it is obvious from the ﬁgure that decision errors are unavoidable. Indeed, there is

“04-Ch02-SA272” 18/9/2008 page 16

16 CHAPTER 2 Classiﬁers Based on Bayes Decision Theory

where P(·, ·) is the joint probability of two events. Recalling, once more, our

probability basics (Appendix A), this becomes

⫽ P(x ∈ R

|␻

)P(␻

) ⫹ P(x ∈ R

|␻

)P(␻

)

⫽ P(␻

)



p(x|␻

) dx ⫹ P(␻

)



p(x|␻

) dx (2.8)

or using the Bayes rule

⫽



P(␻

|x)p(x) dx ⫹



P(␻

|x)p(x) dx (2.9)

It is now easy to see that the error is minimized if the partitioning regions R

and

of the feature space are chosen so that

: P(␻

|x) ⬎ P(␻

|x)

: P(␻

|x) ⬎ P(␻

|x) (2.10)

Indeed,since the union of the regions R

, R

covers all the space,from the deﬁnition

of a probability density function we have that



P(␻

|x)p(x) dx ⫹



P(␻

|x)p(x) dx ⫽ P(␻

) (2.11)

Combining Eqs. (2.9) and (2.11), we get

⫽ P(␻

) ⫺



(

P(␻

|x) ⫺ P(␻

|x)

)

p(x) dx (2.12)

This suggests that the probability of error is minimized if R

is the region of space in

which P(␻

|x) ⬎ P(␻

|x). Then, R

becomes the region where the reverse is true.

So far, we have dealt with the simple case of two classes. Generalizations to

the multiclass case are straightforward. In a classiﬁcation task with M classes,

␻

, ␻

, ..., ␻

,an unknown pattern,represented by the feature vector x,is assigned

to class ␻

P(␻

|x) ⬎ P(␻

|x) ᭙j ⫽ i (2.13)

It turns out that such a choice also minimizes the classiﬁcation error probability

( Problem 2.1).

Minimizing the Average Risk

The classiﬁcation error probability is not always the best criterion to be adopted for

minimization. This is because it assigns the same importance to all errors. However,

there are cases in which some wrong decisions may have more serious implications

than others. For example, it is much more serious for a doctor to make a wrong

decision and a malignant tumor to be diagnosed as a benign one, than the other

way round. If a benign tumor is diagnosed as a malignant one, the wrong decision

will be cleared out during subsequent clinical examinations. However, the results

剩余956页未读，继续阅读

fanbird2008

粉丝: 208
资源: 249

模式识别：理论与实践

模式识别第四版答案（pattern recognition fourth edition solution）

Pattern Recognition

Pattern Recognition and Machine Learning高清pdf

Pattern Recognition 4th Edition模式识别第四版 MATLAB demo以及说明

模式识别权威指南：Pattern Recognition 4th Edition

pattern recognition 4th edition matlab

Pattern Recognition (Fourth Edition) by Sergios Theodoridis 高清pdf_part02

Pattern+Recognition+4th+Edition模式识别第四版_patternrecognition_模式识别_源

Pattern Recognition and Machine Learning 课后习题完整答案

模式识别 Pattern Recognition 英文版 第四版 By Sergios Theodoridis

最新资源

模式识别 Pattern Recognition 英文版第四版 By Sergios Theodoridis