没有合适的资源?快使用搜索试试~ 我知道了~
首页模式识别(Pattern Classification)英文版课后答案与解析
模式识别(Pattern Classification)英文版课后答案与解析
需积分: 9 0 下载量 171 浏览量
更新于2024-07-14
收藏 1.09MB PDF 举报
"模式识别Pattern Classification的英文版课后习题答案分享,这份资源来自于CSDN,并由用户tommyod在GitHub上更新于2018年12月11日。" "Pattern Recognition"是由Richard O. Duda, Peter E. Hart和David G. Stork合著的一本经典书籍,尽管出版于2001年,但其第二版经受住了时间的考验,成为模式识别或机器学习领域的广泛引用和深受欢迎的入门教材。据谷歌学术统计,该书已获得近40,000次引用。 提供的文档包含了书中部分习题的解答,这些解答是用LaTeX排版的,旨在帮助作者自己的学习过程。作者强调,虽然包含章节摘要,但它们不能替代原书的内容,如果没有原书,这些摘要可能难以理解。对于那些希望深入学习机器学习的人来说,作者推荐购买原著。 近年来,机器学习领域获得了巨大的关注,尽管网上有许多有价值的资源,但在全面理解和掌握知识点时,书籍仍然是首选。作者分享的解答应能对遇到困难的读者有所帮助,鼓励读者在寻求答案前先自己尝试解决问题。 这份资源涵盖了图像处理、模式分类和机器学习的相关习题解答,对于正在学习这些主题的读者来说,是一份宝贵的参考资料。通过解答,读者可以加深对模式识别算法的理解,提升解决实际问题的能力,包括但不限于特征提取、分类模型的构建、训练与验证以及性能评估等方面的知识。
资源详情
资源推荐
![](https://csdnimg.cn/release/download_crawler_static/16069516/bgd.jpg)
Figure 7: A dendrogram representation of hierarchical clustering. The source is
Wikipedia.
• Competitive learning is a clustering algorithm implemented by a neural network.
The data x ∈ R
d
is augmented to x := (1, x) and normalized so that it lies on
a (d + 1)-dimensional sphere. The weight w
∗
= arg max
w
w
T
x is updated to be
more similar to the sample x. The algorithm can also create new clusters w
j
if
certain conditions are met. It’s suitable for on-line learning, but will not necessarily
converge with constant learning rate.
• PCA can be implemented as a 3-layer neural network autoencoder. The first k
eigenvalues/eigenvectors of X solves the minimization of kX − X
0
k
k, where X
0
k
is
a rank k approximation of X. Non-linear PCA is an extension of the idea, using a
5-layer non-linear neural network autoencoder.
Figure 8: A neural network autoencoder with 5 layers. The source is Wikipedia.
• Self-organizing feature maps transform from a high dimensional space to low dimen-
sional one, while preserving local topological information. They are implemented as
neural networks, and a key idea is to update neighboring areas when learning the
map. An additional benefit is that they automatically place more points in regions
of high probability.
13
![](https://csdnimg.cn/release/download_crawler_static/16069516/bge.jpg)
2 Solutions to “Pattern Classification”
This section contains solutions to problems in “Pattern Classification” by Duda et al.
There are approximately 8 solved problems from each chapter.
2.2 Bayesian Decision Theory
Problem 2.6
a) We want the probability of choosing action α
2
to be smaller than, or equal to, E
1
,
given that the true state of nature is ω
1
. Let’s assume that µ
1
< µ
2
and that the
decision threshold is x
∗
, so we decide α
2
if x > x
∗
. We then have
P (α
2
| ω
1
) ≤ E
1
p(x > x
∗
| ω
1
) ≤ E
1
1 −
Z
x
∗
0
p(x | ω
1
) dx
≤ E
1
We let Φ : R → [0, 1] denote the cumulative Gaussian distribution, and Φ
−1
:
[0, 1] → R it’s inverse function. Making use of Φ we write
1 − Φ
x
∗
− µ
1
σ
1
≤ E
1
x
∗
≥ µ
1
+ σ
1
Φ
−1
(1 − E
1
) .
If the desired error is close to zero, then x
∗
goes to positive infinity. If the desired
error is close to one, then x
∗
goes to negative infinity.
b) The error rate for classifying ω
2
as ω
1
is
P (α
1
| ω
2
) = p(x ≤ x
∗
| ω
2
) =
Z
x
∗
0
p(x | ω
2
) dx = Φ
x
∗
− µ
2
σ
2
.
Making use of x
∗
from the previous problem, we obtain
Φ
µ
1
+ σ
1
Φ
−1
(1 − E
1
) − µ
2
σ
2
= Φ
µ
1
− µ
2
σ
2
+
σ
1
σ
2
Φ
−1
(1 − E
1
)
.
c) The overall error rate becomes
P (error) = P (α
1
, ω
2
) + P (α
2
, ω
1
)
= P (α
1
| ω
2
)P (ω
2
) + P (α
2
| ω
1
)P (ω
1
)
=
1
2
[P (α
1
| ω
2
) + P (α
2
| ω
1
)]
=
1
2
E
1
+ Φ
µ
1
− µ
2
σ
2
+
σ
1
σ
2
Φ
−1
(1 − E
1
)
.
14
![](https://csdnimg.cn/release/download_crawler_static/16069516/bgf.jpg)
In the last equality we used the results from the previous subproblems.
d) We substitute the given values into the equations, and obtain x
∗
≈ 0.6449. The
total error rate is P (error) ≈ 0.2056.
e) The Bayes error rate, as a function of x
∗
, is given by
P (error) = P (α
2
| ω
1
)P (ω
1
) + P (α
1
| ω
2
)P (ω
2
)
=
1
2
[p(x > x
∗
| ω
1
) + p(x < x
∗
| ω
2
)]
=
1
2
1 − Φ
x
∗
− µ
1
σ
1
+ Φ
x
∗
− µ
2
σ
2
The Bayes error rate for this problem is depicted in Figure 9.
4 3 2 1 0 1 2 3 4
0.0
0.1
0.2
0.3
0.4
0.5
p
(
x
|
1
)
p
(
x
|
2
)
P
(
error
)
x
*
Figure 9: Graf accompanying problem 2.6.
Problem 2.12
a) A useful observation is that the maximal value P (ω
max
| x) is greater than, or equal
to, the average. Therefore we obtain
P (ω
max
| x) ≥
1
c
c
X
i=1
P (ω
i
| x) =
1
c
,
where the last equality is due to probabilities summing to unity.
b) The minimum error rate is achieved by choosing ω
max
, the most likely state of nature.
The average probability of error over the data space is therefore the probability that
ω
max
is not the true state of nature for a given x, i.e.
P (error) = E
x
[1 − P (ω
max
| x)] = 1 −
Z
P (ω
max
| x)p(x) dx.
c) We see that
P (error) = 1 −
Z
P (ω
max
| x)p(x) dx ≤ 1 −
Z
1
c
p(x) dx = 1 −
1
c
=
c − 1
c
,
where we have used the fact that
R
p(x) dx = 1.
d) A situation where P (error) = (c − 1)/c arises when P (ω
i
) = 1/c for every i. Then
the maximum value is equal to the average value, and the inequality in problem a)
becomes an equality.
15
![](https://csdnimg.cn/release/download_crawler_static/16069516/bg10.jpg)
Problem 2.19
a) The entropy is given by H [p(x)] = −
R
p(x) ln p(x) dx. The optimization problem
gives the synthetic function (or Lagrange function)
H
s
= −
Z
p(x) ln p(x) dx +
q
X
k=1
λ
k
Z
b
k
(x)p(x) dx − a
k
,
and since a probability density function has
R
p(x) dx = 1 we add an additional
constraint for k = 0 with b
0
(x) = 1 and a
k
= 1. Collecting terms we obtain
H
s
= −
Z
p(x) ln p(x) dx +
q
X
k=0
λ
k
Z
b
k
(x)p(x) dx −
q
X
k=0
λ
k
a
k
= −
Z
p(x)
"
ln p(x) −
q
X
k=0
λ
k
b
k
(x)
#
dx −
q
X
k=0
λ
k
a
k
,
which is what we were asked to show.
b) Differentiating the equation above with respect to p(x) and equating it to zero we
obtain
−
Z
1
"
ln p(x) −
q
X
k=0
λ
k
b
k
(x)
#
+ p(x)
1
p(x)
!
dx = 0.
This integral is zero if the integrand is zero for every x, so we require that
ln p(x) −
q
X
k=0
λ
k
b
k
(x) + 1 = 0,
and solving this equation for p(x) gives the desired answer.
Problem 2.21
We are asked to compute the entropy of the (1) Gaussian distribution, (2) triangle distri-
bution and (3) uniform distribution. Every probability density function (pdf) has µ = 0
and standard deviation σ, and we must write every pdf parameterized using σ.
Gaussian We use the definition H [p(x)] = −
R
p(x) ln p(x) dx to compute
H [p(x)] = −
Z
1
√
2πσ
exp
−
1
2
x
2
σ
2
ln
1
√
2πσ
−
1
2
x
2
σ
2
dx.
Let us denote the constant by K =
1
√
2πσ
to simplify notation. We obtain
−
Z
K exp
−
1
2
x
2
σ
2
ln K −
1
2
x
2
σ
2
dx =
−K ln K
Z
exp
−
1
2
x
2
σ
2
dx + K
Z
1
2
x
2
σ
2
exp
−
1
2
x
2
σ
2
dx
16
剩余76页未读,继续阅读
![pdf](https://img-home.csdnimg.cn/images/20210720083512.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://csdnimg.cn/download_wenku/file_type_ask_c1.png)
![](https://profile-avatar.csdnimg.cn/5db36ddea9084611b8543232841cee1e_weixin_46869245.jpg!1)
weixin_46869245
- 粉丝: 0
- 资源: 5
上传资源 快速赚钱
我的内容管理 展开
我的资源 快来上传第一个资源
我的收益
登录查看自己的收益我的积分 登录查看自己的积分
我的C币 登录后查看C币余额
我的收藏
我的下载
下载帮助
![](https://csdnimg.cn/release/wenkucmsfe/public/img/voice.245cc511.png)
最新资源
- 解决Eclipse配置与导入Java工程常见问题
- 真空发生器:工作原理与抽吸性能分析
- 爱立信RBS6201开站流程详解
- 电脑开机声音解析:故障诊断指南
- JAVA实现贪吃蛇游戏
- 模糊神经网络实现与自学习能力探索
- PID型模糊神经网络控制器设计与学习算法
- 模糊神经网络在自适应PID控制器中的应用
- C++实现的学生成绩管理系统设计
- 802.1D STP 实现与优化:二层交换机中的生成树协议
- 解决Windows无法完成SD卡格式化的九种方法
- 软件测试方法:Beta与Alpha测试详解
- 软件测试周期详解:从需求分析到维护测试
- CMMI模型详解:软件企业能力提升的关键
- 移动Web开发框架选择:jQueryMobile、jQTouch、SenchaTouch对比
- Java程序设计试题与复习指南
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035711.png)
![](https://img-home.csdnimg.cn/images/20220527035111.png)
安全验证
文档复制为VIP权益,开通VIP直接复制
![](https://csdnimg.cn/release/wenkucmsfe/public/img/green-success.6a4acb44.png)