W.-S. Zheng et al. / Pattern Recognition 42 (2009) 764 -- 779 767
between the expectation values of f
b
(W, n)andf
w
(W, n)withrespect
to
n such that the uncertainty is considered to be over the domain
of
n.Thatis:
˜
W
opt
= arg max
W
E
n
[f
b
(W, n)]/E
n
[f
w
(W, n)]
= arg max
W
f
b
(W)/f
w
(W)
It can be verified that
f
b
(W) = E
n
[f
b
(W, n)] = trace(W
T
˜
S
b
W) (15)
f
w
(W) = E
n
[f
w
(W, n)] = trace(W
T
˜
S
w
W) (16)
So, it is exactly the optimization model formulated in Eq. (12). This
gives an more intuitive understanding of the effects of covariance
matrices
˜
S
w
and
˜
S
b
.ThoughinP-LDA
ˆ
S
w
and
ˆ
S
b
are perturbated
by S
w
and S
b
, respectively, however in Section 5 we will show
˜
S
w
and
˜
S
b
will converge to the precise within-class and between-class
covariance matrices, respectively. This will show the rationality of
P-LDA, since the class empirical mean is almost its expectation value
when sample size is large enough and then the perturbation effect
could be ignored.
2.2. P-LDA under mixture of Gaussian distribution
This section extends Theorem 1 by altering the class distribu-
tion from single Gaussian to mixture of Gaussians [3]. Therefore, the
probability density function of a sample x in class C
k
is:
p(x|C
k
) =
I
k
i=1
P( i|k )N(x|u
i
k
, N
i
k
), (17)
where u
i
k
is the expectation of x in the i th Gaussian component
(GC) N(x|u
i
k
, N
i
k
)ofclassC
k
, N
i
k
is its covariance matrix and P(i|k)is
the prior probability of the ith GC of class C
k
. Such density function
indicates that any sample x in class C
k
mainly distributes in one
of the GC. Therefore, Theorem 1 under single Gaussian distribution
can be extended to learning perturbation in each GC. To do so, the
clusters within each class should be first determined such that data
in each cluster are approximately normally distributed. Then those
clusters are labeled as subclasses, respectively. Finally P-LDA is used
to learn the discriminant information of all those subclasses. It is
similar to the idea of Zhu and Martinez [26] who extended classical
Fisher's LDA to the mixture of Gaussian distribution case.
In details, suppose there are I
k
GCs (clusters) in class C
k
and N
i
k
out of all N samples are in the ith GC of class C
k
.LetC
i
k
denote the ith
GC of class C
k
.Ifwedenotex
k
i,s
as the sth sample of C
i
k
, s = 1, ...,N
i
k
,
then a perturbation random vector
n
k
i,s
for x
k
i,s
can be modeled, where
n
k
i,s
∼ N(0, X
C
i
k
), X
C
i
k
∈
n×n
, so that
˜
x
k
i,s
= x
k
i,s
+ n
k
i,s
is a random
vector stochastically describes the expectation o f subclass C
i
k
, i.e.,
u
i
k
. Then P-LDA can be extended to the mixture of Gaussians case
by classifying the subclasses {C
i
k
}
k=1,...,L
i=1,
...,I
k
. Thus we get the following
theorem
3
a straightforward extension of Theorem 1 and the proof
is omitted.
Theorem 2. Under the Gaussian mixture distribution of data within
each class, the projection matrix of perturbation LDA (P-LDA),
˜
W
opt
,
can be found as follows:
˜
W
opt
= arg max
W
trace(W
T
˜
S
b
W)
trace(W
T
˜
S
w
W)
= arg max
W
trace(W
T
(
ˆ
S
b
+ S
b
)W)
trace(W
T
(
ˆ
S
w
+ S
w
)W)
(18)
where
˜
S
b
= E
n
[
1
2
L
k=1
L
j=1
I
k
i=1
I
j
s=1
N
i
k
N
×
N
s
j
N
(
˜
u
i
k
−
˜
u
s
j
)(
˜
u
i
k
−
˜
u
s
j
)
T
] =
ˆ
S
b
+ S
b
,
S
b
=
L
k=1
I
k
i=1
(N−N
i
k
)
2
N
3
X
C
i
k
+
L
k=1
I
k
i=1
N
i
k
N
3
L
j=1
I
j
s=1,(j,s) (k,i)
(N
s
j
X
C
s
j
),
ˆ
S
b
=
1
2
L
k=1
L
j=1
I
k
i=1
I
j
s=1
N
i
k
N
×
N
s
j
N
(
ˆ
u
i
k
−
ˆ
u
s
j
)(
ˆ
u
i
k
−
ˆ
u
s
j
)
T
,
˜
S
w
=
L
k=1
I
k
i=1
N
i
k
N
˜
S
i
k
=
ˆ
S
w
+ S
w
,
˜
S
i
k
= E
n
k,i
[
N
i
k
s=1
1
N
i
k
(x
k
i,s
−
˜
u
i
k
)(x
k
i,s
−
˜
u
i
k
)
T
],
S
w
=
1
N
L
k=1
I
k
i=1
X
C
i
k
,
ˆ
S
w
=
1
N
L
k=1
I
k
i=1
N
i
k
s=1
(x
k
i,s
−
ˆ
u
i
k
)(x
k
i,s
−
ˆ
u
i
k
)
T
,
ˆ
u
i
k
=
1
N
i
k
N
i
k
s=1
x
k
i,s
,
˜
u
i
k
=
ˆ
u
i
k
+
1
N
i
k
N
i
k
s=1
n
k
i,s
, i = 1, ...,I
k
, k = 1, ...,L,
n
k,i
={n
k
i,1
, ...,n
k
i,N
i
k
}, n
={n
1,1
, ...,n
1,I
1
, ...,n
L,1
, ...,n
L,I
L
}.
3. Estimation of perturbation covariance matrices
For implementation of P-LDA, we need to properly estimate two
perturbation covariance matrices S
b
and S
w
. Parameter estimation
is challenging, since it is always ill-posed [3,23] due to limited sam-
ple size and the curse of high dimensionality. A more robust and
tractable way to overcome this problem is to perform some regu-
larized estimation. It is indeed the motivation here. A method will
be suggested to implement P-LDA with parameter estimation in an
entire PCA subspace without discarding any nonzero principal com-
ponent. Unlike the covariance matrix estimation on sample data, we
will introduce an indirect way for estimation of the covariance ma-
trices of perturbation random vectors, since the observation values
of the perturbation random vectors are hard to be found directly.
For derivation, parameter estimation would focus on P-LDA un-
der single Gaussian distribution, and it could be easily generalized
to the Gaussian mixture distribution case by Theorem 2. This section
3
The designs of
˜
S
b
and
˜
S
w
in the criterion are not restricted to the presented
forms. The goal here is just to present a way how to generalize the analysis under
single Gaussian case.
剩余15页未读，继续阅读
weixin_38519619
- 粉丝: 2
- 资源: 905
我的内容管理
展开
最新资源
- 高效办公必备：可易文件夹批量生成器
- 吉林大学图形学与人机交互课程作业解析
- 8086与8255打造简易乒乓球游戏机教程
- Win10下C++开发工具包：Bongo Cat Mver、GLEW、GLFW
- Bootstrap前端开发：六页果蔬展示页面
- MacOS兼容版VSCode 1.85.1：最后支持10.13.x版本
- 掌握cpp2uml工具及其使用方法指南
- C51单片机星形流水灯设计与Proteus仿真教程
- 深度远程启动管理器使用教程与工具包
- SAAS云建站平台，一台服务器支持数万独立网站
- Java开发的博客API系统：完整功能与接口文档
- 掌握SecureCRT：打造高效SSH超级终端
- JAVA飞机大战游戏实现与源码分享
- SSM框架开发的在线考试系统设计与实现
- MEMS捷联惯导解算与MATLAB仿真指南
- Java实现的学生考试系统开发实战教程
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
