Bayesian classification with normal distributions
When the covariance matrices f or two classes are the same a nd diagonal i.e. Σ
i
= Σ
j
= σ
2
I
then the discrimination functions g
ij
(x) are given by
g
ij
(x) = w
T
(x −x
0
) = (µ
i
− µ
j
)
T
(x − x
0
) , (2)
since the vector w is w = µ
i
− µ
j
in this case. Note that the point x
0
is on the decision
hyperplane i.e. satisfies g
ij
(x) = 0 since g
ij
(x
0
) = w
T
(x
0
− x
0
) = 0. Let x be another point
on the decision hyperplane, then x − x
0
is a vector in the decision hyperplane. Since x is a
point on the decision hyperplane it also must satisfy g
ij
(x) = 0 from the functional form for
g
ij
(·) and the definition of w is this means that
w
T
(x − x
0
) = (µ
i
− µ
j
)
T
(x − x
0
) = 0 .
This is the statement that the line connecting µ
i
and µ
j
is orthogonal to the decision hy-
perplane. In t he same way, when the covariance matrices of each class are not diagonal but
are nevertheless the Σ
i
= Σ
j
= Σ the same logic that we used above states that the decision
hyperplane is a gain orthogonal to the vector w which in this case is Σ
−1
(µ
i
−µ
j
).
The magnitude o f P (ω
i
) relative to P (ω
j
) influences how close t he decision hyperplane is
to t he respective class means µ
i
or µ
j
, in the sense tha t the class with the larger a priori
probability will have a “larger” region of R
l
assigned to it for classification. For example, if
P (ω
i
) < P (ω
j
) then ln
P (ω
i
)
P (ω
j
)
< 0 so the point x
0
which in the case Σ
i
= Σ
j
= Σ is given
by
x
0
=
1
2
(µ
i
+ µ
j
) − ln
P (ω
i
)
P (ω
j
)
µ
i
− µ
j
||µ
i
− µ
j
||
2
Σ
−1
, (3)
we can write as
x
0
=
1
2
(µ
i
+ µ
j
) + α(µ
i
− µ
j
) ,
with the value of α > 0. Since µ
i
− µ
j
is a vector fr om µ
j
to µ
i
the expression f or x
0
above
starts at the midpoint
1
2
(µ
i
+ µ
j
) a nd moves closer to µ
i
. Meaning that the amo unt of R
l
assigned to class ω
j
is “larg er” than the amount assigned to class ω
i
. This is expected since
the prior probability of class ω
j
is larger than that of ω
i
.
Notes on Example 2.2
To see the final lengths of the principal axes we start with the transformed equation of
constant Mahalanobis distance of d
m
=
√
2.952 or
(x
′
1
− µ
′
11
)
2
λ
1
+
(x
′
2
−µ
′
12
)
2
λ
2
= (
√
2.952)
2
= 2.952 .
Since we want the principal axis about (0, 0) we have µ
′
11
= µ
′
12
= 0 and λ
1
and λ
2
are the
eigenvalues given by solving |Σ − λI| = 0. In this case, we get λ
1
= 1 (in direction v
1
) and
λ
2
= 2 (in direction v
2
). Then the above becomes in “ standa r d form” for a conic section
(x
′
1
)
2
2.952λ
1
+
(x
′
2
)
2
2.952λ
2
= 1 .
5
我的内容管理
收起
我的收益 登录查看自己的收益
我的积分
登录查看自己的积分
我的C币
登录后查看C币余额
我的收藏 
我的下载 
下载帮助 
