直接LDA与LDA不等价的原因分析

需积分: 9 134 浏览量更新于2024-10-02 收藏 192KB PDF 举报

"这篇在Pattern Recognition杂志上发表的文章指出，直接线性判别分析（Direct LDA，简称D-LDA）并不等同于传统的线性判别分析（LDA）。作者Hui Gao和James W. Davis来自俄亥俄州立大学的计算机视觉实验室，他们对D-LDA提出了反驳观点，揭示了D-LDA与LDA之间的关键差异，并讨论了在处理小样本大小问题时，D-LDA的局限性。" 文章中，作者首先澄清了一个普遍的误解，即D-LDA与LDA是等价的。他们基于贝叶斯决策理论分析指出，D-LDA实际上是LDA的一种特殊情况，但它直接将类均值的线性空间作为LDA解决方案，而忽略了联合协方差估计。在LDA的传统方法中，联合协方差矩阵用于捕捉类别间的变异性和类内的共变性，这是区分不同类别的重要因素。然而，D-LDA忽视了这一关键统计特性，这可能导致其在某些情况下的分类性能下降。进一步地，作者强调了D-LDA在处理小样本大小问题时与传统子空间基LDA的区别。在小样本情况下，传统的LDA通过考虑样本协方差来调整模型，以提高分类效果。而D-LDA由于没有充分利用样本信息，可能无法充分适应数据的分布特性，从而限制了其在一般应用中的性能表现。这种局限性在数据稀疏或样本数量有限的环境中尤为明显。关键词：线性判别分析；直接LDA；小样本大小问题这篇文章揭示了直接LDA与线性判别分析在方法论和实际应用上的差异，并提醒研究者在选择分类方法时需谨慎考虑数据的特性和样本量，以避免潜在的性能限制。在实际工作中，理解这些差异对于优化模型选择和提高预测准确性至关重要。

Pattern Recognition 39 (2006) 1002 – 1006

www.elsevier.com/locate/patcog

Rapid and brief communication

Why direct LDA is not equivalent to LDA

Hui Gao

∗

, James W. Davis

Computer Vision Laboratory, Department of Computer Science and Engineering, The Ohio State University, 395 Dreese Lab,

2015 Neil Avenue, Columbus, OH 43210, USA

Received 26 August 2005; accepted 25 November 2005

Abstract

In this paper, we present counterarguments against the direct LDA algorithm (D-LDA), which was previously claimed to be equivalent

to Linear Discriminant Analysis (LDA). We show from Bayesian decision theory that D-LDA is actually a special case of LDA by directly

taking the linear space of class means as the LDA solution. The pooled covariance estimate is completely ignored. Furthermore, we

demonstrate that D-LDA is not equivalent to traditional subspace-based LDA in dealing with the Small Sample Size problem. As a result,

D-LDA may impose a signiﬁcant performance limitation in general applications.

Keywords: Linear discriminant analysis; Direct LDA; Small sample size problem

1. Introduction

Recently, an algorithm called direct Linear Discriminant

Analysis (D-LDA) has received considerable interest in Pat-

tern Recognition and Computer Vision. It was ﬁrst proposed

in Ref. [1] to deal with the small sample size (SSS) problem

in face recognition and has been followed with several ex-

tensions, e.g., fractional direct LDA [2], kernel based direct

LDA [3], and regularized direct discriminant analysis [4].

The key idea in this method is that the null space of the

between-class scatter matrix S

contains no useful infor-

mation for recognition and is discarded by diagonalization.

The within-class scatter matrix S

is then projected into the

linear subspace of S

and factorized using eigenanalysis to

obtain the solution. It was claimed in Ref. [1] that

(1) D-LDA gives the “exact solution for Fisher’s criterion”.

(2) D-LDA is equivalent to subspace-based LDA (e.g.,

PCA + LDA) in dealing with the SSS problem.

However, we observe that these claims of D-LDA are

ﬂawed in theory. Although the null components of S

do not

∗

Corresponding author. Tel.: +1 614 247 6095; fax: +1 614 292 2911.

E-mail address: gaoh@cse.ohio-state.edu (H. Gao).

0031-3203/$30.00

䉷

doi:10.1016/j.patcog.2005.11.016

inﬂuence the projection of S

in the feature space, they do

inﬂuence the projection of S

and hence should not be dis-

carded. Since all “direct” approaches share the same idea

(e.g. Refs. [1–4]), we focus on the original work of D-LDA

[1] to simplify the discussion. Similar arguments can be

made to any of the extensions.

Our analysis originates from the viewpoint of Bayesian

decision theory. It is well-known [5] that Fisher’s LDA (ratio

of S

and S

in the projection space) is equivalent to a

classiﬁcation problem of c Gaussians with equal covariance

when the model parameters are estimated in the maximum-

likelihood (ML) fashion. The solution requires a minimum

of c − 1 linear features (assuming input dimension D?c)to

form a sufﬁcient statistic. However in D-LDA, because the

null space of S

is ﬁrst discarded, its solution is constrained

to be in the linear space of S

(no matter the form of S

which is maximally c − 1 dimensional. Hence, the complete

c − 1 dimensional linear space of S

must be kept as the

D-LDA solution in order for it to possibly be a sufﬁcient

statistic. Due to the fact of ignoring S

, D-LDA is a special

case of LDA.

We additionally point out one missing assumption in

the linear algebra derivation of D-LDA given in Ref.

[1]. When any singular matrix (S

or S

) is involved in

the generalized eigenvector and eigenvalue problem, the

下载后可阅读完整内容，剩余4页未读，立即下载

maverickfei

粉丝: 0

直接LDA与LDA不等价的原因分析

Direct LDA and PCA+LDA：LDA、Direct LDA和PCA+LDA的实现。 有关详细信息，请参见描述。-matlab开发

LDA_LDA关键词_主题词提取_

LDA.zip_LDA文档_lda java_lda4085_lda模型_主题模型

Failed to build lda ERROR: Could not build wheels for lda, which is required to install pyproject.toml-based projects

怎么解决TypeError: 'LDA' object is not callable

怎么解决这个报错TypeError: 'LDA' object is not callable

Could not build wheels for lda, which is required to install pyproject.toml-based projects

name 'lda' is not defined 这个问题怎么解决

TypeError: 'LDA' object is not callable

name 'lda_model' is not defined. Did you mean: 'LdaModel'

最新资源

Direct LDA and PCA+LDA：LDA、Direct LDA和PCA+LDA的实现。有关详细信息，请参见描述。-matlab开发