A novel classification method using the combination of FDPS
and flexible neural tree
Bo Yang
b,
, Lin Wang
a
, Zhenxiang Chen
b
, Yuehui Chen
b
, Runyuan Sun
b
a
School of Computer Science and Technology, Shandong University, Jinan 250101, China
b
School of Information Science and Engineering, University of Jinan, Jinan 250022, China
article info
Available online 28 December 2009
Keywords:
Classification
Further division of partition space
Flexible neural tree
abstract
The combination of Further Division of Partition Space (FDPS) and Flexible Neural Tree (FNT) is
proposed to improve the neural network classification performance. FDPS, which divides partition
space into many partitions that will attach to different classes automatically, is a novel technique for
neural network classification. FNT is a neural network’s structure which uses flexible tree model. The
proposed method combines FDPS and FNT to overcome their respective problems by using the other’s
merit. In order to evaluate the performance of this method, four well-known data sets are used for
classification test. Experiment results have shown that this method has favorable performance.
& 2010 Elsevier B.V. All rights reserved.
1. Introduction
Classification is an important research area in data mining. In
supervised classification tasks, a classification model is usually
constructed according to a given training set. Once the model has
been built, it can map a test data to a certain class in the given
class set. In recent years, many classification techniques including
decision tree [1,2], neural network (NN) [3], support vector
machine (SVM) [4,5], rule based classification systems etc. have
been proposed. Among these techniques, decision tree is simple
and easy to be comprehended by human beings. SVM is a new
machine learning method developed on the Statistical Learning
Theory. SVM is gaining popularity due to many attractive features,
and promising empirical performance. SVM is based on the
hypothesis that the training samples obey a certain distribution
which restricts its application scope. Neural network classification
has been proved to be a practical approach with lots of success
stories in many classification tasks. However, its training
efficiency is usually a problem, which is the current focus of our
research in this paper.
For a fully connected NN classifier, it is often a difficult task to
select features for the classification problem, especially when the
feature space is large. Some insignificant features will confuse
classifier and lead to low classification accuracy. FNT [9,10], which
uses flexible tree model as the neural network’s structure, is
proposed to improve the performance of neural network. FNT
allows input features selection, over-layer connections and
different activation functions for different nodes. In the perspec-
tive of FNT framework, the nature of model construction
procedure allows FNT to identify important input features that
is computationally efficient and effective. It has been proved that
this model is effective for classification problems with better
generalization ability. However, for FNT classifier, it is hard to get
a high accuracy model in training stage. Its optimization process
is time consuming, especially in structure optimization. Further-
more, same as NN classifier, all the samples will be classified
using FNT classifier, even including the samples which will be
hardly categorized. In some cases, such as diagnosis, it is good to
regard these samples as unclassified after classification.
Further Division of Partition Space, which was put forward in
our previous research [6], is proposed as a novel improvement
of neural network classifier. The experiments of FDPS have
shown that this method has favorable performance especially
with respect to the optimization speed and classification
accuracy. Nevertheless, generalization accuracy increases by a
smaller margin than training accuracy in FDPS, which restricts its
wider use.
In this research, we combined FNT and FDPS to overcome their
respective problems by using the other’s merit. On the one hand,
the use of FNT instead of NN will help FDPS to select important
features to build appropriate partition space and reduce inter-
ference of insignificant features. If there is no feature selection in
training process, FDPS will adapt itself to all of the features
including some insignificant features which can not distinguish
one class from the others. Therefore, although FDPS has been
proven to achieve high training accuracy, this shortcoming
restricts the increase of generalization accuracy. After combined
ARTICLE IN P RESS
Contents lists available at ScienceDirect
journal homepage: www.elsevier.com/locate/neucom
Neurocomputing
0925-2312/$ - see front matter & 2010 Elsevier B.V. All rights reserved.
doi:10.1016/j.neucom.2009.11.014
Corresponding author. Tel.: + 86 531 82765717.
E-mail addresses: yangbo@ujn.edu.cn (B. Yang).
nic_wanglin1983@ujn.edu.cn (L. Wang).
Neurocomputing 73 (2010) 690–699