Polynomial Neural Network Classifiers Based on Data Preprocessing and Space
Search Optimization
Wei Huang
School Computer and Communication Engineering,
Tianjin University of Technology
Tianjin, China
huangwabc@163.com
Sung-Kwun Oh
Department of Electrical Engineering,
The University of Suwon
Hwaseong-si, Gyeonggi-do, South Korea
ohsk@suwon.ac.kr
Witold Pedrycz
Department of Electrical & Computer Engineering,
University of Alberta, Edmonton T6R 2V4 AB,
Canada
Abstract— In this paper, we propose a novel architecture of
polynomial neural network classifier (PNNC) with the aid of
data preprocessing technique and space search optimization,
which adopts accelerated convergence mechanism instead of
purely random search. Two type of polynomials are adopted
for constructing discriminate functions in the PNNC to
alleviate the limitation of relatively simple geometry using
linear discriminate function in the conventional neural
network classifiers. Space search optimization is exploited here
to realize structure optimizes and parameter optimize in the
design of PNNC. Moreover, data preprocessing techniques are
used to reduce the dimension of training data. The proposed
PNNC is compared with some well-known classifiers based on
several benchmark data sets. Experimental results illustrate
the effectiveness of PNNCs.
Keywords- Polynomial Neural Network Classifiers (PNNCs);
polynomial discriminant function; data preprocessing; space
search optimization
I. INTRODUCTION
I In the past years, many neural network classifiers
including radial basis function (RBF) neural networks, back
propagation (BP) neural network classifier [1][2][3] have
been developed. Among these neural network classifiers,
there have been some models using two order polynomial
discriminate functions [4-7], the problem of using high order
discriminate function remains open.
In general, there are lots of parameters needed to be
determined in the design of many neural network classifiers.
To solve this problem, evolutionary algorithms are
commonly used to estimate such parameters. However, these
evolutionary algorithms use the purely random search
mechanism that leads to long computing time and premature
convergence [8]. In our previous study, we present a space
search optimization algorithm (SSOA) that can alleviate the
drawback of the purely random search mechanism [8].
When dealing with high dimensional training data,
preprocessing techniques are used to address the high
dimensional problem. There are a suite of techniques to
realize data preprocessing for data sets. Principal component
analyses (PCA) [9], linear discriminate analysis (LDA) [10]
are two typical approaches among these data preprocessing
techniques: PCA is used to reduce the dimension of data sets
while LDA utilizes the differences among classes such as the
between-class scatter and the within-class scatter. In short,
PCA and LDA have become the data preprocessing methods
commonly used in the classification field.
In this study, we propose a novel polynomial neural
network classifier (PNNC). Instead of linear discriminate
function in the conventional neural network classifiers, high
order polynomials (more than two orders) are used as
discriminate functions. Space search optimization algorithm
is exploited to optimize the parameters when developing
PNNC. Furthermore, data preprocessing techniques
including PCA or LDA are used to handle the original data
set. The performance of PNNC is compared with some well-
known classifiers based on several benchmark data sets.
II. A
RCHITECTURE OF POLYNOMIAL NEURAL NETWORK
CLASSIFIERS
This section we elaborate on the design and architecture
of the polynomial neural network classifiers. This network
emerges from six components: inputs, outputs, preprocessing
part, premise part, consequence part, and aggregation part.
Figure 1 illustrates a general architecture of PNNCs. As
shown in Figure 1, the inputs come from original data set,
while the outputs are the final classification results calculated
from aggregation part. The premise part is developed based
on Fuzzy C-means (FCM) method, while the consequent part
is realized with the aid of Least Square Estimation (LSE).
Furthermore, the preprocessing part is realized via data
preprocessing technique, and the aggregation part relate to
the polynomial discriminate functions.
PNNCs can also be described as a set of fuzzy rules.
Premise part and consequence part corresponding to the
formation of the fuzzy rules, while the aggregation part relate
to a fuzzy inference (mapping procedure). Let
n
denotes the
2016 Joint 8th International Conference on Soft Computing and Intelligent Systems and 2016 17th International Symposium
on Advanced Intelligent Systems
978-1-5090-2678-4/16 $31.00 © 2016 IEEE
DOI 10.1109/SCIS&ISIS.2016.173
769