Dissimilarity Based Ensemble of Extreme Learning Machine for Gene Expression Data
Classification
I
Hui-juan LU
a,b,∗
, Chun-lin AN
a
, En-hui ZHENG
c
, Yi LU
d
a
College of Information Engineering, China Jiliang University, Hangzhou 310018, China
b
School of Information and Electric Engineering, China University of Mining and technology, Xuzhou 221008, China
c
College of Mechanical and Electric Engineering, China Jiliang University, Hangzhou 310018, China
d
Department of Computer Science, Prairie View A&M University, Prairie View, 77446, U.S.A
Abstract
Extreme Learning Machine (ELM) has salient features such as fast learning speed and excellent generalization performance. How-
ever, a single extreme learning machine is unstable in data classification. To overcome this drawback, more and more researchers
consider using ensemble of ELMs. This paper proposes a method integrating voting-based extreme learning machines (V-ELM)
with dissimilarity (D-ELM). First, based on different dissimilarity measures, we remove number of ELMs from the ensemble pool.
Then, the remaining ELMs are grouped as an ensemble classifier by majority voting. Finally we use disagreement measure and
double-fault measure to validate the D-ELM. The theoretical analysis and experimental results on gene expression data demonstrate
that, 1) the D-ELM can achieve better classification accuracy with less number of ELMs; 2) the double-fault measure based D-ELM
(DF-D-ELM) performs better than disagreement measure based D-ELM (D-D-ELM).
Keywords: extreme learning machine, dissimilarity ensemble, double-fault measure, majority voting, gene expression data
1. Introduction
Human genome project (HGP) was officially launched in
1990. In the short span of 20 years, gene technology obtained
rapid development. Golub et al. [1] were the first to use gene
chips to study the human acute leukemia, and found two sub-
types of acute lymphoblastic leukemia: T2Cell ALL and B2Cell
ALL. The classification methods that were used on gene expres-
sion data early include the support vector machine (SVM) [2],
Artificial Neural Networks (ANN) [3], and Probabilistic Neu-
ral Network (PNN) [4]. Jin et al. [5] used partial least squares
method to establish a classification model. Zhang et al. [6]
applied Non-negative Matrix Factorization (NMF) for the gene
expression data classification. Yang et al. [7] used binary deci-
sion tree to classify gene expression data of tumor.
Extreme learning machine (ELM) [8] was proposed as an
efficient learning algorithm for single-hidden layer feedforward
neural networks (SLFNs). It increases learning speed by means
of randomly generating weights and biases for hidden nodes
rather than iteratively adjusting network parameters which is
commonly adopted by gradient based methods.
However, the stability of single ELM can be improved. To
achieve better generalization performance, Lan et al. [9] pro-
I
This work was supported by the National Natural Science Foundation of
China (No. 61272315, No.60842009, and No. 60905034), Zhejiang Provincial
Natural Science Foundation (No. Y1110342, No. Y1080950) and the Pao Yu-
Kong and Pao Zhao-Long Scholarship for Chinese Students Studying Abroad.
∗
Corresponding author.Tel.:+8657186914580; fax:+8657186914573.
Email address: hjlu@cjlu.edu.cn, huijuanlu29@gmail.com
(Hui-juan LU)
posed an ensemble of online sequential extreme learning ma-
chine (EOS-ELM) which is more stable and accurate than the
original OS-ELM.
Motivated by the ensemble idea, in 2009 Heeswijk et al.
[10] proposed an adaptive ensemble model of ELM which is
adaptive and has low computational cost. In 2010, Tian et
al. proposed a bagging ensemble scheme to combine ELMs
[11], and another ELM ensemble method based on modified
AdaBoost.RT algorithm [12]. In the same year, an ensemble
based ELM (EN-ELM) algorithm was proposed by Liu et al.
[13] which uses the cross-validation scheme to create an en-
semble of ELM classifiers for decision making. Wang and Li
[14] proposed dynamic Adaboost ensemble ELM which has
been successfully applied to problem of function approxima-
tion and classification application. Zhai et al. [15] proposed a
dynamic ensemble of sample entropy based extreme learning
machines, which can alleviate some extent of instability and
over-fitting problem, and increase the prediction accuracy. In
2011, Heeswijk et al. [16] proposed a method which is based on
GPU-accelerated and parallelized ELM ensemble, and is used
in large-scale regression. In 2012, Wang and Alhamdoosh [17]
proposed an algorithm which employs the model diversity as
fitness function to direct the selection of base learners, and pro-
duces an optimal solution with ensemble size control. It im-
proved the generalization power. Cao et al. [18] proposed an
improved learning algorithm for classification that is referred
to as voting based extreme learning machine (V-ELM) which is
adopted widely.
The ensemble classifiers have already been used in gene ex-
pression data classification. Chen et al. [19] used artificial neu-
Preprint submitted to Neurocomputing February 2, 2013