to the antigen. The clone selection is corresponding to an affinity
maturation process, which means immune individuals with high
affinity will gradually increase during clone and mutation process.
At the same time, some immune individuals will polarize into
memory individuals.
Similar to the AIS, evolutionary algorithms (EAs), such as
Genetic Algorithms (GA) (Park & Ryu, 2010), Evolution Strategies
(ES) (Huang, Chang, Hsieh, & Sandnes, 2011) and Differential evo-
lution (DE) (Storn & Price, 1997) are all designed based on the basic
idea of biological evolution to control, and optimize artificial sys-
tems. Evolutionary computation shares many concepts of AIS like
a population, genotype phenotype mapping, and proliferation of
the most fit. On the other hand, AIS models based on immune net-
works resemble the structures and interactions of neural network
models. The key advantages of AIS over neural networks are the
benefits of a population of solutions and the evolutionary selection
pressure and mutation. Meanwhile, the underlying mechanisms
are fundamentally different in many aspects. First and foremost,
the immune system is highly distributed, highly adaptive, self-
organizing, maintains a memory of past encounters and has the
ability to continuously learn about new encounters. AIS is the sys-
tem developed around the current understanding of the immune
system. Second, AIS is a general framework for a distributed adap-
tive system and could, in principle, be applied to many domains.
Compared to most other evolutionary algorithms, AIS is much
more simple and straightforward to be implemented, which is
important for practitioners from other fields. In addition, because
AIS is self-organizing, it requires much less system parameters
than other evolutionary computation methods. Some works have
also pointed out the similarities and the differences between AIS
and other heuristics (Aickelin, Dasgupta, & Gu, 2013; Castro &
Timmis, 2002; Zheng, Chen, & Zhang, 2010).
In recent years, there has been considerable interests in explor-
ing and exploiting the potential of AIS for applications in computer
science and engineering including pattern recognition (Yuan,
Zhang, Zhao, Li, & Zhang, 2012), clustering (de Mello Honorio,
Leite da Silva, & Barbosa, 2012), optimization (Woldemariam &
Yen, 2010), and Remote Sensing (Zhong & Zhang, 2012). However,
the advantage of AIS for Bayesian classification has received very
little attention. In this paper, we propose a new AIS based attribute
weighting method for Naive Bayes classification. The great perfor-
mance of this design is validated through numerous performance
metrics, including classification accuracy, class probability estima-
tion, and class ranking performance. It is worth noting that some
works exist to improve AIS for domain specific problems, such as
an improved Artificial Immune System for seeking the Pareto front
of land-use allocation problem in large areas (Huang, Liu, Li, Liang,
& He, 2013). However, in this paper, we do not consider the
improved AIS for WNB. This is mainly because that we aim at pro-
posing a self-adaptive attribute weighting framework based on the
immune system for WNB, and our designs can be easily general-
ized to any AIS based algorithms.
3. Preliminaries and problem definition
Given a training set D¼fx
1
; ...; x
N
g with N instances, each of
which contains n attribute values and a class label, we use
x
i
¼fx
i;1
; ...x
i;j
; ...x
i;n
; y
i
g to denote the ith instance x
i
in the data
set D. x
i;j
denotes the jth attribute value of x
i
and y
i
denotes the
class label of x
i
. The class space Y¼fc
1
; ...; c
k
; ...; c
L
g denotes
the set of labels that each instance belongs to and c
k
denotes the
kth label of the class space. For ease of understanding, we use
ðx
i
; y
i
Þ as a shorthand to represent an instance and its class label,
and use x
i
as a shorthand of x
i
. We also use a
j
as a shorthand to rep-
resent the jth attribute. For an instance (x
i
; y
i
) in the training set D,
its class label satisfies y
i
2Y, whereas a test instance x
t
only con-
tains attribute values and its class label y
t
needs to be predicted
by a weighted Naive Bayes classification model, which can be for-
mally defined as
cðx
t
Þ¼arg max
c
k
2Y
P ðc
k
Þ
Y
n
j¼1
P ðx
t;j
c
k
Þj
w
j
ð1Þ
In Eq. (1), pðc
k
Þ represents the prior probability of class c
k
in the
whole training set. Pðx
t;j
j c
k
Þ. denotes the conditional probability
distribution of attribute x
t;j
conditioned by the given class c
k
. w
j
denotes the weight value of the jth attribute.
In this paper, we focus on the calculation of the conditional
probability pðx
i;j
j c
k
Þ
w
j
by finding optimal attribute weight values
w
j
; j ¼ 1; ...; n. While all existing attribute weighting approaches
define the weight without considering the uniqueness of the
underlying training data, we intend to resolve the optimal w value
selection problem as an optimization process. Assume that the cal-
culation of each conditional probability value pðx
i;j
j c
k
Þ
w
j
has an
optimal w
j
value, there are nw
j
vectors needed for NB classifica-
tion. As a result, the WNB classification can be transferred to an
optimization problem as follows.
w
¼ arg max
w
j
2w
f ðx
t
; wÞ s:t: 0 6 w
j
6 1 ð2Þ
where w ¼fw
1
; ...; w
j
; ...; w
n
g denotes the attribute weight vector
for WNB. And f ðx
t
; wÞ is calculated by Eq. (1).
4. Self-adaptive attribute weighted Naive Bayes
4.1. AIS symbol definitions and overall framework
4.1.1. AIS symbol definitions
In this paper, we propose to use AIS to learn optimal attribute
weight values for NB classification. In our solution, antigens in AIS-
WNB are simulated as training instances which are presented to
the system during the training process. Antibodies represent attri-
bute weight vector w with different set of values (i.e., candidates).
The binding of the antibodies and antigens will resemble the fit-
ness of a specific weight vector with respect to the given training
data. This can be evaluated by using the affinity score.
During the learning process, the antibodies with good affinity
will experience a form of clonal expansion after being presented
with the training data sets (analogous to antigens). When antibod-
ies are cloned they will undergo a mutation process, in which a
specific mutation function will be designed (and deployed). The
evolving optimization process of the AIS system will help discover
optimal w vector with the best classification performance.
Before introducing algorithm details, we briefly define follow-
ing key notations, which will help understand the learning of the
weight values using AIS principle. In Table 1, we also summarize
Table 1
Symbol mapping between immune system and AISWNB.
Immune systems AISWNB
Antibody Attribute weight vector w
Antigens
Training instances in D
a
Shape-space Possible values of the data vectors
Affinity The fitness of the weight vector w on the testing
datasets
Clonal expansion Reproduction of weight vectors that are well matched
with antigens
Affinity maturation Specific mutation of w vector and removal of lowest
stimulated weight vectors.
Immune memory Memory set of mutated weight vectors
1490 J. Wu et al. / Expert Systems with Applications 42 (2015) 1487–1502