![](https://csdnimg.cn/release/download_crawler_static/15409719/bg4.jpg)
KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS VOL. 11, NO. 4, April 2017 2023
only takes the parameter n from a static and predefined parameter set determined by energy
value and node density. It is irrespective of current energy situation of neighbors. The value
in the set can also not be changed for different type of nodes. It is obviously that this method
is not suitable for all the conditions as the energy value of various nodes is not identical.
In opportunistic networks scenarios, it is impossible to replace or recharge batteries while
nodes are moving around. However, nodes are aware of distinct remaining energy levels of
themselves, and can get this information from their current neighbors by the inquiry method.
Hence, detrimental replication decisions for adjacent nodes can be canceled ahead of
schedule for saving energy. There must be a tradeoff between energy conservation and
message replication. This paper not only considers energy constraints of mobile nodes, but
also takes the current local energy state information of neighbors into account during making
message replication decisions.
2.2 Learning Automata
Learning Automata (LA) is an adaptive decision-making mathematical model which
operates on an unknown random environment [13]. There is a finite set of actions that could
be chosen at every period by a learning automaton. An action probability vector is used to
describe the possibility of each action. The environment will evaluate the chosen action and
give a reinforcement signal with the probability distribution. Then the automaton updates its
action probability vector depending on the reinforcement signal at each cycle, and then
evolves to next desired behavior. The mathematical expression of learning automata is called
variable structure, and is represented by a quadruple (α, β, p, T), in which α = {α
1
, α
2
, … α
r
}
represents the action set of the automata, β = {β
1
, β
2
, … β
r
} represents the input set, p = {p
1
,
p
2
,…p
r
} denotes the action probability set, and p(n + 1) = T[α(n), β(n), p(n)] represents the
learning algorithm.
The environment depending on the class of the reinforcement signal β can be classified
into P-model, Q-model and S-model. The environment in which the reinforcement signal can
only take two binary values 0 and 1 are referred to P-model environment. Another class of
the environment allows a finite number of values in the interval [0, 1] can be taken by the
reinforcement signal. Such an environment is referred to Q-model environment. In S-model
environment, the reinforcement signal lies in the interval [a, b] [15].
Let α
i
be the action chosen at time n, then the recurrence equation for updating p is defined
as:
( 1) () ()(1 ())
( 1) () () () ,
+= + ⋅−
+= − ⋅ ∀ ≠
ii i
jj j
pn pn an pn
pn pn an pn jj i
(1)
for favorable responses, and
( 1) (1 ()) ()
( 1) (1 ()) () ,
1
+=− ⋅
+= +− ⋅ ∀ ≠
−
ii
jj
pn bn pn
b
pn bn pn jj i
r
(2)
for unfavorable ones, where a and b are reward and penalty parameters respectively. If a = b,
learning algorithm is called L
R-P
; if a << b, it is called L
R
ε
P
; if b=0, it is called L
R-I
[16]. The
relationship between the learning automaton and its random environment is shown in Fig. 1.
2.3 Dynamic Irregular Cellular Learning Automata (DICLA)
Cellular Automata (CA) is a dynamic model that is consisted of a lot of identical