An Intelligent Routing Algorithm in Wireless Sensor Networks based on
Reinforcement Learning
Wenjing Guo
1,a *
, Cairong Yan
2,b
, Yanglan Gan
3,c
and Ting Lu
4,d
1,2,3,4
School of Computer Science and Technology, Donghua University, No.2999, North Renmin
Road, Songjiang District, Shanghai, China
a
wjguo@dhu.edu.cn,
b
cryan@dhu.edu.cn,
c
ylgan@dhu.edu.cn,
d
luting@dhu.edu.cn
Keywords: Wireless sensor networks (WSNs); Network lifetime; Intelligent routing; Reinforcement
learning (RL); Packet delivery.
Abstract. Lifetime enhancement has been a hot issue in Wireless Sensor Networks (WSNs). To
prolong the network lifetime of WSNs, this paper proposes an intelligent routing algorithm named
RLLO. RLLO makes uses of the superiority of reinforcement learning (RL) and considers residual
energy and hop count to define the reward function. It is to uniformly distribute the energy
consumption and improve the packet delivery without additional cost. This proposed algorithm has
been compared with Energy Aware Routing (EAR) and improved EAR (I-EAR). Simulation results
show that RLLO gains a significant improvement in terms of network lifetime and packet delivery
over these two algorithms.
Introduction
Due to the specific characteristics of Wireless Sensor Networks (WSNs) in which sensor nodes have
limited energy supply, constrained computation and communication ability [1,2], network lifetime
becomes the mayor concern in WSNs. Therefore, the goal of routing algorithm in WSNs is to prolong
the network lifetime as far as possible.
To achieve such a goal of enhancing network lifetime, many routing algorithms have been
specially designed for WSNs. Among all of the proposed algorithms, Energy Aware Routing (EAR)
[3] is the most typical data-centric routing algorithms. It is based on such an idea that always using the
minimum energy path is not advisable since such a way will deplete the energy of nodes on that path
and the network will get partitioned. It maintains multiple paths between source node and destination
node. Then, sub-optimal paths are occasionally chosen to balance the energy consumption among the
whole network. EAR has its inherent advantage of trying to balance energy consumption between
nodes to postpone the time when the first sensor node dies, and it has been proved in [3] to provide an
overall improvement of 21.5% energy saving and a 44% increase in network lifetime over Directed
Diffusion. Moreover, this algorithm has been compared in the survey [4] to have much stronger
energy efficiency among all the data-centric routing algorithms in WSNs. Thus, we compare our
proposed algorithm with this algorithm.
However, there are some problems in EAR. First, in the determination of routing path, only energy
consumption and remaining energy are considered, but other metrics such as delay and packet
delivery are not taken into account. Second, not just in the setup phase, in the route maintenance
phase, flooding also occurs. Such flooding brings about much more additional overhead. Third, for
the data communication phase, it completely depends on the routing table. This routing table has been
established in advance. It can not absolutely reflect the current condition of the network. Authors in [5]
propose a new algorithm to improve the performance of EAR. In this algorithm I-EAR, sensor nodes
choose one route among several routes with a probability. The probability is determined by the
residual energy of nodes, the energy consumption of the communication and the number of paths
including the forwarding node. This algorithm has been simulated and compared with EAR. Results
show that it outperforms EAR by prolonging the time of first-node-death. However, it has the same
problems as EAR does.
Applied Mechanics and Materials Vol. 678 (2014) pp 487-493 Submitted: 26.08.2014
Online available since 2014/Oct/08 at www.scientific.net Accepted: 26.08.2014
© (2014) Trans Tech Publications, Switzerland
doi:10.4028/www.scientific.net/AMM.678.487
All rights reserved. No part of contents of this paper may be reproduced or transmitted in any form or by any means without the written permission of TTP,
www.ttp.net. (ID: 114.93.68.99-14/10/14,13:58:47)