Physics Letters A 380 (2016) 903–909
Contents lists available at ScienceDirect
Physics Letters A
www.elsevier.com/locate/pla
Energy consumption analysis for various memristive networks
under
different learning strategies
Lei Deng
a
, Dong Wang
a
, Ziyang Zhang
b
, Pei Tang
b
, Guoqi Li
a,∗
, Jing Pei
a,b,∗
a
Center for Brain Inspired Computing Research (CBICR), Department of Precision Instrument, Tsinghua University, Beijing 100084, China
b
Optical Memory National Engineering Research Center, Department of Precision Instrument, Tsinghua University, Beijing 100084, China
a r t i c l e i n f o a b s t r a c t
Article history:
Received
5 September 2015
Received
in revised form 11 December 2015
Accepted
23 December 2015
Available
online 29 December 2015
Communicated
by R. Wu
Keywords:
Memristor
Energy
consumption
Neuromorphic
engineering
Neural
networks
Brain-inspired
computation
Recently, various memristive systems emerge to emulate the efficient computing paradigm of the
brain cortex; whereas, how to make them energy efficient still remains unclear, especially from an
overall perspective. Here, a systematical and bottom-up energy consumption analysis is demonstrated,
including the memristor device level and the network learning level. We propose an energy estimating
methodology when modulating the memristive synapses, which is simulated in three typical neural
networks with different synaptic structures and learning strategies for both offline and online learning.
These results provide an in-depth insight to create energy efficient brain-inspired neuromorphic devices
in the future.
© 2015 Elsevier B.V. All rights reserved.
1. Introduction
Nowadays, brute computing dependent conventional computer
is widely used in lots of fields, such as scientific research, medical
analysis, industrial control, as well as big data and cloud com-
putation
through Internet. However, the low energy efficiency of
von Neumann architecture has become one of its main obstacles
to the progress of artificial intelligence and machine learning. In
stark contrast, human brain consumes only 20 W, but it does well
in versatile tasks including working memory, pattern recognition,
vision processing and adaptive learning, which all seem tough for
computers.
To
this end, many microelectronics technologists have been try-
ing
to build the hardware neuromorphic devices to emulate the
brain computation [1–4]. As is well known, the neural networks
within human brain consist of more than 10
11
neurons, which are
highly interconnected by about 10
15
synapses [5]. Facing so huge
number of connections, choosing a suitable device to model the
synapse becomes the key. On the one side, large-scale networks
require synapses with high density and low energy consumption
to realize a compact system; on the other side, the powerful learn-
ing
function of the brain is believed to result from the adaptive
*
Corresponding authors at: 9003 Building, Tsinghua University, Beijing 100084,
China. Tel.: +86 01062788101.
E-mail
addresses: peij@mail.tsinghua.edu.cn (J. Pei), liguoqi@mail.tsinghua.edu.cn
(G. Li).
tuning of synapses which requires good synaptic plasticity [6]. In
this sense, conventional CMOS synaptic devices, such as SRAM [4],
SDRAM [3] and analog floating-gate transistor [7], occupy large size
or consume high energy, as well as lack the plastic property. For-
tunately,
the emerging memristor device [8] is similar to the bio-
logical
plastic synapse [9,10], which can move itself to a new state
based on its non-volatile memory of the historical states. Besides,
small size and low energy consumption are also its advantages. As
a result, memristor is widely used in various neuromorphic de-
vices [11].
Although
the low-energy advantage of memristor is well
known, the underlying principles of how to reduce the energy
consumption have not been systematically investigated, especially
at the network level. Here we mainly study the energy caused
by memristor, which is simultaneously related to memristor de-
vice
itself, the programming condition, as well as learning strat-
egy
including the modulation scheme and modulation precision.
We conduct a systematical analysis at various levels, from sin-
gle
memristor device to memristive networks, which is organized
in a bottom-up manner. Specifically, a quantitative energy esti-
mating
methodology with regard to the modulation of memris-
tive
synapses is proposed. We simulate it in three typical neural
networks based on different synaptic structures and modulation
schemes for both offline and online learning, including network
built by single memristor-based synapse for sequence learning, as
well as built by multiple memristors-based synapse for pattern
recognition and generation. In all the quantitative estimations, we
http://dx.doi.org/10.1016/j.physleta.2015.12.024
0375-9601/
© 2015 Elsevier B.V. All rights reserved.