Neurocomputing 272 (2018) 677–682
Contents lists available at ScienceDirect
Neurocomputing
journal homepage: www.elsevier.com/locate/neucom
GST-memristor-based online learning neural networks
Shuixin Xiao
a
,
b
, Xudong Xie
a
,
b
, Shiping Wen
a
,
b
,
c
,
d
,
∗
, Zhigang Zeng
a
,
b
, Tingwen Huang
d
,
Jianhua Jiang
a , b , ∗
a
School of Automation, Huazhong University of Science and Technology, Wuhan 430074, China
b
Department of Automation, Huazhong University of Science and Technology, Wuhan 430074, China
c
College of Science, Engineering and Technology, Hamad bin Khalifa University, 23874, Doha, Qatar
d
Science Program, Texas A&M University at Qatar, 23874, Doha, Qatar
a r t i c l e i n f o
Article history:
Received 15 June 2017
Revised 21 July 2017
Accepted 5 August 2017
Available online 18 August 2017
Communicated by Jin-Liang Wang
Keywords:
Artificial neural network
Online learning
CMOS
GST-memristor
a b s t r a c t
At present, it is an urgent issue to effectively train artificial neural network (ANN), especially when the
data is large. Online learning has been used to solve the problem, most of which is based on least mean
square (LMS). However, it is inefficient to implement the LMS on conventional digital hardware, because
of the physical separation between the memory arrays and arithmetic module. To solve this problem,
CMOS has been utilized. However, it costs too many powers and areas while designing CMOS synapses
in the very large scale integrated (VLSI) circuit. As a novel device, memristor is believed to overcome this
shortcoming as memristors could be utilized to store the weights which could be changed by a voltage
pulse. The filamentary bipolar memristive switching in Ge
2
Sb
2
Te
5
(GST) has been proved to be an ideal
choice for memristive materials. And it has two states—amorphous and crystalline, which can be changed
by DC sweep. In this paper, we consider an artificial synapse which includes a GST-memristor and two
MOSFET transistors (p-type and n-type). A number of artificial synapses are employed to form a circuit
which is expected to consume 2 − 8% of the area compared to CMOS-only circuit. And the accuracy is
about 80%, which is good enough in realistic diagnosis and has good robustness with noise.
©2017 Elsevier B.V. All rights reserved.
1. Introduction
Human brain is organized by a large number of parallel nonlin-
ear processing units, which can be self-organized and self-learning.
Compared with the traditional computer, biological neural network
is the most skillful and complex information processing system in
real life because it has unparalleled information processing capa-
bilities [1] . Therefore, the researchers hope to use artificial neu-
ral network to simulate the information processing mechanism of
brain, which can be used in many fields such as pattern recogni-
tion and expert system. Moreover, if we could establish a neural
network model with cognitive function, the artificial intelligence
would be realized [2] .
In 1943, McCulloch and Pitts [3] firstly proposed the mathemat-
ical model of neural by analyzing and summarizing the basic char-
acteristics of neurons. Hebb [4] proposed artificial neural network
(ANN) in 1949, which is a kind of model to simulate the structure
∗
Corresponding authors at: School of Automation, Huazhong University of Sci-
ence and Technology, Key Laboratory of Image Processing and Intelligent Control of
Education Ministry of China, Wuhan, Hubei, 430074, China.
E-mail addresses: wenshiping226@126.com (S. Wen), zgzeng527@126.com (Z.
Zeng).
and function of biological. neural network. The research of artifi-
cial neural network has become a research hotspot in many fields
[5–7] , which is based on mathematics, physics and computer sci-
ence. In recent, a large number of artificial neural networks and
neural network models have been proposed, such as the linear
threshold function neuron model [8] , BP neural network model [9] ,
Hopfield neural network model [10] . And some control strategies
were proposed in [11,12] for neural networks. Adaptive coupling
weights [13] is a good method to control complex dynamical net-
works. However, they all have to solve the problem: how to train
the neural network effectively, especially when the data is large.
Therefore, Zinkevich and coworkers [14] proposed online learning
and it costs little The most popular online learning is based on
least mean square (LMS), which has been used in training neural
networks.
However LMS utilizes large matrices to update the synaptic
weights. Therefore, the power of ANN mainly stems from the
learning rules to update the weights and the locality of LMS stems
from the chain rules used to calculate the gradients. Meanwhile,
it is inefficient to implement the LMS on the conventional digital
hardware for the physical separation between the memory arrays
and arithmetic module, which are used to store the value of the
synaptic weight and calculate the update rules. To overcome this
http://dx.doi.org/10.1016/j.neucom.2017.08.014
0925-2312/© 2017 Elsevier B.V. All rights reserved.