![](https://csdnimg.cn/release/download_crawler_static/20703710/bg11.jpg)
2.1. NEURAL NETWORKS 11
ions to flow out of the cell, reversing the voltage gradient until it returns to its resting state. e
locking behavior of voltage-gated channels causes hysteresis: once a spike has started, any further
external stimuli will be dominated by the rush of ions until the entire process has returned to its
rest state. is phenomenon, combined with the large local charge gradient caused by the ion
flow, allows an action potential to propagate unidirectionally along the surface of a neuron. is
forms the fundamental mechanism of communication between neurons: a small signal received
at dendrites translates into a wave of charge flowing through the cell body and down the axon,
which in turn causes a buildup of charge at the synapses of other neurons to which it is connected.
e discriminative capacity of a biological neuron stems from the mixing of incoming
signals. Synapses actually come in two types: excitatory and inhibitory, which cause the outgoing
action potential sent from one neuron to induce either an increase or decrease of charge on the
dendrite of an adjoining neuron. ese small changes in charge accumulate and decay over short
time scales until the overall cross-membrane voltage reaches the activation threshold for the ion
channels, triggering an action potential. At a system level, these mutually induced pulses of
electrical charge combine to create the storm of neural activity that we call thought.
2.1.2 ARTIFICIAL NEURAL NETWORKS
Computational and mathematical models of neurons have a long, storied history, but most ef-
forts can be broadly grouped into two categories: (1) models that replicate biological neurons to
explain or understand their behavior or (2) solve arbitrary problems using neuron-inspired mod-
els of computation. e former is typically the domain of biologists and cognitive scientists, and
computational models that fall into this group are often described as neuromorphic computing, as
the primary goal is to remain faithful to the original mechanisms. is book deals exclusively
with the latter category, in which loosely bio-inspired mathematical models are brought to bear
on a wide variety of unrelated, everyday problems. e two fields do share a fair amount of
common ground, and both are important areas of research, but the recent rise in practical appli-
cation of neural networks has been driven largely by this second area. ese techniques do not
solve learning problems in the same way that a human brain does, but, in exchange, they can
offer other advantages, such as being simpler for a human to build or mapping more naturally
to modern microprocessors. For brevity, we use “neural network” to refer to this second class of
algorithms in the rest of this book.
We’ll start by looking at a single artificial neuron. One of the earliest and still most widely
used models is the perceptron, first proposed by Rosenblatt in 1957. In modern language, a per-
ceptron is just two terms: a weighted sum of inputs x
i
, and a nonlinear activation function, ':
y D '
X
i
w
i
x
i
!
e vestiges of a biological neuron are visible: the summation reflects charge accumu-
lation in the soma, and ' loosely models the activation threshold of voltage-gated membrane