生物启发的多层卷积脉冲神经网络：无监督学习与噪声鲁棒性

需积分: 50 112 浏览量更新于2024-09-01 收藏 1.07MB PDF 举报

本文主要探讨了一种创新的生物启发式多层卷积脉冲神经网络（Spiking Convolutional Neural Network, SNN），该网络在实践中解决了多层学习训练困难的问题。相比于传统的非脉冲神经网络，SNN在生物仿真性、低功耗硬件实现和理论计算能力方面具有优势。论文的核心在于提出了一种分层次的训练策略，即采用贪心、逐层的方式训练网络。首先，网络结构包括一个卷积/池化层，这一层采用稀疏的、基于脉冲的自编码器进行训练，目标是提取原始视觉特征。自编码器是一种无监督学习模型，通过压缩和重构输入数据来学习重要的特征表示，这对于理解图像中的关键元素至关重要。接着，论文引入了一个特征发现层，这一层利用概率化的突触时间依赖性可塑性（Probabilistic Spike-Timing-Dependent Plasticity, P-STDP）学习规则。P-STDP是一种模拟生物神经元间学习机制的算法，通过调整连接权重来反映神经元之间的同步活动，从而捕捉到更复杂的视觉特征。在这个层中，使用了Winner-Takes-All (WTA) 阈值的漏式积分与放电（Leaky Integrate-and-Fire, LIF）神经元，进一步提升了特征提取的效率和准确性。实验部分，作者将这种新型SNN应用到MNIST手写数字数据集上，对清洁图像和带有噪声的图像进行了测试。结果显示，对于清洁图像，该网络的识别性能超过98%，这主要归功于卷积层和特征发现层逐级提取的独立且信息丰富的视觉特征。然而，在处理噪声图像时，识别性能下降幅度在0.1%到8.5%之间，表明网络具有一定的抗噪能力，能够保持稳健的表现。这篇论文展示了如何通过分层学习和生物启发的方法，设计出一种易于训练且具有高效特征提取能力的多层卷积脉冲神经网络，这对于实际应用中的低功耗、生物模仿型计算有着积极的推动作用。未来的研究可以进一步优化训练策略和噪声处理方法，以提升网络在复杂环境下的鲁棒性和性能。

Multi-Layer Unsupervised Learning in a Spiking

Convolutional Neural Network

Amirhossein Tavanaei

The Center for Advanced Computer Studies

University of Louisiana at Lafayette, LA 70504, USA

Email: tavanaei@louisiana.edu

Anthony S. Maida

The Center for Advanced Computer Studies

University of Louisiana at Lafayette, LA 70504, USA

Email: maida@louisiana.edu

Abstract—Spiking neural networks (SNNs) have advantages

over traditional, non-spiking networks with respect to bio-

realism, potential for low-power hardware implementations, and

theoretical computing power. However, in practice, spiking net-

works with multi-layer learning have proven difﬁcult to train.

This paper explores a novel, bio-inspired spiking convolutional

neural network (CNN) that is trained in a greedy, layer-wise

fashion. The spiking CNN consists of a convolutional/pooling

layer followed by a feature discovery layer, both of which

undergo bio-inspired learning. Kernels for the convolutional layer

are trained using a sparse, spiking auto-encoder representing

primary visual features. The feature discovery layer uses a

probabilistic spike-timing-dependent plasticity (STDP) learning

rule. This layer represents complex visual features using WTA-

thresholded, leaky, integrate-and-ﬁre (LIF) neurons. The new

model is evaluated on the MNIST digit dataset using clean and

noisy images. Intermediate results show that the convolutional

layer is stack-admissible, enabling it to support a multi-layer

learning architecture. The recognition performance for clean

images is above 98%. This performance is accounted for by

the independent and informative visual features extracted in

a hierarchy of convolutional and feature discovery layers. The

performance loss for recognizing the noisy images is in the range

0.1% to 8.5%. This level of performance loss indicates that the

network is robust to additive noise.

I. INTRODUCTION

Hierarchical feature discovery using convolutional neural

networks (CNNs) has attracted much recent interest in ma-

chine learning and computer vision [1], [2]. CNNs have

outperformed previous models in several areas such as im-

age [3] and speech recognition [4]. Most CNNs are trained

by backpropagation, which cannot be computed locally, and

thus seems biologically implausible. This paper addresses

the challenge of training a spiking CNN with biologically

plausible local learning at the synapses (weights). In contrast

to conventional CNNs, spiking CNNs are amenable to low-

power hardware implementations.

One challenge to using spiking CNNs is that they are

difﬁcult to train. To illustrate this difﬁculty, we consider

some designs used in low-power neuromorphic hardware. To

avoid the difﬁculties of directly training a spiking CNN, these

networks are usually trained as a conventional (non-spiking)

CNN and then, after training, are converted to a spiking

network [5], [6], [7]. For instance, Cao et al. (2014) developed

a spiking CNN by converting an already trained, rate-based

CNN to a spike-based implementation [6]. Diehl et al. (2015)

extended the conversion method introduced in [6] to reduce

the performance loss during the conversion using a weight

adjustment approach [7]. However, a number of spiking CNNs

[8], [9], [10], [11] trained by spike-timing-dependent plasticity

(STDP) [12] currently exist. One of their limitations is they

utilize only one trainable layer of unsupervised learning.

The network of Masquelier and Thorpe (2007), which is

possibly the earliest spiking CNN, has this property [8]. It

consists of a convolutional/pooling layer followed by a feature

discovery layer and a classiﬁcation layer. Only the feature

discovery layer uses unsupervised learning. Wysoski et al.

(2008) used a similar design which extracted initial features

using a difference of Gaussian (DoG) ﬁlters in different orien-

tations [9]. This network also had only one trainable layer for

unsupervised learning. Furthermore, neither of these networks

trained the earlier feature extraction layer, but instead used

handcrafted Gabor or DoG ﬁlters. Recent extensions of [8]

providing multi-layer STDP-based networks still utilize the

handcrafted ﬁlters for primary visual feature extraction [13],

[14]. Recent work of [15] developed a backpropagation-trained

spiking auto-encoder for a multi-layer spiking CNN. However,

the backpropagation algorithm is not biologically plausible.

Our interest is in developing spiking CNNs which directly

use multi-layer learning such that the convolutional (feature

extraction) and feature discovery layers are trained locally

using layer-wise, unsupervised learning.

Our ﬁrst contribution replaces handcrafted convolutional

ﬁlters with learned detectors acquired by a biologically plau-

sible, state-of-the-art, sparse coding model [16]. The acquired

detectors represent the model receptive ﬁelds whose shapes

resemble those found in primate visual cortex (area V1).

Sparse representations, where each input state is coded by

a few active units, are a compromise between extremely

localized representations and fully distributed representations,

while being easy to analyze [17]. The construction of sparse

representations that resemble V1 receptive ﬁelds has been

achieved by different methods such as simple Hebbian units

connected by anti-Hebbian feedback synapses [17], minimiz-

ing reconstruction error combined with a sparsity regular-

izer [18], and independent components analysis (ICA) [19].

In terms of spike code formation, recent work has used

spiking networks to study the acquisition of visual sparse

code representations comparable to visual features found in

下载后可阅读完整内容，剩余7页未读，立即下载

小牧户

粉丝: 0
资源: 13

生物启发的多层卷积脉冲神经网络：无监督学习与噪声鲁棒性

脉冲神经网络

脉冲神经网络的一个训练方法

SNN脉冲神经网络理论解析PPT文件

医学图像处理医学图像处理-卷积神经网络卷积神经网络_小白也能看懂的卷积神经网络原理...

多重卷积神经网络的案例

零基础入门深度学习(4) - 卷积神经网络 - 作业部落 cmd markdown 编辑阅读器.pdf

1.掌握卷积操作应用 2.掌握构建卷积核 3.掌握构建tensor 4.掌握使用convd2d里不同参数属性

多层卷积神经网络实现原理

多层卷积网络效果好于单层卷积网络

最新资源