深度卷积稀疏自编码器的多级委员会优化对象识别

119 浏览量更新于2024-08-29 收藏 979KB PDF 举报

本文探讨了深度前馈卷积稀疏降噪自动编码器（Deep Feedforward Convolutional Sparse Denoise Autoencoder, DFCSDA）在对象识别任务中的应用，特别是在提高分类性能方面的创新方法。传统上，深度学习和无监督特征学习系统通过大规模架构和每层众多特征在基准测试中表现出色。然而，研究者发现当特征数量超过某个阈值后，它们对性能的影响变得微小。因此，论文的核心思想是提出一种改进策略，通过深度学习网络的层次结构和多级委员会（Multistage Committees）来优化。 DFCSDA的多级委员会模型通过逐层训练，每个阶段仅使用少量特征。这种方法利用了卷积核的优化，采用L-BFGS算法而非反向传播更新网络参数。关键在于，通过将多个浅层分类器集成到一个委员会中，即使在每个层次上的特征较少，也能提升整体的识别精度。这种设计旨在降低计算复杂度，同时保持良好的性能。具体来说，论文的流程包括以下步骤： 1. **特征提取与压缩**：DFCSDA作为预训练模型，通过自编码器的结构进行特征学习和降噪，提取出关键的低维表示。 2. **分层处理**：网络被设计成多级结构，每一级包含一组小型特征子集，这有助于减少过拟合并提高泛化能力。 3. **级联分类**：每个层次的输出通过级联的方式传递到下一级，每一级负责特定的识别任务，最终由整个委员会决定最终分类结果。 4. **优化策略**：利用L-BFGS算法优化卷积核，避免了反向传播中的梯度消失或爆炸问题，提高了收敛速度。 5. **实验验证**：通过实际的物体识别数据集进行实验，评估了多级委员会DFCSDA模型相对于传统深度学习模型在性能上的提升，以及其在资源效率和准确性之间的平衡。这篇研究论文提供了一种有效的解决方案，利用深度前馈卷积神经网络的稀疏降噪机制和多级模块化策略，改善了对象识别任务中的特征利用率和分类性能，为实际应用中的资源管理和精度提升提供了新的视角。

Multistage Committees of Deep Feedforward

Convolutional Sparse Denoise Autoencoder for

Object Recognition

Shicao Luo

1.College of Information Science and Technology,

Donghua University

Shanghai 201620, China

Email: shicaoLuo@163.com

Yongsheng Ding

1,2

* , Kuangrong Hao

1,2

2. Engineering Research Center of Digitized Textile &

Apparel Technology

Shanghai 201620, China

* Email: ysding@dhu.edu.cn

Abstract—Deep learning and unsupervised feature learning

systems are known to achieve good performance in benchmarks

by using extremely large architectures with many features at

each layer. However, we found that the number of features’

contribution to performance is very small when it is more than

the threshold. Meanwhile, the size of pooling layer has an

important influence on performance. In this paper, we present an

unsupervised method to improve the classification result by going

deep and combining multistage classifiers in a committee with a

small amount of features at each layer. The network is trained

layer-wise via denoise autoencoder (dA) with L-BFGS to optimize

convolutional kernels and no backpropagation is used. In

addition, we regularize the dA encouraging representations to fit

sparse for each coding layer. We apply it on the STL-10 dataset

which has very few training examples and a large amount of

unlabeled data. Experimental results show that our method

presents higher performance than the existing ones on the

condition via individual network.

Keywords—multistage classifiers; sparse denoise autoencoder;

object recognition; deep learning

I. INTRODUCTION

Recent theoretical studies indicate that unsupervised

learning methods are able to automatically build feature

extractors instead of handcrafting them. Classical methods for

dimensionality reduction or clustering, such as principal

component analysis and K-means, have been used routinely in

numerous vision applications [1, 2].

In the context of object recognition, a key problem of

current machine vision systems is whether unsupervised

learning can be used to learn robust and invariant features.

Traditionally, scale invariant feature transform (SITF) or

speed up robust features (SURF) can be understood and

generalized as a way to go from pixels to patch descriptors [3].

However, it is often difficult to adapt to new settings. Unlike

the above mentioned artificial systems for feature learning,

primate visual system [24] can accomplish these tasks

effortlessly in common sense. The groundbreaking work of

Hubel and Wiesel [28] played a major role in the computer

vision community via Marr’s work [29] on building visual

hierarchies analogous to the primate visual system. To narrow

the gap with biological systems and play computer’s own

advantages, much recent research has focused on training deep,

multi-layered networks of feature, such as deep belief nets [4],

deep auto-encoders [5], deep convolutional neural networks

[6], hierarchical sparse coding [7, 8], and SIFT-based MFs

coding [25]. The main benefit of these models is their high

genericity, since deep learning approaches that learn to push

pixels through multiple layers of feature transforms, without

any use of prior knowledge. And, a trick called “receptive

field” dramatically reduces the number of parameters that

must be trained and is a key element of several state-of-the-art

systems [9, 10].

It has been proved that deep models are potentially more

capable than shallow models in handling complex tasks [11].

Deep belief nets [12] learn a hierarchy of features by greedy

layer-wise using the unsupervised restricted Boltzmann

machine. This pre-training method is useful to jump out of

local minimum. The learned weights are then further adjusted

to the current task using supervised information. To make

deep belief nets applicable to full-size images, convolutional

deep belief nets [13] are proposed to use a small receptive

fields and share the weights between the hidden and visible

layers among all locations in an image. Stacked denoising

autoencoders [14] build deep networks by stacking layers of

denoising autoencoders that train one-layer neural network to

reconstruct input data from partial random corruption. Spike-

and-slab sparse coding (S3C) [26] is a preexisting model

which gains very good performance, particularly when the

number of labeled examples is low.

In this paper, we propose an unsupervised learning model

for large size image recognition. In this network, we learn the

following key components: 1) A sparse and overcomplete

feature bank at each layer; 2) The factor of multistage

classifiers for voting. The network is trained layer-wise via

denoise autoencoder and no backpropagation is used. We

apply it on the STL-10 dataset which is an image recognition

dataset for developing unsupervised feature learning, deep

learning, and self-taught learning algorithms. In particular,

each class has few labeled training examples, but a very large

set of unlabeled examples. Zou [15] built a hierarchical

network to learns invariant features via simulated fixations in

video. With this method, it gains 4.5% improvement in STL-

10 classification accuracy. Coates [16] introduced “Selecting

Receptive Fields” method to limit the number of connections

from lower level features to higher ones and achieved the

565

下载后可阅读完整内容，剩余5页未读，立即下载

weixin_38635166

粉丝: 8
资源: 876

深度卷积稀疏自编码器的多级委员会优化对象识别

深度学习 深度前馈网络

CS294A Lecture notes Sparse autoencoder （稀疏自编码器课程讲义，吴恩达）

MATLAB用于体脂数据上训练网络，开发用于模式识别的两层前馈网络，以及在MNIST数据上实现用于降维的自动编码器.rar

orthogonal-denoising-autoencoder:论文“利用正交降噪自动编码器学习多视图”的实现和改进

基于无监督的基于稀疏自动编码器的深度神经网络的频率选择性衰落场景中的AMC鲁棒方法

Log MAP 解码器：用于前馈和反馈卷积码的对数 MAP 解码器-matlab开发

深度前馈网络pdf

不要再纠结卷积的公式啦！0公式深度解析全连接前馈网络与卷积神经网络.pdf

不要再纠结卷积的公式啦！0公式深度解析全连接前馈网络与卷积神经网络.rar

具有基于随机稀疏矩阵的自动编码器的增强型分层极限学习机

最新资源

深度学习深度前馈网络