Character-level Convolutional Networks for Text
Classification
∗
Xiang Zhang Junbo Zhao Yann LeCun
Courant Institute of Mathematical Sciences, New York University
719 Broadway, 12th Floor, New York, NY 10003
{xiang, junbo.zhao, yann}@cs.nyu.edu
Abstract
This article offers an empirical exploration on the use of character-level convolu-
tional networks (ConvNets) for text classification. We constructed several large-
scale datasets to show that character-level convolutional networks could achieve
state-of-the-art or competitive results. Comparisons are offered against traditional
models such as bag of words, n-grams and their TFIDF variants, and deep learning
models such as word-based ConvNets and recurrent neural networks.
1 Introduction
Text classification is a classic topic for natural language processing, in which one needs to assign
predefined categories to free-text documents. The range of text classification research goes from
designing the best features to choosing the best possible machine learning classifiers. To date,
almost all techniques of text classification are based on words, in which simple statistics of some
ordered word combinations (such as n-grams) usually perform the best [12].
On the other hand, many researchers have found convolutional networks (ConvNets) [17] [18] are
useful in extracting information from raw signals, ranging from computer vision applications to
speech recognition and others. In particular, time-delay networks used in the early days of deep
learning research are essentially convolutional networks that model sequential data [1] [31].
In this article we explore treating text as a kind of raw signal at character level, and applying tem-
poral (one-dimensional) ConvNets to it. For this article we only used a classification task as a way
to exemplify ConvNets’ ability to understand texts. Historically we know that ConvNets usually
require large-scale datasets to work, therefore we also build several of them. An extensive set of
comparisons is offered with traditional models and other deep learning models.
Applying convolutional networks to text classification or natural language processing at large was
explored in literature. It has been shown that ConvNets can be directly applied to distributed [6] [16]
or discrete [13] embedding of words, without any knowledge on the syntactic or semantic structures
of a language. These approaches have been proven to be competitive to traditional models.
There are also related works that use character-level features for language processing. These in-
clude using character-level n-grams with linear classifiers [15], and incorporating character-level
features to ConvNets [28] [29]. In particular, these ConvNet approaches use words as a basis, in
which character-level features extracted at word [28] or word n-gram [29] level form a distributed
representation. Improvements for part-of-speech tagging and information retrieval were observed.
This article is the first to apply ConvNets only on characters. We show that when trained on large-
scale datasets, deep ConvNets do not require the knowledge of words, in addition to the conclusion
∗
An early version of this work entitled “Text Understanding from Scratch” was posted in Feb 2015 as
arXiv:1502.01710. The present paper has considerably more experimental results and a rewritten introduction.
1
下载后可阅读完整内容，剩余8页未读，立即下载
Yue_Zengying
- 粉丝: 4
- 资源: 5
我的内容管理
展开
最新资源
- 李兴华Java基础教程：从入门到精通
- U盘与硬盘启动安装教程：从菜鸟到专家
- C++面试宝典：动态内存管理与继承解析
- C++ STL源码深度解析：专家级剖析与关键技术
- C/C++调用DOS命令实战指南
- 神经网络补偿的多传感器航迹融合技术
- GIS中的大地坐标系与椭球体解析
- 海思Hi3515 H.264编解码处理器用户手册
- Oracle基础练习题与解答
- 谷歌地球3D建筑筛选新流程详解
- CFO与CIO携手：数据管理与企业增值的战略
- Eclipse IDE基础教程：从入门到精通
- Shell脚本专家宝典：全面学习与资源指南
- Tomcat安装指南：附带JDK配置步骤
- NA3003A电子水准仪数据格式解析与转换研究
- 自动化专业英语词汇精华：必备术语集锦
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
