Entropy 2015, 17 6010
2.2. Distributed Sentence Composition Model
Distributed Sentence Composition refers to representing a sentence in a low-dimensional space for
conveying the semantic information contained in the sentence. Various types of distributed sentence
representation models have been proposed recently. Socher et al. [16] proposed a recursive neural tensor
network (RNTN) for semantic compositionality over a sentiment Treebank which pushes the binary
classification accuracy on Stanford sentiment tree bank from 80% up to 85.4%. Kalchbrenner et al. [17]
proposed a dynamic convolutional neural network (DCNN) to handles the input sentences with varying
length and induced a feature graph over the sentence that is capable of explicitly capturing short and
long-range relations. It improves the above accuracy from 85.4% to 86.8%. Kim [18] presented two
simple CNN models with little hyper parameter tuning which are trained on pre-trained word vectors for
sentence-level classification tasks. It further improves the above accuracy to 88.1%. Le and Mikolov [19]
proposed an unsupervised algorithm that learns fixed-length feature representations from variable-length
pieces of texts, such as sentences, paragraphs and documents. The recurrent neural network (RNN) may
also be viewed as a sentence model. The layer computed at the last word represents the sentence [11,17].
3. Our Approach
In this study, we propose to learn distributed representation of word sense learning approach by
incorporating WordNet gloss compositionality and context words clustering in large-scale raw text.
The system framework with three main components is shown in Figure 1. The first component, the
Word Embedding Construction Module, takes a large collection of raw text to train a word embedding
model. The word embeddings output by the model are then used by a Sentence Composition Model,
which takes glosses in WordNet as positive training data and randomly replacing part of the words as
negative training data to construct the corresponding word sense vectors based on the one-dimensional
CNN. The learned sense vectors are fed into a variant of the previously proposed Multi-Sense Skip-Gram
Model (MSSG) to generates distributed representations of word senses from a text corpus. We name our
approach as CNN-VMSSG.
Figure 1. Framework of our approach.
剩余17页未读，继续阅读
weixin_38693311
- 粉丝: 4
- 资源: 922
我的内容管理
展开
最新资源
- 中国微型数字传声器：技术革新与市场前景
- 智能安防：基于Hi3515的嵌入式云台控制系统设计
- 手机电量低时辐射真增千倍？解析手机使用谣言
- 56F803型DSP驱动的高精度大功率超声波电源控制策略研究
- ARM与GPRS结合的远程监测系统设计
- GPS与RFID技术结合的智能巡检系统设计
- CPLD驱动的低功耗爆炸场温度测试系统设计
- 基于FPGA的智能驱动控制系统：可扩展设计与工业网络协议
- 基于ATmega128和CH374的嵌入式USB接口设计
- 基于AT89C52的温度补偿超声波测距仪：高精度设计与应用
- MSP430F448单片机在交流数字电压表中的应用
- 提升变频器应用效率的12项实用技巧
- STM32F103在数字电镀电源并联均流系统中的应用
- PSpice仿真下的升压开关电源设计：拓扑分析与CCM稳定性提升
- 轻巧高效：MSP430主导的低成本无线传感器网络节点设计
- FPGA在EDA/PLD中实现LVDS接口的应用解析
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
