4654
Table 1: Frequently used notations.
Notation Description
M Number of documents
K Number of topics
N Size of the vocabulary
L Size of the label set
D Document collection
d
i
∈ D The i-th document
V Vocabulary
v
j
∈ V The j-th word
Z Topic collection
z
k
∈ Z The k-th topic
Y Label collection
y
i
∈ R
L
Labels for document d
i
p(v
j
|d
i
) The probability of v
j
given d
i
and regular lowercase letters (e.g., v
j
) to represent
scalar variables. Based on the above convention,
frequently used notations are shown in Table 1.
Given a document d
i
with labels y
i
, our goal is
to discover topics with a neural network frame-
work. Therefore, we first describe the neural net-
work view of topic models briefly.
Topic modeling is a popular latent variable in-
ference method for co-occurrence data which as-
sociates unobserved classes with observations d
i
and v
j
, where v
j
is a word in d
i
. The conditional
probability p(v
j
|d
i
) is defined as:
p(v
j
|d
i
) =
K
X
k=1
p(v
j
|z
k
)p(z
k
|d
i
). (1)
Let φ(v
j
) = [p(v
j
|z
1
), . . . , p(v
j
|z
K
)] and
θ(d
i
) = [p(z
1
|d
i
), . . . , p(z
K
|d
i
)], then p(v
j
|d
i
)
in Equation 1 can be represented as the following
vector form:
p(v
j
|d
i
) = φ(v
j
) · θ(d
i
). (2)
We represent horizontal stack by commas
and vertical stack by semicolons, thus W
1
=
[θ(d
1
)
T
, . . . , θ(d
M
)
T
] ∈ R
K×M
and W
2
=
[φ(v
1
); . . . ; φ(v
N
] ∈ R
N×K
, which are con-
strained by: W
1
[k, m] ≥ 0, W
2
[n, k] ≥ 0,
P
K
k=1
W
1
[k, m] = 1, and
P
N
j=1
W
2
[j, k] = 1,
where k ∈ [1, K], m ∈ [1, M ], and n ∈ [1, N ].
Then, the vector form in Equation 2 can be ex-
Z
V
-
Z
V
Figure 1: SLTM’s word generation framework.
Document
set
Predicted label
Predicted conditional
probability
Input layer
.
.
.
Topic distribution
Hidden layer Output layer
Figure 2: SLTM’s architecture from the perspective of
neural networks.
tended to:
p(V|D) =
p(v
1
|d
1
) · · · p(v
1
|d
M
)
.
.
.
.
.
.
.
.
.
p(v
N
|d
1
) · · · p(v
N
|d
M
)
=
(φ(v
1
) · θ(d
1
)) · · · (φ(v
1
) · θ(d
M
))
.
.
.
.
.
.
.
.
.
(φ(v
N
) · θ(d
1
)) · · · (φ(v
N
) · θ(d
M
))
= W
2
W
1
. (3)
With Equation 3, topic models can be viewed
as neural networks, where D and V are input sets,
p(V|D) is the output set, and W
1
and W
2
are pa-
rameter matrices of the neural network.
4 Siamese Labeled Topic Model
Similar to generative models such as pLSA, we
propose a Siamese Labeled Topic Model (SLTM)
based on the aforementioned neural network per-
spective of topic models. Figure 1 illustrates the
framework of generating each word in SLTM, and
the process is as follows. For a document d
i
in D,
the topic distribution p(Z|d
i
) is estimated by:
p(Z|d
i
) = W
1
d
i
, (4)
剩余10页未读，继续阅读
weixin_38747025
- 粉丝: 129
- 资源: 1108
我的内容管理
展开
最新资源
- OptiX传输试题与SDH基础知识
- C++Builder函数详解与应用
- Linux shell (bash) 文件与字符串比较运算符详解
- Adam Gawne-Cain解读英文版WKT格式与常见投影标准
- dos命令详解：基础操作与网络测试必备
- Windows 蓝屏代码解析与处理指南
- PSoC CY8C24533在电动自行车控制器设计中的应用
- PHP整合FCKeditor网页编辑器教程
- Java Swing计算器源码示例：初学者入门教程
- Eclipse平台上的可视化开发：使用VEP与SWT
- 软件工程CASE工具实践指南
- AIX LVM详解：网络存储架构与管理
- 递归算法解析：文件系统、XML与树图
- 使用Struts2与MySQL构建Web登录验证教程
- PHP5 CLI模式：用PHP编写Shell脚本教程
- MyBatis与Spring完美整合：1.0.0-RC3详解
资源上传下载、课程学习等过程中有任何疑问或建议，欢迎提出宝贵意见哦~我们会及时处理！
点击此处反馈
