Transformer可视化：字典学习揭示上下文嵌入的线性叠加

版权申诉

5星 · 超过95%的资源 186 浏览量更新于2024-07-19 收藏 2.69MB PDF 举报

"这篇PDF研究论文探讨了使用字典学习方法对Transformer模型进行可视化的创新方式，通过将上下文嵌入表示为Transformer因素的线性叠加，揭示了Transformer内部的深层次语义结构。作者们来自Facebook AI Research、Berkeley AI Research、New York University和Redwood Center for Theoretical Neuroscience，他们的工作旨在提升对Transformer网络理解的深度，特别是在自然语言处理（NLP）领域。 Transformer网络自其引入以来，已经彻底改变了NLP领域的表示学习。尽管已经进行了大量努力来解释Transformer中的表示，但普遍认为我们对其的理解仍然不足。主要问题之一是缺乏用于详细分析的可视化工具。为解决这一问题，论文提出了使用字典学习技术，将Transformer视为线性叠加的“因子”，从而打开这个“黑箱”。通过可视化，作者们展示了Transformer因素捕获的层次语义结构，例如：单词级别的多义词消歧、句子级别的模式形成以及长距离依赖关系。这些模式既证实了传统语言学的先验知识，也揭示了Transformer在理解和处理语言时的新颖之处。字典学习方法的应用使得研究人员能够更直观地理解Transformer如何在不同层次上编码信息。在词级，Transformer因素可以区分单词的不同意义，帮助消除多义性。在句子级别，这些因素展示出如何构建和理解各种语言模式。此外，通过揭示长距离依赖，证明了Transformer即使在复杂语境下也能有效处理远距离的句法和语义关系。这项工作为Transformer的研究提供了新的视角，有助于我们更好地理解Transformer如何在NLP任务中实现高效表示和处理，从而推动未来模型的设计和优化。同时，这也为其他机器学习领域的模型可视化提供了一种可能的方法，促进对深度学习模型内部运作机制的深入理解。"

Adversarial Text Explaination α

(o)

album as "full of exhilarating, ecstatic, thrilling, fun and

sometimes downright silly songs"

the original top-activated words and

their contexts for transformer factor

:,35

9.5

(a)

album as "full of delightful, lively, exciting, interesting

and sometimes downright silly songs"

use different adjective. 9.2

(b)

album as "full of unfortunate, heartbroken, annoying, bor-

ing and sometimes downright silly songs"

Change all adjective to negative adjec-

tive.

8.2

(c)

album as "full of [UNK], [UNK], thrilling, [UNK] and

sometimes downright silly songs"

Mask all adjective with Unknown To-

ken.

5.3

(d)

album as "full of thrilling and sometimes downright silly

songs"

Remove the ﬁrst three adjective. 7.8

(e)

album as "full of natural, smooth, rock, electronic and

sometimes downright silly songs"

Change adjective to neutral adjective. 6.2

(f)

each participant starts the battle with one balloon. these

can be re@-@ inﬂated up to four

Use an random sentence that has quo-

tation mark.

0.0

(g)

The book is described as "innovative, beautiful and bril-

liant". It receive the highest opinion from James Wood

A sentence we created that contain the

pattern of consecutive adjective.

7.9

Table 4: We construct adversarial texts similar but different to the pattern “Consecutive adjective”. The last column

shows the activation of Φ

:,35

, or α

(8)

, w.r.t. the blue-marked word in layer 8.

correspond to a speciﬁc pattern, we can use con-

structed example words and context to probe their

activation. In Table 4, we construct several text

sequences that are similar to the patterns corre-

sponding to a certain transformer factor but with

subtle differences. The result conﬁrms that the con-

text that strictly follows the pattern represented by

that transformer factor triggers a high activation.

On the other hand, the more closer the adversar-

ial example to this pattern, the higher activation it

receives at this transformer factor.

High-Level: Long-Range Dependency.

High-

level transformer factors correspond to those lin-

guistic patterns that span a long-range in the text.

Since the IS curves of mid-level and high-level

transformer factors are similar, it is difﬁcult to dis-

tinguish those transformer factors based on their IS

curves. Thus, we have to manually examine the top-

activation words and contexts for each transformer

factor to distinguish whether they are mid-level or

high-level transformer factors. In order to ease the

process, we can use the black-box interpretation

algorithm LIME (Ribeiro et al., 2016) to identify

the contribution of each token in a sequence.

Given a sequence

s ∈ S

, we can treat

(l)

c,i

, the

activation of

:,c

in layer-

at location

, as a scalar

function of

(l)

c,i

(s)

. Assume a sequence

trig-

gers a high activation

(l)

c,i

, i.e.

(l)

c,i

(s)

is large. We

want to know how much each token (or equivalently

each position) in

contributes to

(l)

c,i

(s)

. To do

so, we generated a sequence set

S(s)

, where each

∈ S(s)

is the same as

except for that several

random positions in

are masked by [‘UNK’] (the

unknown token). Then we learns a linear model

)

with weights

w ∈ R

to approximate

f(s

)

where

is the length of sentence

. This can be

solved as a ridge regression:

min

w∈R

L(f, w, S(s)) + σkwk

The learned weights

can serve as a saliency

map that reﬂects the “contribution” of each token

in the sequence

. Like in Figure 7, the color re-

ﬂects the weights

at each position, red means

the given position has positive weight, and green

means negative weight. The magnitude of weight

is represented by the intensity. The redder a token

is, the more it contributions to the activation of

the transformer factor. We leave more implementa-

tion and mathematical formulation details of LIME

algorithm in the appendix.

We provide detailed visualization for 2 different

transformer factors that show long-range depen-

dency in Figure 7, 8. Since visualization of high-

level information requires longer context, we only

show the top 2 activated words and their contexts

for each such transformer factor. Many more will

be provided in the appendix section G.

We name the pattern for transformer factor

:,297

in Figure 7 as “repetitive pattern detector”. All top

activated contexts for

:,297

contain an obvious

repetitive structure. Speciﬁcally, the text snippet

“can’t get you out of my head" appears twice in

the ﬁrst example, and the text snippet “xxx class

passenger, star alliance” appears 3 times in the sec-

ond example. Compared to the patterns we found

in the mid-level [6], the high-level patterns like

剩余33页未读，继续阅读

Fun_He

粉丝: 19
资源: 104

Transformer可视化：字典学习揭示上下文嵌入的线性叠加

Transformer-Explainability:[CVPR 2021]超越注意力可视化的变压器可解释性的官方PyTorch实施，这是一种通过基于变压器的网络对分类进行可视化的新方法

bertviz：在Transformer模型中可视化注意力的工具（BERT，GPT-2，Albert，XLNet，RoBERTa，CTRL等）

Swin-Transformer+gradcam可视化代码+Windows环境.zip

配电网规划工作的数字化转型研究.pdf

可交互的 Attention 可视化工具！我的Transformer可解释性有救了？.pdf

AI基础：图解Transformer.pdf

基于深度学习的煤矿领域实体关系抽取研究.pdf

音视频-编解码-概念图像化引导对创造性联想影响的研究.pdf

易云维智慧工业云平台让ChatGPT重新认识中国制造业信息化水平.pdf

2023预训练大模型与医疗：从算法研究到应用.pdf

最新资源