Shunted Transformer

Shunted Transformer是一种改进的Transformer模型，与传统的Transformer模型相比，它更加注重自注意层内场景对象的多尺度性质。Shunted Transformer通过多尺度的标记汇聚来实现。具体而言，Shunted Transformer模型分为四个阶段，每个阶段包含多个Shunted Transformer块。在每个阶段中，每个块输出相同大小的特征图。为了连接不同的阶段，采用了带有stride 2的卷积层，这会导致特征图的尺寸减半，但维度增加一倍。这种结构可以帮助模型更好地捕捉不同尺度的信息，提高对场景对象的建模能力。如果你对这个模型更加具体的实现细节感兴趣，可以参考相关的论文和Github链接。<span class="em">1</span><span class="em">2</span><span class="em">3</span>
#### 引用[.reference_title]
- *1* *3* [论文笔记(Shunted-Transformer)](https://blog.csdn.net/Karl51/article/details/129338845)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_1"}}] [.reference_item style="max-width: 50%"]
- *2* [Shunted Transformer 飞桨权重迁移体验](https://blog.csdn.net/m0_63642362/article/details/124335344)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_1"}}] [.reference_item style="max-width: 50%"]
[ .reference_list ]