搭建transformer

搭建transformer模型可以分为Encoder和Decoder两个部分。在搭建Encoder时，可以利用ResNet中的跳层链接技巧和常见的Batch Normalization等技巧，输入和输出的矩阵大小均为512×m。在搭建Decoder时，可以参考Transformer模型的结构，使用自注意力机制进行序列建模。Transformer模型相较于RNN和CNN更高效、更容易并行化，广泛应用于神经机器翻译、文本生成、问答等任务。<span class="em">1</span><span class="em">2</span><span class="em">3</span>
#### 引用[.reference_title]
- *1* *3* [搭建Transformer模型](https://blog.csdn.net/qq_24178985/article/details/118884171)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v92^chatsearchT0_1"}}] [.reference_item style="max-width: 50%"]
- *2* [深度学习实战24-人工智能(Pytorch)搭建transformer模型,真正跑通transformer模型，深刻了解transformer的...](https://blog.csdn.net/weixin_42878111/article/details/130043148)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v92^chatsearchT0_1"}}] [.reference_item style="max-width: 50%"]
[ .reference_list ]