知识蒸馏一点蒸馏、多点蒸馏
时间: 2023-08-29 15:11:12 浏览: 60
知识蒸馏可以采用一点蒸馏或多点蒸馏的策略。一点蒸馏是指在一个候选蒸馏点上进行蒸馏,常见的一点蒸馏策略包括始终进行蒸馏、随机决定是否进行蒸馏以及采用与自适应蒸馏相反的蒸馏策略。而多点蒸馏则是指在多个候选蒸馏点上进行蒸馏,通过在不同的位置进行蒸馏来提高知识转移的效率。<span class="em">1</span><span class="em">2</span><span class="em">3</span>
#### 引用[.reference_title]
- *1* [模型压缩 | TIP 2022 - 蒸馏位置自适应:Spot-adaptive Knowledge Distillation](https://blog.csdn.net/P_LarT/article/details/124923047)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v92^chatsearchT0_1"}}] [.reference_item style="max-width: 50%"]
- *2* *3* [论文阅读:Knowledge Distillation: A Survey 知识蒸馏综述(2021)](https://blog.csdn.net/qq_39710087/article/details/123796740)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v92^chatsearchT0_1"}}] [.reference_item style="max-width: 50%"]
[ .reference_list ]