Scheduler Delay
时间: 2023-10-12 10:01:39 浏览: 47
Scheduler Delay是指任务在调度时的延迟时间。在Spark中,Scheduler Delay的计算可以使用公式:schedulerDelay = math.max(0, duration - runTime - deserializeTime - serializeTime - gettingResultTime)。其中,duration是任务的总运行时间,runTime是任务的实际运行时间,deserializeTime是任务的反序列化时间,serializeTime是任务的序列化时间,gettingResultTime是任务获取结果的时间。Scheduler Delay的目的是通过延迟调度来选择数据本地性,以提高任务的执行效率和公平性。<span class="em">1</span><span class="em">2</span><span class="em">3</span>
#### 引用[.reference_title]
- *1* *3* [spark on yarn 中的延迟调度(delay scheduler)](https://blog.csdn.net/qq403977698/article/details/51084437)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v92^chatsearchT3_1"}}] [.reference_item style="max-width: 50%"]
- *2* [Spark Task的各个动作时间来源以及Task Schedule Delay 问题排查](https://blog.csdn.net/wankunde/article/details/121403842)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v92^chatsearchT3_1"}}] [.reference_item style="max-width: 50%"]
[ .reference_list ]