yarn集群的多队列可以让每个队列指定对应的nodeManger吗
时间: 2023-12-10 09:40:02 浏览: 107
是的,YARN集群的多队列可以让每个队列指定对应的NodeManager。这可以通过在yarn-site.xml配置文件中设置属性来实现。具体来说,可以使用以下属性来为每个队列指定NodeManager:
```
yarn.scheduler.capacity.root.<queue-name>.accessible-node-labels
yarn.scheduler.capacity.root.<queue-name>.capacity
yarn.scheduler.capacity.root.<queue-name>.maximum-capacity
yarn.scheduler.capacity.root.<queue-name>.minimum-user-limit-percent
yarn.scheduler.capacity.root.<queue-name>.ordering-policy
yarn.scheduler.capacity.root.<queue-name>.priority
yarn.scheduler.capacity.root.<queue-name>.queue-mappings
yarn.scheduler.capacity.root.<queue-name>.queue-path
yarn.scheduler.capacity.root.<queue-name>.user-limit-factor
yarn.scheduler.capacity.root.<queue-name>.user-limit-factor-args
yarn.scheduler.capacity.root.<queue-name>.user-limit
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.node-locality-delay
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.percent
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.ratio
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.timeout
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.total
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.total.percent
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.total.ratio
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.total.timeout
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.totalusers
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.users.percent
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.users.ratio
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.users.timeout
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.users.total
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.users.total.percent
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.users.total.ratio
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-per-rack.users.total.timeout
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.enabled
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.max-allowable-limit
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.monitoring-interval
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.natural-termination-factor
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.natural-termination-monitoring-interval
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.natural-termination-multiplier
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.natural-termination-threshold
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.natural-termination-vcore-ratio
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.natural-termination-wait-timeout
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.max-completion-interval
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.monitoring-interval-in-ms
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.preemption-utilization-threshold
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.total-preemption-per-round
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.total-preemption-per-round-ratio
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.total-preemption-per-round.timeout
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.total-preemption-per-user
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.preemption.total-preemption-per-user.timeout
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.resource-calculator
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.root-queue
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.maximum-am-resource-percent
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.maximum-applications
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.maximum-capacity
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.minimum-allocation-mb
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.minimum-allocation-vcores
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.maximum-allocation-mb
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.maximum-allocation-vcores
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.node-locality-delay
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.queue-mappings
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.reserve-mb
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.reserve-vcores
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.root-queue
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.user-limit-factor
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.user-limit-factor-args
yarn.scheduler.capacity.root.<queue-name>.yarn.scheduler.capacity.user-limit
```
在这些属性中,最重要的是`yarn.scheduler.capacity.root.<queue-name>.accessible-node-labels`属性,它可以指定某个队列可以访问哪些节点标签。例如,如果要为队列A指定NodeManager1作为其资源管理器,则需要在yarn-site.xml文件中添加以下配置:
```xml
<property>
<name>yarn.scheduler.capacity.root.A.accessible-node-labels</name>
<value>label1</value>
</property>
<property>
<name>yarn.scheduler.capacity.root.A.queue-mappings</name>
<value>A:nodeManager1</value>
</property>
```
这将使队列A只能访问标签为`label1`的节点,并将其分配给NodeManager1。
阅读全文
相关推荐











