YARN Resource Calculation Based on Labels

MapR 6.1.0 implements correct steady/instantaneous Fair Scheduler shares, headroom, and maximum resource calculation for queues with label-based scheduling (LBS).

This new approach to YARN resource allocation enables the cluster LBS configuration to compute the resources that are allowed for each queue. It also uses LBS to assign containers to the correct queues to preempt containers, and to prevent resource overuse. The LBS approach to YARN resource allocation relies on:
  1. Resource computation (memory, CPU, disks) based on labels for each queue.
  2. Per-label preemption threshold.

Example of LBS Resource Computation

For example, imagine that you have the following resources:



Using the former fair-share resource distribution results in the following:



Using the new LBS resource distribution results in the following:



Understanding LBS preemption thresholds

Before LBS, preemption occurred by default when an entire cluster became 80% full (yarn.scheduler.fair.preemption.cluster-utilization-threshold=0.8f). With LBS, preemption occurs per labeled resource, as that resource becomes 80% full.

Descriptions of New Properties When Using LBS

  • defaultQueueLabel

    Assigned to all new queues and existing queues that do not have a label (excluding the root queue).

    For example, root.%username% queue is created if you submit a new job without queue information and property yarn.scheduler.fair.user-as-default-queue is true.

    You can specify this property in fair-scheduler.xml: 
    <allocations>
  <defaultQueueLabel>LabelA</defaultQueueLabel>
  <queue name="root">
           ...
  <queue name="queue1">
  </queue>
  <queue name="queue2">
    <label>LabelB</label>
   </queue>
           ...
  </queue>
</allocations>

    See Specifying Fair Scheduler Configuration Properties in yarn-site.xml for more information.

  • yarn.scheduler.fair.resources-based-on-labels-enabled

    Used to enable or disable recomputing of fair shares based on labels. It allows container allocation on all nodes.

    You can specify this property in yarn-site.xml: 
    <property>
  <name>yarn.scheduler.fair.resources-based-on-labels-enabled</name>
  <value>false</value>
</property>
  • yarn.scheduler.fair.preemption.cluster-utilization-threshold.based-on-labels-enabled

    Allows enabling or disabling (default) preemption of the threshold per-label. To overcome the default and start container preemption when the threshold of the label is exceeded, change this property to true.

    You can specify this property in yarn-site.xml: 
    <property>
  <name>yarn.scheduler.fair.preemption.cluster-utilization-threshold.based-on-labels-enabled</name>
  <value>false</value>
</property>

See Specifying Fair Scheduler Configuration Properties in yarn-site.xml for more information.

LBS Requirements and Restrictions?

Before you adopt LBS, you should be aware of two issues:
  • Multiple labels for a node and label expression at the queue level are not supported.
  • Except for root, all queues including the default, must be labeled, either independent or through inheritance.