Configuring YARN for Control Groups

Control groups (cgroups) are a Linux kernel feature available through the LinuxContainerExecutor program that you can configure to limit and monitor the CPU resources available to YARN container processes on a node.

To enable cgroups, follow the Apache instructions for Using CGroups with YARN, but note that the default directory for yarn.nodemanager.linux-container-executor.cgroups.hierarchy (hadoop-yarn) is created automatically in the default place (/sys/fs/cgroup/cpu,cpuacct/) by mapr-warden, and permissions are set to the mapr user.

If you run Spark jobs, note the following consideration. When deployed in client mode, Spark driver programs are not controlled by cgroups, but the tasks are. In cluster mode, both the driver and tasks are controlled by cgroups. For more information about these modes, see Deployment Modes.