YARN Container Resources
Provides an overview of YARN.
A YARN application can be a MapReduce version 2 (MRv2) application or a non-MapReduce application. The Warden on each node calculates the resources that can be allocated to process YARN applications. Each application has an ApplicationMaster that negotiates YARN container resources. For MapReduce applications, YARN processes each map or reduce task in a container. The ApplicationMaster requests resources from the ResourceManager based on memory, CPU, and disk requirements for the YARN containers. For YARN containers that process MRv2 tasks, there are additional considerations. See YARN Container Resources for MapReduce Version 2 Applications for details.
The ApplicationMaster requests YARN container resources based on the values of the
following parameters:
- yarn.scheduler.minimum-allocation-mb
- Default: 1024
- yarn.scheduler.maximum-allocation-mb
- Default: 8192
- yarn.nodemanager.resource.memory-mb
- Default: Variable. This value is calculated by Warden.
- yarn.nodemanager.resource.cpu-vcores
- Default: Variable. This value is calculated by Warden.
- yarn.nodemanager.resource.io-spindles
- Default: Variable. This value is calculated by Warden.