YARN Container Resources for MapReduce Version 2 Applications

In addition to the YARN container resource allocation parameters, the MapReduce ApplicationMaster also considers the following container requirements when it sends requests to the ResourceManager for containers to run MapReduce applications:

Parameter Default Description

mapreduce.map.memory.mb

1024 Defines the container size for map tasks in MB.
mapreduce.reduce.memory.mb 3072

Defines the container size for reduce tasks in MB.
mapreduce.reduce.java.opts -Xmx2560m --add-opens java.base/java.lang=ALL-UNNAMED -XX:+UseParallelGC Java options for reduce tasks.
mapreduce.map.java.opts -Xmx900m --add-opens java.base/java.lang=ALL-UNNAMED -XX:+UseParallelGC Java options for map tasks.

mapreduce.map.disk

0.5

Defines the number of disks a map task requires. For example, a node with 4 disks can run 8 map tasks at a time. Note: If I/O intensive tasks do not run on the node, you may want to change this value.

mapreduce.reduce.disk

1.33

Defines the number of disks that a reduce task requires. For example, a node with 4 disks can run 3 reduce tasks at a time. Note: If I/O intensive tasks do not run on the node, you might want to change this value.

You can use one of the following methods to change the default configuration:

  • Provide updated values in the mapred-site.xml file on the node that runs the job. You can use central configuration to change this value on each node that runs the NodeManager in the cluster. Then, restart NodeManager on each node in the cluster. The mapred-site.xml file for MapReduce ve applications is located in the following directory: opt/mapr/hadoop/hadoop-2.x.x/etc/hadoop
  • Override the default values from the command line for each application that requires a non-default value.