Resource Manager Metrics

Every 10 seconds, the collectd service uses a HPE Ezmeral Data Fabric plugin to gather Resource Manager metrics on the active Resource Manager. Collectd gathers metrics on the Resource Manager JVM process, YARN applications, and nodes that are managed by the Resource Manager. The method used to gather the metrics differs based on the metric type.

YARN Application Metrics

Collectd gathers YARN application metrics via JMX and REST API. The application metrics that are collected by JMX have the metric name mapr.rm.<metric_name>. Application metrics collected via REST API have the metric name mapr.rm_queue.<metric_name>.

Metrics Collected Using JMX

The following metrics are collected using JMX. To filter these metrics by queue using the rm_queue tag, see Configure Queue Filters for mapr.rm.<value> Metrics.
mapr.rm.active_applications
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of active applications.
mapr.rm.active_users
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of users with active applications.
mapr.rm.aggregate_containers_allocated
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of allocated containers.
mapr.rm.aggregate_containers_released
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of released containers.
mapr.rm.allocated_MB
Additional Tags: rm_queue: Display values for a specified queue.
Description: The amount of memory allocated to the Resource Manager in MB.
mapr.rm.allocated_vcores
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of CPUs allocated to the Resource Manager.
mapr.rm.apps_completed
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of completed applications.
mapr.rm.apps_failed
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of failed applications.
mapr.rm.apps_killed
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of killed applications.
mapr.rm.apps_pending
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of pending applications.
mapr.rm.apps_running
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of running applications.
mapr.rm.apps_submitted
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of submitted applications.
mapr.rm.available_MB
Additional Tags: rm_queue: Display values for a specified queue.
Description: The amount of memory available to the Resource Manager in MB.
mapr.rm.available_disks
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of disks available to the Resource Manager.
mapr.rm.available_vcores
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of CPUs available to the Resource Manager.
mapr.rm.pending_MB
Additional Tags: rm_queue: Display values for a specified queue.
Description: The amount of memory, in MB, waiting to be allocated by the Resource Manager.
mapr.rm.pending_containers
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of containers waiting to be allocated by the Resource Manager.
mapr.rm.pending_disks
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of disks waiting to be allocated by the Resource Manager.
mapr.rm.pending_vcores
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of CPUs waiting to be allocated by the Resource Manager.
mapr.rm.reserved_MB
Additional Tags: rm_queue: Display values for a specified queue.
Description: The amount of memory reserved by the Resource Manager in MB.
mapr.rm.reserved_containers
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of containers reserved by the Resource Manager.
mapr.rm.reserved_disks
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of disks reserved by the Resource Manager.
mapr.rm.reserved_vcores
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of CPUs reserved by the Resource Manager.

Metrics Collected Using REST API

The following YARN application metrics are collected using REST API.
mapr.rm_queue.aggregate_containers_allocated
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of containers allocated for applications in the default and custom queues.
mapr.rm_queue.appmaster_used_disks
Additional Tags: rm_queue: Display values for a specified queue.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the number of disks used by the Application Master for applications in the default and custom queues.
mapr.rm_queue.appmaster_used_memory
Additional Tags: rm_queue: Display values for a specified queue.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the amount of memory, in MB, used by the Application Master for applications in the default and custom queues.
mapr.rm_queue.appmaster_used_vcores
Additional Tags: rm_queue: Display values for a specified queue.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the number of CPUs used by the Application Master for applications in the default and custom queues.
mapr.rm_queue.apps_pending
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of pending applications in the default and custom queues.
mapr.rm_queue.apps_running
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of applications running in the default and custom queues.
mapr.rm_queue.fairshare_disks
Additional Tags: rm_queue: Display values for a specified queue.
Description: When queue resources are managed by the Fair Scheduler, this parameter is the number of disks allocated to default and custom queues.
mapr.rm_queue.fairshare_memory
Additional Tags: rm_queue: Display values for a specified queue.
Description: When queue resources are managed by the Fair Scheduler, this parameter denotes the amount of memory, in MB, allocated to default and custom queues.
mapr.rm_queue.fairshare_vcores
Additional Tags: rm_queue: Display values for a specified queue.
Description: When queue resources are managed by the Fair Scheduler, this parameter denotes the number of CPUs used by applications in the default and custom queues.
mapr.rm_queue.used_disks
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of disks used by applications in the default and custom queues.
mapr.rm_queue.used_memory
Additional Tags: rm_queue: Display values for a specified queue.
Description: The amount of memory, in MB, used by applications in the default and custom queues.
mapr.rm_queue.used_vcores
Additional Tags: rm_queue: Display values for a specified queue.
Description: The number of CPUs used by applications in the default and custom queues.
mapr.rm_queue.max_disks
Additional Tags: rm_queue: Display values for a specified queue.
Description: When queue resources are managed by the Fair Scheduler, this parameter denotes the maximum number of disks available to default and custom queues.
mapr.rm_queue.max_memory
Additional Tags: rm_queue: Display values for a specified queue.
Description: When queue resources are managed by the Fair Scheduler, this parameter denotes the maximum amount of memory, in MB, available to default and custom queues.
mapr.rm_queue.max_vcores
Additional Tags: rm_queue: Display values for a specified queue.
Description: When queue resources are managed by the Fair Scheduler, this parameter denotes the maximum number of CPUs available to default and custom queues.
mapr.rm_queue.user_allocated_disks
Additional Tags:
  • rm_queue: Display values for a specified queue.
  • rm_user: Display values for a specified user.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the number of disks allocated to the queues.
mapr.rm_queue.user_allocated_memory
Additional Tags:
  • rm_queue: Display values for a specified queue.
  • rm_user: Display values for a specified user.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the amount of memory, in MB, allocated to the queues.
mapr.rm_queue.user_allocated_vcores
Additional Tags:
  • rm_queue: Display values for a specified queue.
  • rm_user: Display values for a specified user.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the number of CPUs allocated to queues.
mapr.rm_queue.user_appmaster_used_disks
Additional Tags: rm_queue: Display values for a specified queue.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the number of disks used by the queues.
mapr.rm_queue.appmaster_used_memory
Additional Tags: rm_queue: Display values for a specified queue.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the amount of memory used by the queues.
mapr.rm_queue.appmaster_used_vcores
Additional Tags: rm_queue: Display values for a specified queue.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the number of CPUs used by the queues.
mapr.rm_queue.user_apps_pending
Additional Tags:
  • rm_queue: Display values for a specified queue.
  • rm_user: Display values for a specified user.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the number of applications pending in the queues.
mapr.rm_queue.user_apps_running
Additional Tags:
  • rm_queue: Display values for a specified queue.
  • rm_user: Display values for a specified user.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the number of applications running in the queues.
mapr.rm_queue.user_used_disks
Additional Tags:
  • rm_queue: Display values for a specified queue.
  • rm_user: Display values for a specified user.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the number of number of disks used by the queues.
mapr.rm_queue.user_used_memory
Additional Tags:
  • rm_queue: Display values for a specified queue.
  • rm_user: Display values for a specified user.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the amount of memory, in MB, used by the queues.
mapr.rm_queue.user_used_vcores
Additional Tags:
  • rm_queue: Display values for a specified queue.
  • rm_user: Display values for a specified user.
Description: When queue resources are managed by the Capacity Scheduler, this parameter denotes the number of CPUs used by the queues.

Resource Manager Node Metrics

The following are the Node metrics:
mapr.rm_cluster.active_nodes
The number of nodes in the cluster where containers are running.
mapr.rm_cluster.total_nodes
The number of nodes in the cluster.
mapr.rm_cluster.unhealthy_nodes
The number of nodes in the cluster that are unable to accept applications.

Resource Manager JVM Metrics

The following Resource Manager metrics are collected using JMX:
mapr.rm.jvm.gc_count
The number of garbage collections.
mapr.rm.jvm.gc_count_ps_mark_sweep
The number of parallel scavenge mark sweep collections.
mapr.rm.jvm.gc_count_ps_scavenge
The number of parallel scavenge collections.
mapr.rm.jvm.gc_time_millis
The amount of time, in milliseconds, spent on garbage collection.
mapr.rm.jvm.gc_time_millis_ps_mark_sweep
The amount of time, in milliseconds, spent on parallel scavenge mark sweep collection.
mapr.rm.jvm.gc_time_millis_ps_scavenge
The amount of time, in milliseconds, spent on parallel scavenge collection.
mapr.rm.jvm.log_error
The total number of ERROR logs.
mapr.rm.jvm.log_fatal
The total number of FATAL logs.
mapr.rm.jvm.log_info
The total number of INFO logs.
mapr.rm.jvm.log_warn
The total number of WARN logs.
mapr.rm.jvm.mem_heap_committed_m
The amount of heap memory, in megabytes, committed to the Resource Manager.
mapr.rm.jvm.mem_heap_max_m
The maximum amount of heap memory, in megabytes, that can be committed to the Resource Manager.
mapr.rm.jvm.mem_heap_used_m
The amount of heap memory, in megabytes, used by the Resource Manager.
mapr.rm.jvm.mem_max_m
The maximum amount of memory, in megabytes, that can be committed to the Resource Manager.
mapr.rm.jvm.mem_non_heap_committed_m
The amount of non-heap memory, in megabytes, committed to the Resource Manager.
mapr.rm.jvm.mem_non_heap_max_m
The maximum amount of non-heap memory, in megabytes, that can be committed to the Resource Manager.
mapr.rm.jvm.mem_non_heap_used_m
The maximum amount of non-heap memory, in megabytes, that can be used by the Resource Manager.
mapr.rm.jvm.threads_blocked
The number of Resource Manager threads in BLOCKED state.
mapr.rm.jvm.threads_new
The number of Resource Manager threads in NEW state.
mapr.rm.jvm.threads_runnable
The number of Resource Manager threads in RUNNABLE state.
mapr.rm.jvm.threads_terminated
The number of Resource Manager threads in TERMINATED state.
mapr.rm.jvm.threads_time_waiting
The number of Resource Manager threads in TIMED_WAITING state.
mapr.rm.jvm.threads_waiting
The number of Resource Manager threads in WAITING state.