Resource Manager Metrics
Every 10 seconds, the collectd
service uses a HPE Ezmeral Data Fabric plugin to gather
Resource Manager metrics on the active Resource Manager. Collectd
gathers
metrics on the Resource Manager JVM process, YARN applications, and nodes that are managed by
the Resource Manager. The method used to gather the metrics differs based on the metric
type.
YARN Application Metrics
Collectd
gathers YARN application metrics via JMX and REST
API. The application metrics that are collected by JMX have the metric name
mapr.rm.<metric_name>
. Application metrics collected via REST API have
the metric name mapr.rm_queue.<metric_name>
.
Metrics Collected Using JMX
The following
metrics are collected using JMX. To filter these metrics by queue using the rm_queue
tag, see Configure Queue Filters for mapr.rm.<value> Metrics.
- mapr.rm.active_applications
- Additional Tags:
rm_queue
: Display values for a specified queue. - mapr.rm.active_users
- Additional Tags:
rm_queue
: Display values for a specified queue. - mapr.rm.aggregate_containers_allocated
- Additional Tags:
rm_queue
: Display values for a specified queue. - mapr.rm.aggregate_containers_released
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.allocated_MB
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.allocated_vcores
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.apps_completed
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.apps_failed
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.apps_killed
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.apps_pending
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.apps_running
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.apps_submitted
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.available_MB
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.available_disks
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.available_vcores
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.pending_MB
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.pending_containers
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.pending_disks
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.pending_vcores
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.reserved_MB
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.reserved_containers
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.reserved_disks
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm.reserved_vcores
- Additional Tags:
rm_queue
: Display values for a specified queue.
Metrics Collected Using REST API
The following YARN application metrics are collected using REST API.
- mapr.rm_queue.aggregate_containers_allocated
- Additional Tags:
rm_queue
: Display values for a specified queue. - mapr.rm_queue.appmaster_used_disks
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.appmaster_used_memory
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.appmaster_used_vcores
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.apps_pending
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.apps_running
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.fairshare_disks
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.fairshare_memory
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.fairshare_vcores
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.used_disks
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.used_memory
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.used_vcores
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.max_disks
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.max_memory
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.max_vcores
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.user_allocated_disks
- Additional Tags:
rm_queue
: Display values for a specified queue.rm_user
: Display values for a specified user.
- mapr.rm_queue.user_allocated_memory
- Additional Tags:
rm_queue
: Display values for a specified queue.rm_user
: Display values for a specified user.
- mapr.rm_queue.user_allocated_vcores
- Additional Tags:
rm_queue
: Display values for a specified queue.rm_user
: Display values for a specified user.
- mapr.rm_queue.user_appmaster_used_disks
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.appmaster_used_memory
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.appmaster_used_vcores
- Additional Tags:
rm_queue
: Display values for a specified queue.
- mapr.rm_queue.user_apps_pending
- Additional Tags:
rm_queue
: Display values for a specified queue.rm_user
: Display values for a specified user.
- mapr.rm_queue.user_apps_running
- Additional Tags:
rm_queue
: Display values for a specified queue.rm_user
: Display values for a specified user.
- mapr.rm_queue.user_used_disks
- Additional Tags:
rm_queue
: Display values for a specified queue.rm_user
: Display values for a specified user.
- mapr.rm_queue.user_used_memory
- Additional Tags:
rm_queue
: Display values for a specified queue.rm_user
: Display values for a specified user.
- mapr.rm_queue.user_used_vcores
- Additional Tags:
rm_queue
: Display values for a specified queue.rm_user
: Display values for a specified user.
Resource Manager Node Metrics
The following are the Node metrics:
- mapr.rm_cluster.active_nodes
- The number of nodes in the cluster where containers are running.
- mapr.rm_cluster.total_nodes
- The number of nodes in the cluster.
- mapr.rm_cluster.unhealthy_nodes
- The number of nodes in the cluster that are unable to accept applications.
Resource Manager JVM Metrics
The following Resource Manager metrics are collected using JMX:
- mapr.rm.jvm.gc_count
- The number of garbage collections.
- mapr.rm.jvm.gc_count_ps_mark_sweep
- The number of parallel scavenge mark sweep collections.
- mapr.rm.jvm.gc_count_ps_scavenge
- The number of parallel scavenge collections.
- mapr.rm.jvm.gc_time_millis
- The amount of time, in milliseconds, spent on garbage collection.
- mapr.rm.jvm.gc_time_millis_ps_mark_sweep
- The amount of time, in milliseconds, spent on parallel scavenge mark sweep collection.
- mapr.rm.jvm.gc_time_millis_ps_scavenge
- The amount of time, in milliseconds, spent on parallel scavenge collection.
- mapr.rm.jvm.log_error
- The total number of ERROR logs.
- mapr.rm.jvm.log_fatal
- The total number of FATAL logs.
- mapr.rm.jvm.log_info
- The total number of INFO logs.
- mapr.rm.jvm.log_warn
- The total number of WARN logs.
- mapr.rm.jvm.mem_heap_committed_m
- The amount of heap memory, in megabytes, committed to the Resource Manager.
- mapr.rm.jvm.mem_heap_max_m
- The maximum amount of heap memory, in megabytes, that can be committed to the Resource Manager.
- mapr.rm.jvm.mem_heap_used_m
- The amount of heap memory, in megabytes, used by the Resource Manager.
- mapr.rm.jvm.mem_max_m
- The maximum amount of memory, in megabytes, that can be committed to the Resource Manager.
- mapr.rm.jvm.mem_non_heap_committed_m
- The amount of non-heap memory, in megabytes, committed to the Resource Manager.
- mapr.rm.jvm.mem_non_heap_max_m
- The maximum amount of non-heap memory, in megabytes, that can be committed to the Resource Manager.
- mapr.rm.jvm.mem_non_heap_used_m
- The maximum amount of non-heap memory, in megabytes, that can be used by the Resource Manager.
- mapr.rm.jvm.threads_blocked
- The number of Resource Manager threads in BLOCKED state.
- mapr.rm.jvm.threads_new
- The number of Resource Manager threads in NEW state.
- mapr.rm.jvm.threads_runnable
- The number of Resource Manager threads in RUNNABLE state.
- mapr.rm.jvm.threads_terminated
- The number of Resource Manager threads in TERMINATED state.
- mapr.rm.jvm.threads_time_waiting
- The number of Resource Manager threads in TIMED_WAITING state.
- mapr.rm.jvm.threads_waiting
- The number of Resource Manager threads in WAITING state.