Kafka JMX Metrics
Starting in EEP 9.0.0, you can enable metrics collection for Kafka consumers and producers. When enabled, JMX collects Kafka producer and consumer metrics from client applications. You can view the metrics through the JConsole UI or JMXTerm CLI.
Metrics Collected
When you enable Kafka JMX metrics, you can view the following producer and consumer metrics
through the JConsole UI or JMXTerm CLI :
- Producer Metrics Collected
- Global metrics collected:
- record-send-rate
- record-send-total
- record-error-rate
- record-error-total
- record-size-max
- record-size-avgNOTERecord size is computed as the sum of topic length, key size, value size, and sizes of keys and values of all headers if any exist.
- Consumer Metrics Collected
-
- fetch-size-avg
- fetch-size-max
- bytes-consumed-rate
- bytes-consumed-total
- records-consumed-rate
- records-consumed-total
NOTE
This list of supported metrics is a subset of all the Kafka JMX metrics. The
unsupported metrics are either not registered or have default values such as 0 or NaN. A
full list of Kafka JMX metrics is available here. Enabling Metrics Collection
To enable metrics collection, set metrics.enabled=true
in the producer and
consumer configuration.
The following sections provide steps to enable metrics collection and view metrics in the JConsole UI and JMXTerm CLI.
Using the JConsole UI to View Metrics
Complete the following steps to enable metrics collection and view metrics in the JConsole
UI:
- Start the console producer with metrics collection
enabled:
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic /s:t --producer-property metrics.enabled=true
- Enter and send data.
- In another terminal, run the following command to start
JConsole:
jconsole
- In JConsole, select the console producer process and connect to it.
- Select Insecure Connection.
- In the MBeans section, look at kafka.producer or kafka.producer domain, and select the MBean of interest to view its attributes.
- Click Refresh to update attribute values.
Using the JMXTerm CLI to View Metrics
JMXTerm is the CLI analog of JConsole. To download JMXTerm, go to https://docs.cyclopsgroup.org/jmxterm.
Complete the following steps to enable metrics collection and view metrics through the
JMXTerm CLI:
- Start the console producer with metrics collection
enabled:
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic /s:t --producer-property metrics.enabled=true
- Run the following command to start JMXTerm in interactive
mode:
java -jar jmxterm-1.0.2-uber.jar
- Run the
jvms
command to identify which process is running the Kafka console producer:jvms WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.cyclopsgroup.jmxterm.utils.WeakCastUtils$2 (file:/home/mapr/jmxterm-1.0.2-uber.jar) to method sun.tools.jconsole.LocalVirtualMachine.getAllVirtualMachines() WARNING: Please consider reporting this to the maintainers of org.cyclopsgroup.jmxterm.utils.WeakCastUtils$2 WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release 4113 ( ) - com.mapr.admin.AdminApplication 18946 (m) - org.apache.hadoop.yarn.server.nodemanager.NodeManager 3045 (m) - org.apache.hadoop.yarn.server.resourcemanager.ResourceManager 23286 ( ) - org.apache.zookeeper.server.quorum.QuorumPeerMain /opt/mapr/zookeeper/zookeeper-3.5.6/conf/zoo.cfg 29545 (m) - com.mapr.fs.cldb.CLDB /opt/mapr/conf/cldb.conf 4090 ( ) - jmxterm-1.0.2-uber.jar 26491 ( ) - com.mapr.warden.WardenMain /opt/mapr/conf/warden.conf 13003 (m) - org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer 2286 (m) - kafka.tools.ConsoleProducer --broker-list localhost:9092 --topic /s:t --producer-property metrics.enabled=true
NOTEIn this example, the process is 2286. On your system, the process number may differ. Use the process number provided by your system. - Run the
open
command with the process number to open the connection:open 2286
- Run the following command to set the domain to
kafka.producer:
domain kafka.producer
- Run the
beans
command to see the beans available in the kafka.producer domain:beans #domain = kafka.producer: kafka.producer:client-id=console-producer,topic="/s:t",type=producer-topic-metrics kafka.producer:client-id=console-producer,type=kafka-metrics-count kafka.producer:client-id=console-producer,type=producer-metricseans
- Set the bean to
kafka.producer:client-id=console-producer,type=producer-metrics:
bean kafka.producer:client-id=console-producer,type=producer-metrics
- Run the following command to get all the metrics for the
kafka.producer:client-id=console-producer,type=producer-metrics
bean:
get * #mbean = kafka.producer:client-id=console-producer,type=producer-metrics: record-send-rate = 0.0; record-retry-total = 0.0; record-size-avg = NaN; batch-split-total = 0.0; record-queue-time-avg = NaN; request-latency-avg = NaN; record-error-total = 0.0; batch-split-rate = 0.0; record-error-rate = 0.0; record-send-total = 3.0; batch-size-max = NaN; compression-rate-avg = NaN; record-queue-time-max = NaN; record-retry-rate = 0.0; request-latency-max = NaN; record-size-max = NaN; batch-size-avg = NaN; records-per-request-avg = NaN;