Monitoring Producers
To get a sense of how quickly producers are sending messages to the producer client
library, you can run the command stream topic info
at intervals.
Doing so will show you changes over time in the rate at which the values for
maxoffset
and maxtimestamp
increase for the partitions in
the topics that your producers are publishing to.
For example, if you have a script that runs the command at intervals of 10 seconds, the
change per second would be (Value at first run - Value at second run)/10
.
This is an indirect measure of the speed of the producers because the producer client library batches messages before publishing them to partitions. The faster the producers send messages to the client, the faster the client publishes message batches, and the greater the change per second.
If producers do not seem to be sending messages quickly enough, and this problem is not caused by server-side I/O bottlenecks, you can spawn more producer threads.
See stream topic info
for the
syntax of this command.