mapr perfproducer

This utility starts a producer and generates data for the producer to publish in messages to a HPE Ezmeral Data Fabric Stream. When starting the utility, you can specify how many topics the producer publishes to, how many partitions to create for each topic, and how many messages to publish to each partition. You can also specify the method for distributing messages among the partitions in each topic.

mapr perfproducer -path /myVolume/myDirectory/stream_a -ntopics 40 -npart -5 
-nmsgs 100000 -rr true

The mapr perfproducer utility uses the default values for all of the configuration parameters that apply to producers. For a list of these parameters, see HPE Ezmeral Data Fabric Streams Configuration Parameters.

Each producer runs as a single thread. You can run multiple instances of the utility at the same time. However, because producers can be CPU-intensive, it is recommended to run at most 4 or 5 on a single cluster node.

When multiple instances of the mapr perfproducer utility publish to a single stream, the separate instances share topics. For example, if -ntopics is set to 40 for each instance that publishes to a single stream, together those instances create no more than 40 topics in the stream and they share those topics.

It is recommended that all producers that publish to a single cluster publish to the same number of topics and partitions within those topics. Therefore, use the same values for -ntopics and -npart for each instance of the mapr perfproducer utility that shares a stream with other instances.

Monitor the performance of the running instances of the mapr perfproducer utility by issuing the maprcli command stream topic info at intervals, as described in Monitoring Producers. The command stream topic info shows statistics for single topics. Because all of the topics that mapr perfproducer creates have the same number of partitions, and because mapr perfproducer writes the same number of messages to each partition, you can assume that the statistics that the command stream topic info displays for any one topic are close to the statistics for any other topic. The naming convention that mapr perfproducer uses when creating topics is simply topicn, which produces the names topic0, topic1, and so on. You can run the command stream topic info with any one of these names as the value of the -topic parameter.

To simulate consumers to estimate the performance of HPE Ezmeral Data Fabric Streams in your network configuration, run one or more instances of the mapr perfconsumer utility against the stream.

Prerequisites for running this utility

• Create a HPE Ezmeral Data Fabric Stream in a Data Fabric cluster for the mapr perfproducer utility to publish messages to. See stream create.

  If you plan to replicate the stream for the purposes of the performance estimate, create the replica stream. Then, start replication from the first stream to the replica stream. For instructions on setting up replication between streams, see Managing Stream Replication.

• Ensure that the user ID that runs the mapr perfproducer utility has the readAce and writeAce permissions on the volume where the stream is located.
• Ensure that the user ID that runs the mapr perfproducer utility has the produceperm and topicperm permissions on the stream.

For information about how to set permissions on volumes, see Setting Whole Volume ACEs.

For information about how to set permissions on streams, see Enabling Table and Stream Authorizations with ACEs.

Syntax

mapr perfproducer
-path <stream-full-name>
[ -ntopics <num topics> (default: 2) ]
[ -npart <numpartitions per topic> (default: 4) ]
[ -nmsgs <num messages per topicfeed> (default: 100000) ]
[ -msgsz <msg value size> (default: 200) ]
[ -rr <round robin true/false> (default: false) ]
[ -hashkey <true/false> (default: false) ]

Parameters