mapr perfconsumer
This utility runs a consumer reading messages from topics in a HPE Ezmeral Data Fabric Stream. Use this utility to run consumers when you want to estimate the performance of consumers for your HPE Ezmeral Data Fabric Streams applications, given your network configuration.
This utility works in conjuction with the mapr perfproducer
utility. When starting this
utility, you can specify how many topics to read from, how many partitions to read from in
each topic, and how many messages to read.
The mapr perfconsumer
utility uses the default values for all of
the configuration parameters that apply to consumers. For a list of these parameters, see
HPE Ezmeral Data Fabric Streams Configuration Parameters.
The utility uses the default values for all of the configuration parameters that apply to consumers.
Each consumer runs as a single thread. You can run multiple instances of the utility at the same time. However, because consumers can be CPU-intensive, it is recommended to run at most 4 or 5 on a single cluster node.
When you run multiple instances of this utility, you can use the -group
parameter to create consumer groups.
Monitor the performance of the running instances of the mapr perfconsumer
utility by following the instructions that are given
in Monitoring Consumers.
Prerequisites for running this utility
- Ensure that there is a HPE Ezmeral Data Fabric Stream
that one or more instances of
mapr perfproducer
have already published messages to or are actively publishing messages to. - Ensure that the user ID that runs the
mapr perfconsumer
utility has theconsumeperm
permission on the stream. - Ensure that the user ID that runs the
mapr perfconsumer
utility has thereadAce
andwriteAce
permissions on the volume where the stream is located.
For information about how to set permissions on volumes, see Setting Whole Volume ACEs.
For information about how to set permissions on streams, see Enabling Table and Stream Authorizations with ACEs.
mapr
user is not treated as a
superuser. HPE Ezmeral Data Fabric Streams does not allow the
mapr
user to run this utility unless that user is given the relevant
permission or permissions with access-control expressions.Syntax
mapr perfconsumer
-path <stream-full-name>
[ -ntopics <num topics> (default: 2) ]
[ -npart <numpartitions per topic> (default: 4)
[ -nmsgs <num messages per topicfeed> (default: 100000) ]
[ -group <consumer group id> (default: null)
[ -topicsubscription <true/false> (default: false) ]
Parameters
Parameter | Description |
---|---|
path |
The path to the stream. |
ntopics |
The number of topics for the consumer to subscribe to. The default is 2. If the number that you specify is greater than the number of topics that are in the stream, the utility hangs. |
npart |
The number of partitions to read from in each topic that is subscribed. The default is 4. If the number that you specify is greater than the number of partitions that are in each topic, the utility hangs. If you specify a group
ID with the If you do not specify a group ID, then the consumer's committed cursors are not saved. If you use the -group parameter to
specify a group ID and you set the -topicsubscription parameter
to true :
|
nmsgs |
The number of messages to read from each partition. The default is 100,000. |
group |
The identifier of a consumer group. When two or more consumers belong to a
consumer group, they must read from the same number of topics. HPE Ezmeral Data Fabric Streams
distributes the partitions for those topics among the consumers in the group. The default is null. |
topicsubscription |
A vaue of true subscribes the consumer to topics. A value of
false subscribes the consumer to topic partitions.The default
is |