Topic Partitions
Partitions, which exist within topics, are parallel, ordered, immutable sequences of messages that are continually appended to.
Topics can contain multiple partitions, which make topics scalable by spreading the load for a topic across multiple servers.
Downstream applications that read messages can read from multiple partitions within a topic for faster performance than would be possible if they read from a single partition per topic. Downstream applications can also scale by having separate instances read from separate partitions.
When creating or editing a stream, a default number of partitions can be specified for that stream's topics. Topics inherit the stream's partition default. However, topics can also override the stream's partition default by setting the number of partitions to be used.
Performance
The default number of partitions for data-fabric streams and topics can impact performance. Depending on the volume of messages being published to a topic, the default number of partitions might be increased for efficient consumption.
- Multiple consumers, in consumer groups, reading from multiple partitions are handled more efficiently.
- Individual consumers each reading from a single partition are handled less efficiently.
Reference
The following lists topics that have more detailed information.
- See the maprcli stream create for information
about creating streams with the
-defaultpartitions
parameter. - See the maprcli stream edit for information about
editing streams with the
-defaultpartitions
parameter. - See the maprcli stream topic create for
information about creating topics with the
-partitions
parameter. - See the maprcli stream topic edit for information
about modifying topics with the
-partitions
parameter. - See the maprcli stream topic info for information
about topic data including the
-partitions
parameter. - See the HPE Ezmeral Data Fabric Streams Java API Library for the methods used to create and edit streams and to create and edit topics.