Consumer Subscriptions
Consumers subscribe to topics. When a consumer subscribes to a topic or partition, it means that the consumer wants to receive messages from that topic or partition. A subscription is the list of the topics, specific partitions, or both to which a consumer is subscribed.
For example, an analytics application might subscribe to the topics
rfids_productA
, rfids_productB
, and more to track movement
of products from factories to distribution centers. A reporting tool might subscribe to the
topics meters_NW
, meters_SW
, and more to get a report of
electricity usage in different geographic regions that a power company services.
- Topics
- When a consumer subscribes to a topic, it reads messages from all of the partitions
that are in the topic. The exception is when a consumer is part of a consumer group.
Consumer groups and this exception are explained in Consumer Groups. Consumers can subscribe to topics in two ways:
- By name
- Consumers specify the names of the topics to which they subscribe.
- By regular expression
- Consumers specify a regular expression and subscribe to all topics with names
that match the regular expression.
The ability to use regular expressions is helpful when the
-autocreate
parameter for a stream is set totrue
and producers are allowed to create topics automatically at runtime.To unsubscribe from topics to which you are subscribed with regular expressions, you must use the same regular expressions.
For example, suppose that you use this regular expression to subscribe totopic0
andtopic1
:
Next, you addtopic[0-1]
topic2
,topic3
, andtopic4
to the subscription, as follows:
Trying subsequently to unsubscribe from, say,topic[0-4]
topic0
has no effect. The consumer remains subscribed to it becausetopic0
was subscribed to as part of a regular expression.Trying to unsubscribe from
topic[0-1]
also has no effect because the regular expressiontopic[0-4]
was used aftertopic[0-1]
, and the latter is a superset of the former.To unsubscribe fromtopic0
, you have to follow these steps:- Unsubscribe from
topic[0-4]
. This step unsubscribes you fromtopic2
,topic3
, andtopic4
. You must follow this step because a) this regular expression was used last, and b) because it is a superset oftopic[0-1]
. The order in which regular expressions are used in subscriptions matters. If you were to unsubscribe fromtopic[0-1]
first, you would still be subscribed totopic[0-4]
. - Unsubscribe from
topic[0-1]
. This step unsubscribes you fromtopic0
andtopic1
.
- Unsubscribe from
- Partitions
- Consumers can subscribe to individual partitions within topics. This is helpful when
you want a consumer to read the messages published to a specific partition. For example,
a producer might publish messages for high-priority data to a specific partition for
processing by a dedicated consumer.
When a consumer subscribes to individual partitions within a topic, the consumer does not receive messages from any of the other partitions in the topic.
Subscriptions to individual partitions can cause problems in consumer groups, as explained in the section Consumer Groups.