Consumer Failure and Recovery

When a consumer that is not associated with a consumer-group ID recovers from failure and comes back online, it can either start reading its partitions from the earliest offsets or from the latest offset. This choice is determined by the auto.offset.reset configuration parameter.

If the consumer reads from the earliest offset in a partition, which is the offset of the message that has been in the partition longest without being deleted because of the expiration of the time-to-live interval for the stream, it might re-read a large number of messages before reading messages that were published after it failed.

If the consumer reads from the latest offset in a partition, which is the offset of the most current message at the time the consumer requests new messages from HPE Ezmeral Data Fabric Streams, the consumer starts off up-to-date, but skips over the messages between its time of failure and the current time.