Hi, I have observed the following scenario (the consumer here has 'enable.auto.commit=false' and offsets are committed using commitSync() if any messages are received):
1. Start a consumer (with a specific group.Id) and send some messages to its subscribed topic. 2. The consumer consumes the messages and the group+consumer has an entry in the __commit_offsets with the latest offsets for this group and consumer. 3. The consumer will keep polling the topic but don't send any more messages to the topic for a long time (longer than one day. The consumer keeps polling the topic in a while loop). The default duration for which a group's entry is retained in the offsets topic is 1 day. 4. Now stop the consumer. (there is no other consumer for this group) 5. Send some more messages to the topic. 6. Start the consumer (with the same group and consumer id as earlier). 7. The consumer does not pick up the new messages sent in step 5 as it has lost the committed offsets and starts with the 'latest' offsets. Is this an expected behavior? Or do I have something wrong in the configurations? Is there a way to ensure that the offsets are retained even if there are no messages flowing in? I had assumed that if the consumer keeps polling the kafka topic, its offsets will be retained even if no messages are received by the corresponding topic. Regards, Rahul This email message and any attachments are intended solely for the use of the addressee. If you are not the intended recipient, you are prohibited from reading, disclosing, reproducing, distributing, disseminating or otherwise using this transmission. If you have received this message in error, please promptly notify the sender by reply email and immediately delete this message from your system. This message and any attachments may contain information that is confidential, privileged or exempt from disclosure. Delivery of this message to any person other than the intended recipient is not intended to waive any right or privilege. Message transmission is not guaranteed to be secure or free of software viruses. ***********************************************************************************************************************
