Hi folks.

I used kafka 8 to create a proof-of-concept, using the SimpleConsumer. In this code,
I locally kept track of the offset of the last message read from a topic.

I'm now refactoring the code for production, and I've also started using version 9 of kafka. There is no more SimpleConsumer, so I've started using the KafkaConsumer class. Using this class, I should no longer need to keep track of the offset myself, as the offset of last message
read is automatically stored in zookeeper for me.

My issue is that my consumer is not always running. When it is restarted, I want it to receive
message n+1 where n is the last one it read. So, in this scenario:

1. there are 5 messages in topic
2. consumer is started, reads 5 messages.
3. consumer is stopped
4. 3 more messages are placed in topic
5. consumer is restarted
6. 2 more messages are placed in topic


I want my consumer to receive message 6. Instead, the first message it receives is message 9. It only receives messages that are written to the topic from the time it starts.

I'm sure I'm doing something incorrect in my (Java) consumer. Can somebody point me at doc that indicates how the server knows the new instance of the consumer is the same one
that was previously receiving messages?

Thanks!

--
Jeff Ramin
Software Engineer
Singlewire Software
2601 W Beltline Hwy #510
Madison, WI 53713

Phone Direct - 608.661.1172
www.singlewire.com

Reply via email to