Hi folks.
I used kafka 8 to create a proof-of-concept, using the SimpleConsumer.
In this code,
I locally kept track of the offset of the last message read from a topic.
I'm now refactoring the code for production, and I've also started using
version 9 of kafka.
There is no more SimpleConsumer, so I've started using the KafkaConsumer
class. Using this
class, I should no longer need to keep track of the offset myself, as
the offset of last message
read is automatically stored in zookeeper for me.
My issue is that my consumer is not always running. When it is
restarted, I want it to receive
message n+1 where n is the last one it read. So, in this scenario:
1. there are 5 messages in topic
2. consumer is started, reads 5 messages.
3. consumer is stopped
4. 3 more messages are placed in topic
5. consumer is restarted
6. 2 more messages are placed in topic
I want my consumer to receive message 6. Instead, the first message it
receives is message 9.
It only receives messages that are written to the topic from the time it
starts.
I'm sure I'm doing something incorrect in my (Java) consumer. Can
somebody point me at
doc that indicates how the server knows the new instance of the consumer
is the same one
that was previously receiving messages?
Thanks!
--
Jeff Ramin
Software Engineer
Singlewire Software
2601 W Beltline Hwy #510
Madison, WI 53713
Phone Direct - 608.661.1172
www.singlewire.com
