Re: are kafka consumer apps guaranteed to see msgs at least once?

2013-11-23 Thread Imran Rashid
yes, you are right. Like I said, I'm counting on some magic from commitOffsets to not commit backwards. I have some vague that this might be possible w/ zookeeper, though I don't know the details. I will be submitting a PR for that shortly, but without those protections, hoping for some help on

Re: are kafka consumer apps guaranteed to see msgs at least once?

2013-11-23 Thread Jason Rosenberg
Interesting, but I think you might run into problems when partitions get rebalanced between processes (not just between threads in the same process). So, you'd need to coordinate your issue of not committing backward, between threads running on different processes. I'm trying to think now, what m