Kafka 0.8.2.1 OffsetCommitRequests and auto.commit=true

2015-10-20 Thread William Grim
Hello, Kafka users! I've been trying to build Apache Kafka into our system, and for one part of our system, we have a use-case where we have to leave auto-commit enabled but would like to reset the offset numbers to an earlier offset if a failure happens in our code. We are using auto-commit beca

Kafka 0.8.2.1 Offsets Restarting when auto.commit=false

2015-10-20 Thread William Grim
Hi, fellow Kafka users, I have another question to ask. In Kafka 0.8.2.1, when we disable auto.commit and listen to data from a single partition, occasionally, we just get our offsets reset back to where we started. Therefore, we can never complete the reading of the data (about a million messag

Re: New Kafka consumer stability

2015-10-20 Thread William Grim
What version of the API are you planning to use? We're finding it extremely unstable. On Tue, Oct 20, 2015 at 2:16 PM, tao xiao wrote: > Hi, > > I am starting a new project that requires heavy use on Kafka consumer. I > did a quick look at the new Kafka consumer and found it provides some of >