Re: Processing older records Kafka Consumer

2016-12-06 Thread Asaf Mesika
Seek will do the trick. Just make sure that when you run it, it only runs on partitions the current reader is assigned (call assignments() and filter only the ones assigned to you now) On Tue, Dec 6, 2016 at 12:30 PM Amit K wrote: > Sorry for not providing complete information. > > I use the aut

Re: Processing older records Kafka Consumer

2016-12-06 Thread Amit K
Sorry for not providing complete information. I use the auto-commit. Most of the other properties are more or less the default one. Actually further analysis reveled that the records are consumed by consumer but some dependent component was down (unfortunately it went completely un-detected :( ).

Re: Processing older records Kafka Consumer

2016-12-06 Thread Asaf Mesika
Do you use auto-commit or committing your self? I'm trying to figure out how the offset moved if it was stuck. On Tue, Dec 6, 2016 at 10:28 AM Amit K wrote: > Hi, > > Is there any way to re-consume the older records from Kafka broker with > kafka consumer? > > I am using kafka 0.9.0.0 In one of

Processing older records Kafka Consumer

2016-12-06 Thread Amit K
Hi, Is there any way to re-consume the older records from Kafka broker with kafka consumer? I am using kafka 0.9.0.0 In one of the scenario, I saw records for 2 days from today were not consumed as consumer was stuck. When the consumer restarted, it started processing records from today but older