Re: Consumer seek on 0.9.0 API

2016-02-22 Thread Péricé Robin
OK I understand the explanation. Thanks you for sharing your knowledge ! Regards, Robin 2016-02-18 18:56 GMT+01:00 Jason Gustafson : > Woops. Looks like Alex got there first. Glad you were able to figure it > out. > > -Jason > > On Thu, Feb 18, 2016 at 9:55 AM, Jason Gustafson > wrote: > > > H

Re: Consumer seek on 0.9.0 API

2016-02-18 Thread Jason Gustafson
Woops. Looks like Alex got there first. Glad you were able to figure it out. -Jason On Thu, Feb 18, 2016 at 9:55 AM, Jason Gustafson wrote: > Hi Robin, > > It would be helpful if you posted the full code you were trying to use. > How to seek largely depends on whether you are using new consumer

Re: Consumer seek on 0.9.0 API

2016-02-18 Thread Jason Gustafson
Hi Robin, It would be helpful if you posted the full code you were trying to use. How to seek largely depends on whether you are using new consumer in "simple" or "group" mode. In simple mode, when you know the partitions you want to consume, you should just be able to do something like the follow

Re: Consumer seek on 0.9.0 API

2016-02-18 Thread Alex Loddengaard
Hi Robin, Glad it's working. I'll explain: When a consumer subscribes to one or many topics using subscribe(), the consumer group coordinator is responsible for assigning partitions to each consumer in the consumer group, to ensure all messages in the topic are being consumed. The coordinator han

Re: Consumer seek on 0.9.0 API

2016-02-18 Thread Péricé Robin
Hi, Ok I did a poll() before my seek() and poll() again and now my consumer starts at offset. Thanks you a lot ! But I don't really understand why I have to do that, if anyone can explain me. Regards, Robin 2016-02-17 20:39 GMT+01:00 Alex Loddengaard : > Hi Robin, > > I believe seek() needs t

Re: Consumer seek on 0.9.0 API

2016-02-17 Thread Alex Loddengaard
Hi Robin, I believe seek() needs to be called after the consumer gets its partition assignments. Try calling poll() before you call seek(), then poll() again and process the records from the latter poll(). There may be a better way to do this -- let's see if anyone else has a suggestion. Alex O

Consumer seek on 0.9.0 API

2016-02-17 Thread Péricé Robin
Hi, I'm trying to use the new Consumer API with this example : https://github.com/apache/kafka/tree/trunk/examples/src/main/java/kafka/examples With a Producer I sent 1000 messages to my Kafka broker. I need to know if it's possible, for example, to read message from offset 500 to 1000. What I d