Re: Fetch API Offset

2015-04-22 Thread Piotr Husiatyński
I have found out what was the issue -- it was bug in the library. Thanks everyone for help. On Tue, Apr 21, 2015 at 9:02 PM, Andrei wrote: > @Piotr I can confirm that in native (Java/Scala) library returned message > offsets start with the one specified in request, not the next one. Note > sure a

Re: Fetch API Offset

2015-04-21 Thread Andrei
@Piotr I can confirm that in native (Java/Scala) library returned message offsets start with the one specified in request, not the next one. Note sure about underlying protocol, though. As long as you also know Python, I would suggest you looking at kafka-python [1] library that is pretty readable

Re: Fetch API Offset

2015-04-21 Thread Piotr Husiatyński
I'm using Go with kafka library** and sending fetch request as described in wire protocol documentation returns messages off by one. Are you sure that code that you wrote is not sending fetch request with offset being one less than you pass? ** https://github.com/optiopay/kafka On Tue, Apr 21, 20

Re: Fetch API Offset

2015-04-21 Thread Manoj Khangaonkar
Hi, I have used code like FetchRequest req = new FetchRequestBuilder() .clientId(clientname) .addFetch(topic, partition, offset_in_partition, 10).build(); FetchResponse fetchResponse = consumer.fetch(req); And it returns the message that I was looking for. The offset_in_

Re: Fetch API Offset

2015-04-21 Thread Piotr Husiatyński
What do you mean that message has expired? If I will do fetch request with offset 4, I will receive first message with offset 5. I think fetch is always excluding requested offset, while documentation is saying it will be included in response. On Tue, Apr 21, 2015 at 3:35 PM, Manoj Khangaonkar w

Re: Fetch API Offset

2015-04-21 Thread Manoj Khangaonkar
Hi, I suspect If some message from the given offset have expired, then they will not be returned. regards On Tue, Apr 21, 2015 at 5:14 AM, Piotr Husiatyński wrote: > According to documentation, sending fetch request with offset value > result in messages starting with given offset (including)

Fetch API Offset

2015-04-21 Thread Piotr Husiatyński
According to documentation, sending fetch request with offset value result in messages starting with given offset (including) and greater. https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-FetchAPI > Logically one specifies the topics, partit