Hi Tanguy, Just updated the wording on this protocol wiki a bit:
https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetAPI Hope it is now clearer. Guozhang On Thu, Jul 17, 2014 at 11:48 PM, Tanguy tlrx <tlrx....@gmail.com> wrote: > Hi Jay, Hi Guozhang, > > Thanks for your response. I was suspecting something like that but did not > found the documentation entry that explain that. > > Thanks again! > > > 2014-07-17 23:41 GMT+02:00 Jay Kreps <jay.kr...@gmail.com>: > > > This makes sense if you think about it. If you want to start "now" you > > don't want the last message in the log, which could be seconds, > > minutes, days, or weeks old, you actually want the next message that > > comes in. > > > > -Jay > > > > On Thu, Jul 17, 2014 at 10:00 AM, Guozhang Wang <wangg...@gmail.com> > > wrote: > > > Hello Tanguy, > > > > > > With "latest" the returned offset is actually the "logEndOffset", which > > is > > > the offset of the next message that will be appended to the log. > > > > > > Guozhang > > > > > > > > > On Thu, Jul 17, 2014 at 5:26 AM, Tanguy tlrx <tlrx....@gmail.com> > wrote: > > > > > >> Hi all, > > >> > > >> I have a strange behavior with Kafka 0.8.1.1 where consumers do not > > >> retrieve the latest messages. > > >> > > >> I created a topic called "my-topic" with 8 partitions. My application > > >> creates a SimpleConsumer instance for each topic-partition pair and > > >> executes an OffsetRequest to retrieve the earliest and latest offsets > > for > > >> each partition. > > >> > > >> As an example, the partition 5 of the topic "my-topic" has an earliest > > >> offset at 4 and a latest offset at 9. > > >> > > >> Then, my application uses the same SimpleConsumer instance and > executes > > a > > >> FetchRequest to read messages (starting from earliest) for that > > partition. > > >> > > >> Still with the example with partition 5, my application can read > > messages > > >> with offset 4,5,6,7,8 and then no more messages can be read. The same > > >> behavior exists for other partitions. > > >> > > >> I'm not sure that the message with offset "9" really exists, but the > > Kafka > > >> API returns a latest offset of 9 that let me thinking that this > message > > >> exists. I tested with a HighLevelConsumer and it works the same way, > > >> reading messages 4...8 but not the last one. > > >> > > >> I did not notice this behavior with the previous version of Kafka I > used > > >> (0.8.0-beta1). > > >> > > >> Am i missing something? Anyone with the same problem? > > >> > > >> Thanks > > >> > > >> -- Tanguy > > >> twitter @tlrx > > >> https://github.com/tlrx > > >> > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > -- > -- Tanguy > twitter @tlrx > https://github.com/tlrx > -- -- Guozhang
