Hi Jay, Hi Guozhang, Thanks for your response. I was suspecting something like that but did not found the documentation entry that explain that.
Thanks again! 2014-07-17 23:41 GMT+02:00 Jay Kreps <jay.kr...@gmail.com>: > This makes sense if you think about it. If you want to start "now" you > don't want the last message in the log, which could be seconds, > minutes, days, or weeks old, you actually want the next message that > comes in. > > -Jay > > On Thu, Jul 17, 2014 at 10:00 AM, Guozhang Wang <wangg...@gmail.com> > wrote: > > Hello Tanguy, > > > > With "latest" the returned offset is actually the "logEndOffset", which > is > > the offset of the next message that will be appended to the log. > > > > Guozhang > > > > > > On Thu, Jul 17, 2014 at 5:26 AM, Tanguy tlrx <tlrx....@gmail.com> wrote: > > > >> Hi all, > >> > >> I have a strange behavior with Kafka 0.8.1.1 where consumers do not > >> retrieve the latest messages. > >> > >> I created a topic called "my-topic" with 8 partitions. My application > >> creates a SimpleConsumer instance for each topic-partition pair and > >> executes an OffsetRequest to retrieve the earliest and latest offsets > for > >> each partition. > >> > >> As an example, the partition 5 of the topic "my-topic" has an earliest > >> offset at 4 and a latest offset at 9. > >> > >> Then, my application uses the same SimpleConsumer instance and executes > a > >> FetchRequest to read messages (starting from earliest) for that > partition. > >> > >> Still with the example with partition 5, my application can read > messages > >> with offset 4,5,6,7,8 and then no more messages can be read. The same > >> behavior exists for other partitions. > >> > >> I'm not sure that the message with offset "9" really exists, but the > Kafka > >> API returns a latest offset of 9 that let me thinking that this message > >> exists. I tested with a HighLevelConsumer and it works the same way, > >> reading messages 4...8 but not the last one. > >> > >> I did not notice this behavior with the previous version of Kafka I used > >> (0.8.0-beta1). > >> > >> Am i missing something? Anyone with the same problem? > >> > >> Thanks > >> > >> -- Tanguy > >> twitter @tlrx > >> https://github.com/tlrx > >> > > > > > > > > -- > > -- Guozhang > -- -- Tanguy twitter @tlrx https://github.com/tlrx
