This makes sense if you think about it. If you want to start "now" you don't want the last message in the log, which could be seconds, minutes, days, or weeks old, you actually want the next message that comes in.
-Jay On Thu, Jul 17, 2014 at 10:00 AM, Guozhang Wang <wangg...@gmail.com> wrote: > Hello Tanguy, > > With "latest" the returned offset is actually the "logEndOffset", which is > the offset of the next message that will be appended to the log. > > Guozhang > > > On Thu, Jul 17, 2014 at 5:26 AM, Tanguy tlrx <tlrx....@gmail.com> wrote: > >> Hi all, >> >> I have a strange behavior with Kafka 0.8.1.1 where consumers do not >> retrieve the latest messages. >> >> I created a topic called "my-topic" with 8 partitions. My application >> creates a SimpleConsumer instance for each topic-partition pair and >> executes an OffsetRequest to retrieve the earliest and latest offsets for >> each partition. >> >> As an example, the partition 5 of the topic "my-topic" has an earliest >> offset at 4 and a latest offset at 9. >> >> Then, my application uses the same SimpleConsumer instance and executes a >> FetchRequest to read messages (starting from earliest) for that partition. >> >> Still with the example with partition 5, my application can read messages >> with offset 4,5,6,7,8 and then no more messages can be read. The same >> behavior exists for other partitions. >> >> I'm not sure that the message with offset "9" really exists, but the Kafka >> API returns a latest offset of 9 that let me thinking that this message >> exists. I tested with a HighLevelConsumer and it works the same way, >> reading messages 4...8 but not the last one. >> >> I did not notice this behavior with the previous version of Kafka I used >> (0.8.0-beta1). >> >> Am i missing something? Anyone with the same problem? >> >> Thanks >> >> -- Tanguy >> twitter @tlrx >> https://github.com/tlrx >> > > > > -- > -- Guozhang
