Hi all, I have a strange behavior with Kafka 0.8.1.1 where consumers do not retrieve the latest messages.
I created a topic called "my-topic" with 8 partitions. My application creates a SimpleConsumer instance for each topic-partition pair and executes an OffsetRequest to retrieve the earliest and latest offsets for each partition. As an example, the partition 5 of the topic "my-topic" has an earliest offset at 4 and a latest offset at 9. Then, my application uses the same SimpleConsumer instance and executes a FetchRequest to read messages (starting from earliest) for that partition. Still with the example with partition 5, my application can read messages with offset 4,5,6,7,8 and then no more messages can be read. The same behavior exists for other partitions. I'm not sure that the message with offset "9" really exists, but the Kafka API returns a latest offset of 9 that let me thinking that this message exists. I tested with a HighLevelConsumer and it works the same way, reading messages 4...8 but not the last one. I did not notice this behavior with the previous version of Kafka I used (0.8.0-beta1). Am i missing something? Anyone with the same problem? Thanks -- Tanguy twitter @tlrx https://github.com/tlrx
