Hello, Using kafka 0.8.2.1 with reactive-kafka wrapper around Java client we observe that some consumers in a group are getting stuck every once in a while. One characteristic of these stuck consumers is that (once restarted) they are attached to partitions with expired offsets, i.e. I see the following log message:
[xxx-leader-finder-thread] INFO kafka.consumer.ConsumerFetcherManager - [ConsumerFetcherManager-1483059556669] Added fetcher for partitions ArrayBuffer([[xxx,11], initOffset 77619349 to broker id:2,host:kafka2,port:6667] , [[xxx,10], initOffset -1 to broker id:1,host:kafka1,port:6667] ) I assume that offset -1 means that there is no offset for this consumer group and partition found. I'm certain that the group consumed from this partition before. Sometimes we get a lot of data in which case consumer may fall behind (even beyond topic retention policy), but in this case I expect it to pick up from the beginning of the partition (since auto.offset.reset is set to smallest), but instead it is getting stuck. Before getting stuck it consumes a few messages (I assume from the other partitions it's consuming). Appreciate any advice on how to fix this issue. Thanks, Timur
