I am seeing a very wired issue with my consumer group when running consumer 0.9 with broker 0.9.0.1. On adding or removing consumers, it seems that for certain partitions the commit offset in kafka alway gets rewinded to a fixed point.
When I run the below command over time to look at my consumer group: bin/kafka-consumer-groups.sh --describe --group consumer-gp-1 --new-consumer --bootstrap-server localhost:9092 I notice that for certain partitions the consumer offsets gets always reset back to a fixed point. See an example below(Each line represents the result of running the above command at different times (~1 min apart) for a particular partition): consumer-gp-1, topic-1, 54, 6682915, 8264593, 1581678, consumer-5_/ 172.18.5.49 consumer-gp-1, topic-1, 54, 6852270, 8265027, 1412757, consumer-5_/ 172.18.5.49 consumer-gp-1, topic-1, 54, 6682915, 8265252, 1582337, consumer-5_/ 172.18.5.49 consumer-gp-1, topic-1, 54, 6865345, 8265457, 1400112, consumer-5_/ 172.18.5.49 consumer-gp-1, 54, 6682915, 8266152, 1583237, consumer-5_/172.18.5.49 In the above case, for partition 54 - the consumer offset will move ahead but then always come back again to 6682915 Any inputs on how could I go about debugging this?
