Re: Kafka Consumer Committing Offset Even After Re-Assignment

2017-12-08 Thread Saïd Bouras
Hi Praveen, I don't know if you that's your case but if you know that consumers will lost ownership of partitions, you have to use the *ConsumerRebalanceListener* to drop the last offset of record processed in a clean way. If you don't do that, the rebalance will start when the GroupCoordinator s

Kafka Consumer Committing Offset Even After Re-Assignment

2017-12-07 Thread Praveen
I have 4 consumers on 2 boxes (running two consumers each) and 16 partitions. Each consumer takes 4 partitions. In Kafka 0.9.0.1, I'm noticing that even when a consumer is no longer assigned the partition, it is able to commit offset to it. *Box 1 Started* t1 - Box 1, Consumer 1 - Owns 8 partitio