Hi, Per consumer group we have many consumers that aggregate data for a predefined amount of time and then write the aggregated data to a db. We're looking to set enable.auto.commit to false and write the offsets only when the aggregated data has been written to db. In that context we're not sure what happens when a rebalance occurs.
The rebalance can occur at any point in time, e.g. when a new consumer joins the party. What will happen with the existing consumer at such a moment? It is likely that they have consumed messages from the partitions that they were owning and suddenly they will own different partitions. What does the Kafka 0.9 client do if we don't provide a ConsumerRebalanceListener? Will we have to implement a ConsumerRebalanceListener to act on such an event? Kind regards, Balthasar Schopman DevOps Software Engineer LeaseWeb Technologies B.V. T: +31 20 316 0232 M: E: b.schop...@tech.leaseweb.com W: http://www.leaseweb.com Luttenbergweg 8, 1101 EC Amsterdam, Netherlands
