Hey everyone, I'd like to bring to your attention a general design document that was just published in Apache Kafka's wiki space:
https://cwiki.apache.org/confluence/display/KAFKA/Incremental+Cooperative+Rebalancing%3A+Support+and+Policies It deals with the subject of Rebalancing of groups in Kafka and proposes basic infrastructure to support improvements on the current rebalancing protocol as well as a set of policies that can be implemented to optimize rebalancing under a number of real-world scenarios. Currently, this wiki page is meant to serve as a reference to the proposition of Incremental Cooperative Rebalancing overall. Specific KIPs will follow in order to describe in more detail - using the standard KIP format - the basic infrastructure and the first policies that will be proposed for implementation in components such as Connect, the Kafka Consumer and Streams. Stay tuned! Konstantine
