partitions are never automatically moved. They are assigned to broker(s) and stay that way unless reassignments are triggered by external tools. (leadership can move automatically though, if RF>1).
There's more info on partitions and moving partitions at these two links: https://kafka.apache.org/documentation/#operations https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools 1) you need to do partition reassignments for the new broker to be involved in existing topics. 2) no. it does leadership changes though if RF>1 3) if the data was deleted, when it comes back up with the same broker ID if RF > 1 the partitions will automatically be synced back on to the broker. I'm not sure with RF=1 but I guess it would be created as empty. 4) Gwen Shapira does a great talk on "when it absolutely, positively has to be there" which covers failure scenarios: https://www.confluent.io/resources/kafka-summit-2016/absolutely-positively-reliability-guarantees-kafka/ There's a 5 part series from Gwen here too: https://www.confluent.io/online-talk/best-practices-for-apache-kafka-in-production-confluent-online-talk-series And I believe part 2 "Reliability Guarantees in Ahache Kafka" is largely a similar talk, and a bit more up to date. So maybe just go straight there. And probably start with part 1. :) On Mon, Apr 23, 2018 at 3:47 AM, Raghu Arur <raghua...@gmail.com> wrote: > I have a few questions on behavior of kafka w.r.t to broker life-cycle > changes. > > 1. When a new broker is added to the cluster, do we need to manually invoke > rebalancing or based on the load, kafka automatically invokes rebalancing ? > > 2. When an active broker goes down, does Kafka moves the partitions to > other brokers ? And when the broker comes back again, does it get back the > ownership of the partitions (or partition replicas) automatically without a > manual rebalance ? > > 3. If a broker goes down and comes back up clean (for eg. disk went bad and > the partitions got cleaned up) with the same broker ID, would the > partitions be automatically moved to this broker or do we need to do a > manual rebalance to get this broker handle some partitions ? > > 4. Is there a document/blog post/link that talks about Kafka broker > behavior on failure scenarios. > > Thanks, > Raghu. >
