Hi, There was a Jira to add "remove broker" option to the partition-reassignment tool. I think it died in a long discussion trying to solve a harder problem...
To your work-around - it is an acceptable work-around. Few improvements: 1. Manually edit the resulting assignment json to avoid unnecessary moves. Or even create your own assignment (either manually or using a small script). 2. We don't throttle the partition move automatically, so it can easily take over the network if you are not careful. Therefore running the reassignment tools multiple times to move partitions one-by-one is often safer. 3. If you don't mean to permanently reduce the number of brokers but rather to replace a broker, don't reassign. Just take down the existing broker and give the new one the same ID. Hope this helps, Gwen On Wed, Jan 6, 2016 at 10:14 AM, Tom Crayford <tcrayf...@heroku.com> wrote: > Hi there, > > Kafka's `kafka-reassign-partitions.sh` tool currently has no mechanism for > removing brokers. However, it does have the ability to generate partition > plans across arbitrary sets of brokers, by using `--generate`, passing all > the topics in the cluster into it, then passing the generated plan to > --execute. > > This isn't ideal, because it (from my understanding), potentially moves all > the partitions in the entire cluster around, but it should work fine, and > stop Kafka from having the partitions assigned to a broker that no longer > exists. > > Am I missing something there? Or is this a reasonable workaround until > better partition reassignment tools turn up in the future? > > Thanks > > Tom >
