Hi hayoung, This is the problem that KIP-1066 <https://cwiki.apache.org/confluence/display/KAFKA/KIP-1066%3A+Mechanism+to+cordon+brokers+and+log+directories> is trying to address. It is under discussion, and should be included in the upcoming releases.
Thanks. Luke On Tue, Oct 29, 2024 at 12:45 AM hayoung lee <l4314...@gmail.com> wrote: > Hi Team, > > I am currently operating a Kafka cluster in KRaft mode and would like to > raise a few questions and suggestions regarding the cluster scale-down > process. > > Recently, I scaled down the cluster by removing one broker; however, I > encountered a situation where the removed broker still appeared in the > cluster, causing new partitions to be assigned to it. To address this, I > was able to remove the broker from the controller metadata using the > kafka-cluster.sh command. However, I could not find any documentation on > this method, so I had to refer to the Kafka source code to discover this > solution. The issue is now resolved, but I would like to ask if there is a > more streamlined approach for scaling down. > > Additional Questions and Suggestions: > > 1. > > *Automatic Cluster Exclusion* > I initially expected that a removed broker would be automatically > excluded from the cluster after a certain period. However, the broker > remained part of the cluster for quite some time. I am curious whether > Kafka has a feature that automatically detects and excludes removed > brokers > from the cluster. > 2. > > *Atomic Scale Down* > If the above feature is not available, I would like to inquire if there > is a way to atomically exclude a broker and simultaneously remove it > from > the cluster metadata. Since we manage a Kafka cluster with continuous > topic > creation, a non-atomic process might lead to new topics being assigned > to > the broker between its exclusion and metadata removal, necessitating > manual > partition reassignments. > 3. > > *Documentation Enhancement (Suggestion)* > While addressing the scale-down issue, I could not find any relevant > information in the Kafka User Guide. I had to refer to the source code, > specifically the unregister functionality, to learn that this could be > done using kafka-cluster.sh. Including documentation on this process > would likely make it easier for other users to manage their clusters > effectively. > > Thank you, and I look forward to any guidance or recommendations. > > Best regards, > > hayoung >
