Hi Team,

I am currently operating a Kafka cluster in KRaft mode and would like to
raise a few questions and suggestions regarding the cluster scale-down
process.

Recently, I scaled down the cluster by removing one broker; however, I
encountered a situation where the removed broker still appeared in the
cluster, causing new partitions to be assigned to it. To address this, I
was able to remove the broker from the controller metadata using the
kafka-cluster.sh command. However, I could not find any documentation on
this method, so I had to refer to the Kafka source code to discover this
solution. The issue is now resolved, but I would like to ask if there is a
more streamlined approach for scaling down.

Additional Questions and Suggestions:

   1.

   *Automatic Cluster Exclusion*
   I initially expected that a removed broker would be automatically
   excluded from the cluster after a certain period. However, the broker
   remained part of the cluster for quite some time. I am curious whether
   Kafka has a feature that automatically detects and excludes removed brokers
   from the cluster.
   2.

   *Atomic Scale Down*
   If the above feature is not available, I would like to inquire if there
   is a way to atomically exclude a broker and simultaneously remove it from
   the cluster metadata. Since we manage a Kafka cluster with continuous topic
   creation, a non-atomic process might lead to new topics being assigned to
   the broker between its exclusion and metadata removal, necessitating manual
   partition reassignments.
   3.

   *Documentation Enhancement (Suggestion)*
   While addressing the scale-down issue, I could not find any relevant
   information in the Kafka User Guide. I had to refer to the source code,
   specifically the unregister functionality, to learn that this could be
   done using kafka-cluster.sh. Including documentation on this process
   would likely make it easier for other users to manage their clusters
   effectively.

Thank you, and I look forward to any guidance or recommendations.

Best regards,

hayoung

Reply via email to