Thank you. You are right about the limitations of Kafka's own assignments. Thank you for letting me know about Cruise Control.
I think given that Kafka's own assignment is useful enough to be exposed via the local reassign command, it should also be exposed on the Admin client. It is likely sufficient for our use case. Will look into creating a KIP for this. Den man. 5. okt. 2020 kl. 11.34 skrev Tom Bentley <tbent...@redhat.com>: > Hi Stig, > > AdminUtils is not a public API, so you're right to not want to depend on > it. Unfortunately the Admin client API doesn't offer functionality > equivalent to `kafka-reassign-partitions.sh --generate`. That is, while you > can get the broker to generate the assignments for you when you create a > topic or add partitions, you can't say "reassign these existing topics as > if they were new". > > One of the problems with the naive assignments generated by > AdminUtils.assignReplicasToBrokers is that the assignments it returns can > result in an unbalanced cluster, where some brokers have much higher load > than others. Many people running Kafka use external tools, such as Cruise > Control, to manage replica assignments so that the cluster is balanced. > > You could try to open a KIP to provide the functionality. I can see it > might be useful for clusters where it doesn't matter if the brokers have > similar load. The risk of adding this functionality is that people end up > using it inappropriately and ending up with very unbalanced clusters in > production. > > Kind regards, > > Tom > > On Mon, Oct 5, 2020 at 10:05 AM Stig Rohde Døssing <stigdoess...@gmail.com > > > wrote: > > > Hi, > > > > Kafka can recommend an assignment via the ReassignPartitionsCommand (with > > --generate), and it is also possible to get this assignment directly by > > calling AdminUtils.assignReplicasToBrokers. > > > > We would like our application to be able to get the suggested partition > > assignment and apply it to a topic. We're currently doing this by calling > > AdminUtils directly and passing the result to > > Admin.alterPartitionReassignments. > > > > Is there a way to do this solely using the kafka-clients Admin client? > We'd > > like to avoid depending directly on the Kafka server jar. > > >
