Re: KRaft Migration Prod Ready?

2024-07-05 Thread Luke Chen
Hi Upesh, Yes, it's supported. The doc will be updated soon (after v3.8.0 released). Thanks. Luke On Wed, Jul 3, 2024 at 6:02 AM Upesh Desai wrote: > Hello, > > > > We have been a bit confused as to if KRaft migration from ZooKeeper is > production ready? It seems that the Kafka docs still ind

Re: MM2, how to configure offset-syncs topic name

2024-07-05 Thread Franck L.
Hello, If it's possible for your conf : could you try something like: *Setting 2 partitions on your topic A *run both MM2 at the same time with the same group of consumer but different clientID. In this way, Kafka Broker shoulds affect one partition by MM2 clientID, and so, messages will be

Re: Kafka 20k topics metadata update taking long time

2024-07-05 Thread Amit Chopra
But 20K topics is also not a realistic assumption to have. This is our existing scale with AWS kinesis 20k shards. And we are moving over to kafka and thus testing with equivalent scale. We are looking at an ingestion rate of 2.5 GBps. I don't see an alarming difference in the latency results from

Re: Kafka 20k topics metadata update taking long time

2024-07-05 Thread Amit Chopra
Just to make sure I am following correctly, we do not see issues with 1 topic 20k partitions. With 25 brokers, the load is fairly distributed and we can achieve aggregate throughput of ~2.5 GBps. With 20k topics with 1 partition each, we see much reduced throughput. Thanks, Amit On Fri, Jul 5,