Balancing traffic between multiple directories

2022-10-25 Thread Lehar Jain
Hey, We run Kafka brokers with multiple log directories. I wanted to know how Kafka balances traffic between various directories. Can we have our own strategy to distribute different partitions to different directories. As currently, we are facing an imbalance in sizes of the aforementioned direct

Re: Balancing traffic between multiple directories

2022-10-25 Thread Andrew Grant
Hey Lehar, I don’t think there’s a way to control this during topic creation. I just took a look through https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/admin/AdminUtils.scala and it does appear partition assignment does not account for each broker’s different log directories

Re: Balancing traffic between multiple directories

2022-10-25 Thread Lehar Jain
Hey Andrew, Thanks for the reply. Currently, we are using the same method as you described. Wanted to make sure if there is a better way. It seems there isn't currently. So we will keep using this only. On Tue, Oct 25, 2022 at 7:23 PM Andrew Grant wrote: > Hey Lehar, > > > I don’t think there’