Good Day, I'm looking for someone to poke holes in my theory.
I want to balance my disk usage across brokers. I want to maintain order per partition. Yes there are tools but they require manual intervention. What if created a custom partition strategy. The strategy is to take the existing partitioning strategy but add the ability to rotate the writing of partitions by 1. If then then ect... A -> A A->B A->C B -> B B->C B->A C -> C C->A C->B The idea is to simply rotate the partition after some measure is reached.(Time sounds like the most likely way to do it, but to do it the 'correct way' to avoid race conditions would have to be part of the strategy.) This should help ensure that the ordering of the partitions is maintained, but probably requires extra logic on the consumer side to undo the partitioning strategy and take advantage of the ordering. This should help with a more balanced disk usage on a per topic level. Has anyone tried this? Is there a pitfall I should consider?
