Hey all! I was wondering how you all might recommend expanding your log.dirs settings safely.
Specifically, when you add new entries to your log.dirs setting on your brokers, and execute a rolling restart to have the brokers take the new configuration, how do you spread out your partitions into the new directory that was added? I've noticed that this will not happen automatically for existing partitions(on version 1.1.1). A relatively straightforward, albeit dangerous solution is to simply remove all contents from the existing log.dirs defined in the brokers configuration(including the new one). I'd really rather not do that going forward if I could avoid it, as it does put additional strain on the cluster for what I feel like should be possible to do mostly within a single broker. I am aware that there is a more manual method available by modifying the files in the partition directories, outlined for the most part here: https://youtu.be/qDjEveb8vzk?t=153 If possible, I'm not looking to move these partitions out of the broker itself, more just have it balance the number of partitions more evenly across the available log.dirs. Thanks in advance for any insight! Best, - Will
