Hi, We have the similar solution that have been working in production since 2014. You can see it here: https://github.com/resetius/kafka/commit/20658593e246d2184906879defa2e763c4d413fb The idea is very simple 1. Disk balancer runs in a separate thread inside scheduler pool. 2. It does not touch empty partitions 3. Before it moves a partition it forcibly creates new segment on a destination disk 4. It moves segment by segment from new to old. 5. Log class works with segments on both disks
Your approach seems too complicated, moreover it means that you have to patch different components of the system Could you clarify what do you mean by topicPartition.log? Is it topic-paritition/segment.log ? 12.01.2017, 21:47, "Dong Lin" <lindon...@gmail.com>: > Hi all, > > We created KIP-113: Support replicas movement between log directories. > Please find the KIP wiki in the link > *https://cwiki.apache.org/confluence/display/KAFKA/KIP-113%3A+Support+replicas+movement+between+log+directories > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-113%3A+Support+replicas+movement+between+log+directories>.* > > This KIP is related to KIP-112 > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-112%3A+Handle+disk+failure+for+JBOD>: > Handle disk failure for JBOD. They are needed in order to support JBOD in > Kafka. Please help review the KIP. You feedback is appreciated! > > Thanks, > Dong
