After running, I see partitions on the new volumes that were added to my brokers. Both mount points are listed in log.dirs comma separated. After running kafka-reassign-partitions.sh, I now see partitions in the second additional volume.
-jeremy > On Oct 20, 2016, at 10:30 PM, Manikumar <manikumar.re...@gmail.com> wrote: > > kafka-reassign-partitions.sh is for migrating data to new/other brokers. > How it is useful for migrating data within same broker? > you reassigned some partitions to other broker and moved back to original > broker? > > On Fri, Oct 21, 2016 at 10:46 AM, Jeremy Hansen <jer...@skidrow.la> wrote: > >> So I seemed to have solved this issue just by running >> kafka-reassign-partitions.sh. I now see even distribution of partitions >> across both disk volumes. Does this make sense? >> >> Thanks >> -jeremy >> >>> On Oct 20, 2016, at 10:07 PM, Manikumar <manikumar.re...@gmail.com> >> wrote: >>> >>> Kafka does not automatically migrate the the existing partition data to >> new >>> volumes. Only new >>> partitions will placed on on new volumes. >>> >>> For now, you can manually copy some the partition dirs(careful with >>> checkpoint files) to new disk, >>> or you can increase the partitions. >>> >>> Or we can just delete the old disk contents and add new disk to >> log.dirs, >>> kafka replicates the the >>> data from other brokers and places on available disks. This may take time >>> depending on data size. >>> >>> JIRA related to auto migration: https://issues.apache.org/ >>> jira/browse/KAFKA-1689 >>> >>> On Fri, Oct 21, 2016 at 3:01 AM, Jeremy Hansen <jer...@skidrow.la> >> wrote: >>> >>>> I’ve read several posts on this but it seemed to be old versions of >>>> kafka. I wanted to get the latest proper way to do this. >>>> >>>> If I add a volume to my brokers running kafka 0.10.0 and I’ve adjusted >>>> log.dirs with the new volume mount point, what do I have to do to >> rebalance >>>> so kafka takes advantage of the new volume? Also, does it make sense to >>>> increase partitions when adding an additional volume? >>>> >>>> Thank you >>>> -jeremy >> >>
