You can stop the broker and copy some of the log directories to the new volumes. You have to be a bit careful when you do that. There are two metadata files recovery-point-offset-checkpoint and replication-offset-checkpoint that you have to manually split and copy over.
Ideally, we should be able to do this automatically. Perhaps when a broker starts up, if a "storage.rebalancing" option is enabled, the broker can automatically copy some of the log dirs around to balance the load among the different volumes. Could you file a jira to track this? Thanks, Jun On Tue, Oct 7, 2014 at 7:02 AM, Steve Morin <st...@stevemorin.com> wrote: > Neha, > I log volume or can it be volumes plural? > -Steve > > On Tue, Oct 7, 2014 at 6:41 AM, Neha Narkhede <neha.narkh...@gmail.com> > wrote: > > > Is it possible to perform this migration without losing the data > currently > > stored in the kafka cluster? > > > > Though I haven't tested this, the way this is designed should allow you > to > > shut down a broker, move some partition directories over to the new log > > volume and restart the broker. You will have to do this manually per > broker > > though. > > > > Thanks, > > Neha > > > > On Tue, Oct 7, 2014 at 3:31 AM, Javier Alba <m...@fjavieralba.com> wrote: > > > > > Hi, > > > > > > I have a Kafka 0.8.1.1 cluster consisting in 4 servers with several > > topics > > > on it. > > > > > > The cluster was initially configured to store kafka log data in a > single > > > directory on each server (log.dirs = /tmp/kafka-logs) > > > > > > Now, I have assigned 3 new disks to each server and I would like to use > > > them to store log data, instead the old directory. (logs.dirs = > > > /srv/data/1,/srv/data/2,/srv/data/3) > > > > > > What would be the recommended way of doing such a migration? > > > > > > Is it possible to perform this migration without losing the data > > currently > > > stored in the kafka cluster? > > > > > > Would it be possible to achieve that kind of change without having to > > stop > > > the cluster and losing service? > > > > > > > > > Thanks, > > > > > > > > >
