Thanks guys for your support. @Jay, this approach seems the easiest right now, I will try it.
@Jun, following your suggestion I’ve submitted a Jira: https://issues.apache.org/jira/browse/KAFKA-1689 thanks, Javier On Wednesday 8 October 2014 at 00:45, Jay Kreps wrote: > I think the more automated/lazy way right now would be to shutdown one > broker, rm -rf all its data, add the data directories in config, and > restart to let the broker restore off the replicas. This may actually be > okay though it is a little slower. > > -Jay > > On Tue, Oct 7, 2014 at 3:25 PM, Jun Rao <jun...@gmail.com > (mailto:jun...@gmail.com)> wrote: > > > You can stop the broker and copy some of the log directories to the new > > volumes. You have to be a bit careful when you do that. There are two > > metadata files recovery-point-offset-checkpoint and > > replication-offset-checkpoint that you have to manually split and copy > > over. > > > > Ideally, we should be able to do this automatically. Perhaps when a broker > > starts up, if a "storage.rebalancing" option is enabled, the broker can > > automatically copy some of the log dirs around to balance the load among > > the different volumes. Could you file a jira to track this? > > > > Thanks, > > > > Jun > > > > On Tue, Oct 7, 2014 at 7:02 AM, Steve Morin <st...@stevemorin.com > > (mailto:st...@stevemorin.com)> wrote: > > > > > Neha, > > > I log volume or can it be volumes plural? > > > -Steve > > > > > > On Tue, Oct 7, 2014 at 6:41 AM, Neha Narkhede <neha.narkh...@gmail.com > > > (mailto:neha.narkh...@gmail.com)> > > > wrote: > > > > > > > Is it possible to perform this migration without losing the data > > > currently > > > > stored in the kafka cluster? > > > > > > > > Though I haven't tested this, the way this is designed should allow you > > > to > > > > shut down a broker, move some partition directories over to the new log > > > > volume and restart the broker. You will have to do this manually per > > > > > > > > > > broker > > > > though. > > > > > > > > Thanks, > > > > Neha > > > > > > > > On Tue, Oct 7, 2014 at 3:31 AM, Javier Alba <m...@fjavieralba.com > > > > (mailto:m...@fjavieralba.com)> > > wrote: > > > > > > > > > Hi, > > > > > > > > > > I have a Kafka 0.8.1.1 cluster consisting in 4 servers with several > > > > topics > > > > > on it. > > > > > > > > > > The cluster was initially configured to store kafka log data in a > > > single > > > > > directory on each server (log.dirs = /tmp/kafka-logs) > > > > > > > > > > Now, I have assigned 3 new disks to each server and I would like to > > use > > > > > them to store log data, instead the old directory. (logs.dirs = > > > > > /srv/data/1,/srv/data/2,/srv/data/3) > > > > > > > > > > What would be the recommended way of doing such a migration? > > > > > > > > > > Is it possible to perform this migration without losing the data > > > > currently > > > > > stored in the kafka cluster? > > > > > > > > > > Would it be possible to achieve that kind of change without having to > > > > stop > > > > > the cluster and losing service? > > > > > > > > > > > > > > > Thanks,
