Re: [zfs-discuss] thoughts on parallel backups, rsync, and send/receive

2009-01-26 Thread Ian Collins
Richard Elling wrote: > Ian Collins wrote: > >> One thing I have yet to do is find the optimum number of parallel >> transfers when there are 100s of filesystems. I'm looking into making >> this dynamic, based on throughput. >> > > I'm not convinced that a throughput throttle or metric will be

Re: [zfs-discuss] thoughts on parallel backups, rsync, and send/receive

2009-01-26 Thread Richard Elling
Ahmed Kamal wrote: > Did anyone share a script to send/recv zfs filesystems tree in > parallel, especially if a cap on concurrency can be specified? > Richard, how fast were you taking those snapshots, how fast were the > syncs over the network. For example, assuming a snapshot every 10mins, > is i

Re: [zfs-discuss] thoughts on parallel backups, rsync, and send/receive

2009-01-26 Thread Richard Elling
Ian Collins wrote: Richard Elling wrote: Recently, I've been working on a project which had agressive backup requirements. I believe we solved the problem with parallelism. You might consider doing the same. If you get time to do your own experiments, please share your observations with the

Re: [zfs-discuss] thoughts on parallel backups, rsync, and send/receive

2009-01-26 Thread Ahmed Kamal
Did anyone share a script to send/recv zfs filesystems tree in parallel, especially if a cap on concurrency can be specified? Richard, how fast were you taking those snapshots, how fast were the syncs over the network. For example, assuming a snapshot every 10mins, is it reasonable to expect to syn

Re: [zfs-discuss] thoughts on parallel backups, rsync, and send/receive

2009-01-25 Thread Ian Collins
Richard Elling wrote: > Recently, I've been working on a project which had agressive backup > requirements. I believe we solved the problem with parallelism. You > might consider doing the same. If you get time to do your own experiments, > please share your observations with the community. > htt

[zfs-discuss] thoughts on parallel backups, rsync, and send/receive

2009-01-25 Thread Richard Elling
Recently, I've been working on a project which had agressive backup requirements. I believe we solved the problem with parallelism. You might consider doing the same. If you get time to do your own experiments, please share your observations with the community. http://richardelling.blogspot.com/2