Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-05 Thread Brandon High
On Thu, May 5, 2011 at 11:17 AM, Giovanni Tirloni wrote: > What I find it curious is that it only happens with incrementals. Full > send's go as fast as possible (monitored with mbuffer). I was just wondering > if other people have seen it, if there is a bug (b111 is quite old), etc. I missed tha

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-05 Thread Paul Kraus
On Thu, May 5, 2011 at 2:17 PM, Giovanni Tirloni wrote: > What I find it curious is that it only happens with incrementals. Full > send's go as fast as possible (monitored with mbuffer). I was just wondering > if other people have seen it, if there is a bug (b111 is quite old), etc. I have b

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-05 Thread Giovanni Tirloni
On Wed, May 4, 2011 at 9:04 PM, Brandon High wrote: > On Wed, May 4, 2011 at 2:25 PM, Giovanni Tirloni > wrote: > > The problem we've started seeing is that a zfs send -i is taking hours > to > > send a very small amount of data (eg. 20GB in 6 hours) while a zfs send > full > > transfer everyt

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Randy Jones
On 05/03/11 22:45, Rich Teer wrote: True, but the SB1000 only supports 2GB of RAM IIRC! I'll soon be Actually you can get up to 16GB ram in a SB1000 (or SB2000). The 4GB dimms are most likely not too common however the 1GB and 2GB dimms seem to be common. At one time Dataram and maybe Kingst

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Brandon High
On Wed, May 4, 2011 at 2:25 PM, Giovanni Tirloni wrote: >   The problem we've started seeing is that a zfs send -i is taking hours to > send a very small amount of data (eg. 20GB in 6 hours) while a zfs send full > transfer everything faster than the incremental (40-70MB/s). Sometimes we > just gi

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Giovanni Tirloni
On Tue, May 3, 2011 at 11:42 PM, Peter Jeremy < peter.jer...@alcatel-lucent.com> wrote: > - Is the source pool heavily fragmented with lots of small files? > Peter, We've some servers holding Xen VMs and the setup was create to have a default VM from where others would be cloned so the space s

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Rich Teer
On Wed, 4 May 2011, Edward Ned Harvey wrote: > 4G is also lightweight, unless you're not doing much of anything. No dedup, > no L2ARC, just simple pushing bits around. No services running... Just ssh Yep, that's right. This is a repurposed workstation for use in my home network. > I don't un

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Rich Teer
On Wed, 4 May 2011, Edward Ned Harvey wrote: > I suspect you're using a junky 1G slow-as-dirt usb thumb drive. Nope--unless an IOMega Prestige Desktop Hard Drive (containing an Hitachi 7200K RPM hard drive with 32MB of cache) counts as a slow as dirt USB thumb drive! -- Rich Teer, Publisher V

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Edward Ned Harvey
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Rich Teer > > Not such a silly question. :-) The USB1 port was indeed the source of > much of the bottleneck. The same 50 MB file system took only 8 seconds > to copy when I plugged the driv

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread Edward Ned Harvey
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Rich Teer > > Also related to this is a performance question. My initial test involved > copying a 50 MB zfs file system to a new disk, which took 2.5 minutes > to complete. The strikes me as

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-04 Thread David Dyer-Bennet
On Tue, May 3, 2011 19:39, Rich Teer wrote: > I'm playing around with nearline backups using zfs send | zfs recv. > A full backup made this way takes quite a lot of time, so I was > wondering: after the initial copy, would using an incremental send > (zfs send -i) make the process much quick beca

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-03 Thread Rich Teer
On Wed, 4 May 2011, Peter Jeremy wrote: > Possibilities I can think of: > - Do you have lots of snapshots? There's an overhead of a second or so > for each snapshot to be sent. > - Is the source pool heavily fragmented with lots of small files? Nope, and I don't think so. > Hopefully a silly

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-03 Thread Peter Jeremy
On 2011-May-04 08:39:39 +0800, Rich Teer wrote: >Also related to this is a performance question. My initial test involved >copying a 50 MB zfs file system to a new disk, which took 2.5 minutes >to complete. The strikes me as being a bit high for a mere 50 MB; >are my expectation realistic or is

Re: [zfs-discuss] Quick zfs send -i performance questions

2011-05-03 Thread Eric D. Mudama
On Tue, May 3 at 17:39, Rich Teer wrote: Hi all, I'm playing around with nearline backups using zfs send | zfs recv. A full backup made this way takes quite a lot of time, so I was wondering: after the initial copy, would using an incremental send (zfs send -i) make the process much quick becau