Re: [zfs-discuss] Zvol vs zfs send/zfs receive
> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- > boun...@opensolaris.org] On Behalf Of Bill Sommerfeld > > > But simply creating the snapshot on the sending side should be no > problem. > > By default, zvols have reservations equal to their size (so that writes > don't fail due to the pool being out of space). > > Creating a snapshot in the presence of a reservation requires reserving > enough space to overwrite every block on the device. This is surprising, because it's not like the "normal" zfs behavior. Normal ZFS does not reserve snapshot space to guarantee you can always completely overwrite every single used block of every single file in the system. It just starts consuming space for changed blocks, and if you fill up the zpool, further writes are denied until you delete some snaps. But you're saying it handles zvols differently - That when you create a zvol, it reserves enough space for it, and when you snapshot it, it reserves enough space to completely overwrite it and keep both the snapshot and the current live version without running out of storage space. I never heard that before - and I can see some good reasons to do it this way - But it's surprising. Based on what I'm hearing now, it also seems - Upon zvol creation, you create a reservation. Upon the first snapshot, you double the reservation, but upon subsequent snapshots, you don't need to increase the reservation each time. Because snapshots are read-only, the system is able to account for all the used space of the snapshots, plus a reservation for the "live" current version. Total space reserved will be 2x the size of the zvol, plus the actual COW consumed space for all the snapshots. The point is to guarantee that writes to a zvol will never be denied, presumably because there's an assumption zvol's are being used by things like VM's and iscsi shares, which behave very poorly if write is denied. Unlike "normal" files, where write denied is generally an annoyance but doesn't cause deeper harm, such as virtual servers crashing. There's another lesson to be learned here. As mentioned by Matthew, you can tweak your reservation (or refreservation) on the zvol, but you do so at your own risk, possibly putting yourself into a situation where writes to the zvol might get denied. But the important implied meaning is the converse - If you have guest VM's in the filesystem (for example, if you're sharing NFS to ESX, or if you're running VirtualBox) then you might want to set the reservation (or refreservation) for those filesystems modeled after the zvol behavior. In other words, you might want to guarantee that ESX or VirtualBox can always write. It's probably a smart thing to do, in a lot of situations. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Zvol vs zfs send/zfs receive
On Sun, Sep 16, 2012 at 7:43 PM, Edward Ned Harvey (opensolarisisdeadlongliveopensolaris) wrote: > There's another lesson to be learned here. > > As mentioned by Matthew, you can tweak your reservation (or refreservation) > on the zvol, but you do so at your own risk, possibly putting yourself into a > situation where writes to the zvol might get denied. > > But the important implied meaning is the converse - If you have guest VM's in > the filesystem (for example, if you're sharing NFS to ESX, or if you're > running VirtualBox) then you might want to set the reservation (or > refreservation) for those filesystems modeled after the zvol behavior. In > other words, you might want to guarantee that ESX or VirtualBox can always > write. It's probably a smart thing to do, in a lot of situations. I'd say just do what you normally do. In my case, I use sparse files or dynamic disk images anyway, so when I use zvols I use "zfs create -s". That single switch sets reservation and refreservation to "none", -- Fajar ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Zvol vs zfs send/zfs receive
On Sep 15, 2012, at 6:03 PM, Bob Friesenhahn wrote: > On Sat, 15 Sep 2012, Dave Pooser wrote: > >> The problem: so far the send/recv appears to have copied 6.25TB of >> 5.34TB. >> That... doesn't look right. (Comparing zfs list -t snapshot and looking at >> the 5.34 ref for the snapshot vs zfs list on the new system and looking at >> space used.) >> Is this a problem? Should I be panicking yet? > > Does the old pool use 512 byte sectors while the new pool uses 4K sectors? > Is there any change to compression settings? > > With volblocksize of 8k on disks with 4K sectors one might expect very poor > space utilization because metadata chunks will use/waste a minimum of 4k. > There might be more space consumed by the metadata than the actual data. With a zvol of 8K blocksize, 4K sector disks, and raidz you will get 12K (data plus parity) written for every block, regardless of how many disks are in the set. There will also be some metadata overhead, but I don't know of a metadata sizing formula for the general case. So the bad news is, 4K sector disks with small blocksize zvols tend to have space utilization more like mirroring. The good news is that performance is also more like mirroring. -- richard -- illumos Day & ZFS Day, Oct 1-2, 2012 San Fransisco www.zfsday.com richard.ell...@richardelling.com +1-760-896-4422 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Zvol vs zfs send/zfs receive
On 9/16/12 10:40 AM, "Richard Elling" wrote: >With a zvol of 8K blocksize, 4K sector disks, and raidz you will get 12K >(data >plus parity) written for every block, regardless of how many disks are in >the set. >There will also be some metadata overhead, but I don't know of a metadata >sizing formula for the general case. > >So the bad news is, 4K sector disks with small blocksize zvols tend to >have space utilization more like mirroring. The good news is that >performance >is also more like mirroring. > -- richard Ok, that makes sense. And since there's no way to change the blocksize of a zvol after creation (AFAIK) I can either live with the size, find 3TB drives with 512byte sectors (I think Seagate Constellations would work) and do yet another send/receive, or create a new zvol with a larger blocksize and copy the files from one zvol to the other. (Leaning toward option 3 because the files are mostly largish graphics files and the like.) Thanks for the help! -- Dave Pooser Manager of Information Services Alford Media http://www.alfordmedia.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
