On 5/4/2011 4:14 PM, Ray Van Dolson wrote:
On Wed, May 04, 2011 at 02:55:55PM -0700, Brandon High wrote:
On Wed, May 4, 2011 at 12:29 PM, Erik Trimble<erik.trim...@oracle.com> wrote:
I suspect that NetApp does the following to limit their resource
usage: they presume the presence of some sort of cache that can be
dedicated to the DDT (and, since they also control the hardware, they can
make sure there is always one present). Thus, they can make their code
AFAIK, NetApp has more restrictive requirements about how much data
can be dedup'd on each type of hardware.
See page 29 of http://media.netapp.com/documents/tr-3505.pdf - Smaller
pieces of hardware can only dedup 1TB volumes, and even the big-daddy
filers will only dedup up to 16TB per volume, even if the volume size
is 32TB (the largest volume available for dedup).
NetApp solves the problem by putting rigid constraints around the
problem, whereas ZFS lets you enable dedup for any size dataset. Both
approaches have limitations, and it sucks when you hit them.
-B
That is very true, although worth mentioning you can have quite a few
of the dedupe/SIS enabled FlexVols on even the lower-end filers (our
FAS2050 has a bunch of 2TB SIS enabled FlexVols).
Stupid question - can you hit all the various SIS volumes at once, and
not get horrid performance penalties?
If so, I'm almost certain NetApp is doing post-write dedup. That way,
the strictly controlled max FlexVol size helps with keeping the resource
limits down, as it will be able to round-robin the post-write dedup to
each FlexVol in turn.
ZFS's problem is that it needs ALL the resouces for EACH pool ALL the
time, and can't really share them well if it expects to keep performance
from tanking... (no pun intended)
The FAS2050 of course has a fairly small memory footprint...
I do like the additional flexibility you have with ZFS, just trying to
get a handle on the memory requirements.
Are any of you out there using dedupe ZFS file systems to store VMware
VMDK (or any VM tech. really)? Curious what recordsize you use and
what your hardware specs / experiences have been.
Ray
Right now, I use it for my Solaris 8 containers and VirtualBox images.
the VB images are mostly Windows (XP and Win2003).
I tend to put the OS image in one VMdisk, and my scratch disks in
another. That is, I generally don't want my apps writing much to my OS
images. My scratch/data disks aren't dedup.
Overall, I'm running about 30 deduped images served out over NFS. My
recordsize is set to 128k, but, given that they're OS images, my actual
disk block usage has a significant 4k presence. One way I reduced this
initally was to have the VMdisk image stored on local disk, then copied
the *entire* image to the ZFS server, so the server saw a single large
file, which meant it tended to write full 128k blocks. Do note, that my
30 images only takes about 20GB of actual space, after dedup. I figure
about 5GB of dedup space per OS type (and, I have 4 different setups).
My data VMdisks, however, chew though about 4TB of disk space, which is
nondeduped. I'm still trying to determine if I'm better off serving
those data disks as NFS mounts to my clients, or as VMdisk images
available over iSCSI or NFS. Right now, I'm doing VMdisks over NFS.
The setup I'm using is an older X4200 (non-M2), with 3rd-party SSDs as
L2ARC, hooked to an old 3500FC array. It has 8GB of RAM in total, and
runs just fine with that. I definitely am going to upgrade to something
much larger in the near future, since I expect to up my number of VM
images by at least a factor of 5.
That all said, if you're relatively careful about separating OS installs
from active data, you can get really impressive dedup ratios using a
relatively small amount of actual space. In my case, I expect to
eventually be serving about 10 different configs out to a total of maybe
100 clients, and probably never exceed 100GB max on the deduped end.
Which means that I'll be able to get away with 16GB of RAM for the whole
server, comfortably.
--
Erik Trimble
Java System Support
Mailstop: usca22-123
Phone: x17195
Santa Clara, CA
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
