Even the most expensive decompression algorithms generally run
significantly faster than I/O to disk -- at least when real disks are
involved. So, as long as you don't run out of CPU and have to wait for
CPU to be available for decompression, the decompression will win. The
same concept is true for dedup, although I don't necessarily think of
dedup as a form of compression (others might reasonably do so though.)
- Garrett
On Sat, 2010-07-10 at 19:09 -0400, Edward Ned Harvey wrote:
> > From: Roy Sigurd Karlsbakk [mailto:r...@karlsbakk.net]
> > > increases the probability of arc/ram cache hit. So dedup allows you
> > to
> > > stretch your disk, and also stretch your ram cache. Which also
> > > benefits performance.
> >
> > Theoretically, yes, but there will be an overhead in cpu/memory that
> > can reduce this benefit to a penalty.
>
> That's why a really fast compression algorithm is used in-line, in hopes that
> the time cost of compression is smaller than the performance gain of
> compression. Take for example, v.42bis and v.44 which was used to accelerate
> 56K modems. (Probably still are, if you actually have a modem somewhere. ;-)
>
> Nowadays we have faster communication channels; in fact when talking about
> dedup we're talking about local disk speed, which is really fast. But we
> also have fast processors, and the algorithm in question can be really fast.
>
> I recently benchmarked lzop, gzip, bzip2, and lzma for some important data on
> our fileserver that I would call "typical." No matter what I did, lzop was
> so ridiculously light weight that I could never get lzop up to 100% cpu.
> Even reading data 100% from cache and filtering through lzop to /dev/null,
> the kernel overhead of reading ram cache was higher than the cpu overhead to
> compress.
>
> For the data in question, lzop compressed to 70%, gzip compressed to 42%,
> bzip 32%, and lzma something like 16%. bzip2 was the slowest (by a factor of
> 4). lzma -1 and gzip --fast were closely matched in speed but not
> compression. So the compression of lzop was really weak for the data in
> question, but it contributed no significant cpu overhead. The point is:
> It's absolutely possible to compress quickly, if you have a fast algorithm,
> and gain performance. I'm boldly assuming dedup performs this fast. It
> would be nice to actually measure and prove it.
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss@opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
