Bill,
I did the same test on the Thumper I'm working on with the NFS vols
converted from ZFS stripes to SVM stripes. In both cases same
number/type of disks in the stripe. In my very simple test ,time for
file in frame*; do cp /inq/$file /outq/$file; done, UFS did
approximately 64 MB/s, the best run for ZFS was approx 58 MB/s. Not a
huge difference for sure, but enough to make you think about switching.
This was single stream over a 10GE link. (x4600 mounting vols from an x4500)
Matt
Bill Moore wrote:
On Thu, Nov 23, 2006 at 03:37:33PM +0100, Roch - PAE wrote:
Al Hopper writes:
> Hi Roch - you are correct in that the data presented was incomplete. I
> did'nt present data for the same test with an NFS mount from the same
> server, for a UFS based filesystem. So here is that data point:
>
> $ ptime gunzip -c /tmp/emacs-21.4a.tar.gz |tar xf -
>
> real 12.671
> user 2.356
> sys 0.228
>
> This test is not totally fair, in that the UFS filesystem being shared is
> on a single 400Gb SATA drive being used as the boot device - versus the
> 5-way raidz config which consists of 5 of those same 400Gb SATA drives.
> But the data clearly shows the NFS/ZFS is a bad combination: 2 minutes 33
> Seconds for NFS/ZFS versus 13 Seconds (rouding up) for NFS/UFS.
I'd put 100$ on the table that the WCE is enabled on the
SATA drive backing UFS. Even if format says it's not, are
there not some drives which just ignore the WC disable
commands ?
I agree with Roch here. With UFS, if WCE is enabled on the drives
(which I'm sure it is on Al's SATA drives), UFS is fooled into thinking
that when it writes a block to disk, it's safe. The drive returns from
the write amazingly fast (since the data only landed in cache - not the
media), so you get quick turnarounds (low latency) on NFS, which is the
only thing that matters on single-threaded performance.
With ZFS, on the other hand, not only do we write data to the drive when
NFS tells us to, but we issue a DKIOCFLUSHWRITECACHE ioctl to the
underlying device (FLUSH_CACHE on ATA, SYNCHRONIZE_CACHE on SCSI) to
ensure that the data that's supposed to be on the disk is really, truly
on the disk. This takes typically around 4-6ms, which is quite a while.
Again, this dictates the single-threaded NFS performance.
If you want an apples-to-apples comparison, either try the UFS/ZFS tests
on a drive that has WCE disabled, or turn off the ZIL on a drive that
has WCE enabled. I'll bet the difference will be rather slight, perhaps
in favor of ZFS.
--Bill
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
