On Thu, Nov 23, 2006 at 03:37:33PM +0100, Roch - PAE wrote: > Al Hopper writes: > > Hi Roch - you are correct in that the data presented was incomplete. I > > did'nt present data for the same test with an NFS mount from the same > > server, for a UFS based filesystem. So here is that data point: > > > > $ ptime gunzip -c /tmp/emacs-21.4a.tar.gz |tar xf - > > > > real 12.671 > > user 2.356 > > sys 0.228 > > > > This test is not totally fair, in that the UFS filesystem being shared is > > on a single 400Gb SATA drive being used as the boot device - versus the > > 5-way raidz config which consists of 5 of those same 400Gb SATA drives. > > But the data clearly shows the NFS/ZFS is a bad combination: 2 minutes 33 > > Seconds for NFS/ZFS versus 13 Seconds (rouding up) for NFS/UFS. > > I'd put 100$ on the table that the WCE is enabled on the > SATA drive backing UFS. Even if format says it's not, are > there not some drives which just ignore the WC disable > commands ?
I agree with Roch here. With UFS, if WCE is enabled on the drives (which I'm sure it is on Al's SATA drives), UFS is fooled into thinking that when it writes a block to disk, it's safe. The drive returns from the write amazingly fast (since the data only landed in cache - not the media), so you get quick turnarounds (low latency) on NFS, which is the only thing that matters on single-threaded performance. With ZFS, on the other hand, not only do we write data to the drive when NFS tells us to, but we issue a DKIOCFLUSHWRITECACHE ioctl to the underlying device (FLUSH_CACHE on ATA, SYNCHRONIZE_CACHE on SCSI) to ensure that the data that's supposed to be on the disk is really, truly on the disk. This takes typically around 4-6ms, which is quite a while. Again, this dictates the single-threaded NFS performance. If you want an apples-to-apples comparison, either try the UFS/ZFS tests on a drive that has WCE disabled, or turn off the ZIL on a drive that has WCE enabled. I'll bet the difference will be rather slight, perhaps in favor of ZFS. --Bill _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
