Another thing to keep an eye out for is disk caching. With ZFS, whenever the NFS server tells us to make sure something is on disk, we actually make sure it's on disk by asking the drive to flush dirty data in its write cache out to the media. Needless to say, this takes a while.
With UFS, it isn't aware of the extra level of caching, and happily pretends it's in a world where once the drive ACKs a write, it's on stable storage. If you use format(1M) and take a look at whether or not the drive's write cache is enabled, that should shed some light on this. If it's on, try turning it off and re-run your NFS tests on ZFS vs. UFS. Either way, let us know what you find out. --Bill On Tue, Jan 02, 2007 at 12:40:26PM -0800, Brad Plecs wrote: > I had a user report extreme slowness on a ZFS filesystem mounted over NFS > over the weekend. > After some extensive testing, the extreme slowness appears to only occur when > a ZFS filesystem is mounted over NFS. > > One example is doing a 'gtar xzvf php-5.2.0.tar.gz'... over NFS onto a ZFS > filesystem. this takes: > > real 5m12.423s > user 0m0.936s > sys 0m4.760s > > Locally on the server (to the same ZFS filesystem) takes: > > real 0m4.415s > user 0m1.884s > sys 0m3.395s > > The same job over NFS to a UFS filesystem takes > > real 1m22.725s > user 0m0.901s > sys 0m4.479s > > Same job locally on server to same UFS filesystem: > > real 0m10.150s > user 0m2.121s > sys 0m4.953s > > > This is easily reproducible even with single large files, but the multiple > small files > seems to illustrate some awful sync latency between each file. > > Any idea why ZFS over NFS is so bad? I saw the threads that talk about an > fsync penalty, > but they don't seem relevant since the local ZFS performance is quite good. > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
