I've submitted these to Roch and co before on the NFS list and off list. My favorite case was writing 6250 8k files (randomly generated) over NFS from a solaris or linux client. We originally were getting 20K/sec when I was using RAIDZ, but between switching to RAID-5 backed iscsi luns in a zpool stripe and B40/41, we saw our performance approach a more reasonable 300-400K/sec average. I get closer to 1-3MB/sec with UFS as the backend vs ZFS. Of course, if its locally attached storage (not iSCSI) performance starts to be parallel to that of UFS or better. There is some built in latency and some major penalties for streaming writes of various sizes with the NFS implementation and its fsync happiness (3 fsyncs per write from an NFS client). Its all very true that its stable/safe, but its also very slow in various use cases!
On 8/1/06, eric kustarz <[EMAIL PROTECTED]> wrote:
Joe Little wrote: > On 7/31/06, Dale Ghent <[EMAIL PROTECTED]> wrote: > >> On Jul 31, 2006, at 8:07 PM, eric kustarz wrote: >> >> > >> > The 2.6.x Linux client is much nicer... one thing fixed was the >> > client doing too many commits (which translates to fsyncs on the >> > server). I would still recommend the Solaris client but i'm sure >> > that's no surprise. But if you'r'e stuck on Linux, upgrade to the >> > latest stable 2.6.x and i'd be curious if it was better. >> >> I'd love to be on kernel 2.6 but due to the philosophical stance >> towards OpenAFS of some people on the lkml list[1], moving to 2.6 is >> a tough call for us to do. But that's another story for another list. >> The fact is that I'm stuck on 2.4 for the time being and I'm having >> problems with a Solaris/ZFS NFS server that I'm (and Jan) are not >> having with Solaris/UFS and (in my case) Linux/XFS NFS server. >> >> [1] https://lists.openafs.org/pipermail/openafs-devel/2006-July/ >> 014041.html >> >> /dale >> > > First, OpenAFS 1.4 works just fine with 2.6 based kernels. We've > already standardized on that over 2.4 kernels (deprecated) at > Stanford. Second, I had similar fsync fatality when it came to NFS > clients (linux or solaris mind you) and non-local backed clients using > ZFS on a Solaris 10U2 (or B40+) server. My case was iscsi and it was > chalked up to low latency on iSCSI, but I still to this day find NFS > write performance on small or multititudes of files at a time with ZFS > as a back end to be rather iffy. Its perfectly fast for NFS reads and > and its always speedly local to the box, but the NFS/ZFS integration > seems problematic. I can always test w/ UFS and get great performance. > Its the roundtrips with many fsyncs to the backend storage that ZFS > requires for commits that get ya. Do you have a reproducable test case for this? If so, i would be interested... I wonder if you're hitting: 6413510 zfs: writing to ZFS filesystem slows down fsync() on other files in the same FS http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6413510 which Neil is finishing up as we type. The problem basically is that fsyncs can get slowed down by non-related I/O, so if you had a process/NFS client that was doing lots of I/O and another doing fsyncs, the fsyncs would get slowed down by the other process/client. eric > > > >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss@opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >>
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
