Thanks for the script. Here is some sample output from 'sudo ./nfssvrtop -b 512 5' (my disks are 512B-sector emulated and the pool is ashift=9, some benchmarking didn't show much difference with ashift=12 other than giving up 8% of available space) during a copy operation from 37.30 with sync=standard:
2012 Jun 14 13:59:13, load: 0.68, read: 0 KB, swrite: 0 KB, awrite: 557056 KB Ver Client NFSOPS Reads SWrites AWrites Commits Rd_bw SWr_bw AWr_bw Rd_t SWr_t AWr_t Com_t Align% 3 xxx.xxx.37.30 108 0 0 108 0 0 0 111206 0 0 396 1917419 100 a bit later... 3 xxx.xxx.37.30 109 0 0 108 0 0 0 111411 0 0 427 0 100 sample output from the end of 'zpool iostat -v 5 mainpool' concurrently: logs - - - - - - c31t3d0s0 260M 9.68G 0 1.21K 0 85.3M c31t4d0s0 260M 9.68G 0 1.21K 0 85.1M In case the alignment fails, the nonzero entries are under NFSOPS, AWrites, AWr_bw, AWr_t, Com_t and Align%. The Com_t (average commit time?) column alternates between zero and a million or two (the other columns stay about the same, the zeros stay zero), while the "Commits" column stays zero during the copy. The write throughput to the logs varies quite a bit, that sample is a very high mark, it mainly alternates between almost zero and 30M each, which is kind of odd considering the copy speed (using gigabit network, copy speed averages around 110MB/s). When I 'zfs set sync=disabled', the output of nfssrvtop stays about the same, except the Com_t stays 0, and the log devices also stay 0 for throughput. Could you enlighten me as to what "Com_t" measures when "Commits" stays zero? Perhaps the nfs server caches asynchronous nfs writes how I expect, but flushes its cache with synchronous writes? Thanks, Tim On Thu, Jun 14, 2012 at 9:24 AM, Richard Elling <richard.ell...@gmail.com> wrote: > On Jun 13, 2012, at 4:51 PM, Daniel Carosone wrote: > >> On Wed, Jun 13, 2012 at 05:56:56PM -0500, Timothy Coalson wrote: >>> client: ubuntu 11.10 >>> /etc/fstab entry: <server>:/mainpool/storage /mnt/myelin nfs >>> bg,retry=5,soft,proto=tcp,intr,nfsvers=3,noatime,nodiratime,async 0 >>> 0 >> >> nfsvers=3 >> >>> NAME PROPERTY VALUE SOURCE >>> mainpool/storage sync standard default >> >> sync=standard >> >> This is expected behaviour for this combination. NFS 3 semantics are >> for persistent writes at the server regardless - and mostly also >> for NFS 4. > > NB, async NFS was introduced in NFSv3. To help you easily see NFSv3/v4 > async and sync activity, try nfssvrtop > https://github.com/richardelling/tools > > -- richard > > -- > > ZFS and performance consulting > http://www.RichardElling.com > > > > > > > > > > > > > > > > _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
