Thanks, I will watch out for the MFC and test. Lars
On Sep 11, 2013, at 13:54, Rick Macklem <rmack...@uoguelph.ca> wrote:
> Lars Eggert wrote:
>> Hi,
>>
>> I'm seeing extremely high CPU usage withssh-st the new nfsd:
>>
>> PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU
>> COMMAND
>> 2280 root 102 0 9932K 1376K *nfs_c 0 320:11 100.00%
>> nfsd{nfsd: service}
>> 2280 root 102 0 9932K 1376K CPU7 7 319:47 100.00%
>> nfsd{nfsd: service}
>> 2280 root 102 0 9932K 1376K CPU5 5 318:25 100.00%
>> nfsd{nfsd: service}
>> 2280 root 102 0 9932K 1376K CPU6 6 318:20 100.00%
>> nfsd{nfsd: service}
>> 2280 root 52 0 9932K 1376K CPU0 0 317:32 100.00%
>> nfsd{nfsd: service}
>> 2280 root 102 0 9932K 1376K *nfs_c 1 315:41 99.17%
>> nfsd{nfsd: service}
>> 2280 root 52 0 9932K 1376K *nfs_c 4 320:22 98.78%
>> nfsd{nfsd: master}
>> 2280 root 102 0 9932K 1376K *nfs_c 1 317:10 98.10%
>> nfsd{nfsd: service}
>>
>> And this is at a few hundred KB/s with only a few clients:
>>
>> ifstat -i igb1 10
>> igb1
>> KB/s in KB/s out
>> 796.56 208.66
>> 431.19 232.36
>> 316.11 280.31
>> 1005.96 523.42
>> 1077.74 342.25
>> 340.63 217.73
>> 1067.96 330.56
>> 487.91 235.61
>>
>> Any ideas?
>>
>> FreeBSD stanley.muccbc.hq.netapp.com 9.2-PRERELEASE FreeBSD
>> 9.2-PRERELEASE #7: Wed Sep 4 11:06:31 CEST 2013
>> r...@stanley.muccbc.hq.netapp.com:/usr/obj/usr/src/sys/STANLEY
>> amd64
>>
>> Thanks,
>> Lars
>>
> There is a patch in head (r254337) that I believe handles this.
> It will be MFC'd to stable/9 in about a week, unless someone finds
> problems with it before then.
> If you want a semantically equivalent (but uglier code) patch,
> you can find it here:
> http://people.freebsd.org/~rmacklem/drc4-stable9.patch
> After applying the patch, you need to set sysctl variable(s),
> to avoid the aggressive trimming of stale DRC entries. Garrett
> Wollman suggests the following for a large server:
> vfs.nfsd.tcphighwater=100000
> vfs.nfsd.tcpcachetimeout=300 (5 minutes instead of default of several hrs)
>
> You can also use the sysctl
> vfs.nfsd.cachetcp=0
> to disable use of the DRC for TCP.
>
> The old nfs server did not use the DRC for TCP. The assumption being that
> TCP layer retransmits are good enough to maintain reliable RPC transport.
> Unfortuantely, you can get file corruption when the server reboots or
> there is a network partitioning, if the client chooses to redo the RPC
> over TCP (clients always do this after having to create a new TCP connection).
> In other words, vfs.nfsd.cachetcp=0 is roughly what the old nfsd did.
>
> If you don't want to patch the 9.2 code, you can edit the sources
> (sys/fs/nfsserver/nfs_nfsdcache.c) and change the line:
> static int nfsrc_tcpnonidempotent = 1;
> to
> static int nfsrc_tcpnonidempotent = 0;
> to do the same thing as vfs.nfsd.cachetcp=0
>
> rick
signature.asc
Description: Message signed with OpenPGP using GPGMail
