Mohan Srinivasan wrote:
mohans 2007-10-12 19:12:21 UTC
FreeBSD src repository
Modified files:
sys/nfsclient nfs.h nfs_socket.c nfs_subs.c nfsmount.h
Log:
NFS MP scaling changes.
- Eliminate the hideous nfs_sndlock that serialized NFS/TCP request senders
thru the sndlock.
- Institute a new nfs_connectlock that serializes NFS/TCP reconnects. Add
logic to wait for pending request senders to finish sending before
reconnecting. Dial down the sb_timeo for NFS/TCP sockets to 1 sec.
- Break out the nfs xid manipulation under a new nfs xid lock, rather than
over loading the nfs request lock for this purpose.
- Fix some of the locking in nfs_request.
Many thanks to Kris Kennaway for his help with this and for initiating the
MP scaling analysis and work. Kris also tested this patch thorougly.
Approved by: re@ (Ken Smith)
For the benefit of others: this change improved throughput by about 10%
at high I/O loads with a dual core client, and by a factor of 10 on an 8
core client (this was mostly the home-brew nfs_sndlock, which mohan
correctly describes :-).
Mohan's previous commit that increases the nfs server socket buffer size
is also very important for NFS performance. Without it I was only
getting 1-2MB/sec throughput over 10Gb ethernet with UDP mounts, because
the minuscule 32kb socket buffer was constantly filling up and forcing
retransmits. With the new default of 256KB I still get full buffers
with 10ge, so you may need to increase this further to eliminate this
problem. It might be OK for gige speeds, although I was still seeing
some buffer full events, so maybe we should consider increasing the
default sockbuf size to 512KB or so if this is widespread.
As a side comment there is a bug in either the nfs client or server that
corrupts I/O when there is packet loss with UDP mounts (the default).
TCP mounts handle this at the TCP layer.
Kris
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
