Hi, a while ago I noticed something odd on my 64bit FreeBSD 6 systems - two amd64, one alpha. I can't make head or tails of it, so I post it in hope that someone can.
Box #1:
[EMAIL PROTECTED]: ~% uname -srm
FreeBSD 6.1-RELEASE-p7 amd64
[EMAIL PROTECTED]: ~% grep kern.ipc /boot/loader.conf
kern.ipc.nsfbufs="8192"
kern.ipc.nmbclusters="49152"
[EMAIL PROTECTED]: ~% sysctl kern.ipc.nsfbufs kern.ipc.nmbclusters
kern.ipc.nsfbufs: 0
kern.ipc.nmbclusters: 32768
[EMAIL PROTECTED]: ~% netstat -m
350/132/482/49152 mbuf clusters in use (current/cache/total/max)
0/0/0 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
2864770 requests for I/O initiated by sendfile
Box #2:
[EMAIL PROTECTED]: ~% uname -srm
FreeBSD 6.2-RC1 alpha
[EMAIL PROTECTED]: ~% grep kern.ipc /boot/loader.conf
kern.ipc.nsfbufs="4294967300"
[EMAIL PROTECTED]: ~% sysctl kern.ipc.nsfbufs kern.ipc.nmbclusters
kern.ipc.nsfbufs: 0
kern.ipc.nmbclusters: 1664
[EMAIL PROTECTED]: ~% netstat -m
64/80/144/1664 mbuf clusters in use (current/cache/total/max)
0/0/0 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
Box #3:
[EMAIL PROTECTED]: ~% uname -srm
FreeBSD 6.1-RELEASE-p10 amd64
[EMAIL PROTECTED]: ~% grep kern.ipc /boot/loader.conf
kern.ipc.nsfbufs="8192"
kern.ipc.nmbclusters="49152"
[EMAIL PROTECTED]: ~% sysctl kern.ipc.nsfbufs kern.ipc.nmbclusters
kern.ipc.nsfbufs: 0
kern.ipc.nmbclusters: 32768
[EMAIL PROTECTED]: ~% grep NSF /root/kernel/RIVENDELL
options NSFBUFS=8192
[EMAIL PROTECTED]: ~% netstat -m
64/118/182/49152 mbuf clusters in use (current/cache/total/max)
0/0/0 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
35 requests for I/O initiated by sendfile
For comparison, Box #4:
[EMAIL PROTECTED]: ~% uname -srm
FreeBSD 6.1-STABLE i386
[EMAIL PROTECTED]: ~% grep kern.ipc /boot/loader.conf
kern.ipc.nsfbufs="8192"
[EMAIL PROTECTED]: ~% sysctl kern.ipc.nsfbufs kern.ipc.nmbclusters
kern.ipc.nsfbufs: 8192
kern.ipc.nmbclusters: 32768
[EMAIL PROTECTED]: ~% netstat -m
104/272/376/32768 mbuf clusters in use (current/cache/total/max)
1598/2635/8192 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
8938512 requests for I/O initiated by sendfile
So, looking at the `netstat -m' output, setting kern.ipc.nmbclusters
in the loader.conf seems to work, but the sysctl reports the default
value that would be in place, if it wasn't overridden in the loader.conf.
For the sendfile buffers, the stats are completely out of whack.
The sysctl reports 0 for every value (`netstat-m' just prints the sysctl
fields for these values if I understood the source right (unlikely).
But sendfile() works and the accounting on the number of sendfile calls
seems to work too.
So I believe the NSFBUFS kernel option and the loader.conf setting work,
but it is quite impossible to properly tune the buffers to the workload.
I did not bother trying the NMBCLUSTERS kernel option, it was removed
a while ago anyway.
The loader.conf kern.ipc.nsfbufs value on box #2 is ridiculously high,
yes, I tried to get beyond the boundary of a u_int_32 to see if maybe
something good or bad happens. It did not.
At this point, I wanted to include links to two mails on amd64@ I
found when I first searched for this oddity, both of which had no
reply. Instead I found a posting to net@ that occured in the meantime,
and a reply to it by Yar Tikhiy two days ago explaining the nsfbufs issue.
There are no such buffers on amd64, ia64 and alpha as they aren't needed.
http://docs.freebsd.org/cgi/getmsg.cgi?fetch=26091+0+current/freebsd-net
Ok, that leaves the question why `sysctl kern.ipc.nmbclusters' reports a
different value compared to the `netstat -m' output on box #1 and #3.
Thanks for any pointers,
Joerg
PS: documenting the nsfbufs issue on amd64/ia64/alpha might be a good
PS: idea. loader(8) seems a good place. Opinions?
--
| /"\ ASCII ribbon | GnuPG Key ID | e86d b753 3deb e749 6c3a |
| \ / campaign against | 0xbbcaad24 | 5706 1f7d 6cfd bbca ad24 |
| X HTML in email | .the next sentence is true. |
| / \ and news | .the previous sentence was a lie. |
signature.asc
Description: PGP signature
