rihad wrote:
Oleg Bulyzhin wrote:
On Wed, Oct 07, 2009 at 03:16:27PM +0500, rihad wrote:
Oleg Bulyzhin wrote:
On Wed, Oct 07, 2009 at 02:23:47PM +0500, rihad wrote:
Few questions:
1) why are you not using fastforwarding?
2) search_steps/searches ratio is not that good, are you using
'buckets'
keyword in your pipe configuration?
3) you have net.inet.ip.fw.one_pass = 0, is it intended?
1) and 3): the box does traffic accounting and shaping, so I need
one_pass=0 to do both ngtee and pipes.
Still can not see any objection for not using fastforwarding, and usually
ipfw ruleset can be rearranged for using dummynet & netgraph with
one_pass=1.
You probably have some special sources of documentation ;-) According to
man ipfw, both "netgraph/ngtee" and "pipe" decide the fate of the packet
unless one_pass=0. Or do you mean sprinkling smart skiptos here and
there? ;-)
Could you show your 'ipfw show' output? (hide ip addresses if you wish
but
keep counters please).
Here it is, in its whole glory:
00100 10434423 1484891105 allow ip from any to any via lo0
00200 2 14 deny ip from any to 127.0.0.0/8
00300 1 4 deny ip from 127.0.0.0/8 to any
01000 3300039938 327603104711 allow ip from any to any in
01010 26214900 421138433 allow ip from me to any out
01020 5453857 46806278 allow icmp from any to any out
01030 3268289053 327224694165 ngtee 1 ip from any to any out
01040 18681181 1089636054 skipto 1100 ip from table(127) to any out
recv bce0 xmit bce1
01060 777488848 76743392754 pipe tablearg ip from any to table(0) out
recv bce0 xmit bce1
01070 776831109 76682499457 allow ip from any to table(0) out recv
bce0 xmit bce1
01100 13102697 808411842 pipe tablearg ip from any to table(2) out
65535 662648946 66711487830 allow ip from any to any
table(127) is static in nature and is under 100 entries.
table(0) and table(2) have the same IP clients' addresses but different
pipe IDs.
2) Hm, I'm not using "buckets", but rather
net.inet.ip.dummynet.hash_size. It's at default, 64. I've tried
setting net.inet.ip.dummynet.hash_size=65536 in sysctl.conf but
somehow it was still 64 after reboot, so I left it at 64. Should I
make it 128? 256? Does it matter that much? The load is at approx.
70-120 consumers per pipe, so I thought 64 bucket size was enough.
It depends on traffic pattern, try to increase it and watch
search_steps/searches ratio (~1.001 is good enough)
After reconfiguring all pipes with hash_size=256 (4 times as much as
before), the ratio has started decreasing slowly. Run by me every 5-100
seconds:
[ri...@billing ~]$ echo "$(sysctl -n
net.inet.ip.dummynet.search_steps)/$(sysctl -n
net.inet.ip.dummynet.searches)" | bc -l
1.10639566354978963640
[ri...@billing ~]$ echo "$(sysctl -n
net.inet.ip.dummynet.search_steps)/$(sysctl -n
net.inet.ip.dummynet.searches)" | bc -l
1.10638988711274017516
[ri...@billing ~]$ echo "$(sysctl -n
net.inet.ip.dummynet.search_steps)/$(sysctl -n
net.inet.ip.dummynet.searches)" | bc -l
1.10637649664889937145
[ri...@billing ~]$ echo "$(sysctl -n
net.inet.ip.dummynet.search_steps)/$(sysctl -n
net.inet.ip.dummynet.searches)" | bc -l
1.10636898392044547569
[ri...@billing ~]$ echo "$(sysctl -n
net.inet.ip.dummynet.search_steps)/$(sysctl -n
net.inet.ip.dummynet.searches)" | bc -l
1.10634798328730542254
[ri...@billing ~]$ echo "$(sysctl -n
net.inet.ip.dummynet.search_steps)/$(sysctl -n
net.inet.ip.dummynet.searches)" | bc -l
1.10608591323771604268
[ri...@billing ~]$ echo "$(sysctl -n
net.inet.ip.dummynet.search_steps)/$(sysctl -n
net.inet.ip.dummynet.searches)" | bc -l
1.10600110020578292697
but the number of drops is still there. Run every minute:
Wed Oct 7 11:00:44 UTC 2009 34630 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:01:44 UTC 2009 34630 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:02:44 UTC 2009 34729 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:03:44 UTC 2009 34729 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:04:44 UTC 2009 34861 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:05:44 UTC 2009 34932 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:06:44 UTC 2009 35499 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:07:45 UTC 2009 35780 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:08:45 UTC 2009 35841 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:09:45 UTC 2009 36348 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:10:45 UTC 2009 36568 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:11:45 UTC 2009 36673 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:12:45 UTC 2009 36673 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:13:46 UTC 2009 36673 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:14:46 UTC 2009 36673 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:15:46 UTC 2009 36673 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:16:46 UTC 2009 36849 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:17:46 UTC 2009 37234 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:18:46 UTC 2009 37949 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:19:47 UTC 2009 38043 output packets dropped due to no
bufs, etc.
Wed Oct 7 11:20:47 UTC 2009 38549 output packets dropped due to no
bufs, etc.
2200-2350 users online (ipfw table load). I'll wait and see if the drop
rate approaches 500-1000 per second as the number of online users comes
close to 3-4K.
net.isr.direct=0
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
