--- On Wed, 10/7/09, rihad <ri...@mail.ru> wrote:
> From: rihad <ri...@mail.ru> > Subject: Re: dummynet dropping too many packets > To: "Oleg Bulyzhin" <o...@freebsd.org> > Cc: freebsd-net@freebsd.org > Date: Wednesday, October 7, 2009, 7:23 AM > rihad wrote: > > Oleg Bulyzhin wrote: > >> On Wed, Oct 07, 2009 at 03:16:27PM +0500, rihad > wrote: > >>> Oleg Bulyzhin wrote: > >>>> On Wed, Oct 07, 2009 at 02:23:47PM +0500, > rihad wrote: > >>>> > >>>> Few questions: > >>>> 1) why are you not using fastforwarding? > >>>> 2) search_steps/searches ratio is not that > good, are you using 'buckets' > >>>> keyword in your pipe > configuration? > >>>> 3) you have net.inet.ip.fw.one_pass = 0, > is it intended? > >>>> > >>> 1) and 3): the box does traffic accounting and > shaping, so I need one_pass=0 to do both ngtee and pipes. > >> Still can not see any objection for not using > fastforwarding, and usually > >> ipfw ruleset can be rearranged for using dummynet > & netgraph with one_pass=1. > > > > You probably have some special sources of > documentation ;-) According to man ipfw, both > "netgraph/ngtee" and "pipe" decide the fate of the packet > unless one_pass=0. Or do you mean sprinkling smart skiptos > here and there? ;-) > > > >> Could you show your 'ipfw show' output? (hide ip > addresses if you wish but > >> keep counters please). > >> > > Here it is, in its whole glory: > > > > > 00100 10434423 1484891105 > allow ip from any to any via lo0 > > 00200 2 > 14 deny ip from any to > 127.0.0.0/8 > > 00300 1 > 4 deny ip from 127.0.0.0/8 to > any > > 01000 3300039938 327603104711 allow ip from any to any > in > > 01010 26214900 421138433 > allow ip from me to any out > > 01020 5453857 > 46806278 allow icmp from any to any out > > 01030 3268289053 327224694165 ngtee 1 ip from any to > any out > > > 01040 18681181 1089636054 > skipto 1100 ip from table(127) to any out recv bce0 xmit > bce1 > > 01060 777488848 76743392754 pipe tablearg > ip from any to table(0) out recv bce0 xmit bce1 > > 01070 776831109 76682499457 allow ip from > any to table(0) out recv bce0 xmit bce1 > > 01100 13102697 808411842 > pipe tablearg ip from any to table(2) out > > 65535 662648946 66711487830 allow ip from > any to any > > > > table(127) is static in nature and is under 100 > entries. > > table(0) and table(2) have the same IP clients' > addresses but different pipe IDs. > > > >>> 2) Hm, I'm not using "buckets", but rather > net.inet.ip.dummynet.hash_size. It's at default, 64. I've > tried setting net.inet.ip.dummynet.hash_size=65536 in > sysctl.conf but somehow it was still 64 after reboot, so I > left it at 64. Should I make it 128? 256? Does it matter > that much? The load is at approx. 70-120 consumers per pipe, > so I thought 64 bucket size was enough. > >> It depends on traffic pattern, try to increase it > and watch > >> search_steps/searches ratio (~1.001 is good > enough) > >> > After reconfiguring all pipes with hash_size=256 (4 times > as much as before), the ratio has started decreasing slowly. > Run by me every 5-100 seconds: > [ri...@billing ~]$ echo "$(sysctl -n > net.inet.ip.dummynet.search_steps)/$(sysctl -n > net.inet.ip.dummynet.searches)" | bc -l > 1.10639566354978963640 > [ri...@billing ~]$ echo "$(sysctl -n > net.inet.ip.dummynet.search_steps)/$(sysctl -n > net.inet.ip.dummynet.searches)" | bc -l > 1.10638988711274017516 > [ri...@billing ~]$ echo "$(sysctl -n > net.inet.ip.dummynet.search_steps)/$(sysctl -n > net.inet.ip.dummynet.searches)" | bc -l > 1.10637649664889937145 > [ri...@billing ~]$ echo "$(sysctl -n > net.inet.ip.dummynet.search_steps)/$(sysctl -n > net.inet.ip.dummynet.searches)" | bc -l > 1.10636898392044547569 > [ri...@billing ~]$ echo "$(sysctl -n > net.inet.ip.dummynet.search_steps)/$(sysctl -n > net.inet.ip.dummynet.searches)" | bc -l > 1.10634798328730542254 > [ri...@billing ~]$ echo "$(sysctl -n > net.inet.ip.dummynet.search_steps)/$(sysctl -n > net.inet.ip.dummynet.searches)" | bc -l > 1.10608591323771604268 > [ri...@billing ~]$ echo "$(sysctl -n > net.inet.ip.dummynet.search_steps)/$(sysctl -n > net.inet.ip.dummynet.searches)" | bc -l > 1.10600110020578292697 > > > > but the number of drops is still there. Run every minute: > Wed Oct 7 11:00:44 UTC 2009 34630 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:01:44 UTC 2009 34630 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:02:44 UTC 2009 34729 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:03:44 UTC 2009 34729 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:04:44 UTC 2009 34861 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:05:44 UTC 2009 34932 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:06:44 UTC 2009 35499 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:07:45 UTC 2009 35780 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:08:45 UTC 2009 35841 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:09:45 UTC 2009 36348 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:10:45 UTC 2009 36568 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:11:45 UTC 2009 36673 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:12:45 UTC 2009 36673 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:13:46 UTC 2009 36673 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:14:46 UTC 2009 36673 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:15:46 UTC 2009 36673 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:16:46 UTC 2009 36849 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:17:46 UTC 2009 37234 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:18:46 UTC 2009 37949 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:19:47 UTC 2009 38043 output > packets dropped due to no bufs, etc. > Wed Oct 7 11:20:47 UTC 2009 38549 output > packets dropped due to no bufs, etc. > > 2200-2350 users online (ipfw table load). I'll wait and see > if the drop rate approaches 500-1000 per second as the > number of online users comes close to 3-4K. > > net.isr.direct=0 Its frightening to me that someone is managing such a large network with dummynet. Talk about stealing your customer's money. BC _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
