Luigi Rizzo wrote:
On Mon, Oct 05, 2009 at 10:55:21AM +0800, Eugene Grosbein wrote:
On Sun, Oct 04, 2009 at 06:47:23PM +0500, rihad wrote:
sysctls:
kern.ipc.nmbclusters=50000
net.inet.ip.dummynet.io_fast=1
I guess you should also try to increase pipes length:
net.inet.ip.dummynet.hash_size=65536
net.inet.ip.dummynet.pipe_slot_limit=1000
in fact, i forgot to ask, we'd need to know the output of
"ipfw pipe show" just to get the idea if there is any
known reason for the drop (e.g. queues too short, etc.)
The pipes are fine, each normally having 100-120 concurrent consumers
(i.e. active users). (The max number of consumers allowed per pipe is
hash_size*max_chain_len, normally 64*16==1024.)
kern.ipc.nmbclusters=111111
And these are by default:
#net.inet.ip.dummynet.hash_size=64
#net.inet.ip.dummynet.max_chain_len=16
Yesterday I set up a cronjob logging the number of drops in the past 15
minutes:
*/15 * * * * (echo -n "$(date) "; netstat -z -s 2>/dev/null | fgrep
-w "output packets dropped") >> /tmp/bufs.log
And here's bufs.log:
Sun Oct 4 21:45:00 AZST 2009 418869 output packets dropped due to no
bufs, etc.
Sun Oct 4 22:00:00 AZST 2009 851693 output packets dropped due to no
bufs, etc.
Sun Oct 4 22:15:01 AZST 2009 932885 output packets dropped due to no
bufs, etc.
Sun Oct 4 22:30:00 AZST 2009 890522 output packets dropped due to no
bufs, etc.
Sun Oct 4 22:45:00 AZST 2009 1065931 output packets dropped due to no
bufs, etc.
Sun Oct 4 23:00:00 AZST 2009 937863 output packets dropped due to no
bufs, etc.
Sun Oct 4 23:15:01 AZST 2009 1018822 output packets dropped due to no
bufs, etc.
Sun Oct 4 23:30:00 AZST 2009 981922 output packets dropped due to no
bufs, etc.
Sun Oct 4 23:45:00 AZST 2009 1015124 output packets dropped due to no
bufs, etc.
Mon Oct 5 00:00:01 AZST 2009 1123926 output packets dropped due to no
bufs, etc.
Mon Oct 5 00:15:01 AZST 2009 948161 output packets dropped due to no
bufs, etc.
Mon Oct 5 00:30:00 AZST 2009 937277 output packets dropped due to no
bufs, etc.
Mon Oct 5 00:45:00 AZST 2009 875218 output packets dropped due to no
bufs, etc.
Mon Oct 5 01:00:00 AZST 2009 803527 output packets dropped due to no
bufs, etc.
Mon Oct 5 01:15:00 AZST 2009 728639 output packets dropped due to no
bufs, etc.
Mon Oct 5 01:30:00 AZST 2009 626154 output packets dropped due to no
bufs, etc.
Mon Oct 5 01:45:00 AZST 2009 519441 output packets dropped due to no
bufs, etc.
Mon Oct 5 02:00:00 AZST 2009 371098 output packets dropped due to no
bufs, etc.
Mon Oct 5 02:15:00 AZST 2009 681243 output packets dropped due to no
bufs, etc.
Mon Oct 5 02:30:00 AZST 2009 562909 output packets dropped due to no
bufs, etc.
Mon Oct 5 02:45:00 AZST 2009 426734 output packets dropped due to no
bufs, etc.
Mon Oct 5 03:00:00 AZST 2009 344619 output packets dropped due to no
bufs, etc.
Mon Oct 5 03:15:00 AZST 2009 90006 output packets dropped due to no
bufs, etc.
Mon Oct 5 03:30:00 AZST 2009 17064 output packets dropped due to no
bufs, etc.
Mon Oct 5 03:45:00 AZST 2009 3851 output packets dropped due to no
bufs, etc.
Mon Oct 5 04:00:00 AZST 2009 1323 output packets dropped due to no
bufs, etc.
Mon Oct 5 04:15:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 04:30:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 04:45:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 05:00:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 05:15:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 05:30:01 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 05:45:01 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 06:00:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 06:15:01 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 06:30:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 06:45:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 07:00:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 07:15:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 07:30:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 07:45:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 08:00:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 08:15:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 08:30:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 08:45:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 09:00:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 09:15:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 09:30:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 09:45:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 10:00:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 10:15:00 AZST 2009 0 output packets dropped due to no bufs, etc.
Mon Oct 5 10:30:00 AZST 2009 177 output packets dropped due to no
bufs, etc.
Mon Oct 5 10:45:00 AZST 2009 1701 output packets dropped due to no
bufs, etc.
Mon Oct 5 11:00:01 AZST 2009 19933 output packets dropped due to no
bufs, etc.
Mon Oct 5 11:15:00 AZST 2009 30003 output packets dropped due to no
bufs, etc.
Mon Oct 5 11:30:00 AZST 2009 56712 output packets dropped due to no
bufs, etc.
Mon Oct 5 11:45:00 AZST 2009 78721 output packets dropped due to no
bufs, etc.
Mon Oct 5 12:00:01 AZST 2009 112518 output packets dropped due to no
bufs, etc.
Mon Oct 5 12:15:00 AZST 2009 7229 output packets dropped due to no
bufs, etc.
Mon Oct 5 12:30:01 AZST 2009 24965 output packets dropped due to no
bufs, etc.
Mon Oct 5 12:45:00 AZST 2009 75900 output packets dropped due to no
bufs, etc.
Mon Oct 5 13:00:00 AZST 2009 45002 output packets dropped due to no
bufs, etc.
Mon Oct 5 13:15:00 AZST 2009 67161 output packets dropped due to no
bufs, etc.
Mon Oct 5 13:30:00 AZST 2009 112591 output packets dropped due to no
bufs, etc.
As you can see the drops gradually went away completely at about 4:00
a.m., and started coming up at about 10:30 a.m., although at a lower
rate, probably thanks to me bumping "ipfw ... queue NNN" up to 5000 at
10a.m. this morning. The traffic flow between 4a.m. and 10:30a.m., the
"quiet" times, is about 200-330 mbit/s 5 minute average, without a
single drop. But after that, in come the drops, no matter how high I set
the queue. Should I try 10000 slots? 20000? I'm really sure there are
plenty of heavy downloaders between 4a.m. and 10a.m., but still without
a single drop before approx. 330 mbit/s! Strange, isn't it? This makes
me believe I'm hitting some other global memory limit, rather than
per-user limit, at around 340-350 mbit/s, causing drops. top and netstat
-m are OK right now, though:
Mem: 1870M Active, 1220M Inact, 481M Wired, 201M Cache, 214M Buf, 164M Free
15595/5385/20980/111112 mbuf clusters in use (current/cache/total/max)
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
