On Feb 9, 2013, at 2:41 PM, Olivier Cochard-Labbé <oliv...@cochard.me> wrote:
> On Sat, Feb 9, 2013 at 11:19 AM, Daniel Kalchev <dan...@digsys.bg> wrote:
>>
>> By the way, I just checked two of my routers, and it seems I am reaching the
>> current BSDRP limit:
>>
>> router1:
>>
>> 229742/4768/234510 mbufs in use (current/cache/total)
>> 229320/3006/232326/262144 mbuf clusters in use (current/cache/total/max)
>> 229320/3000 mbuf+clusters out of packet secondary zone in use (current/cache)
>> 516096K/8320K/524416K bytes allocated to network (current/cache/total)
>> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
>> 0/0/0 requests for jumbo clusters denied (4k/9k/16k)
>>
>> router2:
>>
>> 229352/22678/252030 mbufs in use (current/cache/total)
>> 229324/18988/248312/262144 mbuf clusters in use (current/cache/total/max)
>> 229324/6321 mbuf+clusters out of packet secondary zone in use (current/cache)
>> 515986K/43645K/559632K bytes allocated to network (current/cache/total)
>> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
>>
>> Both have 12 igb interfaces.
>
>
> Hi Daniel,
>
> with 12 igb interfaces… you need a lot's of mbuf.
> But on your output I didn't see any "requests for mbufs denied".
>
> What's the output of:
> vmstat -z | head -1 ; vmstat -z | grep -i mbuf
> => If there is no "FAIL" counter, it should be OK
>
> What about the throughput (in pps) that you reach with your setup ?
>
These boxes are not very loaded. Both have single E5620 Xeon with 6GB of RAM
(3x2GB in order to utilise all channels). CPU load is never over 20% (all
cores).
# vmstat -z | head -1 ; vmstat -z | grep -i mbuf
ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP
mbuf_packet: 256, 0, 229323, 6322,166435162273, 0, 0
mbuf: 256, 0, 28, 16357,1602810218, 0, 0
mbuf_cluster: 2048, 262144, 235645, 12667,129037740, 0, 0
mbuf_jumbo_page: 4096, 12800, 0, 0, 0, 0, 0
mbuf_jumbo_9k: 9216, 6400, 0, 0, 0, 0, 0
mbuf_jumbo_16k: 16384, 3200, 0, 0, 0, 0, 0
mbuf_ext_refcnt: 4, 0, 0, 1176,79331737, 0, 0
I don't see any allocation falls, and the boxes generally never starve -- but
again, the load at the moment is light.
# netstat -w1
input (Total) output
packets errs idrops bytes packets errs bytes colls
166211 0 0 105125723 176973 0 137031820 0
163726 0 0 104487562 173713 0 135682449 0
170029 0 0 109345496 180797 0 142914734 0
157287 0 0 99326922 166606 0 127626241 0
155415 0 0 97220074 162638 0 122009528 0
157834 0 0 97700455 166001 0 124284638 0
152080 0 0 94515101 158383 0 117204095 0
158367 0 0 100541511 166076 0 127295652 0
But both boxes run full BGP with about dozen of peers and rather huge OSPF
network, both IPv4 and IPv6.
Unfortunately, these were installed years ago with vanilla FreeBSD build of my
own and just recently moved to BSDRP as it stabilised. But I am going to have
new spare box or two and will especially make sure it has 10G interfaces -- so
will be interested to test and optimise throughly that configuration.
Anyway, with regards to the number of mbufs, one has to keep in mind that
adding more interfaces will require more mbufs, or the box will just not
transmit any data after boot. Perhaps it is good idea to add this to the
documentation. It seems the current default is just about right for 12 igb
interfaces.
Daniel
------------------------------------------------------------------------------
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
_______________________________________________
Bsdrp-users mailing list
Bsdrp-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bsdrp-users
