Hey Seth, On Mon, Jan 26, 2015 at 8:38 AM, Seth Mos <[email protected]> wrote: > Sorry to reply to myself here, but 2.2 in combination with the Intel > X540-2 card isn't very stable. The card keeps dropping the Phy which is > fine on 2.1.5. >
That's surprising, we've seen much better results on our systems with ix cards, and I know of one system where they push near 2 Gbps across tens to hundreds of thousands of simultaneous connections through ix and have had great results where with 2.1.5 they were at times hitting the limit of what pf could push. They have a good deal of head room to spare now. You using any of those cards on stock FreeBSD? I think most of the Intel 10G cards we have are X520 rather than 540, which might be the difference. >> [zone: mbuf_jumbo_9k] kern.ipc.nmbjumbo9 limit reached >> ix1: Could not setup receive structures >> >> That didn't happen on 2.1.5 at all, apparently the limits have changed. >> >> In FreeBSD 10 these changes need to into loader.conf during boot, >> different from before. >> https://pleiades.ucsc.edu/hyades/FreeBSD_Network_Tuning >> >> kern.ipc.nmbclusters=262144 >> kern.ipc.nmbjumbop=262144 >> kern.ipc.nmbjumbo9=65536 >> kern.ipc.nmbjumbo16=32768 >> Some of that does like higher limits in 10.x. Though some of that actually isn't much different from 8.x/2.1.5, like I'm surprised you were getting by with < 262144 mbufs. We've taken to gnn@'s "million packet march" as they referred to it at a past employer of his, and recommended nmbclusters=1000000 in the NIC tuning guide. https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards#Intel_ix.284.29_Cards We've seen some bad behavior with low-ish nmbclusters on ix NICs which are similar to what you describe, I'm curious if things would be fine if you bumped nmbclusters to 1 million. You have a test setup where you could try that on the specific hardware you're using? _______________________________________________ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
