On 2023-06-29, Lyndon Nerenberg (VE7TFX/VE6BBM) <lyn...@orthanc.ca> wrote: > We are about to discover the joys of upstream BGP routing :-P The > current plan is to use a pair of OpenBSD+bgpd hosts as the routers. > > Each host will require 4x10gig ports (SFP+). One of those links > (to AWS) will be close to saturated, along with the downlink to our > switches. The other two will only need to carry ~1Gb/s of traffic. > > We are pretty much a Supermicro shop, and I'm wondering if anyone > out there is running a similar setup on SM hardware. My main concern > is finding NICs that will let us squeeze every last drop of bandwidth > on the 10gig links.
I don't need full 10G and haven't benchmarked anything recently, but Hrvoje has done a lot of testing in this area, see comments at https://marc.info/?l=openbsd-misc&m=167665861931266&w=2 For servers, look at the AMD boards e.g. M11SDV-based systems like https://www.supermicro.com/Aplus/system/Embedded/AS-5019D-FTN4.cfm Sadly Supermicro seem to have stopped doing boards with 4x fibre module slots, so you'll be stuck with needing PCIe NICs for the newer boards. (Newer xeon d boards have 2xSFP28 plus copper; networking on their AMD boards tend to be copper only). I would probably favour ix(4) i.e. X520 (for one thing, firmware is less of a moving target..) > I did run some brief ttcp tests on a pair of SM 1Us (don't have the > model number handy, maybe 5018-FTN4s?) with add-in Intel cards > (550s?) and was able to get 700 MBytes/s of throughput. This would > have been circa the 6.7 or 6.8 releases. A lot changed since then. See some stats over time at http://bluhm.genua.de/perform/results/perform.html (especially forwarding tests). Don't test packet generation on the box itself if you care about forwarding. Generate packets elsewhere and pass them through the device under test.
