On Wed, Mar 8, 2017 at 6:28 AM, Mike Tancsa <m...@sentex.net> wrote: > On 3/7/2017 9:08 PM, Navdeep Parhar wrote: >> On Tue, Mar 7, 2017 at 5:46 PM, Mike Tancsa <m...@sentex.net> wrote: >> >>> >>> # dmesg | grep netm >>> netmap: loaded module >>> vcxl0: netmap queues/slots: TX 2/1023, RX 2/1024 >>> vcxl0: 1 txq, 1 rxq (NIC); 1 txq, 1 rxq (TOE); 2 txq, 2 rxq (netmap) >>> vcxl1: netmap queues/slots: TX 2/1023, RX 2/1024 >>> vcxl1: 1 txq, 1 rxq (NIC); 1 txq, 1 rxq (TOE); 2 txq, 2 rxq (netmap) >>> igb0: netmap queues/slots: TX 4/1024, RX 4/1024 >>> igb1: netmap queues/slots: TX 4/1024, RX 4/1024 >>> >>> It maxes out at about 800Kpps with and without netmap. Is there a way >> >> Are you actually using a netmap based application that acts as a >> packet router or is this just the vcxl interface running as a normal >> ifnet? > > the later, vcxl running normal ifnet. I thought there would be a benefit > to utilizing netmap ? Sorry, this is not clear to me.
The kernel's routing code does not utilize netmap even if it's available. You'll need something like netmap-fwd for netmap based routing. If you're not using netmap there is no need to create the extra vcxl interfaces. > >> >>> to increase the queues for the Chelsio nic, like the onboard igb ? >> >> If you're not running a netmap based router get rid of the num_vis=2 >> and simply try with the cxl0/cxl1 interfaces. They should each have 4 >> rxq/4 txq on your system. In case you want to increase the number of >> queues, use this: > > The tests with the regular cxl also show the box topping out at 0.8Mpps > for forwarding. I would have expected multiple streams to do better. There is a lot of information about forwarding on the bsdrp.net website. Have you tried the tips there? The numbers there are significantly better than what you observe. I suspect your router is CPU-bound. https://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_hp_proliant_dl360p_gen8_with_10-gigabit_with_10-gigabit_chelsio_t540-cr https://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_superserver_5018a-ftn4_with_10-gigabit_chelsio_t540-cr There's a projects/routing branch that does much better than the stock kernel. I'm not sure what work remains to be done before it can be merged into head. https://github.com/ocochard/netbenches/blob/master/Xeon_E5-2650-8Cores-Chelsio_T540-CR/forwarding-pf-ipfw/results/fbsd11-routing.r287531/README.md Regards, Navdeep > >> >> The "NIC" queues are the normal tx/rx queues, the "netmap" queues are >> active when the interface is in netmap mode. >> >> Does netsend generate a single flow or multiple flows? If it's a >> single flow it will use a single queue only. > > I think its as a single flow. However, I was using a separate box to > generate a second flow as well. It still topped out at about 800Kpps > before dropping packets. > > ---Mike > > > > -- > ------------------- > Mike Tancsa, tel +1 519 651 3400 > Sentex Communications, m...@sentex.net > Providing Internet services since 1994 www.sentex.net > Cambridge, Ontario Canada http://www.tancsa.com/ _______________________________________________ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
