Greetings, VPP folks. I am continuing to work on my vpp + router-plugin (+FRRouting) set-up.
I have things mostly working with very large routing tables (source from multiple BGP peers), but am having some challenges when trying to use multi-threaded (additional worker threads) for increasing overall VPP forwarding performance. When using just a single thread, the BGP peers take a long time to sync up but it is relatively stable. Forwarding performance on a 10G NIC (i40e driver and vfio-pci selected), is pretty decent, but I am interested in finding ways to improve performance (and getting things to the point where I can use a 40G card also in the system). The limit seems to be packets per second, and maxes out at about 11Mpps. The problem is, when I try to use worker threads, I start running into issues with rtnetlink buffers, and BGP, ICMP, ARP, etc, all become "flaky". My suspicion is that it has something to do with which thread(s) handle the netlink traffic, and which thread(s) handle the TCP port 179 (BGP) traffic, which needs to go via the tap-inject path to the kernel, and then to the BGP speaking application (FRR sub-unit "bgpd"). Is there anyone who can provide information or advice on this issue? NB: the flakiness is in a COMPLETELY unloaded environment - no other traffic is being handled, nothing else is consuming CPU cycles. It is just the BGP traffic itself plus related stuff (ARP) and any diagnostic traffic I use (ping). Is this a case where I need to adjust the RSS to direct incoming packets to the right subset of cores, and do I also need to direct particular traffic (TCP 179) to the main core? Do I need to ensure anything else, like using a separate core (and set the core afinity with taskset -c ) for my BGP speaker? Any suggestions or advice would be greatly appreciated. Also, any updates on bringing netlink and router plugins into the main vppsb tree? Building them on anything other than 18.07 just doesn't work for me, and even on 18.07 is rather brittle, and I'm not 100% sure about the build steps, which actually involve passing CFLAGS in to make, which suggests something isn't quite right... Thanks in advance, Brian
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11573): https://lists.fd.io/g/vpp-dev/message/11573 Mute This Topic: https://lists.fd.io/mt/28729084/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
