Hoi folks, I did a deep dive today on VXLAN, GENEVE and compared it to GRE and L2XC - the full read is here: https://ipng.ch/s/articles/2022/01/13/vpp-l2.html
One thing that I observed is that both VXLAN and GENEVE use static source ports. In the case of VLLs, (an l2 xconnect from a customer ethernet interface into a tunnel), the customer port will be receiving IPv4 or IPv6 traffic (either tagged or untagged) and this allows the NIC to use RSS to assign this inbound traffic to multiple queues, and thus multiple CPU threads. That’s great, it means linear encapsulation performance. However,, once the traffic is encapsulated, it’ll become single flow with respect to the remote host, ie we're sending from 10.0.0.0:4789 to the remote 10.0.0.1:4789 and it is for this reason, that all decapsulation is single threaded. One common approach is to use an ingress hash algorithm to choose from a pool of source ports, or possibly a simpler round-robin over a pool of ports 4000-5000, say, based on the inner payload. That way, the remote would be able to use multiple RSS queues. However, VPP currently does not implement that. I think the original author has this in mind as a future improvement based on the comment on L295 in vxlan.c /* UDP header, randomize src port on something, maybe? */ udp->src_port = clib_host_to_net_u16 (t->src_port); udp->dst_port = clib_host_to_net_u16 (t->dst_port); What would it take for src_port to not be static? It would greatly improve VXLAN (and similarly, GENEVE) throughput on ingress. groet, Pim -- Pim van Pelt <p...@ipng.nl> PBVP1-RIPE - http://www.ipng.nl/
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20708): https://lists.fd.io/g/vpp-dev/message/20708 Mute This Topic: https://lists.fd.io/mt/88408739/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
