Hi Christopher, This looks like an MTU issue from afar, the VPP config you have seems fine, but it's surprising that your `show hardware-interfaces` doesn't show the "carrier up ... mtu 9001" part for VirtualFunctionEthernet0/7/0 The "Link speed: unknown" part should be fine thought.
Maybe you can try running an iperf to another VirtualFunctionEthernet seeing if the problem still persist, and/or reducing the MTU on the iperf side to something smaller (e.g. 1500). You can also try to rebuild vpp on the latest master (or v21.10) to see if this fixes the issue. On the performance side of things, you shouldn't expect this much of an improvement regarding single flow throughput, as on AWS "Bandwidth for single-flow (5-tuple) traffic is limited to 5 Gbps, regardless of the destination of the traffic." [0]. But you should definitely be able to reach it with VPP. Hope this helps -Nathan [0] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-network-bandwidth.html Le lun. 29 nov. 2021 à 22:23, Christopher Adigun <future...@gmail.com> a écrit : > Hello, > > I am currently trying to run a UPF application via VPP, I decided to use > DPDK and Kubernetes but the speeds I am getting with iperf is not > realistic, I am getting 0 Mbps when I use the VPP as a gateway, if I remove > the VPP as gateway the throughput speed goes around 4Gbps > > iperf3 -c 10.0.7.167 -i 1 -t 10 > Connecting to host 10.0.7.167, port 5201 > [ 5] local 10.0.4.48 port 40466 connected to 10.0.7.167 port 5201 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 323 KBytes 2.65 Mbits/sec 7 8.74 KBytes > [ 5] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec 1 8.74 KBytes > [ 5] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes > [ 5] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec 1 8.74 KBytes > [ 5] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes > [ 5] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes > [ 5] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec 1 8.74 KBytes > [ 5] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes > [ 5] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes > [ 5] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec 0 8.74 KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 323 KBytes 265 Kbits/sec 10 > sender > [ 5] 0.00-10.00 sec 0.00 Bytes 0.00 bits/sec > receiver > > The kubernetes worker node is running on AWS (c5.4xlarge), I used sriov-dp > to add the DPDK interfaces into the pods. > > Below is my startup.conf > > unix { > nodaemon > log /tmp/vpp.log > full-coredump > gid vpp > interactive > cli-listen /run/vpp/cli.sock > exec /openair-upf/etc/init.conf > } > > api-trace { > on > } > > dpdk { > uio-driver vfio-pci > dev default { > num-rx-queues 1 > num-rx-queues 1 > } > dev 0000:00:06.0 > dev 0000:00:07.0 > dev 0000:00:08.0 > } > > api-segment { > gid vpp > } > > plugins { > path /usr/lib/x86_64-linux-gnu/vpp_plugins/ > plugin dpdk_plugin.so { enable } > plugin gtpu_plugin.so { disable } > plugin upf_plugin.so { enable } > } > > > Init conf: > > ip table add 1 > ip table add 2 > > set interface ip table VirtualFunctionEthernet0/6/0 1 > set interface mtu 9001 VirtualFunctionEthernet0/6/0 > set interface ip address VirtualFunctionEthernet0/6/0 10.0.4.11/24 > set interface state VirtualFunctionEthernet0/6/0 up > > set interface ip table VirtualFunctionEthernet0/7/0 0 > set interface mtu 9001 VirtualFunctionEthernet0/7/0 > set interface ip address VirtualFunctionEthernet0/7/0 10.0.6.11/24 > set interface state VirtualFunctionEthernet0/7/0 up > > set interface ip table VirtualFunctionEthernet0/8/0 2 > set interface mtu 9001 VirtualFunctionEthernet0/8/0 > set interface ip address VirtualFunctionEthernet0/8/0 10.0.7.11/24 > set interface state VirtualFunctionEthernet0/8/0 up > > ip route add 0.0.0.0/0 table 2 via 10.0.7.167 VirtualFunctionEthernet0/8/0 > > trace add dpdk-input 100 > > > First of all I noticed the following errors in the container logs: > > interface [error ]: hw_add_del_mac_address: > dpdk_add_del_mac_address: mac address add/del failed: -95 > interface [error ]: hw_add_del_mac_address: > dpdk_add_del_mac_address: mac address add/del failed: -95 > interface [error ]: hw_add_del_mac_address: > dpdk_add_del_mac_address: mac address add/del failed: -95 > interface [error ]: hw_add_del_mac_address: > dpdk_add_del_mac_address: mac address add/del failed: -95 > interface [error ]: hw_add_del_mac_address: > dpdk_add_del_mac_address: mac address add/del failed: -95 > interface [error ]: hw_add_del_mac_address: > dpdk_add_del_mac_address: mac address add/del failed: -95 > > format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95 > format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95 > format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95 > format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95 > format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95 > format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95 > format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95 > format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95 > format_dpdk_device:590: rte_eth_dev_rss_hash_conf_get returned -95 > > > I am a bit confused about the errors because I am able to see the correct > MAC address for all the DPDK interfaces in vpp console, is it safe not to > worry about them? > > vpp# show hardware-interfaces > Name Idx Link Hardware > VirtualFunctionEthernet0/6/0 1 up VirtualFunctionEthernet0/6/0 > Link speed: unknown > *Ethernet address 02:3d:a7:51:90:bc* > AWS ENA VF > carrier up full duplex mtu 9001 > flags: admin-up pmd rx-ip4-cksum > > VirtualFunctionEthernet0/7/0 2 up VirtualFunctionEthernet0/7/0 > Link speed: unknown > * Ethernet address 02:87:8d:0f:e2:20* > AWS ENA VF > > VirtualFunctionEthernet0/8/0 3 up VirtualFunctionEthernet0/8/0 > Link speed: unknown > * Ethernet address 02:20:55:04:c1:76* > AWS ENA VF > carrier up full duplex mtu 9001 > flags: admin-up pmd rx-ip4-cksum > > > [ec2-user@ip-10-0-0-53 ~]$ kubectl -n oai exec -ti > oai-vpp-upf-57d4fbdcb5-xnslq bin/vppctl show version > kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future > version. Use kubectl kubectl exec [POD] -- [COMMAND] instead. > vpp v21.01.1-release built by root on 4e3c1ed23a8e at 2021-11-25T16:47:55 > > [image: image.png] > > Ping tests are working normally on the VPP.. > > Also will the *Link speed: unknown* have affect it also? > > Any idea what could be causing the low iperf speed? > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20570): https://lists.fd.io/g/vpp-dev/message/20570 Mute This Topic: https://lists.fd.io/mt/87388694/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
