Hi, >From your vcl configs it seems your fifo sizes are only 32kB. You should >probably increase those, for instance, try with 1MB.
If that’s not the issue, other suggestions: - try to pin with taskset iperf to a cpu on the same numa as your nic (check that with sh hardware) - dump the state of the sessions during the test “show session verbose 2” and see in the stats section if we’re heavily retransmitting and in the congestion section if for whatever reason the congestion control algorithm is not increasing the cwnd. - check the error counters to see if tcp or something else is dropping and show run report to see how busy your systems are. - if you’re not running master latest, make sure to change in startup.conf the default tcp congestion control algorithm to cubic For reference, here’s the output of iperf on my skylake testbed with XL710s (40Gbps) and 4MB fifos Connecting to host 6.0.1.1, port 5201 [ 33] local 6.0.1.2 port 17468 connected to 6.0.1.1 port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 33] 0.00-1.00 sec 4.36 GBytes 37.5 Gbits/sec 0 0.00 Bytes [ 33] 1.00-2.00 sec 4.37 GBytes 37.6 Gbits/sec 0 0.00 Bytes [ 33] 2.00-3.00 sec 4.37 GBytes 37.5 Gbits/sec 0 0.00 Bytes [ 33] 3.00-4.00 sec 4.37 GBytes 37.5 Gbits/sec 0 0.00 Bytes [ 33] 4.00-5.00 sec 4.37 GBytes 37.6 Gbits/sec 0 0.00 Bytes [ 33] 5.00-6.00 sec 4.37 GBytes 37.6 Gbits/sec 0 0.00 Bytes Regards, Florin > On Apr 18, 2020, at 3:12 AM, jiangxiaom...@outlook.com wrote: > > [Edited Message Follows] > > Iperf3 throughtput with vcl on vpp-20.05-rc0 is only half of vpp-20.01. > Below is the test topology: > > > +-------------+ +-------------+ > > |iperf3/Server| |iperf3/client| > > +-------------+ +-------------+ > > | | > > +-------------+ +-------------+ > > | VPP Server | | VPP Client | > > +-------------+ +-------------+ > > | | > > | | > > | | > > +------+ +------+ > > | Eth0 +-------- Connect ---------+ Eth1 | > > +------+ +------+ > > > VPP 20.05 Server: > > vpp# show version > > vpp v20.05-rc0 built by dev on localhost.localdomain at 2020-04-18T09:40:12 > > vpp# sh int addr > > eth0 (up): > > L3 192.168.7.100/24 > > local0 (dn): > > vpp# sh hard > > Name Idx Link Hardware > > eth0 1 up eth0 > > Link speed: 10 Gbps > > Ethernet address 90:e2:ba:93:2d:48 > > Intel 82599 > > carrier up full duplex mtu 1500 > > flags: admin-up pmd maybe-multiseg tx-offload intel-phdr-cksum > rx-ip4-cksum > > Devargs: > > VPP 20.05 Client: > vpp# show version > vpp v20.05-rc0 built by dev on localhost.localdomain at 2020-04-18T09:40:12 > vpp# sh int addr > eth0 (up): > L3 192.168.7.101/24 > local0 (dn): > vpp# sh hard > Name Idx Link Hardware > eth0 1 up eth0 > Link speed: 10 Gbps > Ethernet address 00:00:00:00:00:02 > Intel 82599 > carrier up full duplex mtu 1500 > flags: admin-up pmd maybe-multiseg tx-offload intel-phdr-cksum > rx-ip4-cksum > Devargs: > iperf server: > > sudo env \ > > LD_LIBRARY_PATH=/home/dev/net-base/script/../lib: \ > > VCL_CONFIG=/tmp/vcl-ts-96a9a.conf \ > > VCL_DEBUG=0 \ > > LDP_DEBUG=0 \ > > LDP_SID_BIT=9 > > LDP_TLS_CERT_FILE= \ > > LDP_TLS_KEY_FILE= \ > > LD_PRELOAD=/home/dev/net-base/script/../lib/libvcl_ldpreload.so \ > > iperf3 -4 -s 0.0.0.0 -p 1000 > > iperf client: > > sudo env \ > LD_LIBRARY_PATH=/home/dev/net-base/script/../lib: \ > VCL_CONFIG=/tmp/vcl-tc-bc511.conf \ > VCL_DEBUG=0 \ > LDP_DEBUG=0 \ > LDP_SID_BIT=9 > LDP_TLS_CERT_FILE= \ > LDP_TLS_KEY_FILE= \ > LD_PRELOAD=/home/dev/net-base/script/../lib/libvcl_ldpreload.so \ > iperf3 -4 -c 192.168.7.100 -p 1000 -t 30 > iperf3 client output with vpp 20.05-rc0: > > Connecting to host 192.168.7.100, port 1000 > > [513] local 192.168.7.101 port 4557 connected to 192.168.7.100 port 1000 > > [ ID] Interval Transfer Bandwidth Retr Cwnd > > [513] 0.00-1.00 sec 577 MBytes 4.84 Gbits/sec 0 0.00 Bytes > > [513] 1.00-2.00 sec 322 MBytes 2.70 Gbits/sec 0 0.00 Bytes > > [513] 2.00-3.00 sec 594 MBytes 4.99 Gbits/sec 0 0.00 Bytes > > [513] 3.00-4.00 sec 596 MBytes 5.00 Gbits/sec 0 0.00 Bytes > > [513] 4.00-5.00 sec 599 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 5.00-6.00 sec 452 MBytes 3.79 Gbits/sec 0 0.00 Bytes > > [513] 6.00-7.00 sec 599 MBytes 5.02 Gbits/sec 0 0.00 Bytes > > [513] 7.00-8.00 sec 449 MBytes 3.77 Gbits/sec 0 0.00 Bytes > > [513] 8.00-9.00 sec 599 MBytes 5.02 Gbits/sec 0 0.00 Bytes > > [513] 9.00-10.00 sec 600 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 10.00-11.00 sec 600 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 11.00-12.00 sec 599 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 12.00-13.00 sec 599 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 13.00-14.00 sec 600 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 14.00-15.00 sec 599 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 15.00-16.00 sec 599 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 16.00-17.00 sec 454 MBytes 3.81 Gbits/sec 0 0.00 Bytes > > [513] 17.00-18.00 sec 600 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 18.00-19.00 sec 599 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 19.00-20.00 sec 599 MBytes 5.02 Gbits/sec 0 0.00 Bytes > > [513] 20.00-21.00 sec 600 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 21.00-22.00 sec 600 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 22.00-23.00 sec 599 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 23.00-24.00 sec 599 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 24.00-25.00 sec 599 MBytes 5.02 Gbits/sec 0 0.00 Bytes > > [513] 25.00-26.00 sec 452 MBytes 3.79 Gbits/sec 0 0.00 Bytes > > [513] 26.00-27.00 sec 599 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 27.00-28.00 sec 600 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 28.00-29.00 sec 599 MBytes 5.03 Gbits/sec 0 0.00 Bytes > > [513] 29.00-30.00 sec 599 MBytes 5.02 Gbits/sec 0 0.00 Bytes > > - - - - - - - - - - - - - - - - - - - - - - - - - > > [ ID] Interval Transfer Bandwidth Retr > > [513] 0.00-30.00 sec 16.7 GBytes 4.78 Gbits/sec 0 sender > > [513] 0.00-30.00 sec 16.7 GBytes 4.78 Gbits/sec receiver > > > iperf3 client output with vpp 20.01: > > [513] local 192.168.7.101 port 10800 connected to 192.168.7.100 port 1000 > [ ID] Interval Transfer Bandwidth Retr Cwnd > > [513] 0.00-1.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 1.00-2.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 2.00-3.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 3.00-4.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 4.00-5.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 5.00-6.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 6.00-7.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 7.00-8.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 8.00-9.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 9.00-10.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 10.00-11.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 11.00-12.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 12.00-13.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 13.00-14.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 14.00-15.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 15.00-16.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 16.00-17.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 17.00-18.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 18.00-19.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 19.00-20.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 20.00-21.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 21.00-22.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 22.00-23.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 23.00-24.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 24.00-25.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 25.00-26.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 26.00-27.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 27.00-28.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 28.00-29.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > [513] 29.00-30.00 sec 1.10 GBytes 9.41 Gbits/sec 0 0.00 Bytes > > - - - - - - - - - - - - - - - - - - - - - - - - - > > [ ID] Interval Transfer Bandwidth Retr > > [513] 0.00-30.00 sec 32.9 GBytes 9.41 Gbits/sec 0 sender > > [513] 0.00-30.00 sec 32.9 GBytes 9.41 Gbits/sec receiver > > <vcl-tc-bc511.conf><vcl-ts-96a9a.conf>
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16109): https://lists.fd.io/g/vpp-dev/message/16109 Mute This Topic: https://lists.fd.io/mt/73112071/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
