Hi, Chakri,
You can change the VPP worker assignments by doing:
vpp# show interface rx-placement
vpp# set interface rx-placement ?
set interface rx-placement set interface rx-placement
<interface> [queue <n>] [worker <n> | main]
Thanks very much!
Regards,
Yichen
From: <vpp-dev@lists.fd.io> on behalf of "chakravarthy.arise...@viasat.com"
<chakravarthy.arise...@viasat.com>
Date: Wednesday, August 1, 2018 at 11:22 AM
To: "vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] tx-errors on VPP controlled dpdk device
Hi Damjan,
Thanks for your feedback. I'm running the test in AWS instances. Thus, I have
got only VFs. I do not have access to PF. So, I'm trying to get help from AWS
to find out.
Once I get the info, I'll post it over here. In the mean time, I looked at the
counters that you suggested me to focus on. It looks like the packets are
scheduled on only one core in transmit direction. Is there a way to change?
I have 3 dedicated cores (1 main core thread for stats/mgmt and 2 cores for the
worker threads). All the Tx queues are pinned to worker thread 1. So, worker
thread 2 is not used for transmit path at all. Is there way to spread the
transmit queues across the threads?
Thanks
Chakri
vpp# sh threads
ID Name Type LWP Sched Policy (Priority)
lcore Core Socket State
0 vpp_main 1733 other (0) 1
1 0
1 vpp_wk_0 workers 1745 other (0) 2
2 0
2 vpp_wk_1 workers 1746 other (0) 3
3 0
3 stats 1747 other (0)
0 0 0
vpp# sh run
Thread 0 vpp_main (lcore 1)
Time 5125.9, average vectors/node 0.00, last 128 main loops 0.00 per node 0.00
vector rates in 0.0000e0, out 0.0000e0, drop 0.0000e0, punt 0.0000e0
Name State Calls Vectors
Suspends Clocks Vectors/Call
api-rx-from-ring any wait 0 0
364 1.19e4 0.00
cdp-process any wait 0 0
992 1.98e3 0.00
dhcp-client-process any wait 0 0
51 3.41e3 0.00
dns-resolver-process any wait 0 0
5 4.06e3 0.00
dpdk-process any wait 0 0
1709 5.13e4 0.00
fib-walk any wait 0 0
2563 1.37e3 0.00
ikev2-manager-process any wait 0 0
5124 7.25e2 0.00
ip-route-resolver-process any wait 0 0
51 2.64e3 0.00
ip4-reassembly-expire-walk any wait 0 0
513 3.85e3 0.00
ip6-icmp-neighbor-discovery-ev any wait 0 0
5124 6.92e2 0.00
ip6-reassembly-expire-walk any wait 0 0
513 3.84e3 0.00
lisp-retry-service any wait 0 0
2563 1.57e3 0.00
memif-process any wait 0 0
1709 2.10e3 0.00
rd-cp-process any wait 0 0
237212380 3.21e2 0.00
unix-cli-local:17 active 0 0
580 2.05e5 0.00
unix-epoll-input polling 96172305 0
0 1.19e4 0.00
vpe-oam-process any wait 0 0
2513 1.23e3 0.00
---------------
Thread 1 vpp_wk_0 (lcore 2)
Time 5125.9, average vectors/node 4.82, last 128 main loops 0.00 per node 0.00
vector rates in 9.5578e3, out 8.4052e3, drop 0.0000e0, punt 0.0000e0
Name State Calls Vectors
Suspends Clocks Vectors/Call
VirtualFunctionEthernet0/6/0-o active 91 91
0 8.59e2 1.00
VirtualFunctionEthernet0/6/0-t active 91 91
0 2.82e3 1.00
VirtualFunctionEthernet0/7/0-o active 5334164 32661561
0 4.33e1 6.12
VirtualFunctionEthernet0/7/0-t active 5334164 26753703
0 3.83e2 5.02
arp-input active 182 182
0 7.25e3 1.00
dpdk-input polling 16550217513 16330917
0 4.05e5 0.00
ethernet-input active 5334255 32661652
0 7.97e1 6.12
interface-output active 182 182
0 6.58e2 1.00
ip4-input active 4685453 16330735
0 9.48e1 3.49
ip4-load-balance active 5334073 32661470
0 4.85e1 6.12
ip4-local active 4685453 16330735
0 9.92e1 3.49
ip4-lookup active 4685453 16330735
0 1.05e2 3.49
ip4-rewrite active 5334073 32661470
0 5.57e1 6.12
ip4-udp-lookup active 4685453 16330735
0 8.96e1 3.49
l2-fwd active 10019526 48992205
0 5.56e1 4.89
l2-input active 10019526 48992205
0 6.03e1 4.89
l2-learn active 10019526 48992205
0 6.75e1 4.89
l2-output active 10019526 48992205
0 6.16e1 4.89
memif-input polling 16550217513 32661470
0 2.33e5 0.00
unix-epoll-input polling 1817493 0
0 1.18e4 0.00
vxlan4-encap active 5334073 32661470
0 1.09e2 6.12
vxlan4-input active 4685453 16330735
0 1.19e2 3.49
memif1/1-output active 4685453 16330735
0 1.34e2 3.49
memif1/1-tx active 4685453 16330735
0 1.53e3 3.49
---------------
Thread 2 vpp_wk_1 (lcore 3)
Time 5125.9, average vectors/node 1.67, last 128 main loops 0.00 per node 0.00
vector rates in 3.1859e3, out 3.1859e3, drop 0.0000e0, punt 0.0000e0
Name State Calls Vectors
Suspends Clocks Vectors/Call
dpdk-input polling 16679496489 16330735
0 4.24e5 0.00
ip4-input active 9785099 16330735
0 1.47e2 1.67
ip4-local active 9785099 16330735
0 1.33e2 1.67
ip4-lookup active 9785099 16330735
0 1.25e2 1.67
ip4-udp-lookup active 9785099 16330735
0 1.18e2 1.67
l2-fwd active 9785099 16330735
0 1.06e2 1.67
l2-input active 9785099 16330735
0 1.32e2 1.67
l2-learn active 9785099 16330735
0 1.31e2 1.67
l2-output active 9785099 16330735
0 9.05e1 1.67
memif-input polling 16679496489 0
0 4.38e2 0.00
unix-epoll-input polling 1130721 0
0 1.14e4 0.00
vxlan4-input active 9785099 16330735
0 1.45e2 1.67
memif1/1-output active 9785099 16330735
0 1.01e2 1.67
memif1/1-tx active 9785099 16330735
0 2.09e3 1.67
vpp# sh hardware
[80/1816]
Name Idx Link Hardware
VirtualFunctionEthernet0/6/0 1 up VirtualFunctionEthernet0/6/0
Ethernet address 06:3a:20:ff:aa:d0
AWS ENA VF
carrier up full duplex speed 10000 mtu 9216
rx queues 2, rx desc 1024, tx queues 3, tx desc 1024
cpu socket 0
tx frames ok 2
tx bytes ok 84
rx frames ok 21146632
rx bytes ok 33665435044
extended stats:
rx good packets 21146632
tx good packets 2
rx good bytes 33665435044
tx good bytes 84
VirtualFunctionEthernet0/7/0 2 up VirtualFunctionEthernet0/7/0
Ethernet address 06:90:5e:ca:8f:6c
AWS ENA VF
carrier up full duplex speed 10000 mtu 9216
rx queues 2, rx desc 1024, tx queues 3, tx desc 1024
cpu socket 0
tx frames ok 17322383
tx bytes ok 27577230636
rx frames ok 2
rx bytes ok 84
extended stats:
rx good packets 2
tx good packets 17322383
rx good bytes 84
tx good bytes 27577230636
local0 0 down local0
local
loop1 3 up loop1
Ethernet address de:ad:00:00:00:01
loop2 5 up loop2
Ethernet address de:ad:00:00:00:02
memif1/1 7 up memif1/1
Ethernet address 02:fe:95:70:02:bc
MEMIF interface
instance 0
memif2/2 8 up memif2/2
Ethernet address 02:fe:6d:04:8f:40
MEMIF interface
instance 1
vxlan_tunnel1 4 up vxlan_tunnel1
VXLAN
vxlan_tunnel2 6 up vxlan_tunnel2
VXLAN
vpp# show int
Name Idx State Counter Count
VirtualFunctionEthernet0/6/0 1 up rx packets
21146633
rx bytes
33665435086
tx packets
3
tx bytes
126
ip4
21146630
VirtualFunctionEthernet0/7/0 2 up rx packets
3
rx bytes
126
tx packets
21146633
tx bytes
33665435086
tx-error
3824249
local0 0 up
loop1 3 up
loop2 5 up
memif1/1 7 up tx packets
21146630
tx bytes
32608103460
memif2/2 8 up rx packets
21146630
rx bytes
32608103460
vxlan_tunnel1 4 up rx packets
21146630
rx bytes
32608103460
vxlan_tunnel2 6 up tx packets
21146630
tx bytes
33369382140
vpp# show error
Count Node Reason
120878944 vxlan4-input good packets decapsulated
241757861 vxlan4-encap good packets encapsulated
362636805 l2-output L2 output packets
362636805 l2-learn L2 learn packets
362636805 l2-input L2 input packets
2616 arp-input ARP replies sent
8 arp-input ARP probe or announcement
dropped
18841192 VirtualFunctionEthernet0/7/0-tx Tx packet drops (dpdk tx
failure)
120878917 vxlan4-input good packets decapsulated
120878917 l2-output L2 output packets
120878917 l2-learn L2 learn packets
120878917 l2-input L2 input packets
Startup config snippet
=================
dev 0000:00:06.0 {
num-rx-queues 2
num-rx-desc 1024
num-tx-desc 1024
}
dev 0000:00:07.0 {
num-rx-queues 2
num-rx-desc 1024
num-tx-desc 1024
}
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#10013): https://lists.fd.io/g/vpp-dev/message/10013
Mute This Topic: https://lists.fd.io/mt/23982730/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
