Hi, We are using VPP for one of our products where we have used below VPP and DPDK versions for achieving packet flow between a router and this product/Network entity.
VPP-18.07 DPDK 19.11.2 The packet flow is all fine for a few days/months and all of sudden we see packets are unable to send out of this network element and below VPP counter is getting incremented(full show error output is attached). 19672 EthernetSwitch2/0/0-tx Tx packet drops (dpdk tx failure) Could you please share some inputs on why this error can come and packet drops in VPP? We also see that the incoming traffic to this network entity is fine. Only outgoing is affected and it recovers only if we reboot the system. We are not testing any large traffic flow, but only nominal traffic of about less than 1Gbps (supported is 10Gbps) is being tested. Please note this issue is seen very rarely and we can't reproduce at ease or we just have to wait for the system to come to this state. Regards, Girish
vppctl show node counters Count Node Reason 1 ip4-reassembly Reassembly Timeout 1 ip4-reassembly-expire-walk Reassembly Timeout 6468 ip4-lookup ip4 enqueued pkts drop - due to FAILED arp 955 ip4-lookup ip4 arp resolution failed - host -> fp 72567382 dpdk-crypto-input Crypto ops dequeued 72567382 dpdk-esp-decrypt ESP pkts received 3898657381 ipsec-input-ip4 IPSEC pkts received 49 ipsec-input-ip4 IPSec policy (no match) 3826089949 ipsec-input-ip4 IPSec policy bypass 1 ipsec-input-ip4 IPSec default policy 72567431 ipsec-input-ip4 IPSec received ESP before Decryption 28 ipsec-input-ip4 ESP Packet matching invalid policy index 21852736 ip4-reassembly valid ip4 packets 21852725 ip4-reassembly Reassembled Succesfully 3876804672 ip4-reassembly Non Fragmented 16 ip4-classify Classify misses 1 ethernet-input subinterface down 322718 arp-input ARP replies received 171696 arp-input ARP req to host 72567382 dpdk-esp-decrypt-post ESP post pkts 384270 ipsec-output-ip4 IPSec pkts received 384270 ipsec-output-ip4 IPSec policy bypass 72567382 ipsec-input-ip4 IPSEC pkts received 72567382 ipsec-input-ip4 IPSec policy protect 383318 ip4-frag packet smaller than MTU 383318 ip4-frag number of sent fragments 72567382 ip4-reassembly Non Fragmented 952 ip4-lookup ip4 enqueued pkts till arp resolve 1 ip4-lookup ip4 enqueued pkts drop - due to queue full 162 ip4-lookup ip4 arp stale Entry - fp -> host 75429233 ip4-classify Classify misses 3823228098 ip4-classify Classify hits 2836708 kni-receive Tx TCP packet 3256171078 kni-receive Tx TCP octet 70991412 kni-receive Tx SCTP packet 2950841788 kni-receive Tx SCTP octet 327044 kni-receive ARP req from Host 171695 kni-receive ARP resp from Host 70988586 dpdk-crypto-input Crypto ops dequeued 70988586 dpdk-esp-encrypt ESP pkts received 8 dpdk-input no error 3227149161 ipsec-output-ip4 IPSec pkts received 2 ipsec-output-ip4 IPSec policy (no match) 70988586 ipsec-output-ip4 IPSec policy protect 3156157747 ipsec-output-ip4 IPSec policy bypass 2826 ipsec-output-ip4 IPSec Failed SAE 3298128872 ip4-frag packet smaller than MTU 3298128910 ip4-frag number of sent fragments 6115 ip4-lookup ip4 enqueued pkts till arp resolve 70 ip4-lookup ip4 dequeued pkts after arp resolve 518 ip4-lookup ip4 enqueued pkts drop - due to queue full 3 ip4-lookup ip4 arp Resolved - host -> fp 326942 ip4-lookup ip4 arp stale Entry - fp -> host 19672 EthernetSwitch2/0/0-tx Tx packet drops (dpdk tx failure) vppctl show interface Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count EthernetSwitch1/0/6 1 up 10000/0/0/0 rx packets 37515381297 rx bytes 11589070399930 tx packets 46773499196 tx bytes 20695997472608 drops 3760080 EthernetSwitch2/0/0 2 up 1560/1560/1560/0 rx packets 46870677504 rx bytes 19405277934804 tx packets 37587760749 tx bytes 10559772160412 drops 21953041 rx-miss 2 tx-error 19672
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20894): https://lists.fd.io/g/vpp-dev/message/20894 Mute This Topic: https://lists.fd.io/mt/89341261/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-