Hi Laurent, VPP interface and sub-interface come up in L3 mode by default, unless it is put into L2 mode for either bridging or cross connect:
DBGvpp# set interface l2 bridge ? set interface l2 bridge set interface l2 bridge <interface> <bridge-domain-id> [bvi|uu-fwd] [shg] DBGvpp# set interface l2 xconnect ? set interface l2 xconnect set interface l2 xconnect <interface> <peer interface> Thus, your setup has the interface in VirtualFunctionEthernet0/5/0.101 in L3 mode with its assigned IP/subnet as shown by your “sho interface address” output: set interface state VirtualFunctionEthernet0/5/0 up create sub-interfaces VirtualFunctionEthernet0/5/0 101 set interface state VirtualFunctionEthernet0/5/0.101 up set interface ip address VirtualFunctionEthernet0/5/0.101 100.100.101.2/24<http://100.100.101.2/24> How is the interface on TOR connected to VirtualFunctionEthernet0/5/0 configured? Is it in L3 mode with the same VLAN sub-interface configured with IP address of 100.100.101.1? If it is in L2 mode with VLANs, is there a host on that VLAN which owns IP address 100.100.100.1 which will respond to your ping. It would be helpful to get a packet trace using the following VPP CLI to show how packets are received by your DPDK interface and handled by VPP: clear trace trace add dpdk-input 10 show trace Regards, John From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Laurent Dumont Sent: Wednesday, May 20, 2020 10:55 AM To: Balaji Venkatraman (balajiv) <bala...@cisco.com> Cc: Mrityunjay Kumar <kumarn...@gmail.com>; vpp-dev <vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] VPP - DPDK - No ARP learning on VPP and no ARP reply sent. Yep And that's on VLAN 101. I just did a few tests and as soon as I remove the vlan from both sides, it starts working and I can ping across. Just to be sure this isn't me with a bad VPP configuration. Is the following enough to have basic L2 connectivity with a vlan? set interface state VirtualFunctionEthernet0/5/0 up create sub-interfaces VirtualFunctionEthernet0/5/0 101 set interface state VirtualFunctionEthernet0/5/0.101 up set interface ip address VirtualFunctionEthernet0/5/0.101 100.100.101.2/24<http://100.100.101.2/24> That's assuming that the TOR config is functional and correct as well :) Thanks! On Wed, May 20, 2020 at 1:46 AM Balaji Venkatraman (balajiv) <bala...@cisco.com<mailto:bala...@cisco.com>> wrote: Hi Laurent, Trying to understand your setup: Do you have : 100.100.101.x/24 TOR[.1] <-------------------->[.2] VLAN So, is the interface on TOR end also on a vlan (with same id)? -- Regards, Balaji. From: Laurent Dumont <laurentfdum...@gmail.com<mailto:laurentfdum...@gmail.com>> Date: Tuesday, May 19, 2020 at 4:58 PM To: "Balaji Venkatraman (balajiv)" <bala...@cisco.com<mailto:bala...@cisco.com>> Cc: Mrityunjay Kumar <kumarn...@gmail.com<mailto:kumarn...@gmail.com>>, vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> Subject: Re: [vpp-dev] VPP - DPDK - No ARP learning on VPP and no ARP reply sent. Hey everyone, Thank you for all the comments. Just trying to work my way through it! :) Just as a sanity check, here is what it looks like on a fresh VPP (without any config). # Configure the VPP client with the proper vlan and IP address. set interface state VirtualFunctionEthernet0/5/0 up create sub-interfaces VirtualFunctionEthernet0/5/0 101 set interface state VirtualFunctionEthernet0/5/0.101 up set interface ip address VirtualFunctionEthernet0/5/0.101 100.100.101.2/24<http://100.100.101.2/24> I then have the following : vpp# show interface address VirtualFunctionEthernet0/5/0 (up): VirtualFunctionEthernet0/5/0.101 (up): L3 100.100.101.2/24<http://100.100.101.2/24> local0 (dn): TOR IP : 100.100.101.1 - VLAN 101 vpp# ping 100.100.101.1 Statistics: 5 sent, 0 received, 100% packet loss vpp# show hardware-interfaces Name Idx Link Hardware VirtualFunctionEthernet0/5/0 1 up VirtualFunctionEthernet0/5/0 Link speed: 10 Gbps Ethernet address fa:16:3e:92:30:f1 Intel X710/XL710 Family VF carrier up full duplex mtu 9206 promisc flags: admin-up promisc pmd maybe-multiseg subif tx-offload intel-phdr-cksum rx-ip4-cksum Devargs: rx: queues 1 (max 16), desc 1024 (min 64 max 4096 align 32) tx: queues 1 (max 16), desc 1024 (min 64 max 4096 align 32) pci: device 8086:154c subsystem 103c:0000 address 0000:00:05.00 numa 0 max rx packet len: 9728 promiscuous: unicast on all-multicast on vlan offload: strip off filter on qinq off rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum qinq-strip outer-ipv4-cksum vlan-filter jumbo-frame scatter rx offload active: ipv4-cksum jumbo-frame scatter tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum sctp-cksum tcp-tso outer-ipv4-cksum qinq-insert vxlan-tnl-tso gre-tnl-tso ipip-tnl-tso geneve-tnl-tso multi-segs tx offload active: udp-cksum tcp-cksum multi-segs rss avail: ipv4-frag ipv4-tcp ipv4-udp ipv4-sctp ipv4-other ipv6-frag ipv6-tcp ipv6-udp ipv6-sctp ipv6-other l2-payload rss active: none tx burst function: i40e_xmit_pkts rx burst function: i40e_recv_scattered_pkts_vec_avx2 tx frames ok 5 tx bytes ok 230 rx frames ok 5 rx bytes ok 320 extended stats: rx good packets 5 tx good packets 5 rx good bytes 320 tx good bytes 230 rx bytes 320 rx unicast packets 5 tx bytes 230 tx broadcast packets 5 local0 0 down local0 Link speed: unknown local vpp# show interface address VirtualFunctionEthernet0/5/0 (up): VirtualFunctionEthernet0/5/0.101 (up): L3 100.100.101.2/24<http://100.100.101.2/24> local0 (dn): I can see that I have 5 packets IN (5 ARP IN and 5 replies I assume) No output from : vpp# show ip neighbor vpp# show ip neighbors vpp# Would that basic L3 configuration be enough to ping across the TOR to VPP? Thanks! On Fri, May 15, 2020 at 10:47 AM Balaji Venkatraman (balajiv) <bala...@cisco.com<mailto:bala...@cisco.com>> wrote: As Neale replied earlier, adding L3 addr to the interface should implicitly enable arp on it. Thanks! -- Regards, Balaji. From: Mrityunjay Kumar <kumarn...@gmail.com<mailto:kumarn...@gmail.com>> Date: Friday, May 15, 2020 at 7:19 AM To: "Balaji Venkatraman (balajiv)" <bala...@cisco.com<mailto:bala...@cisco.com>> Cc: Laurent Dumont <laurentfdum...@gmail.com<mailto:laurentfdum...@gmail.com>>, vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> Subject: Re: [vpp-dev] VPP - DPDK - No ARP learning on VPP and no ARP reply sent. Hi Balaji Its working for me. I was just trying to help "Laurent Dumont" -- @Laurent Dumont<mailto:laurentfdum...@gmail.com>, can try it, even this arp issue, is it working for you if you are adding neghbor IP? set ip neighbor set ip neighbor [del] <intfc> <ip-address> <mac-address> [static] [no-fib-entry] [count <count>] [fib-id <fib-id>] [proxy <lo-addr> - <hi-addr>] vpp# Please try it with adding route and arp. you can also use this, to set vlan on vf. #ip link set eth0 vf 18 valn 101 --> u not required to create vlan interface in vpp. Regards, Mrityunjay Kumar. Mobile: +91 - 9731528504 On Fri, May 15, 2020 at 6:59 PM Balaji Venkatraman (balajiv) <bala...@cisco.com<mailto:bala...@cisco.com>> wrote: Hi Mrityunjay, Could you try adding an ip route and recheck. I think ARP is enabled once ip routing is enabled. Thanks -- Regards, Balaji. From: <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> on behalf of Laurent Dumont <laurentfdum...@gmail.com<mailto:laurentfdum...@gmail.com>> Date: Friday, May 15, 2020 at 4:57 AM To: Mrityunjay Kumar <kumarn...@gmail.com<mailto:kumarn...@gmail.com>> Cc: vpp-dev <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> Subject: Re: [vpp-dev] VPP - DPDK - No ARP learning on VPP and no ARP reply sent. Hi everyone! Thank you for the tips The VF is created with the Trust On and spoofcheck disabled during the instance creation. vf 18 MAC fa:16:3e:92:30:f1, spoof checking off, link-state auto, trust on I had a change to test this a bit more. It looks like the issue is specifically with passing tagged traffic between the TOR and VPP. I've added the VLAN101 on the TOR and the following on the VPP side. set interface state VirtualFunctionEthernet0/5/0 up create sub-interfaces VirtualFunctionEthernet0/5/0 101 set interface state VirtualFunctionEthernet0/5/0.101 up set interface ip address VirtualFunctionEthernet0/5/0.101 100.100.101.2/24<http://100.100.101.2/24> vpp# show int address VirtualFunctionEthernet0/5/0 (up): VirtualFunctionEthernet0/5/0.101 (up): L3 100.100.101.2/24<http://100.100.101.2/24> local0 (dn): But now, if I look at the packets in error. vpp# show error Count Node Reason 5 dpdk-input no error 451 arp-disabled ARP Disabled on this interface Is there a way to "enable" ARP on the l3 interface? On Fri, May 15, 2020 at 1:59 AM Mrityunjay Kumar <kumarn...@gmail.com<mailto:kumarn...@gmail.com>> wrote: This issue, we have seen on centos as well. Just for work around, ip link set promisc on eth0 #[eth0 --- PF interface] ip link set eth00 vf 0 spoofchk off #[give your vf id ] ip link set eth0 vf 0 trust on, , But you will see other packet on vpp VF interface as well. Can you try with IPv6, you will see more surprises. :) //MJ Regards, Mrityunjay Kumar. Mobile: +91 - 9731528504 On Fri, May 15, 2020 at 7:55 AM Laurent Dumont <laurentfdum...@gmail.com<mailto:laurentfdum...@gmail.com>> wrote: Hi! I was doing some initial experimentation with VPP/SRIOV/DPDK and I just wanted to see if some of the things I was experiencing we're expected. I was looking at understanding the baseline behavior for something like VPP + DPDK. I have a small test POC with the following topology. VM Ubuntu 18.04( with the "linux-image-extra-virtual" package) ---> (VPP + DPDK + IP address on the VPP interface) ---> VF ---> PF (compute with SRIOV on X710 cards) ---> TOR interface - untagged. What I am seeing : 1. VPP is telling me I have both RX and TX packets. 2. That said, no ARP entry for the TOR is seen on VPP 3. The interface on the TOR is also not seeing any ARP entry for the VPP interface. The TOR is also indicating that RX and TX packets are present. Are there any inherent limitations to the ARP with VPP/DPDK? One weird thing is that trying to show the ARP from VPP doesn't seem to be a valid command : vpp# sh ip arp show ip: unknown input `arp' Here are some more debugging output. vpp# show version vpp v20.01-release built by root on 980ae64453f3 at 2020-01-29T22:13:47 vpp# show int address VirtualFunctionEthernet0/5/0 (up): L3 1.2.3.4/31<http://1.2.3.4/31> vpp# show hardware-interfaces Name Idx Link Hardware VirtualFunctionEthernet0/5/0 1 up VirtualFunctionEthernet0/5/0 Link speed: 10 Gbps Ethernet address fa:16:3e:92:30:f1 Intel X710/XL710 Family VF carrier up full duplex mtu 9206 flags: admin-up pmd maybe-multiseg tx-offload intel-phdr-cksum rx-ip4-cksum Devargs: rx: queues 1 (max 16), desc 1024 (min 64 max 4096 align 32) tx: queues 1 (max 16), desc 1024 (min 64 max 4096 align 32) pci: device 8086:154c subsystem 103c:0000 address 0000:00:05.00 numa 0 max rx packet len: 9728 promiscuous: unicast off all-multicast on vlan offload: strip off filter off qinq off rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum qinq-strip outer-ipv4-cksum vlan-filter jumbo-frame scatter rx offload active: ipv4-cksum jumbo-frame scatter tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum sctp-cksum tcp-tso outer-ipv4-cksum qinq-insert vxlan-tnl-tso gre-tnl-tso ipip-tnl-tso geneve-tnl-tso multi-segs tx offload active: udp-cksum tcp-cksum multi-segs rss avail: ipv4-frag ipv4-tcp ipv4-udp ipv4-sctp ipv4-other ipv6-frag ipv6-tcp ipv6-udp ipv6-sctp ipv6-other l2-payload rss active: none tx burst function: i40e_xmit_pkts rx burst function: i40e_recv_scattered_pkts_vec_avx2 tx frames ok 36 tx bytes ok 1512 rx frames ok 26 rx bytes ok 3014 extended stats: rx good packets 26 tx good packets 36 rx good bytes 3014 tx good bytes 1512 rx bytes 3014 rx broadcast packets 26 tx bytes 1512 tx broadcast packets 36 local0 0 down local0 Link speed: unknown local vpp# show interface Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count VirtualFunctionEthernet0/5/0 1 up 9000/0/0/0 rx packets 26 rx bytes 2910 tx packets 36 tx bytes 1512 drops 14 ip4 25 local0 0 down 0/0/0/0 drops 25 vpp# RP/0/RP0/CPU0:tor-4.qa-site1#show run int tenGigE 0/0/0/37 Fri May 15 02:22:57.640 UTC interface TenGigE0/0/0/37 mtu 9216 ipv4 address 1.2.3.5 255.255.255.252 load-interval 30 !
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16462): https://lists.fd.io/g/vpp-dev/message/16462 Mute This Topic: https://lists.fd.io/mt/74220020/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
