I tried disabling autoneg on R740 side. It is not allowed too. If vpp cannot allow two nics to be successfully added to the same vpp instance, it seems to be a bug. Is it something which can be easily spotted in the code base?
It is also not possible to enforce symmetricity on internet. The other party can do anything as long as basic ping works. On Thu, Oct 17, 2019 at 3:55 PM Chuan Han <chuan...@google.com> wrote: > If I only put one phy nic, i.e., eth0, to vpp, 'sh hardware' shows it is > up. If I put both eth0 and eth1 in vpp, eth0 is always down. It seems > something is wrong with the nic or vpp does not support this type of > hardware? > > We tried enabling autoneg on R230. It is not allowed. To avoid asymmetric > settings, disabling autoneg on R740 will help? > > On Thu, Oct 17, 2019 at 3:46 PM Balaji Venkatraman (balajiv) < > bala...@cisco.com> wrote: > >> It plays a role if it is asymmetric at both ends. You could enable it at >> both ends and check. >> >> On Oct 17, 2019, at 3:15 PM, Chuan Han <chuan...@google.com> wrote: >> >> >> I rebooted the r230 machine and found the phy nic corresponding to eth >> has autoneg off. >> >> root@esdn-relay:~/gnxi/perf_testing/r230# ethtool enp6s0f1 >> Settings for enp6s0f1: >> Supported ports: [ FIBRE ] >> Supported link modes: 10000baseT/Full >> Supported pause frame use: Symmetric >> Supports auto-negotiation: No >> Supported FEC modes: Not reported >> Advertised link modes: 10000baseT/Full >> Advertised pause frame use: Symmetric >> Advertised auto-negotiation: No >> Advertised FEC modes: Not reported >> Speed: 10000Mb/s >> Duplex: Full >> * Port: Direct Attach Copper* >> PHYAD: 0 >> Transceiver: internal >> * Auto-negotiation: off* >> Supports Wake-on: d >> Wake-on: d >> Current message level: 0x00000007 (7) >> drv probe link >> Link detected: yes >> root@esdn-relay:~/gnxi/perf_testing/r230# >> >> On r740, autoneg is on. It is copper. >> >> root@esdn-lab:~/gnxi/perf_testing/r740/vpp# ethtool eno3 >> Settings for eno3: >> Supported ports: [ TP ] >> Supported link modes: 100baseT/Full >> 1000baseT/Full >> 10000baseT/Full >> Supported pause frame use: Symmetric >> Supports auto-negotiation: Yes >> Supported FEC modes: Not reported >> Advertised link modes: 100baseT/Full >> 1000baseT/Full >> 10000baseT/Full >> Advertised pause frame use: Symmetric >> Advertised auto-negotiation: Yes >> Advertised FEC modes: Not reported >> Speed: 10000Mb/s >> Duplex: Full >> * Port: Twisted Pair* >> PHYAD: 0 >> Transceiver: internal >> * Auto-negotiation: on* >> MDI-X: Unknown >> Supports Wake-on: umbg >> Wake-on: g >> Current message level: 0x00000007 (7) >> drv probe link >> Link detected: yes >> root@esdn-lab:~/gnxi/perf_testing/r740/vpp# >> >> not clear if this plays a role or not. >> >> On Thu, Oct 17, 2019 at 2:41 PM Chuan Han via Lists.Fd.Io <chuanhan= >> google....@lists.fd.io> wrote: >> >>> Restarting ixia controller does not help. We ended up with both ixia >>> ports having '!'. >>> >>> We are not sure how ixia port plays a role here. eth0 interfaces are the >>> interfaces connecting two servers, not to ixia. >>> >>> On Thu, Oct 17, 2019 at 11:26 AM Balaji Venkatraman (balajiv) < >>> bala...@cisco.com> wrote: >>> >>>> Hi Chuan, >>>> >>>> >>>> >>>> Could you please try to reset the ixia controller connected to port 4? >>>> >>>> I have seen issues with ‘!’ on ixia. Given the carrier on eth0 is >>>> down, I suspect the ixia port. >>>> >>>> >>>> >>>> -- >>>> >>>> Regards, >>>> >>>> Balaji. >>>> >>>> >>>> >>>> >>>> >>>> *From: *Chuan Han <chuan...@google.com> >>>> *Date: *Thursday, October 17, 2019 at 11:09 AM >>>> *To: *"Balaji Venkatraman (balajiv)" <bala...@cisco.com> >>>> *Cc: *"vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io>, Arivudainambi >>>> Appachi gounder <aappa...@google.com>, Jerry Cen <zhiw...@google.com> >>>> *Subject: *Re: [vpp-dev] Basic l2 bridging does not work >>>> >>>> >>>> >>>> Yes. It is unidirectional stream from port 1 to port 4. >>>> >>>> >>>> >>>> Another engineer, Nambi, configured ixia. What he showed me yesterday >>>> is that xia port connected to port 1 is green and good. ixia port connected >>>> to port 4 is green but has a red exclamation mark, which means ping does >>>> not work. >>>> >>>> >>>> >>>> We also found eth0 on R230 is down shown by "show hardware eth0" >>>> command. However "show int" shows it is up. >>>> >>>> >>>> >>>> >>>> >>>> vpp# sh hardware-interfaces eth0 >>>> Name Idx Link Hardware >>>> eth0 2 down eth0 >>>> Link speed: unknown >>>> Ethernet address b4:96:91:23:1e:d6 >>>> Intel 82599 >>>> carrier down >>>> flags: admin-up promisc pmd rx-ip4-cksum >>>> rx: queues 1 (max 128), desc 512 (min 32 max 4096 align 8) >>>> tx: queues 3 (max 64), desc 512 (min 32 max 4096 align 8) >>>> pci: device 8086:154d subsystem 8086:7b11 address 0000:06:00.01 >>>> numa 0 >>>> max rx packet len: 15872 >>>> promiscuous: unicast on all-multicast on >>>> vlan offload: strip off filter off qinq off >>>> rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum >>>> tcp-lro >>>> macsec-strip vlan-filter vlan-extend jumbo-frame >>>> scatter >>>> security keep-crc >>>> rx offload active: ipv4-cksum >>>> tx offload avail: vlan-insert ipv4-cksum udp-cksum tcp-cksum >>>> sctp-cksum >>>> tcp-tso macsec-insert multi-segs security >>>> tx offload active: none >>>> rss avail: ipv4-tcp ipv4-udp ipv4 ipv6-tcp-ex ipv6-udp-ex >>>> ipv6-tcp >>>> ipv6-udp ipv6-ex ipv6 >>>> rss active: none >>>> tx burst function: (nil) >>>> rx burst function: ixgbe_recv_pkts_vec >>>> >>>> rx frames ok 33278 >>>> rx bytes ok 3960082 >>>> extended stats: >>>> rx good packets 33278 >>>> rx good bytes 3960082 >>>> rx q0packets 33278 >>>> rx q0bytes 3960082 >>>> rx size 65 to 127 packets 33278 >>>> rx multicast packets 33278 >>>> rx total packets 33278 >>>> rx total bytes 3960082 >>>> vpp# sh int >>>> Name Idx State MTU (L3/IP4/IP6/MPLS) >>>> Counter Count >>>> eth0 2 up 9000/0/0/0 rx >>>> packets 33279 >>>> rx >>>> bytes 3960201 >>>> >>>> drops 5 >>>> >>>> punt 1 >>>> >>>> tx-error 33274 >>>> eth1 1 up 9000/0/0/0 rx >>>> packets 33274 >>>> rx >>>> bytes 3959606 >>>> tx >>>> packets 33273 >>>> tx >>>> bytes 3959487 >>>> >>>> drops 33274 >>>> >>>> tx-error 3 >>>> local0 0 down 0/0/0/0 >>>> vpp# >>>> >>>> >>>> >>>> On Thu, Oct 17, 2019 at 10:54 AM Balaji Venkatraman (balajiv) < >>>> bala...@cisco.com> wrote: >>>> >>>> Hi Chuan, >>>> >>>> >>>> >>>> I assume u have unidirectional stream ? ixia->1->2->3->4->ixia? >>>> >>>> >>>> >>>> vpp# sh int >>>> Name Idx State MTU (L3/IP4/IP6/MPLS) >>>> Counter Count >>>> eth0 2 up 9000/0/0/0 rx >>>> packets 30925 >>>> rx >>>> bytes 3680075 >>>> >>>> drops 5 >>>> >>>> punt 1 >>>> >>>> tx-error 30920 >>>> eth1 1 up 9000/0/0/0 rx >>>> packets 30920 <<< packets are received on port 3 >>>> rx >>>> bytes 3679480 >>>> tx >>>> packets 30919 >>>> tx >>>> bytes 3679361 >>>> >>>> drops 30920 <<< all dropped at port 3 >>>> >>>> tx-error 3 >>>> local0 0 down 0/0/0/0 >>>> >>>> >>>> >>>> On sh error logs on R 230 we see >>>> >>>> 1 ethernet-input l3 mac mismatch <<<< >>>> 3 eth1-output interface is down >>>> 30922 eth0-output interface is down >>>> >>>> >>>> >>>> >>>> >>>> Do u see the arp getting resolved on ixia? The mac on ixia at port with >>>> 172.16.1.2/24 should be seen on its other port. Are the ixia ports up >>>> at both ends? >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Regards, >>>> >>>> Balaji. >>>> >>>> >>>> >>>> >>>> >>>> *From: *<vpp-dev@lists.fd.io> on behalf of "Chuan Han via Lists.Fd.Io" >>>> <chuanhan=google....@lists.fd.io> >>>> *Reply-To: *"chuan...@google.com" <chuan...@google.com> >>>> *Date: *Thursday, October 17, 2019 at 9:59 AM >>>> *To: *"Balaji Venkatraman (balajiv)" <bala...@cisco.com> >>>> *Cc: *"vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io> >>>> *Subject: *Re: [vpp-dev] Basic l2 bridging does not work >>>> >>>> >>>> >>>> It seems R740 vpp works fine. All packets coming from port 1 go to port >>>> 2. >>>> >>>> >>>> >>>> vpp# sh int >>>> Name Idx State MTU (L3/IP4/IP6/MPLS) >>>> Counter Count >>>> eth0 2 up 9000/0/0/0 tx >>>> packets 30895 >>>> tx >>>> bytes 3676505 >>>> eth1 1 up 9000/0/0/0 rx >>>> packets 30895 >>>> rx >>>> bytes 3676505 >>>> local0 0 down 0/0/0/0 >>>> vpp# sh int >>>> Name Idx State MTU (L3/IP4/IP6/MPLS) >>>> Counter Count >>>> eth0 2 up 9000/0/0/0 tx >>>> packets 30897 >>>> tx >>>> bytes 3676743 >>>> eth1 1 up 9000/0/0/0 rx >>>> packets 30897 >>>> rx >>>> bytes 3676743 >>>> local0 0 down 0/0/0/0 >>>> vpp# sh error >>>> Count Node Reason >>>> 30899 l2-output L2 output packets >>>> 30899 l2-learn L2 learn packets >>>> 1 l2-learn L2 learn misses >>>> 30899 l2-input L2 input packets >>>> 30899 l2-flood L2 flood packets >>>> vpp# >>>> >>>> >>>> >>>> The drop happened on R230 vpp. Port 3 dropped all pkts complaining >>>> about down interface. However, show command shows interfaces are up. >>>> >>>> >>>> >>>> vpp# sh int >>>> Name Idx State MTU (L3/IP4/IP6/MPLS) >>>> Counter Count >>>> eth0 2 up 9000/0/0/0 rx >>>> packets 30925 >>>> rx >>>> bytes 3680075 >>>> >>>> drops 5 >>>> >>>> punt 1 >>>> >>>> tx-error 30920 >>>> eth1 1 up 9000/0/0/0 rx >>>> packets 30920 >>>> rx >>>> bytes 3679480 >>>> tx >>>> packets 30919 >>>> tx >>>> bytes 3679361 >>>> >>>> drops 30920 >>>> >>>> tx-error 3 >>>> local0 0 down 0/0/0/0 >>>> vpp# sh error >>>> Count Node Reason >>>> 2 llc-input unknown llc ssap/dsap >>>> 61846 l2-output L2 output packets >>>> 61846 l2-learn L2 learn packets >>>> 2 l2-learn L2 learn misses >>>> 61846 l2-input L2 input packets >>>> 61846 l2-flood L2 flood packets >>>> 1 ethernet-input l3 mac mismatch >>>> 3 eth1-output interface is down >>>> 30922 eth0-output interface is down >>>> vpp# >>>> >>>> >>>> >>>> Not sure how to check mac issues. Can you explain a bit more? Here is >>>> what I can see on R230 vpp. >>>> >>>> >>>> >>>> vpp# show bridge-domain 1 detail >>>> BD-ID Index BSN Age(min) Learning U-Forwrd UU-Flood >>>> Flooding ARP-Term arp-ufwd BVI-Intf >>>> 1 1 0 off on on flood on >>>> off off N/A >>>> >>>> Interface If-idx ISN SHG BVI TxFlood >>>> VLAN-Tag-Rewrite >>>> eth0 2 1 0 - * >>>> none >>>> eth1 1 1 0 - * >>>> none >>>> vpp# sh l2fib verbose >>>> Mac-Address BD-Idx If-Idx BSN-ISN Age(min) static filter bvi >>>> Interface-Name >>>> 28:99:3a:f4:3a:a6 1 2 0/1 - - - - >>>> eth0 >>>> 28:99:3a:f4:3a:9c 1 1 0/1 - - - - >>>> eth1 >>>> L2FIB total/learned entries: 2/2 Last scan time: 0.0000e0sec Learn >>>> limit: 4194304 >>>> vpp# >>>> >>>> >>>> >>>> On Wed, Oct 16, 2019 at 6:01 PM Balaji Venkatraman (balajiv) < >>>> bala...@cisco.com> wrote: >>>> >>>> >>>> +-------------------------------------------------------------------------+ >>>> >>>> | >>>> | >>>> >>>> | >>>> | >>>> >>>> | >>>> IXIA | >>>> >>>> | >>>> | >>>> >>>> | >>>> | >>>> >>>> >>>> +-----------------------------------------------------------^-------------+ >>>> >>>> |172.16.1.1/24 | >>>> 172.16.1.2/24 >>>> >>>> | | >>>> >>>> | | >>>> >>>> |eth0 | eth0 >>>> >>>> +-----------v-------------+ >>>> +------------+-----------+ >>>> >>>> | 1 | | 4 >>>> | >>>> >>>> | | | >>>> | >>>> >>>> | | | >>>> | >>>> >>>> | | | >>>> | >>>> >>>> | |eth1 eth1 | >>>> | >>>> >>>> | VPP1 2 +--------------------> 3 VPP 2 >>>> | >>>> >>>> | | | >>>> | >>>> >>>> | | | >>>> | >>>> >>>> | | | >>>> | >>>> >>>> | | | >>>> | >>>> >>>> +-------------------------+ >>>> +------------------------+ >>>> >>>> R 740 R 230 >>>> >>>> >>>> >>>> >>>> >>>> Might help if you could check if the packet counts at ingress (port 1) >>>> & egress (port 2) match. Similarly 3 & 4. And the mac entries seen on both >>>> vpp(s). ARP req/rep tracing might also help. >>>> >>>> >>>> >>>> >>>> >>>> /- >>>> >>>> Balaji >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> *From: *<vpp-dev@lists.fd.io> on behalf of "Damjan Marion via >>>> Lists.Fd.Io" <dmarion=me....@lists.fd.io> >>>> *Reply-To: *"dmar...@me.com" <dmar...@me.com> >>>> *Date: *Wednesday, October 16, 2019 at 5:12 PM >>>> *To: *"chuan...@google.com" <chuan...@google.com> >>>> *Cc: *"vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io> >>>> *Subject: *Re: [vpp-dev] Basic l2 bridging does not work >>>> >>>> >>>> >>>> >>>> >>>> On 16 Oct 2019, at 16:14, Chuan Han via Lists.Fd.Io < >>>> chuanhan=google....@lists.fd.io> wrote: >>>> >>>> >>>> >>>> Hi, vpp experts, >>>> >>>> >>>> >>>> We are trying to make basic l2 bridge works within vpp. >>>> >>>> >>>> >>>> We have two servers: r230 and r740, each of which has two phy nics. Two >>>> servers are connected via cable. On each server, we bring these two nics >>>> into the same vpp instance and put them into the same l2 bridge. We tried >>>> sending traffic using ixia. However, ixia shows ping does not work. >>>> >>>> >>>> >>>> I attached the topology, vpp conf files, startup conf file, and logs. >>>> >>>> >>>> >>>> Please advise where we could make it wrong. >>>> >>>> >>>> >>>> Thanks. >>>> >>>> Chuan >>>> >>>> <r230 vpp.conf><r740 vpp.conf><r230 vpp startup.cfg><r740 vpp >>>> startup.cfg><r740.log><r230.log><vpp testbed - >>>> bridge.pdf>-=-=-=-=-=-=-=-=-=-=-=- >>>> Links: You receive all messages sent to this group. >>>> >>>> View/Reply Online (#14189): https://lists.fd.io/g/vpp-dev/message/14189 >>>> Mute This Topic: https://lists.fd.io/mt/34655826/675642 >>>> Group Owner: vpp-dev+ow...@lists.fd.io >>>> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [dmar...@me.com] >>>> -=-=-=-=-=-=-=-=-=-=-=- >>>> >>>> >>>> >>>> On the 1st look everything look ok including the packet trace. >>>> >>>> Can you try to clear counters* and enable packet trace on both >>>> instances. >>>> >>>> Then send known number of packets and find put where drop happens by >>>> looking into same outputs you already shared..... >>>> >>>> >>>> >>>> * "clear int", "clear run", "clear trace" >>>> >>>> >>>> >>>> -- >>>> Damjan >>>> >>>> >>>> >>>> -=-=-=-=-=-=-=-=-=-=-=- >>> Links: You receive all messages sent to this group. >>> >>> View/Reply Online (#14211): https://lists.fd.io/g/vpp-dev/message/14211 >>> Mute This Topic: https://lists.fd.io/mt/34655826/1991531 >>> Group Owner: vpp-dev+ow...@lists.fd.io >>> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [chuan...@google.com] >>> -=-=-=-=-=-=-=-=-=-=-=- >>> >>
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14219): https://lists.fd.io/g/vpp-dev/message/14219 Mute This Topic: https://lists.fd.io/mt/34655826/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
