Hi everyone: I met a problem in bonding ( mode : active-backup) when I use VPP. I am not know how bonding(mode:avtive-backup) works in linux exactly. So I just did a test for this problem .
Topology: linux-A | linux-B /------------ em3 ---------- | ---------- em3(10.10.1.99/24) -------- Vxlan1(192..192.1.99/24) Vxlan1(192..192.1.100/24) -------- bond0(10.10.1.100/24) -----/ | \------------- em4 --------- | ---------- em4(172.16.152.60) --------- Wireshark When *bond0* 's active slave in *linux-A* is *em3* , I ping *192.192.1.100* from *linux-B* 's terminal, ping is *ok* ! I down the *em3* in *linux* - *A* and up it later on. I do this just want to change *bond0* 's active slave in *linux* - *A* to *em4*. I use cmd " *cat /proc/net/bonding/bond0* " to make sure it works well. So *bond0* 's active slave in *linux-A* is *em4* , I ping *192.192.1.100* from *linux-B* 's terminal, ping is *not ok* (that makes sense). But Wireshart softwear in *linux* - *B* can't capture any packets about ping on *em4* in *linux-B*. At the same time, *bond0* and *em4* in *linux* - *A* does not send any packets, and *me3* and *bond0* in *linux* - *A* drop packets with Same frequency like ping. So I think the linux's bond( mode : active-backup ) drop packtets incoming from inactive-slave ! First, I create a bond on linux-A(CentOS Linux release 7.7.1908 (Core)), the relsut like this: *[root@localhost ~]# cat /proc/net/bonding/bond0* *Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)* ** *Bonding Mode: fault-tolerance (active-backup)* *Primary Slave: None* *Currently Active Slave: em3* *MII Status: up* *MII Polling Interval (ms): 100* *Up Delay (ms): 0* *Down Delay (ms): 0* ** *Slave Interface: em3* *MII Status: up* *Speed: 100 Mbps* *Duplex: full* *Link Failure Count: 2* *Permanent HW addr: 20:04:0f:f5:8a:3a* *Slave queue ID: 0* ** *Slave Interface: em4* *MII Status: up* *Speed: 100 Mbps* *Duplex: full* *Link Failure Count: 3* *Permanent HW addr: 20:04:0f:f5:8a:3b* *Slave queue ID: 0* (We can see than bond0 has two slave interface, em3 is the active slave and em4 is the inactive slave) Second, I create a vxlan interface on linux-A, the CMD and result like this: *[root@localhost ~]#* *ip link add vxlan1 type vxlan id 139265 remote 10.10.1.99 dstport 4789 dev bond0* *[root@localhost ~]#* *ip link set vxlan1 up* *[root@localhost ~]#* *ip addr add 192.192.1.100/24 dev vxlan1* *[root@localhost ~]# ifconfig* *bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST> mtu 1500* *inet 10.10.1.100 netmask 255.255.255.0 broadcast 10.10.1.255* *inet6 fe80::2204:fff:fef5:8a3a prefixlen 64 scopeid 0x20<link>* *ether 20:04:0f:f5:8a:3a txqueuelen 0 (Ethernet)* *RX packets 9441 bytes 981899 (958.8 KiB)* *RX errors 0 dropped 2320 overruns 0 frame 0* *TX packets 343 bytes 42390 (41.3 KiB)* *TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0* *em3: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500* *ether 20:04:0f:f5:8a:3a txqueuelen 1000 (Ethernet)* *RX packets 7064 bytes 729818 (712.7 KiB)* *RX errors 0 dropped 1067 overruns 0 frame 0* *TX packets 269 bytes 36586 (35.7 KiB)* *TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0* *device interrupt 72* ** *em4: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500* *ether 20:04:0f:f5:8a:3a txqueuelen 1000 (Ethernet)* *RX packets 2377 bytes 252081 (246.1 KiB)* *RX errors 0 dropped 20893 overruns 0 frame 0* *TX packets 74 bytes 5804 (5.6 KiB)* *TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0* *device interrupt 73* ** *vxlan1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450* *inet 192.192.1.100 netmask 255.255.255.0 broadcast 0.0.0.0* *inet6 fe80::f077:8cff:fe45:89b9 prefixlen 64 scopeid 0x20<link>* *ether f2:77:8c:45:89:b9 txqueuelen 0 (Ethernet)* *RX packets 230 bytes 18256 (17.8 KiB)* *RX errors 0 dropped 0 overruns 0 frame 0* *TX packets 305 bytes 30322 (29.6 KiB)* *TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0* ** Thrid, I create a vxlan interface on *linux-B* , the CMD and result like this: *[root@localhost ~]# ip link add vxlan1 type vxlan id 139265 remote 10.10.1.100 dstport 4789 dev em3* *[root@localhost ~]# ip link set vxlan1 up* *[root@localhost ~]# ip addr add 192.192.1.99/24 dev vxlan1* *[root@localhost ~]# ifconfig* *em3: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500* *inet 10.10.1.99 netmask 255.255.255.0 broadcast 10.10.1.255* *ether 20:04:0f:f2:79:0f txqueuelen 1000 (Ethernet)* *RX packets 0 bytes 0 (0.0 B)* *RX errors 0 dropped 0 overruns 0 frame 0* *TX packets 0 bytes 0 (0.0 B)* *TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0* *device interrupt 73* *vxlan1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450* *inet 192.192.1.99 netmask 255.255.255.0 broadcast 0.0.0.0* *inet6 fe80::f077:8cff:fe45:89b9 prefixlen 64 scopeid 0x20<link>* *ether f2:77:8c:45:89:b9 txqueuelen 0 (Ethernet)* *RX packets 230 bytes 18256 (17.8 KiB)* *RX errors 0 dropped 0 overruns 0 frame 0* *TX packets 305 bytes 30322 (29.6 KiB)* *TX errors 0 dropped 0 overruns 0 carrier 0 collision ****************************************************************************************************************************************************************** * This is my problem: * * ----- *TenGigabitEthernet3b/0/1(active)* --------------- | port a port c | * * *VTEP* (vxlan)------- *bridge domain* ------ *BondEthernet0* --- / | switch | \___ *TenGigabitEthernet3b/0/0(passive)* ---------------| port b port d | When a *ARP Request* come from *VTEP* and output from *TenGigabitEthernet3b/0/1(active) ,* be transfered to "port A of switch", in switch it will be flooded to "port B/C/D of switch" 。 "Port B of switch" will send it back to *TenGigabitEthernet3b/0/0(inactive)* , then to *BondEthernet0* 。 If bond-input do not drop this packet, loopback happen!(MAC address flapping ) In my test, when *ARP Request* loopback from bond's inactive slave interface, the bond-input graph node will put it to L2 bridge(do L2 learn...), then MAC address flapping happen! So the corresponding ARP Reply come from *TenGigabitEthernet3b/0/1(active) will be dropped in L2 bridge!* * * So should these packets come from *TenGigabitEthernet3b/0/0(passive) be dropped ? or* What wrong config did I have? ro .... Please tell me, thanks! I use VPP's stable version 19.08 and do some config like this: 1. I create a bond interface and set bond mode as “active-backup” ； 2. I add two phy port in bond as bond’s slavies; 3. I set all of the three interface up; after do these configures, I run " *show bond details* " ： vpp# show bond details BondEthernet0 mode: active-backup load balance: active-backup number of active slaves: 2 *TenGigabitEthernet3b/0/1* *TenGigabitEthernet3b/0/0* number of slaves: 2 TenGigabitEthernet3b/0/0 TenGigabitEthernet3b/0/1 device instance: 0 interface id: 0 sw_if_index: 3 hw_if_index: 3 We know that, BondEthernet0's two slaves ： *TenGigabitEthernet3b/0/1 --- active TenGigabitEthernet3b/0/0 --- passive When I use “trace add dpdk-input 100* *、* *show trace” to see how packets be processed. I see the packets which input frome TenGigabitEthernet3b/0/0(passive) still processed by vpp(such as "l2_learn"). Shouldn't these packets just be droped on bond-input graph node? * Regards, gencli Liu * *
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14822): https://lists.fd.io/g/vpp-dev/message/14822 Mute This Topic: https://lists.fd.io/mt/67466668/21656 Mute #vnet: https://lists.fd.io/mk?hashtag=vnet&subid=1480452 Mute #vapi: https://lists.fd.io/mk?hashtag=vapi&subid=1480452 Mute #vpp: https://lists.fd.io/mk?hashtag=vpp&subid=1480452 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
