Hi Neale, I think I mistakenly sent a reply just to you instead of to the group, so I'm sending the reply again.
We are using ip session redirect for management traffic to make sure that packets are reachable between the vpps. The ip session redirect is setup to reach the remote vpp's management IP and a different src port per each ipsec tunnel that's configured. When we remove the tunnels, the ip session redirect match are removed. User routes uses the remote management IP as its next hop, so if there are multiple tunnels, there are multiple path with equal cost to the remote end. So it looks like ip session redirect is mapping the ipipX to the wrong next hop, see the ip session redirect in bold below. When this happens, if I restart vpp without any config changes, the ipip session redirect output matches what was setup. vpp0's management IP is 192.168.3.2 and vpp2's management IP is 192.168.3.6. So in the case of vpp2, ip session redirect is set to match dest ip 192.168.3.2, src port 2157, sw_if_index 6 (ipip0), which is profile vpp0-vpp2_6_0 (192.168.30.6->192.168.10.6) dest ip 192.168.3.2, src port 2158, sw_if_index 8 (ipip2), which is profile vpp0-vpp2_7_1 (192.168.31.6->192.168.11.6) dest ip 192.168.3.2, src port 2159, sw_if_index 7 (ipip1), which is profile vpp0-vpp2_8_2 (192.168.32.6->192.168.12.6) vpp2:~# vppctl sh ip sess redir [0] table 0 key 00000000: 000000000000000000000000000000000000000000000000000000000000c0a8 00000020: 0302086f000000000000000000000000 <========(0x086f=2159) via: path-list:[35] locks:1 flags:shared,no-uRPF, uPRF-list:47 len:1 itfs:[7, ] path:[61] pl-index:35 ip4 weight=1 pref=5 attached: oper-flags:resolved, ipip1 forwarding [@2]: ipv4 via 0.0.0.0 *ipip1* : mtu:9000 next:8 flags:[fixup-ip4o4 ] 45000000000000004004cf9dc0a81f06c0a80b06 stacked-on entry:32: [@2]: ipv4 via *192.168.31.1 tn-eth1* : mtu:1500 next:4 flags:[] fa163f78e754fa163febca520800 <======ipip1, sw_if_index 7, was set up for tunnel 192.168.32.6==192.168.12.6 [1] table 0 key 00000000: 000000000000000000000000000000000000000000000000000000000000c0a8 00000020: 0302086e000000000000000000000000 <======(0x086e=2158) via: path-list:[38] locks:1 flags:shared,no-uRPF, uPRF-list:10 len:1 itfs:[8, ] path:[54] pl-index:38 ip4 weight=1 pref=5 attached: oper-flags:resolved, ipip2 forwarding [@2]: ipv4 via 0.0.0.0 *ipip2* : mtu:9000 next:8 flags:[fixup-ip4o4 ] 45000000000000004004cd9dc0a82006c0a80c06 stacked-on entry:35: [@2]: ipv4 via *192.168.32.1 tn-eth3* : mtu:1500 next:6 flags:[] fa163f1fe606fa163fa0c5d30800 <===== ipip2, sw_if_index 8, was setup for tunnel 192.168.31.6==192.168.11.6 [2] table 0 key 00000000: 000000000000000000000000000000000000000000000000000000000000c0a8 00000020: 0302086d000000000000000000000000 via: path-list:[42] locks:1 flags:shared,no-uRPF, uRPF-list: None path:[52] pl-index:42 ip4 weight=1 pref=5 attached: oper-flags:resolved, ipip0 forwarding [@2]: ipv4 via 0.0.0.0 ipip0: mtu:9000 next:8 flags:[fixup-ip4o4 ] 45000000000000004004d19dc0a81e06c0a80a06 stacked-on entry:31: [@2]: ipv4 via 192.168.30.1 tn-eth0: mtu:1500 next:3 flags:[] fa163f12b28cfa163f4e184f0800 Here's the int addr output for the tunnel interfaces: ipip1 (up): unnumbered, use tn-eth3 L3 192.168.32.6/24 ipip2 (up): unnumbered, use tn-eth1 L3 192.168.31.6/24 Thank you, Sonia
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21254): https://lists.fd.io/g/vpp-dev/message/21254 Mute This Topic: https://lists.fd.io/mt/90427455/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
