Hello David. I was able to solve it while troubleshooting some fragmentation issue. The VTI interfaces had MTU of 1480 by default. I reduced to them to the real PMTUD (1366) and now its all working just fine. I am not sure how its related and why, but seems like it solved the issue.
P.S: while reading the relevant code in the kernel, i think i spotted some mistake in net/ipv4/route.c, in function "update_or_create_fnhe". It looks like it loops over all the exceptions for the nexthop entry, but always overwriting the first (and only) entry, so effectively only 1 exception can exist per nexthop entry. Line 678: "if (fnhe) {" Should probably be: "if (fnhe && fnhe->fnhe_daddr == daddr) {" Thank you for your efforts, Kfir Itzhak On Fri, Aug 14, 2020 at 10:08 AM mastertheknife <masterthekn...@gmail.com> wrote: > > Hello David, > > It's on a production system, vmbr2 is a bridge with eth.X VLAN > interface inside for the connectivity on that 252.0/24 network. vmbr2 > has address 192.168.252.5 in that case > 192.168.252.250 and 192.168.252.252 are CentOS8 LXCs on another host, > with libreswan inside for any/any IPSECs with VTi interfaces. > > Everything is kernel 5.4.44 LTS > > I wish i could fully reproduce all of it in a script, but i am not > sure how to create such hops that return this ICMP > > Thank you, > Kfir > > > On Wed, Aug 12, 2020 at 10:21 PM David Ahern <dsah...@gmail.com> wrote: > > > > On 8/12/20 6:37 AM, mastertheknife wrote: > > > Hello David, > > > > > > I tried and it seems i can reproduce it: > > > > > > # Create test NS > > > root@host:~# ip netns add testns > > > # Create veth pair, veth0 in host, veth1 in NS > > > root@host:~# ip link add veth0 type veth peer name veth1 > > > root@host:~# ip link set veth1 netns testns > > > # Configure veth1 (NS) > > > root@host:~# ip netns exec testns ip addr add 192.168.252.209/24 dev veth1 > > > root@host:~# ip netns exec testns ip link set dev veth1 up > > > root@host:~# ip netns exec testns ip route add default via 192.168.252.100 > > > root@host:~# ip netns exec testns ip route add 192.168.249.0/24 > > > nexthop via 192.168.252.250 nexthop via 192.168.252.252 > > > # Configure veth0 (host) > > > root@host:~# brctl addif vmbr2 veth0 > > > > vmbr2's config is not defined. > > > > ip li add vmbr2 type bridge > > ip li set veth0 master vmbr2 > > ip link set veth0 up > > > > anything else? e.g., address for vmbr2? What holds 192.168.252.250 and > > 192.168.252.252