Hi, following [1],[2] I'm playing with dual wan/ecmp for the first time. The problem is that some destinations are unreachable via one or both WAN interfaces. Some destinations are available via both WANs and others are unavailable at all. There is no ifstated yet, it's a lab, a fresh basic install. My two lab_ISP are simple tplink routers connectet to my home router. Could someone explain me why there is such a random walk and what should I do to reach the whole internet via both WANs simultaneously.
1. https://www.openbsd.org/faq/faq6.html#Multipath 2. https://www.neant.ro/nix/failover-loadbalancing-outgoing-traffic-on-openbsd-with-pf-and-ifstated.html [root@@ecmp~:]cat /etc/hostname.re0 inet 192.168.1.101 255.255.255.0 NONE !route add -mpath default 192.168.1.1 [root@@ecmp~:]cat /etc/hostname.re1 inet 192.168.123.101 255.255.255.0 NONE !route add -mpath default 192.168.123.1 [root@@ecmp~:]cat /etc/sysctl.conf net.inet.ip.multipath=1 [root@@ecmp~:]dmesg | head OpenBSD 7.7 (GENERIC.MP) #625: Sun Apr 13 08:30:20 MDT 2025 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 4259627008 (4062MB) avail mem = 4107096064 (3916MB) random: good seed from bootblocks mpath0 at root scsibus0 at mpath0: 256 targets mainbus0 at root bios0 at mainbus0: SMBIOS rev. 3.0 @ 0x7fe4e040 (9 entries) bios0: vendor coreboot version "v4.17.0.3" date 08/22/2022 Routing tables Internet: Destination Gateway Flags Refs Use Mtu Prio Iface default 192.168.1.1 UGSP 4 77 - 8 re0 default 192.168.123.1 UGSP 0 42 - 8 re1 224/4 127.0.0.1 URS 1 4 32768 8 lo0 127/8 127.0.0.1 UGRS 0 0 32768 8 lo0 127.0.0.1 127.0.0.1 UHhl 1 2 32768 1 lo0 192.168.1/24 192.168.1.101 UCn 1 0 - 4 re0 192.168.1.1 54:e6:fc:ad:53:f8 UHLch 1 5 - 3 re0 192.168.1.101 00:0d:b9:34:0f:14 UHLl 0 10 - 1 re0 192.168.1.255 192.168.1.101 UHb 0 0 - 1 re0 192.168.123/24 192.168.123.101 UCn 1 0 - 4 re1 192.168.123.1 64:66:b3:77:b4:c4 UHLch 1 23 - 3 re1 192.168.123.101 00:0d:b9:34:0f:15 UHLl 0 7 - 1 re1 192.168.123.255 192.168.123.101 UHb 0 0 - 1 re1 [root@@ecmp~:]sh ping.sh + < ip.list + read ip + dig +short he.net + head -1 + ping -c3 216.218.236.2 PING 216.218.236.2 (216.218.236.2): 56 data bytes 64 bytes from 216.218.236.2: icmp_seq=1 ttl=54 time=164.472 ms 64 bytes from 216.218.236.2: icmp_seq=2 ttl=54 time=160.024 ms --- 216.218.236.2 ping statistics --- 3 packets transmitted, 2 packets received, 33.3% packet loss round-trip min/avg/max/std-dev = 160.024/162.248/164.472/2.224 ms + dig +short he.net + head -1 + ping -c3 -I 192.168.1.101 216.218.236.2 PING 216.218.236.2 (216.218.236.2): 56 data bytes --- 216.218.236.2 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss + dig +short he.net + head -1 + ping -c3 -I 192.168.123.101 216.218.236.2 PING 216.218.236.2 (216.218.236.2): 56 data bytes 64 bytes from 216.218.236.2: icmp_seq=0 ttl=54 time=164.395 ms 64 bytes from 216.218.236.2: icmp_seq=1 ttl=54 time=175.492 ms 64 bytes from 216.218.236.2: icmp_seq=2 ttl=54 time=176.851 ms --- 216.218.236.2 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 164.395/172.246/176.851/5.579 ms ################################################################# + read ip + dig +short google.com + head -1 + ping -c3 216.58.215.78 PING 216.58.215.78 (216.58.215.78): 56 data bytes 64 bytes from 216.58.215.78: icmp_seq=0 ttl=57 time=17.724 ms 64 bytes from 216.58.215.78: icmp_seq=1 ttl=57 time=17.586 ms 64 bytes from 216.58.215.78: icmp_seq=2 ttl=57 time=16.538 ms --- 216.58.215.78 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 16.538/17.283/17.724/0.530 ms + dig +short google.com + head -1 + ping -c3 -I 192.168.1.101 216.58.215.78 PING 216.58.215.78 (216.58.215.78): 56 data bytes 64 bytes from 216.58.215.78: icmp_seq=0 ttl=57 time=19.688 ms 64 bytes from 216.58.215.78: icmp_seq=1 ttl=57 time=15.164 ms 64 bytes from 216.58.215.78: icmp_seq=2 ttl=57 time=26.757 ms --- 216.58.215.78 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 15.164/20.536/26.757/4.771 ms + dig +short google.com + head -1 + ping -c3 -I 192.168.123.101 216.58.215.78 PING 216.58.215.78 (216.58.215.78): 56 data bytes --- 216.58.215.78 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss ################################################################# + read ip + head -1 + dig +short cloudflare.com + ping -c3 104.16.133.229 PING 104.16.133.229 (104.16.133.229): 56 data bytes --- 104.16.133.229 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss + dig +short cloudflare.com + head -1 + ping -c3 -I 192.168.1.101 104.16.132.229 PING 104.16.132.229 (104.16.132.229): 56 data bytes --- 104.16.132.229 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss + dig +short cloudflare.com + head -1 + ping -c3 -I 192.168.123.101 104.16.132.229 PING 104.16.132.229 (104.16.132.229): 56 data bytes 64 bytes from 104.16.132.229: icmp_seq=0 ttl=56 time=21.154 ms 64 bytes from 104.16.132.229: icmp_seq=1 ttl=56 time=19.130 ms 64 bytes from 104.16.132.229: icmp_seq=2 ttl=56 time=14.952 ms --- 104.16.132.229 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 14.952/18.412/21.154/2.582 ms ################################################################# + read ip + dig +short onet.pl + head -1 + ping -c3 13.227.146.64 PING 13.227.146.64 (13.227.146.64): 56 data bytes --- 13.227.146.64 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss + dig +short onet.pl + head -1 + ping -c3 -I 192.168.1.101 13.227.146.66 PING 13.227.146.66 (13.227.146.66): 56 data bytes 64 bytes from 13.227.146.66: icmp_seq=0 ttl=247 time=23.393 ms 64 bytes from 13.227.146.66: icmp_seq=1 ttl=247 time=17.524 ms 64 bytes from 13.227.146.66: icmp_seq=2 ttl=247 time=24.362 ms --- 13.227.146.66 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 17.524/21.760/24.362/3.021 ms + dig +short onet.pl + head -1 + ping -c3 -I 192.168.123.101 13.227.146.66 PING 13.227.146.66 (13.227.146.66): 56 data bytes --- 13.227.146.66 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss ################################################################# + read ip + dig +short wp.pl + head -1 + ping -c3 212.77.98.9 PING 212.77.98.9 (212.77.98.9): 56 data bytes 64 bytes from 212.77.98.9: icmp_seq=0 ttl=53 time=14.172 ms 64 bytes from 212.77.98.9: icmp_seq=1 ttl=53 time=14.620 ms 64 bytes from 212.77.98.9: icmp_seq=2 ttl=53 time=19.956 ms --- 212.77.98.9 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 14.172/16.249/19.956/2.627 ms + dig +short wp.pl + head -1 + ping -c3 -I 192.168.1.101 212.77.98.9 PING 212.77.98.9 (212.77.98.9): 56 data bytes 64 bytes from 212.77.98.9: icmp_seq=0 ttl=53 time=28.616 ms 64 bytes from 212.77.98.9: icmp_seq=1 ttl=53 time=24.379 ms 64 bytes from 212.77.98.9: icmp_seq=2 ttl=53 time=24.340 ms --- 212.77.98.9 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 24.340/25.778/28.616/2.006 ms + dig +short wp.pl + head -1 + ping -c3 -I 192.168.123.101 212.77.98.9 PING 212.77.98.9 (212.77.98.9): 56 data bytes --- 212.77.98.9 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss ################################################################# + read ip + dig +short a.root-servers.net + head -1 + ping -c3 198.41.0.4 PING 198.41.0.4 (198.41.0.4): 56 data bytes 64 bytes from 198.41.0.4: icmp_seq=0 ttl=54 time=30.272 ms 64 bytes from 198.41.0.4: icmp_seq=1 ttl=54 time=43.853 ms 64 bytes from 198.41.0.4: icmp_seq=2 ttl=54 time=23.450 ms --- 198.41.0.4 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 23.450/32.525/43.853/8.480 ms + dig +short a.root-servers.net + head -1 + ping -c3 -I 192.168.1.101 198.41.0.4 PING 198.41.0.4 (198.41.0.4): 56 data bytes 64 bytes from 198.41.0.4: icmp_seq=0 ttl=54 time=26.507 ms 64 bytes from 198.41.0.4: icmp_seq=1 ttl=54 time=27.931 ms 64 bytes from 198.41.0.4: icmp_seq=2 ttl=54 time=30.361 ms --- 198.41.0.4 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 26.507/28.266/30.361/1.591 ms + dig +short a.root-servers.net + head -1 + ping -c3 -I 192.168.123.101 198.41.0.4 PING 198.41.0.4 (198.41.0.4): 56 data bytes 64 bytes from 198.41.0.4: icmp_seq=0 ttl=54 time=27.174 ms 64 bytes from 198.41.0.4: icmp_seq=1 ttl=54 time=36.560 ms 64 bytes from 198.41.0.4: icmp_seq=2 ttl=54 time=28.829 ms --- 198.41.0.4 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 27.174/30.855/36.560/4.091 ms ################################################################# + read ip + dig +short b.root-servers.net + head -1 + ping -c3 170.247.170.2 PING 170.247.170.2 (170.247.170.2): 56 data bytes 64 bytes from 170.247.170.2: icmp_seq=0 ttl=55 time=47.790 ms 64 bytes from 170.247.170.2: icmp_seq=1 ttl=55 time=38.032 ms 64 bytes from 170.247.170.2: icmp_seq=2 ttl=55 time=32.215 ms --- 170.247.170.2 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 32.215/39.346/47.790/6.426 ms + dig +short b.root-servers.net + head -1 + ping -c3 -I 192.168.1.101 170.247.170.2 PING 170.247.170.2 (170.247.170.2): 56 data bytes 64 bytes from 170.247.170.2: icmp_seq=0 ttl=55 time=34.736 ms 64 bytes from 170.247.170.2: icmp_seq=1 ttl=55 time=41.941 ms 64 bytes from 170.247.170.2: icmp_seq=2 ttl=55 time=29.856 ms --- 170.247.170.2 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 29.856/35.511/41.941/4.964 ms + dig +short b.root-servers.net + head -1 + ping -c3 -I 192.168.123.101 170.247.170.2 PING 170.247.170.2 (170.247.170.2): 56 data bytes --- 170.247.170.2 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss ################################################################# + read ip + dig +short c.root-servers.net + head -1 + ping -c3 192.33.4.12 PING 192.33.4.12 (192.33.4.12): 56 data bytes 64 bytes from 192.33.4.12: icmp_seq=0 ttl=50 time=35.299 ms 64 bytes from 192.33.4.12: icmp_seq=1 ttl=50 time=30.066 ms 64 bytes from 192.33.4.12: icmp_seq=2 ttl=50 time=31.890 ms --- 192.33.4.12 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 30.066/32.418/35.299/2.169 ms + dig +short c.root-servers.net + head -1 + ping -c3 -I 192.168.1.101 192.33.4.12 PING 192.33.4.12 (192.33.4.12): 56 data bytes 64 bytes from 192.33.4.12: icmp_seq=0 ttl=50 time=31.169 ms 64 bytes from 192.33.4.12: icmp_seq=2 ttl=50 time=35.877 ms --- 192.33.4.12 ping statistics --- 3 packets transmitted, 2 packets received, 33.3% packet loss round-trip min/avg/max/std-dev = 31.169/33.523/35.877/2.354 ms + dig +short c.root-servers.net + head -1 + ping -c3 -I 192.168.123.101 192.33.4.12 PING 192.33.4.12 (192.33.4.12): 56 data bytes --- 192.33.4.12 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss ################################################################# + read ip + dig +short d.root-servers.net + head -1 + ping -c3 199.7.91.13 PING 199.7.91.13 (199.7.91.13): 56 data bytes 64 bytes from 199.7.91.13: icmp_seq=1 ttl=59 time=28.583 ms 64 bytes from 199.7.91.13: icmp_seq=2 ttl=59 time=21.744 ms --- 199.7.91.13 ping statistics --- 3 packets transmitted, 2 packets received, 33.3% packet loss round-trip min/avg/max/std-dev = 21.744/25.164/28.583/3.419 ms + dig +short d.root-servers.net + head -1 + ping -c3 -I 192.168.1.101 199.7.91.13 PING 199.7.91.13 (199.7.91.13): 56 data bytes --- 199.7.91.13 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss + dig +short d.root-servers.net + head -1 + ping -c3 -I 192.168.123.101 199.7.91.13 PING 199.7.91.13 (199.7.91.13): 56 data bytes 64 bytes from 199.7.91.13: icmp_seq=0 ttl=59 time=20.056 ms 64 bytes from 199.7.91.13: icmp_seq=1 ttl=59 time=26.050 ms 64 bytes from 199.7.91.13: icmp_seq=2 ttl=59 time=28.739 ms --- 199.7.91.13 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 20.056/24.949/28.739/3.630 ms ################################################################# + read ip + dig +short e.root-servers.net + head -1 + ping -c3 192.203.230.10 PING 192.203.230.10 (192.203.230.10): 56 data bytes 64 bytes from 192.203.230.10: icmp_seq=0 ttl=59 time=22.710 ms 64 bytes from 192.203.230.10: icmp_seq=1 ttl=59 time=19.804 ms 64 bytes from 192.203.230.10: icmp_seq=2 ttl=59 time=31.891 ms --- 192.203.230.10 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 19.804/24.802/31.891/5.151 ms + dig +short e.root-servers.net + head -1 + ping -c3 -I 192.168.1.101 192.203.230.10 PING 192.203.230.10 (192.203.230.10): 56 data bytes 64 bytes from 192.203.230.10: icmp_seq=0 ttl=59 time=17.495 ms 64 bytes from 192.203.230.10: icmp_seq=1 ttl=59 time=20.463 ms 64 bytes from 192.203.230.10: icmp_seq=2 ttl=59 time=26.192 ms --- 192.203.230.10 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 17.495/21.383/26.192/3.610 ms + dig +short e.root-servers.net + head -1 + ping -c3 -I 192.168.123.101 192.203.230.10 PING 192.203.230.10 (192.203.230.10): 56 data bytes 64 bytes from 192.203.230.10: icmp_seq=0 ttl=59 time=19.784 ms 64 bytes from 192.203.230.10: icmp_seq=1 ttl=59 time=19.601 ms 64 bytes from 192.203.230.10: icmp_seq=2 ttl=59 time=23.967 ms --- 192.203.230.10 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 19.601/21.117/23.967/2.017 ms ################################################################# + read ip + dig +short f.root-servers.net + head -1 + ping -c3 192.5.5.241 PING 192.5.5.241 (192.5.5.241): 56 data bytes 64 bytes from 192.5.5.241: icmp_seq=0 ttl=56 time=26.166 ms 64 bytes from 192.5.5.241: icmp_seq=1 ttl=56 time=20.163 ms 64 bytes from 192.5.5.241: icmp_seq=2 ttl=56 time=21.769 ms --- 192.5.5.241 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 20.163/22.699/26.166/2.537 ms + dig +short f.root-servers.net + head -1 + ping -c3 -I 192.168.1.101 192.5.5.241 PING 192.5.5.241 (192.5.5.241): 56 data bytes 64 bytes from 192.5.5.241: icmp_seq=0 ttl=56 time=23.226 ms 64 bytes from 192.5.5.241: icmp_seq=1 ttl=56 time=29.341 ms 64 bytes from 192.5.5.241: icmp_seq=2 ttl=56 time=23.783 ms --- 192.5.5.241 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 23.226/25.450/29.341/2.761 ms + dig +short f.root-servers.net + head -1 + ping -c3 -I 192.168.123.101 192.5.5.241 PING 192.5.5.241 (192.5.5.241): 56 data bytes 64 bytes from 192.5.5.241: icmp_seq=0 ttl=56 time=25.888 ms 64 bytes from 192.5.5.241: icmp_seq=1 ttl=56 time=22.124 ms 64 bytes from 192.5.5.241: icmp_seq=2 ttl=56 time=24.163 ms --- 192.5.5.241 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 22.124/24.058/25.888/1.538 ms ################################################################# + read ip + dig +short g.root-servers.net + head -1 + ping -c3 192.112.36.4 PING 192.112.36.4 (192.112.36.4): 56 data bytes --- 192.112.36.4 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss + dig +short g.root-servers.net + head -1 + ping -c3 -I 192.168.1.101 192.112.36.4 PING 192.112.36.4 (192.112.36.4): 56 data bytes --- 192.112.36.4 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss + dig +short g.root-servers.net + head -1 + ping -c3 -I 192.168.123.101 192.112.36.4 PING 192.112.36.4 (192.112.36.4): 56 data bytes --- 192.112.36.4 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss ################################################################# + read ip + dig +short h.root-servers.net + head -1 + ping -c3 198.97.190.53 PING 198.97.190.53 (198.97.190.53): 56 data bytes 64 bytes from 198.97.190.53: icmp_seq=0 ttl=56 time=36.725 ms 64 bytes from 198.97.190.53: icmp_seq=1 ttl=56 time=27.487 ms 64 bytes from 198.97.190.53: icmp_seq=2 ttl=56 time=30.701 ms --- 198.97.190.53 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 27.487/31.638/36.725/3.829 ms + dig +short h.root-servers.net + head -1 + ping -c3 -I 192.168.1.101 198.97.190.53 PING 198.97.190.53 (198.97.190.53): 56 data bytes 64 bytes from 198.97.190.53: icmp_seq=0 ttl=56 time=26.644 ms 64 bytes from 198.97.190.53: icmp_seq=1 ttl=56 time=30.718 ms 64 bytes from 198.97.190.53: icmp_seq=2 ttl=56 time=36.418 ms --- 198.97.190.53 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 26.644/31.260/36.418/4.008 ms + dig +short h.root-servers.net + head -1 + ping -c3 -I 192.168.123.101 198.97.190.53 PING 198.97.190.53 (198.97.190.53): 56 data bytes 64 bytes from 198.97.190.53: icmp_seq=0 ttl=56 time=27.605 ms 64 bytes from 198.97.190.53: icmp_seq=1 ttl=56 time=32.021 ms 64 bytes from 198.97.190.53: icmp_seq=2 ttl=56 time=24.948 ms --- 198.97.190.53 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 24.948/28.191/32.021/2.917 ms ################################################################# + read ip + dig +short i.root-servers.net + head -1 + ping -c3 192.36.148.17 PING 192.36.148.17 (192.36.148.17): 56 data bytes --- 192.36.148.17 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss + dig +short i.root-servers.net + head -1 + ping -c3 -I 192.168.1.101 192.36.148.17 PING 192.36.148.17 (192.36.148.17): 56 data bytes --- 192.36.148.17 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss + dig +short i.root-servers.net + head -1 + ping -c3 -I 192.168.123.101 192.36.148.17 PING 192.36.148.17 (192.36.148.17): 56 data bytes --- 192.36.148.17 ping statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss ################################################################# Thanks! -- Please do not CC me Radek
