It is only a coincidence that we have two gateways and the LAN apparently on the same sub-network. When opening an account with an ISP, their gateway/router comes as part of the contract, it is a hardware device, and it may have *any* *non-customizable* RFC-1918 address. One cannot and must not change a whole LAN to adjust to any specific ISP. ---For example, if I put the LAN on RFC-1918 10/8, egress goes automatically to the first gateway on em0. As soon as I put "up" em1, where the second gateway is attached, the network collapses, because ISP #2 happens to be using the same sub-network of ISP #1. If tomorrow I close ISP #2 and get ISP #3, whose gateway is on 10/8, for example, then OpenBSD may put em0 and em1 on their respective egress, but I would have to change the LAN again. Changing a LAN, on a live production network, each time you change ISP, or add a new one, is the shot on one's own head.--- Hence the firewall also serves as gateway. I have one gateway attached to em0, and the other attached to em1. Each gateway happens to be on 192.168.1.1/24 and it is only a coincidence. My LAN also happens to be on 192.168.1.1/24. They happen to be on the same RFC-1918 addressing, but they are three physically different networks, and the firewall sits in between.
I understand from this thread that OpenBSD attaches egress automatically to the gateway. Nobody wrote it explicitly, but I understand this to be the case. I also understand, on the same fashion, that if I attach two gateways, OpenBSD expects them to be on different networks. This means, if I understand correctly, that OpenBSD wants your LAN to be on a third network. This is a whack-a-mole game, because each time you add a gateway (only two in this case, but you may have more) or change ISP, or the ISP changes its own settings, you must change your LAN as a consequence. One needs one's LAN to be independent from the specific ISPs. I hope this clarifies at least part of the "untold" story. So, does OpenBSD make me a slave, or am I independent from specific ISPs and have a LAN that is resilient to ISP changes? On Tuesday, April 1st, 2025 at 8:34 AM, Peter N. M. Hansteen <pe...@bsdly.net> wrote: > On Tue, Apr 01, 2025 at 07:47:09AM +0000, otto.cooper wrote: > > > > Then all I and Peter Hansteen said stand true. Having both interfaces > > > on the same subnetwork won't work easily without unnecessarily > > > complicated routing "hacks". Simply move one of the sides of the > > > network to a different subnet and go from there. > > > > It has been working for 20+ years and never had a single problem with it. > > > This sounds to me like the previous setup had some routing magic in place > that for > some reason or other does not (yet?) have an equivalent in the thing you are > building now. > > > I need to put ix0 out of group egress, and em0 and em1 in group egress. How > > do I do it? > > > The egress group consists of interfaces that have a default route. You could > try to force > the issue with, for each of the interfaces you want to have default routes, > add to the > config file > > !route -n add -inet default $GATEWAY_FOR_THIS_ROUTE > > (assuming that $GATEWAY_FOR_THIS_ROUTE is reachable) and possibly add > corresponding > "!route -n delete" and so on to the ones you want to not have default route. > > This comes with plenty of potential for foot-shooting, of course. > > I would recommend taking a long hard look at whether the network design you > describe > is actually suited for your purpose. > > That said, there may well be factors in play that have not come to light here. > > Have fun! > > -- > Peter N. M. Hansteen, member of the first RFC 1149 implementation team > https://bsdly.blogspot.com/ https://www.bsdly.net/ https://www.nuug.no/ > "Remember to set the evil bit on all malicious network traffic" > delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
