On Tue, Apr 01, 2025 at 10:32:26AM +0000, otto.cooper wrote:
> It is only a coincidence that we have two gateways and the LAN
> apparently on the same sub-network. When opening an account with an
> ISP, their gateway/router comes as part of the contract, it is a
> hardware device, and it may have *any* *non-customizable* RFC-1918
> address. One cannot and must not change a whole LAN to adjust to any
> specific ISP. ---For example, if I put the LAN on RFC-1918 10/8,
> egress goes automatically to the first gateway on em0. As soon as I
> put "up" em1, where the second gateway is attached, the network
> collapses, because ISP #2 happens to be using the same sub-network of
> ISP #1. If tomorrow I close ISP #2 and get ISP #3, whose gateway is on
> 10/8, for example, then OpenBSD may put em0 and em1 on their
> respective egress, but I would have to change the LAN again. Changing
> a LAN, on a live production network, each time you change ISP, or add
> a new one, is the shot on one's own head.--- Hence the firewall also
> serves as gateway. I have one gateway attached to em0, and the other
> attached to em1. Each gateway happens to be on 192.168.1.1/24 and it
> is only a coincidence. My LAN also happens to be on 192.168.1.1/24.
> They happen to be on the same RFC-1918 addressing, but they are three
> physically different networks, and the firewall sits in between.
>
> I understand from this thread that OpenBSD attaches egress
> automatically to the gateway. Nobody wrote it explicitly, but I
> understand this to be the case. I also understand, on the same
> fashion, that if I attach two gateways, OpenBSD expects them to be on
> different networks. This means, if I understand correctly, that
> OpenBSD wants your LAN to be on a third network. This is a
> whack-a-mole game, because each time you add a gateway (only two in
> this case, but you may have more) or change ISP, or the ISP changes
> its own settings, you must change your LAN as a consequence. One needs
> one's LAN to be independent from the specific ISPs.
Your LAN does *not* have to be in the same network segment as your
ISP gateway.
If your ISP changes the configuration of the gateway it provides, or
you change ISP or whatever, just use autoconf on em0, and let it be
configured to whatever your ISP sees fit. Then, if your firewall is
meant to be a gateway between your ISP-provided router(s) and your LAN
(and by LAN I mean everything connected to your "internal" interface,
ix0), your LAN can have whatever numbering you want. It will make your
life much easier if you make it different from whatever the ISP's
gateway is giving you, say 192.168.77.0/24, or something like that.
Then simply setup NAT on the firewall's on pf.conf (lots of examples on
the man page for pf.conf and on Peter's book). And that's it: now your
immune to changes in the gateway<->firewall link.
The only caveat is that if you want to open ports on your firewall
you'll have to do it twice: in your ISP's router you'll have to forward
them to the firewall's "external" IP, and then on the firewall you'll
need to redirect them to the appropriate machine on the LAN. Still,
this is a much less esoteric setup than running everything in the same
network segment.
Also, if you attach two gateways, pf can handle this fine. From the man
page for pf.conf:
In this example, a NAT gateway is set up to translate internal
addresses using a pool of public addresses (192.0.2.16/28). A given
source address is always translated to the same pool address by
using the source-hash keyword. The gateway also translates incoming
web server connections to a group of web servers on the internal
network.
match out on $ext_if inet from any to any nat-to 192.0.2.16/28 \
source-hash
match in on $ext_if proto tcp from any to any port 80 \
rdr-to { 10.1.2.155 weight 2, 10.1.2.160 weight 1, \
10.1.2.161 weight 8 } round-robin
> I hope this clarifies at least part of the "untold" story.
>
> So, does OpenBSD make me a slave, or am I independent from specific
> ISPs and have a LAN that is resilient to ISP changes?
>
>
>
> On Tuesday, April 1st, 2025 at 8:34 AM, Peter N. M. Hansteen
> <pe...@bsdly.net> wrote:
>
> > On Tue, Apr 01, 2025 at 07:47:09AM +0000, otto.cooper wrote:
> >
> > > > Then all I and Peter Hansteen said stand true. Having both interfaces
> > > > on the same subnetwork won't work easily without unnecessarily
> > > > complicated routing "hacks". Simply move one of the sides of the
> > > > network to a different subnet and go from there.
> > >
> > > It has been working for 20+ years and never had a single problem with it.
> >
> >
> > This sounds to me like the previous setup had some routing magic in place
> > that for
> > some reason or other does not (yet?) have an equivalent in the thing you
> > are building now.
> >
> > > I need to put ix0 out of group egress, and em0 and em1 in group egress.
> > > How do I do it?
> >
> >
> > The egress group consists of interfaces that have a default route. You
> > could try to force
> > the issue with, for each of the interfaces you want to have default routes,
> > add to the
> > config file
> >
> > !route -n add -inet default $GATEWAY_FOR_THIS_ROUTE
> >
> > (assuming that $GATEWAY_FOR_THIS_ROUTE is reachable) and possibly add
> > corresponding
> > "!route -n delete" and so on to the ones you want to not have default route.
> >
> > This comes with plenty of potential for foot-shooting, of course.
> >
> > I would recommend taking a long hard look at whether the network design you
> > describe
> > is actually suited for your purpose.
> >
> > That said, there may well be factors in play that have not come to light
> > here.
> >
> > Have fun!
> >
> > --
> > Peter N. M. Hansteen, member of the first RFC 1149 implementation team
> > https://bsdly.blogspot.com/ https://www.bsdly.net/ https://www.nuug.no/
> > "Remember to set the evil bit on all malicious network traffic"
> > delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
>
--
