Hi,
So, what I have done is that I have /31 subnets for jails (every jail has it's
own subnet, with host on lower address and jail on higher).
I just put IP addresses directly on epair interfaces and use host as a router.
No bridge interfaces at all.
So every jail can reach any other jail via it's address, and if I transfer said
jail to another host, it retains it's address.
The jail's /31 subnets are announced via BIRD running on host to the rest of my
infrastructure.
I also have DNS at address 10.0.0.0 and 10.0.0.255 - these two addresses are as
/32 on loopback in one jail, and host has route set to them (again via bird
with BGP).
Each host has the same jail with dns at the same anycast address 10.0.0.0 and
10.0.0.255 (and of course different unicast address so that I can actually ssh
into them) and BGP takes care of figuring out which one is the closest (usually
the one on the same host), and if one goes down, then BIRD recalculates route
to the next best one.
That's basically anycast setup with host as router. And the host would have to
peer with another host(s) and probably upstream router/firewall as well - so
that the rest of the network knows routes to your jails.
This also works well with ipv6 however I stopped using it due to unrelated
problems with tunnelbroker.net (and my ISP doesn't support it)
And if you wouldn't want to have the DNS address accessible outside of host -
you can just set up BIRD so that it doesn't redistribute that prefix
Keep in mind that this setup makes it so that the jail's network is NOT the
same network that is on the wire - it's completely virtual, and traffic from it
only appears on wire if it leaves host. And thus jails don't see broadcast
traffic from real network so that would for example prevent you from running
DHCP server inside of jail. Or basically anything that requires L2 connectivity
for one reason or another.
This may be very much overkill for most use cases (mine included) but it has
been rock solid for the past 3 years that I had this setup. Currently it's
scaled down to one host but if I would need it, I could add another one in a
couple of hours.
Best regards
Łukasz
On 12/10/25 12:10, Anthony Pankov wrote:
Hello,
I'm again facing the problem of providing host-wide interface to all jails in a
host.
I want (for example) that there is a jail with unbound (DNS resolver/cacher)
inside, binded to a host-wide interface. Host-wide interface has a well known
address (IPv6 1::1/128 for example).
All other jails simply put "nameserver 1::1" in its resolv.conf.
The problem that jails can't share one ip address ("address clashes") and can't
"inherit" one interface only. Inheriting is for a whole network as I see.
May be someone have some suggestions?
P.S. Sometime ago I solve this in a not a clean manner by supplying own loopback
interface and unique address (127.0.X.1) to every jail in a host. Then jail with unbound
bind to its 127.0.U.1, other jail must set "nameserver 127.0.U.1". This
solution has administrative problems because of a dynamic nature of addresses
(127.0.X.1). Thing get worse if jails transferred to a different host.
