*sigh*
Found this is the code:
#ifdef VIR_NFTABLES_INCLUDE_HOST_RULES
/* These rules copied from the iptables backend, have been removed
* from the nftab because they are redundant since we are using our
own
* table that is default accept; there are no other users that
* could add a reject rule that we would need to / be able to
* override with these rules
*/
/* allow DHCP requests through to dnsmasq & back out */
nftablesAddTcpInput(fw, VIR_FIREWALL_LAYER_IPV4, def->bridge, 67);
nftablesAddUdpInput(fw, VIR_FIREWALL_LAYER_IPV4, def->bridge, 67);
nftablesAddTcpOutput(fw, VIR_FIREWALL_LAYER_IPV4, def->bridge, 68);
nftablesAddUdpOutput(fw, VIR_FIREWALL_LAYER_IPV4, def->bridge, 68);
Turns out this is correct, so it *was* an nftables ordering problem, I just
didn't know that libvirt didn't lay those rules down in nftables mode so it
looked broken.
Once I removed the relevant reject in my nftables config, it worked.
On Fri, Feb 21, 2025 at 12:06 AM Robin Powell <robinleepow...@gmail.com>
wrote:
> Setting `firewall_backend = "iptables"` and allowing it to use the
> iptables-nft shim *did* get the right rules into place.
>
> I then ran smack into the "my main input chain has a default deny rule",
> but that's my problem. :)
>
> On Thu, Feb 20, 2025 at 10:33 PM <robinleepow...@gmail.com> wrote:
>
>> I do not think this is any of the similar issues that have been
>> posted to this list; I've checked.
>>
>> In particular, this in *NOT* an issue with nft rules precedence; the
>> rules are simply not being written by libvirt.
>>
>> I'm running vagrant which is running libvirt on a Fedora 41 host. I
>> do not think this is a vagrant problem, but if people want me to run
>> virsh commands directly I certainly can.
>>
>> Anyway, vagrant was failing until I noticed that the default for
>> /etc/libvirt/network.conf is now nftables, which I did not have set
>> up. When I set `firewall_backend = "iptables"`, everything worked
>> fine.
>>
>> I want to emphasize that: the same vagrant / libvirt setup *was*
>> working with iptables.
>>
>> But I took that as a sign that it was time to move to nftables, so I
>> moved everything on this host and stuff is back to working.
>>
>> But libvirt just isn't writing out the right nft rules. Like, at
>> all.
>>
>> Here's the network vagrant creates:
>>
>> $ sudo virsh net-dumpxml vagrant-libvirt
>> <network connections='1' ipv6='yes'>
>> <name>vagrant-libvirt</name>
>> <uuid>b2d93ef4-b305-4382-a380-c1eca92d8ebd</uuid>
>> <forward mode='nat'>
>> <nat>
>> <port start='1024' end='65535'/>
>> </nat>
>> </forward>
>> <bridge name='virbr1' stp='on' delay='0'/>
>> <mac address='52:54:00:63:d3:f6'/>
>> <ip address='192.168.121.1' netmask='255.255.255.0'>
>> <dhcp>
>> <range start='192.168.121.1' end='192.168.121.254'/>
>> </dhcp>
>> </ip>
>> </network>
>>
>> And here's the *entire* libvirt-related ruleset in nftables:
>>
>> table ip libvirt_network {
>> chain forward {
>> type filter hook forward priority filter; policy
>> accept;
>> counter packets 0 bytes 0 jump guest_cross
>> counter packets 0 bytes 0 jump guest_input
>> counter packets 0 bytes 0 jump guest_output
>> }
>>
>> chain guest_output {
>> ip saddr 192.168.121.0/24 iif "virbr1" counter
>> packets 0 bytes 0 accept
>> iif "virbr1" counter packets 0 bytes 0 reject
>> }
>>
>> chain guest_input {
>> oif "virbr1" ip daddr 192.168.121.0/24 ct state
>> established,related counter packets 0 bytes 0 accept
>> oif "virbr1" counter packets 0 bytes 0 reject
>> }
>>
>> chain guest_cross {
>> iif "virbr1" oif "virbr1" counter packets 0 bytes 0
>> accept
>> }
>>
>> chain guest_nat {
>> type nat hook postrouting priority srcnat; policy
>> accept;
>> ip saddr 192.168.121.0/24 ip daddr 224.0.0.0/24
>> counter packets 1 bytes 187 return
>> ip saddr 192.168.121.0/24 ip daddr 255.255.255.255
>> counter packets 0 bytes 0 return
>> meta l4proto tcp ip saddr 192.168.121.0/24 ip daddr
>> != 192.168.121.0/24 counter packets 0 bytes 0 masquerade to :1024-65535
>> meta l4proto udp ip saddr 192.168.121.0/24 ip daddr
>> != 192.168.121.0/24 counter packets 0 bytes 0 masquerade to :1024-65535
>> ip saddr 192.168.121.0/24 ip daddr !=
>> 192.168.121.0/24 counter packets 0 bytes 0 masquerade
>> }
>> }
>> table ip6 libvirt_network {
>> chain forward {
>> type filter hook forward priority filter; policy
>> accept;
>> counter packets 0 bytes 0 jump guest_cross
>> counter packets 0 bytes 0 jump guest_input
>> counter packets 0 bytes 0 jump guest_output
>> }
>>
>> chain guest_output {
>> iif "virbr1" counter packets 0 bytes 0 reject
>> }
>>
>> chain guest_input {
>> oif "virbr1" counter packets 0 bytes 0 reject
>> }
>>
>> chain guest_cross {
>> iif "virbr1" oif "virbr1" counter packets 0 bytes 0
>> accept
>> }
>>
>> chain guest_nat {
>> type nat hook postrouting priority srcnat; policy
>> accept;
>> }
>> }
>>
>> Here's some log output (I have virtnetworkd running with -v):
>>
>> $ sudo journalctl -u virtnetworkd.service | grep -i nft
>> [snip]
>> Feb 20 21:36:42 stodi.digitalkingdom.org virtnetworkd[89119]: using
>> firewall_backend: 'nftables'
>> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft list table ip libvirt_network'
>> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft list table ip6 libvirt_network'
>> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip libvirt_network guest_output iif virbr1
>> counter reject'
>> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip libvirt_network guest_input oif virbr1
>> counter reject'
>> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip libvirt_network guest_cross iif virbr1 oif
>> virbr1 counter accept'
>> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip6 libvirt_network guest_output iif virbr1
>> counter reject'
>> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip6 libvirt_network guest_input oif virbr1
>> counter reject'
>> Feb 20 21:36:46 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip6 libvirt_network guest_cross iif virbr1
>> oif virbr1 counter accept'
>> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip libvirt_network guest_output ip saddr
>> 192.168.121.0/24 iif virbr1 counter accept'
>> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip libvirt_network guest_input oif virbr1 ip
>> daddr 192.168.121.0/24 ct state related,established counter accept'
>> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip libvirt_network guest_nat ip saddr
>> 192.168.121.0/24 ip daddr '!=' 192.168.121.0/24 counter masquerade'
>> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip libvirt_network guest_nat meta l4proto udp
>> ip saddr 192.168.121.0/24 ip daddr '!=' 192.168.121.0/24 counter
>> masquerade to :1024-65535'
>> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip libvirt_network guest_nat meta l4proto tcp
>> ip saddr 192.168.121.0/24 ip daddr '!=' 192.168.121.0/24 counter
>> masquerade to :1024-65535'
>> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip libvirt_network guest_nat ip saddr
>> 192.168.121.0/24 ip daddr 255.255.255.255/32 counter return'
>> Feb 20 21:36:47 stodi.digitalkingdom.org virtnetworkd[89119]:
>> Applying 'nft -ae insert rule ip libvirt_network guest_nat ip saddr
>> 192.168.121.0/24 ip daddr 224.0.0.0/24 counter return'
>>
>> So it's not like trying to create the rules and failing, that I can
>> see; it just isn't trying.
>>
>> To clarify this setup appears to be missing every rule that would be
>> needed for DHCPv4 and DNS to work.
>>
>> Like https://gitlab.com/libvirt/libvirt/-/issues/88#note_694493261
>> shows rules for udp 67 and 53 that are just 100% not being created
>> at all.
>>
>> I have no idea what's going wrong here or even where to look; the
>> code at
>>
>> https://lists.libvirt.org/archives/list/de...@lists.libvirt.org/thread/GT75XY7H3VJB5LTNRG4WYK47WGTQYYCZ/
>> sure *looks like* it should be unconditionally adding those rules.
>>
>> Help?
>>
>
