Gave this a spin on my test cluster, and it worked as advertised. As already mentioned and talked off-list we should imo add documentation for this setting and mention the current restrictions (no migration of NAT entries). We might also want to mention that only CT entries created *after* the firewall is enabled for a VM get migrated, although that is probably rather uncommon.
Other than that consider this: Tested-by: Stefan Hanreich <s.hanre...@proxmox.com> I've also mainly looked at the firewall + pve-manager code, maybe someone with more knowledge about QEMU can chime in on the DBus server parts. Consider the firewall and ui patches: Reviewed-by: Stefan Hanreich <s.hanre...@proxmox.com> On 4/24/25 13:19, Christoph Heiss wrote: > Fixes #5180 [0]. > > This implements migration of per-VM conntrack state on live-migration. > > The core of the implementation are in patch #7 & #8. See there for more > details. > > Patch #1 - #3 implement CONNMARK'ing any VM traffic with their unique > VMID. This is needed later on to filter conntrack entries for the > migration. These three patches can be applied independently, > CONNMARK'ing traffic does not have any visible impact. > > Currently, remote/inter-cluster migration is not supported and indicated > to the user with a warning. See also patch #8 for a bit more in-depth > explanation. > > Needed dependency bumps between packages are indicated in the notes > appropriately. > > [0] https://bugzilla.proxmox.com/show_bug.cgi?id=5180 > > Testing > ======= > > I've primarily tested intra-cluster live-migrations, with both the > iptables-based and nftables-based firewall), using the reproducer as > described in #5180. I further verified that the D-Bus servers get > started as expected and are _always_ stopped, even in the case of some > migration error. > > Finally, I also checked using `conntrack -L -m <vmid>` tool that the > conntrack entries are > a) added/updated on the target node and > b) removed from the source node afterwards > > Also tested was the migration from/to an "old" (unpatched) node, which > results in the issue as per #5180 & appropriate warnings in the UI. > > For remote migrations, only tested that the warning is logged as > expected. > > History > ======= > > v1: > https://lore.proxmox.com/pve-devel/20250317141152.1247324-1-c.he...@proxmox.com/ > > Changes v1 -> v2: > * rebased as necessary > * "un-rfc'd" firewall conntrack flushing patches > * use an instanced systemd service instead of fork+exec for the > pve-dbus-vmstate helper > > Diffstat > ======== > > pve-firewall: > > Christoph Heiss (2): > firewall: add connmark rule with VMID to all guest chains > firewall: helpers: add sub for flushing conntrack entries by mark > > debian/control | 3 ++- > src/PVE/Firewall.pm | 7 +++++-- > src/PVE/Firewall/Helpers.pm | 11 +++++++++++ > 3 files changed, 18 insertions(+), 3 deletions(-) > > proxmox-firewall: > > Christoph Heiss (1): > firewall: add connmark rule with VMID to all guest chains > > proxmox-firewall/src/firewall.rs | 14 +++- > .../integration_tests__firewall.snap | 84 +++++++++++++++++++ > proxmox-nftables/src/expression.rs | 9 ++ > proxmox-nftables/src/statement.rs | 10 ++- > 4 files changed, 114 insertions(+), 3 deletions(-) > > proxmox-ve-rs: > > Christoph Heiss (1): > config: guest: allow access to raw Vmid value > > proxmox-ve-config/src/guest/types.rs | 4 ++++ > 1 file changed, 4 insertions(+) > > qemu-server: > > Christoph Heiss (5): > qmp helpers: allow passing structured args via qemu_objectadd() > api2: qemu: add module exposing node migration capabilities > fix #5180: libexec: add QEMU dbus-vmstate daemon for migrating > conntrack > fix #5180: migrate: integrate helper for live-migrating conntrack info > migrate: flush old VM conntrack entries after successful migration > > Makefile | 7 +- > PVE/API2/Qemu.pm | 72 +++++++++++ > PVE/API2/Qemu/Makefile | 2 +- > PVE/API2/Qemu/Migration.pm | 46 +++++++ > PVE/CLI/qm.pm | 5 + > PVE/QemuMigrate.pm | 69 ++++++++++ > PVE/QemuServer.pm | 6 + > PVE/QemuServer/DBusVMState.pm | 120 ++++++++++++++++++ > PVE/QemuServer/Makefile | 1 + > PVE/QemuServer/QMPHelpers.pm | 4 +- > dbus-vmstate/Makefile | 7 ++ > dbus-vmstate/dbus-vmstate | 168 +++++++++++++++++++++++++ > dbus-vmstate/org.qemu.VMState1.conf | 11 ++ > dbus-vmstate/pve-dbus-vmstate@.service | 10 ++ > debian/control | 7 +- > 15 files changed, 530 insertions(+), 5 deletions(-) > create mode 100644 PVE/API2/Qemu/Migration.pm > create mode 100644 PVE/QemuServer/DBusVMState.pm > create mode 100644 dbus-vmstate/Makefile > create mode 100755 dbus-vmstate/dbus-vmstate > create mode 100644 dbus-vmstate/org.qemu.VMState1.conf > create mode 100644 dbus-vmstate/pve-dbus-vmstate@.service > > pve-manager: > > Christoph Heiss (4): > api2: capabilities: explicitly import CPU capabilities module > api2: capabilities: proxy index endpoints to respective nodes > api2: capabilities: expose new qemu/migration endpoint > ui: window: Migrate: add checkbox for migrating VM conntrack state > > PVE/API2/Capabilities.pm | 9 +++++ > www/manager6/window/Migrate.js | 73 ++++++++++++++++++++++++++++++++-- > 2 files changed, 78 insertions(+), 4 deletions(-) > _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
