v4 posted: https://lore.proxmox.com/pve-devel/20250717141530.1471199-1-c.he...@proxmox.com/
On Thu Jul 3, 2025 at 1:54 PM CEST, Christoph Heiss wrote: > Fixes #5180 [0]. > > This implements migration of per-VM conntrack state on live-migration. > > The core of the implementation are in patch #7 & #8. See there for more > details. > > Patch #1 - #3 implement CONNMARK'ing any VM traffic with their unique > VMID. This is needed later on to filter conntrack entries for the > migration. These three patches can be applied independently, > CONNMARK'ing traffic does not have any visible impact. > > Regarding the use of connlabels instead, as was discussed in v2: > Connlabels are (unfortunately) pure bitmasks and cannot be used for > storing arbitrary values such as VMIDs. Thus we are basically stuck with > using connmarks. > > Currently, remote/inter-cluster migration is not supported and indicated > to the user with a warning. See also patch #8 for a bit more in-depth > explanation. > > [0] https://bugzilla.proxmox.com/show_bug.cgi?id=5180 > > Dependencies > ============ > > proxmox-firewall depends on the proxmox-ve-rs changes. > qemu-server depends on the pve-firewall/proxmox-firewall changes. > > pve-manager only soft-depends on the other, as it will detect whether > conntrack migration is supported. > > Testing > ======= > > I've primarily tested intra-cluster live-migrations, with both the > iptables-based and nftables-based firewall), using the reproducer as > described in #5180. I further verified that the D-Bus services get > started as expected and are _always_ stopped, even in the case of some > migration error. > > Finally, I also checked using `conntrack -L -m <vmid>` tool that the > conntrack entries are > a) added/updated on the target node and > b) removed from the source node afterwards > > Also tested was the migration from/to an "old" (unpatched) node, which > results in the issue as per #5180 & appropriate warnings in the UI. > > For remote migrations, tested that the warning is logged as expected. > > History > ======= > > v1: > https://lore.proxmox.com/pve-devel/20250317141152.1247324-1-c.he...@proxmox.com/ > v2: > https://lore.proxmox.com/pve-devel/20250424111941.730528-1-c.he...@proxmox.com/ > > Changes v1 -> v2: > * rebased as necessary > * "un-rfc'd" firewall conntrack flushing patches > * use an instanced systemd service instead of fork+exec for the > pve-dbus-vmstate helper > > Changes v2 -> v3: > * rebased on trixie/latest masters > * added documentation patch > * moved node capability module to > PVE::API2::NodeCapabilities::Qemu::Migration, based on Fiona's > suggestion > > Diffstat > ======== > > proxmox-ve-rs: > > Christoph Heiss (1): > config: guest: allow access to raw Vmid value > > proxmox-ve-config/src/guest/types.rs | 4 ++++ > 1 file changed, 4 insertions(+) > > proxmox-firewall: > > Christoph Heiss (1): > firewall: add connmark rule with VMID to all guest chains > > proxmox-firewall/src/firewall.rs | 14 +++- > .../integration_tests__firewall.snap | 84 +++++++++++++++++++ > proxmox-nftables/src/expression.rs | 9 ++ > proxmox-nftables/src/statement.rs | 10 ++- > 4 files changed, 114 insertions(+), 3 deletions(-) > > pve-firewall: > > Christoph Heiss (2): > firewall: add connmark rule with VMID to all guest chains > firewall: helpers: add sub for flushing conntrack entries by mark > > debian/control | 3 ++- > src/PVE/Firewall.pm | 14 ++++++++++++-- > src/PVE/Firewall/Helpers.pm | 12 ++++++++++++ > 3 files changed, 26 insertions(+), 3 deletions(-) > > qemu-server: > > Christoph Heiss (5): > qmp helpers: allow passing structured args via qemu_objectadd() > api2: qemu: add module exposing node migration capabilities > fix #5180: dbus-vmstate: add daemon for QEMUs dbus-vmstate interface > fix #5180: migrate: integrate helper for live-migrating conntrack info > migrate: flush old VM conntrack entries after successful migration > > Makefile | 4 +- > debian/control | 7 +- > src/Makefile | 1 + > src/PVE/API2/Makefile | 1 + > src/PVE/API2/NodeCapabilities/Makefile | 9 + > .../API2/NodeCapabilities/Qemu/Migration.pm | 48 +++++ > src/PVE/API2/Qemu.pm | 75 ++++++++ > src/PVE/CLI/qm.pm | 5 + > src/PVE/QemuMigrate.pm | 78 ++++++++ > src/PVE/QemuServer.pm | 6 + > src/PVE/QemuServer/DBusVMState.pm | 125 +++++++++++++ > src/PVE/QemuServer/Makefile | 1 + > src/PVE/QemuServer/QMPHelpers.pm | 4 +- > src/dbus-vmstate/Makefile | 11 ++ > src/dbus-vmstate/dbus-vmstate | 168 ++++++++++++++++++ > src/dbus-vmstate/org.qemu.VMState1.conf | 11 ++ > src/dbus-vmstate/pve-dbus-vmstate@.service | 10 ++ > 17 files changed, 560 insertions(+), 4 deletions(-) > create mode 100644 src/PVE/API2/NodeCapabilities/Makefile > create mode 100644 src/PVE/API2/NodeCapabilities/Qemu/Migration.pm > create mode 100644 src/PVE/QemuServer/DBusVMState.pm > create mode 100644 src/dbus-vmstate/Makefile > create mode 100755 src/dbus-vmstate/dbus-vmstate > create mode 100644 src/dbus-vmstate/org.qemu.VMState1.conf > create mode 100644 src/dbus-vmstate/pve-dbus-vmstate@.service > > pve-manager: > > Christoph Heiss (4): > api2: capabilities: explicitly import CPU capabilities module > api2: capabilities: proxy index endpoints to respective nodes > api2: capabilities: expose new qemu/migration endpoint > ui: window: Migrate: add checkbox for migrating VM conntrack state > > PVE/API2/Capabilities.pm | 11 ++++- > www/manager6/window/Migrate.js | 82 ++++++++++++++++++++++++++++++++-- > 2 files changed, 89 insertions(+), 4 deletions(-) > > pve-docs: > > Christoph Heiss (1): > qm: document conntrack state migration for live migrations > > qm.adoc | 16 ++++++++++++++++ > 1 file changed, 16 insertions(+) _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
