Fixes #5180 [0]. This implements migration of per-VM conntrack state on live-migration.
The core of the implementation are in patch #7 & #8. See there for more details. Patch #1 - #3 implement CONNMARK'ing any VM traffic with their unique VMID. This is needed later on to filter conntrack entries for the migration. These three patches can be applied independently, CONNMARK'ing traffic does not have any visible impact. Patch #13 & #14 are marked RFC, as I'm not sure if we need/should implement that. But it's working well and cleanup of old resources is always good IMHO. Currently, remote/inter-cluster migration is not supported and indicated to the user with a warning. See also patch #8 for a bit more in-depth explanation. Needed dependency bumps between packages are indicated in the notes appropriately. [0] https://bugzilla.proxmox.com/show_bug.cgi?id=5180 Testing ======= I've primarily tested intra-cluster live-migrations, with both the iptables-based and nftables-based firewall), using the reproducer as described in #5180. I further verified that the D-Bus servers get started as expected and are _always_ stopped, even in the case of some migration error. Finally, I also checked using `conntrack -L -m <vmid>` tool that the conntrack entries are a) added/updated on the target node and b) removed from the source node (w/ patch #13/#14 applied). Also tested was the migration from/to an "old" (unpatched) node, which results in the issue as per #5180 & appropriate warnings in the UI. For remote migrations, only tested that the warning is logged as expected. Diffstat ======== pve-firewall: Christoph Heiss (2): firewall: add connmark rule with VMID to all guest chains firewall: helpers: add sub for flushing conntrack entries by mark debian/control | 3 ++- src/PVE/Firewall.pm | 7 +++++-- src/PVE/Firewall/Helpers.pm | 11 +++++++++++ 3 files changed, 18 insertions(+), 3 deletions(-) proxmox-firewall: Christoph Heiss (1): firewall: add connmark rule with VMID to all guest chains proxmox-firewall/src/firewall.rs | 14 ++- .../integration_tests__firewall.snap | 85 ++++++++++++++++++- proxmox-nftables/src/expression.rs | 9 ++ proxmox-nftables/src/statement.rs | 10 ++- 4 files changed, 114 insertions(+), 4 deletions(-) proxmox-ve-rs: Christoph Heiss (1): config: guest: allow access to raw Vmid value proxmox-ve-config/src/guest/types.rs | 4 ++++ 1 file changed, 4 insertions(+) qemu-server: Christoph Heiss (5): qmp helpers: allow passing structured args via qemu_objectadd() api2: qemu: add module exposing node migration capabilities fix #5180: libexec: add QEMU dbus-vmstate daemon for migrating conntrack fix #5180: migrate: integrate helper for live-migrating conntrack info migrate: flush old VM conntrack entries after successful migration Makefile | 3 + PVE/API2/Qemu.pm | 72 +++++++++++++++ PVE/API2/Qemu/Makefile | 2 +- PVE/API2/Qemu/Migration.pm | 46 ++++++++++ PVE/CLI/qm.pm | 5 ++ PVE/QemuMigrate.pm | 69 ++++++++++++++ PVE/QemuServer.pm | 6 ++ PVE/QemuServer/DBusVMState.pm | 124 +++++++++++++++++++++++++ PVE/QemuServer/Makefile | 1 + PVE/QemuServer/QMPHelpers.pm | 4 +- debian/control | 7 +- libexec/dbus-vmstate | 164 ++++++++++++++++++++++++++++++++++ org.qemu.VMState1.conf | 11 +++ 13 files changed, 510 insertions(+), 4 deletions(-) create mode 100644 PVE/API2/Qemu/Migration.pm create mode 100644 PVE/QemuServer/DBusVMState.pm create mode 100755 libexec/dbus-vmstate create mode 100644 org.qemu.VMState1.conf pve-common: Christoph Heiss (1): tools: add run_fork_detached() for spawning daemons src/PVE/Tools.pm | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) pve-manager: Christoph Heiss (4): api2: capabilities: explicitly import CPU capabilities module api2: capabilities: proxy index endpoints to respective nodes api2: capabilities: expose new qemu/migration endpoint ui: window: Migrate: add checkbox for migrating VM conntrack state PVE/API2/Capabilities.pm | 9 +++++ www/manager6/window/Migrate.js | 73 ++++++++++++++++++++++++++++++++-- 2 files changed, 78 insertions(+), 4 deletions(-) -- 2.47.1 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
