[pve-devel] [PATCH v3 qemu-server 09/10] migrate: add remote migration handling

2021-12-22 Thread Fabian Grünbichler
remote migration uses a websocket connection to a task worker running on the target node instead of commands via SSH to control the migration. this websocket tunnel is started earlier than the SSH tunnel, and allows adding UNIX-socket forwarding over additional websocket connections on-demand. the

[pve-devel] [PATCH v3 qemu-server 10/10] api: add remote migrate endpoint

2021-12-22 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- Notes: I left the checks here, but for the final version / proper CLI they should move to a separate API endpoint or even just done client-side. PVE/API2/Qemu.pm | 206 ++- debian/control | 2 + 2 f

[pve-devel] [PATCH v3 qemu-server 08/10] migrate: refactor remote VM/tunnel start

2021-12-22 Thread Fabian Grünbichler
no semantic changes intended, except for: - no longer passing the main migration UNIX socket to SSH twice for forwarding - dropping the 'unix:' prefix in start_remote_tunnel's timeout error message Signed-off-by: Fabian Grünbichler --- Notes: v2: fixed nits PVE/QemuMigrate.pm | 158 +++

[pve-devel] [PATCH v3 storage 3/4] storage_migrate: pull out import/export_prepare

2021-12-22 Thread Fabian Grünbichler
for re-use with remote migration, where import and export happen on different clusters connected via a websocket instead of SSH tunnel. no semantic changes intended. Signed-off-by: Fabian Grünbichler --- new in v3 using $opts across the board in storage_migrate to prevent future bugs where opti

[pve-devel] [PATCH v3 storage 4/4] add volume_import/export_start helpers

2021-12-22 Thread Fabian Grünbichler
exposing the two halves of a storage migration for usage across cluster boundaries. Signed-off-by: Fabian Grünbichler --- Notes: new in v3 PVE/Storage.pm | 66 ++ 1 file changed, 66 insertions(+) diff --git a/PVE/Storage.pm b/PVE/Storage.pm

[pve-devel] [PATCH v3 qemu-server 06/10] migrate: move tunnel-helpers to pve-guest-common

2021-12-22 Thread Fabian Grünbichler
besides the log calls these don't need any parts of the migration state, so let's make them generic and re-use them for container migration and replication in the future. Signed-off-by: Fabian Grünbichler --- Notes: new in v3, requires bumped libpve-guest-common-perl PVE/QemuMigrate.pm

[pve-devel] [PATCH v3 proxmox-websocket-tunnel 4/4] add packaging

2021-12-22 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- Notes: v3: rebased/regenerated Makefile | 44 + debian/changelog | 5 debian/control | 58 debian/copyright | 16 debian/debcargo.t

[pve-devel] [PATCH v3 qemu-server 05/10] nbd alloc helper: allow passing in explicit format

2021-12-22 Thread Fabian Grünbichler
and make $volname optional, to support remote migration usage without code duplication. Signed-off-by: Fabian Grünbichler --- Notes: v2: new requires - pve-storage with UNIX import support - pve-access-control with tunnel ticket support - pve-http-server with websocket f

[pve-devel] [PATCH v3 proxmox-websocket-tunnel 1/4] initial commit

2021-12-22 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- .gitignore| 1 + .cargo/config | 5 + Cargo.toml| 11 +++ 3 files changed, 17 insertions(+) create mode 100644 .gitignore create mode 100644 .cargo/config create mode 100644 Cargo.toml diff --git a/.gitignore b/.gitignore new file mo

[pve-devel] [PATCH v3 proxmox-websocket-tunnel 3/4] add fingerprint validation

2021-12-22 Thread Fabian Grünbichler
in case we have no explicit fingerprint, we use openssl's regular "PEER" verification. if we have a fingerprint, we ignore openssl altogether and just verify the fingerprint of the presented leaf certificate. Signed-off-by: Fabian Grünbichler --- Notes: v3: switch to using hex instead of no-

[pve-devel] [PATCH v3 qemu-server++ 0/21] remote migration

2021-12-22 Thread Fabian Grünbichler
this series adds remote migration for VMs. both live and offline migration including NBD and storage-migrated disks should work. groundwork for extending to pve-container and pvesr already laid. new in v3: lots of refactoring and edge-case handling new in v2: dropped parts already applied, incor

[pve-devel] [PATCH v3 qemu-server 07/10] mtunnel: add API endpoints

2021-12-22 Thread Fabian Grünbichler
the following two endpoints are used for migration on the remote side POST /nodes/NODE/qemu/VMID/mtunnel which creates and locks an empty VM config, and spawns the main qmtunnel worker which binds to a VM-specific UNIX socket. this worker handles JSON-encoded migration commands coming in via thi

[pve-devel] [PATCH v3 guest-common 2/3] add tunnel helper module

2021-12-22 Thread Fabian Grünbichler
lifted from PVE::QemuMigrate, abstracting away use-case specific data. Signed-off-by: Fabian Grünbichler --- src/Makefile | 1 + debian/control| 1 + src/PVE/Tunnel.pm | 356 ++ 3 files changed, 358 insertions(+) create mode 100644 src/PV

[pve-devel] [PATCH v3 guest-common 3/3] add storage tunnel module

2021-12-22 Thread Fabian Grünbichler
encapsulating storage-related tunnel methods, currently - source-side storage-migrate helper - target-side disk-import handler - target-side query-disk-import handler to be extended further with replication-related handlers and helpers. Signed-off-by: Fabian Grünbichler --- Notes: new in v3

[pve-devel] [PATCH v3 storage 1/4] volname_for_storage: parse volname before calling

2021-12-22 Thread Fabian Grünbichler
to allow reusing this with remote migration, where parsing of the source volid has to happen on the source node, but this call has to happen on the target node. Signed-off-by: Fabian Grünbichler --- new in v3 PVE/Storage.pm | 16 +--- 1 file changed, 9 insertions(+), 7 deletions(-)

[pve-devel] [PATCH v3 storage 2/4] storage_migrate: pull out snapshot decision

2021-12-22 Thread Fabian Grünbichler
into new top-level helper for re-use with remote migration. Signed-off-by: Fabian Grünbichler --- new in v3 PVE/Storage.pm | 14 ++ 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/PVE/Storage.pm b/PVE/Storage.pm index 05be3dd..4f3a44a 100755 --- a/PVE/Storage.pm +++ b

[pve-devel] [PATCH v3 qemu-server 03/10] parse_config: optional strict mode

2021-12-22 Thread Fabian Grünbichler
when passing a config from one cluster to another, we want to be strict when parsing - it's better to fail the migration early and upgrade the target node instead of failing the migration later (when significant work for transferring disks and/or state has already been done) or not at all, but sile

[pve-devel] [PATCH v3 qemu-server 04/10] update_vm: allow simultaneous setting of boot-order and dev

2021-12-22 Thread Fabian Grünbichler
else this fails if we check 'boot' before the device was put into the config or pending section. Signed-off-by: Fabian Grünbichler --- Notes: as happens when doing a remote migration and the full config is passed through update_vm_api PVE/API2/Qemu.pm | 2 +- 1 file changed, 1 inserti

[pve-devel] [PATCH v3 qemu-server 02/10] schema: use pve-bridge-id

2021-12-22 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- Notes: requires pve-common with pve-bridge-id PVE/QemuServer.pm | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index db3a0b1..f5beb1b 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.p

[pve-devel] [PATCH v3 proxmox-websocket-tunnel 2/4] add tunnel implementation

2021-12-22 Thread Fabian Grünbichler
the websocket tunnel helper accepts control commands (encoded as single-line JSON) on stdin, and prints responses on stdout. the following commands are available: - "connect" a 'control' tunnel via a websocket - "forward" a local unix socket to a remote socket via a websocket -- if requested, this

[pve-devel] [PATCH v3 qemu-server 01/10] refactor map_storage to map_id

2021-12-22 Thread Fabian Grünbichler
since we are going to reuse the same mechanism/code for network bridge mapping. Signed-off-by: Fabian Grünbichler --- PVE/QemuMigrate.pm | 6 +++--- PVE/QemuServer.pm | 6 -- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index c9bc39d

[pve-devel] [PATCH v3 guest-common 1/3] migrate: handle migration_network with remote migration

2021-12-22 Thread Fabian Grünbichler
remote migration always has an explicit endpoint from the start which gets used for everything. Signed-off-by: Fabian Grünbichler --- src/PVE/AbstractMigrate.pm | 37 + 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/src/PVE/AbstractMigrate.pm

Re: [pve-devel] [PATCH v2 flutter_frontend] fix android splash screen logo

2021-12-22 Thread Dominik Csapak
tested this patch (by manually fixing the line breaking issue) and it looked good (no cut off part of the logo anymore) so Tested-by: Dominik Csapak ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/li

Re: [pve-devel] [PATCH v2 flutter_frontend] fix android splash screen logo

2021-12-22 Thread Aaron Lauterer
Looks like Thunderbird is splitting some long SVG lines when saving the email, making it impossible to apply the patch right away Line 188, 307, 472 are affected. Be careful as there usually is some whitespace at the beginning of those lines which is needed. __