[pve-devel] [PATCH manager] API: add node address(es) API endpoint

2021-04-13 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- PVE/API2/Nodes.pm | 70 +++ 1 file changed, 70 insertions(+) diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm index ba6621c6..b30d0739 100644 --- a/PVE/API2/Nodes.pm +++ b/PVE/API2/Nodes.pm @@ -222,6 +222,7 @@ __

[pve-devel] [PATCH common 1/2] schema: pull out abstract 'id-pair' verifier

2021-04-13 Thread Fabian Grünbichler
we'll need another one for guest bridge IDs Signed-off-by: Fabian Grünbichler --- src/PVE/JSONSchema.pm | 22 ++ 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/src/PVE/JSONSchema.pm b/src/PVE/JSONSchema.pm index 4864549..f2ddb50 100644 --- a/src/PVE/JSONSchema

[pve-devel] [PATCH cluster 4/4] get_remote_info: also return FP if available

2021-04-13 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- data/PVE/RemoteConfig.pm | 21 - 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/data/PVE/RemoteConfig.pm b/data/PVE/RemoteConfig.pm index 563e5c1..c4b8499 100644 --- a/data/PVE/RemoteConfig.pm +++ b/data/PVE/RemoteConfig.pm

[pve-devel] [PATCH qemu-server 6/7] migrate: add remote migration handling

2021-04-13 Thread Fabian Grünbichler
remote migration uses a websocket connection to a task worker running on the target node instead of commands via SSH to control the migration. this websocket tunnel is started earlier than the SSH tunnel, and allows adding UNIX-socket forwarding over additional websocket connections on-demand. the

[pve-devel] [PATCH qemu-server 5/7] migrate: refactor remote VM/tunnel start

2021-04-13 Thread Fabian Grünbichler
no semantic changes intended, except for: - no longer passing the main migration UNIX socket to SSH twice for forwarding - dropping the 'unix:' prefix in start_remote_tunnel's timeout error message Signed-off-by: Fabian Grünbichler --- PVE/QemuMigrate.pm | 154 +++

[pve-devel] [PATCH access-control 1/2] tickets: add tunnel ticket

2021-04-13 Thread Fabian Grünbichler
just like VNC ticket, but different prefix to prevent confusion. Signed-off-by: Fabian Grünbichler --- PVE/AccessControl.pm | 50 +--- 1 file changed, 38 insertions(+), 12 deletions(-) diff --git a/PVE/AccessControl.pm b/PVE/AccessControl.pm index 8b5be1e

[pve-devel] [PATCH qemu-server 2/7] refactor map_storage to map_id

2021-04-13 Thread Fabian Grünbichler
since we are going to reuse the same mechanism/code for network bridge mapping. Signed-off-by: Fabian Grünbichler --- PVE/QemuMigrate.pm | 8 PVE/QemuServer.pm | 6 -- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index 5c019

[pve-devel] [PATCH access-control 2/2] ticket: normalize path for verification

2021-04-13 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- PVE/AccessControl.pm | 2 ++ 1 file changed, 2 insertions(+) diff --git a/PVE/AccessControl.pm b/PVE/AccessControl.pm index 9d9a4bd..7949fde 100644 --- a/PVE/AccessControl.pm +++ b/PVE/AccessControl.pm @@ -461,6 +461,8 @@ my $assemble_short_lived_ticket = su

[pve-devel] [PATCH qemu-server 1/7] migrate: factor out storage checks

2021-04-13 Thread Fabian Grünbichler
to re-use them for incoming remote migrations. Signed-off-by: Fabian Grünbichler --- PVE/API2/Qemu.pm | 26 ++ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index c56b609..a789456 100644 --- a/PVE/API2/Qemu.pm +++ b/PV

[pve-devel] [RFC qemu-server++ 0/22] remote migration

2021-04-13 Thread Fabian Grünbichler
this series adds remote migration for VMs. there's still plenty of TODOs/FIXMEs/stuff that requires discussion, hence the RFC. live migration with NBD and storage-migrated disks should work already. the performance bottle neck (~190MB/s on loopback) for the websocket connection seems to be in pvep

[pve-devel] [PATCH cluster 1/4] remote.cfg: add new config file

2021-04-13 Thread Fabian Grünbichler
with two section/entry types: pve-cluster, referencing at least one node + an API token pve-node, containing the connection information (address + optional fingerprint) Signed-off-by: Fabian Grünbichler --- data/PVE/Makefile | 2 +- data/src/status.c | 1 +

[pve-devel] [PATCH qemu-server 3/7] schema: use pve-bridge-id

2021-04-13 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- Notes: requires pve-common with pve-bridge-id PVE/QemuServer.pm | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 685a191..d323d3d 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.p

[pve-devel] [PATCH guest-common] migrate: handle migration_network with remote migration

2021-04-13 Thread Fabian Grünbichler
we only want to use an explicitly provided migration network, not one for the local cluster. Signed-off-by: Fabian Grünbichler --- PVE/AbstractMigrate.pm | 51 +- 1 file changed, 31 insertions(+), 20 deletions(-) diff --git a/PVE/AbstractMigrate.pm b/PVE/

[pve-devel] [PATCH proxmox 1/2] websocket: make field public

2021-04-13 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- proxmox/src/tools/websocket.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proxmox/src/tools/websocket.rs b/proxmox/src/tools/websocket.rs index 57e2591..8685aab 100644 --- a/proxmox/src/tools/websocket.rs +++ b/proxmox/src/tools/webso

[pve-devel] [PATCH cluster 3/4] remote: add option/completion

2021-04-13 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- data/PVE/RemoteConfig.pm | 20 ++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/data/PVE/RemoteConfig.pm b/data/PVE/RemoteConfig.pm index 7c395ba..563e5c1 100644 --- a/data/PVE/RemoteConfig.pm +++ b/data/PVE/RemoteConfig.pm

[pve-devel] [PATCH qemu-server 4/7] mtunnel: add API endpoints

2021-04-13 Thread Fabian Grünbichler
the following two endpoints are used for migration on the remote side POST /nodes/NODE/qemu/VMID/mtunnel which creates and locks an empty VM config, and spawns the main qmtunnel worker which binds to a VM-specific UNIX socket. this worker handles JSON-encoded migration commands coming in via thi

[pve-devel] [PATCH proxmox-websocket-tunnel 2/2] add tunnel implementation

2021-04-13 Thread Fabian Grünbichler
the websocket tunnel helper accepts control commands (encoded as single-line JSON) on stdin, and prints responses on stdout. the following commands are available: - "connect" a 'control' tunnel via a websocket - "forward" a local unix socket to a remote socket via a websocket -- if requested, this

[pve-devel] [PATCH qemu-server 7/7] api: add remote migrate endpoint

2021-04-13 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- Notes: requires - pve-manager with 'addr' API endpoint on target node - pve-cluster with RemoteConfig support on local node - pve-common with bridgepair format - pve-guest-common with AbstractMigrate handling remote migration PVE/API2/Qe

[pve-devel] [PATCH cluster 2/4] add get_remote_info

2021-04-13 Thread Fabian Grünbichler
as a unified helper for talking to a remote node. if the requested node has an entry in the remote config, the information from that entry is used. else, the first locally defined node of the requested cluster is used as proxy. Signed-off-by: Fabian Grünbichler --- data/PVE/RemoteConfig.pm | 55

[pve-devel] [PATCH storage] import: allow import from UNIX socket

2021-04-13 Thread Fabian Grünbichler
this allows forwarding over websockets without requiring a (free) port. Signed-off-by: Fabian Grünbichler --- PVE/CLI/pvesm.pm | 26 +- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/PVE/CLI/pvesm.pm b/PVE/CLI/pvesm.pm index 7b46897..9206188 100755 --- a/PV

[pve-devel] [PATCH proxmox 2/2] websocket: adapt for client connection

2021-04-13 Thread Fabian Grünbichler
previously, this was only used for the server side handling of web sockets. by making the mask part of the WebSocket struct and making some of the fns associated, we can re-use this for client-side connections such as in proxmox-websocket-tunnel. Signed-off-by: Fabian Grünbichler --- proxmox/src

[pve-devel] [PATCH common 2/2] schema: add pve-bridge-id option/format/pair

2021-04-13 Thread Fabian Grünbichler
for re-use in qemu-server/pve-container, which already have this option duplicated. the '-pair' is needed for remote migration, but can also be a nice addition to regular intra-cluster migration to lift the restriction of having identically named bridges. Signed-off-by: Fabian Grünbichler --- sr

[pve-devel] [PATCH proxmox-websocket-tunnel 1/2] initial commit

2021-04-13 Thread Fabian Grünbichler
Signed-off-by: Fabian Grünbichler --- .gitignore| 1 + .cargo/config | 5 + Cargo.toml| 11 +++ 3 files changed, 17 insertions(+) create mode 100644 .gitignore create mode 100644 .cargo/config create mode 100644 Cargo.toml diff --git a/.gitignore b/.gitignore new file mo

Re: [pve-devel] [PATCH installer] highlight focused button with dark orange

2021-04-13 Thread Stoiko Ivanov
Thanks for the patch! can see that it could help users, who (have to) navigate the installer with a keyboard... gave it a (very quick) spin with the check-pve make-target - in general it seems ok to me one thing that does not look too nice to me is the +/- buttons for integer entry (click on Opti

Re: [pve-devel] [PATCH] Add GUI to import disk & VM

2021-04-13 Thread Oguz Bektas
hi, tested along with the qemu-server patch, it seems to work but had some issues in some cases (especially with windows VMs) we discussed already off-list with dominic during the testing, but i'll just put these down here anyway: - the "Next" button should be grayed out until remote manifest i

[pve-devel] [RFC qemu-server] avoid setting lun number for drives when pvscsi controller is used

2021-04-13 Thread Fabian Ebner
Reported in the community forum[0]. In QEMU's hw/scsi/vmw_pvscsi.c in the SCSIBusInfo struct, the max_lun property is set to 0. This means that in our stack, one cannot have multiple disks and use 'scsihw: pvscsi' currently, as kvm would fail with bad scsi device lun: 1 Instead of increasing

Re: [pve-devel] [PATCH v6 storage 1/1] add disk reassign feature

2021-04-13 Thread Dominic Jäger
Needs a rebase since the rbd patches On Fri, Apr 02, 2021 at 12:19:19PM +0200, Aaron Lauterer wrote: > +sub reassign_volume { > +my ($class, $scfg, $storeid, $volname, $target_vmid) = @_; > + > +my $base;; > +(undef, $volname, undef, $base) = $class->parse_volname($volname); > + > +

Re: [pve-devel] [pbs-devel] applied: [PATCH proxmox-backup 2/2] api/datastore: allow pxar file download of entire archive

2021-04-13 Thread Thomas Lamprecht
On 13.04.21 09:23, Dominik Csapak wrote: > On 4/13/21 08:39, Thomas Lamprecht wrote: >> But that API is definitively weird in general... > > just fyi > >> >> 1. old style API definition, should use the #[api()] macro instead > > the api macro cannot handle AsyncHttp api calls (yet?), but this is

Re: [pve-devel] [pbs-devel] applied: [PATCH proxmox-backup 2/2] api/datastore: allow pxar file download of entire archive

2021-04-13 Thread Dominik Csapak
On 4/13/21 08:39, Thomas Lamprecht wrote: On 12.04.21 17:32, Stefan Reiter wrote: Treat filepaths like "/root.pxar.didx" without a trailing slash as wanting to download the entire archive content instead of erroring. The zip-creation code already works fine for this scenario. Signed-off-by: Ste

[pve-devel] applied: [pbs-devel] [PATCH proxmox-backup 3/3] use FileBrowser from proxmox-widget-toolkit

2021-04-13 Thread Thomas Lamprecht
On 01.04.21 17:34, Stefan Reiter wrote: > Signed-off-by: Stefan Reiter > --- > > Requires bumped proxmox-widget-toolkit of course. > > www/Makefile | 1 - > www/datastore/Content.js | 13 +- > www/window/FileBrowser.js | 252 -- > 3 files cha

[pve-devel] applied: [PATCH widget-toolkit] toolkit: override email VType validation

2021-04-13 Thread Thomas Lamprecht
On 08.04.21 14:41, Stoiko Ivanov wrote: > ExtJS email validation regex has a length limit of 6 on the TLD. > This breaks some new gTLDs (e.g. .systems) > > Override the validation function and verify against the EMAILRE from > PVE::Tools (used for 'e-mail' in JSONSchema) > > Reported in our commu

[pve-devel] applied: [PATCH v5 widget-toolkit 1/3] window/edit: add option to disable reset button

2021-04-13 Thread Thomas Lamprecht
On 15.02.21 16:26, Aaron Lauterer wrote: > Sometimes the reset button does not make sense and the isCreate option > does not fit as well because with it, the submit button will be enabled > right away instead of waiting for the form to be valid. > > Signed-off-by: Aaron Lauterer > --- > > v1 ->

[pve-devel] applied: [PATCH proxmox-widget-toolkit 1/2] FileBrowser: allow downloading root folder and simplify code

2021-04-13 Thread Thomas Lamprecht
On 12.04.21 17:32, Stefan Reiter wrote: > Signed-off-by: Stefan Reiter > --- > > Smoke tested by downloading an entire container archive as zip, extracting and > checking the files. If there was a reason for this to be disabled let me know. > > Based on my previous series to move the FileBrowser