[pve-devel] applied: [PATCH common v7 1/2] JSONSchema: refactor tag regex

2022-09-20 Thread Thomas Lamprecht
Am 21/06/2022 um 11:19 schrieb Dominik Csapak: > we'll use that elsewhere too > > Signed-off-by: Dominik Csapak > --- > src/PVE/JSONSchema.pm | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > applied this one for now already, thanks! _

[pve-devel] applied: [PATCH common] remove PVE::Subscription and friends

2022-09-20 Thread Thomas Lamprecht
Am 13/09/2022 um 14:46 schrieb Fabian Grünbichler: > this has been taken over by Proxmox::RS::Subscription, which is now used > by pve-manager and pmg-api. > > Signed-off-by: Fabian Grünbichler > --- > debian/control | 4 +- > src/PVE/INotify.pm | 78 -- > src/PVE/Su

[pve-devel] applied: [PATCH qemu-server] cleanup pci devices in more situations

2022-09-20 Thread Thomas Lamprecht
Am 27/07/2022 um 15:28 schrieb Dominik Csapak: > if the preparing of pci devices or the start of the vm fails, we need > to cleanup the pci devices (reservations *and* mdevs), or else > it might happen that there are leftovers which must be manually removed. > > to include also mdevs now, refactor

[pve-devel] [PATCH access-control v3 1/1] PVE/AccessControl: add Hardware.* privileges and /hardware/ paths

2022-09-20 Thread Dominik Csapak
so that we can assign privileges on hardware level this will generate a new role (PVEHardwareAdmin) Signed-off-by: Dominik Csapak --- src/PVE/AccessControl.pm | 13 + src/PVE/RPCEnvironment.pm | 3 ++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/src/PVE/AccessCo

[pve-devel] [PATCH qemu-server v3 10/13] PVE/API2/Qemu: migrate preconditions: use new check_local_resources info

2022-09-20 Thread Dominik Csapak
restrict the nodes also for mapped devices, and return them in their own property Signed-off-by: Dominik Csapak --- PVE/API2/Qemu.pm | 15 +++ 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index d6d393f..cc726bd 100644 --- a/PVE/AP

[pve-devel] [PATCH qemu-server v3 02/13] PCI: make mediated device path independent of pci id

2022-09-20 Thread Dominik Csapak
mdevs must have a host-unique uuid, and they appear in /sys/bus/mdev/devices/, so there is no need to reference the pciid with it Signed-off-by: Dominik Csapak --- PVE/QemuServer/PCI.pm | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/PVE/QemuServer/PCI.pm b/PVE/QemuServer/P

[pve-devel] [PATCH qemu-server v3 12/13] fix #3574: enable multi pci device mapping from config

2022-09-20 Thread Dominik Csapak
The hardware config now supports multiple devices as a semicolon seperated list. With this, instead of only having one device in a pci mapping, we now have a list of which we can choose from on vm start. This way one can dynamically start vms with a pool of (identical) pci devices without having to

[pve-devel] [PATCH many v3] add cluster-wide hardware device mapping

2022-09-20 Thread Dominik Csapak
this series aims to add a cluster-wide device mapping for pci and usb devices. so that an admin can configure a device to be availble for migration and configuring for uses that are non-root this version is mostly the same as v2, aside from some bugfixes, rebase and preventing from having mdev: 1

[pve-devel] [PATCH qemu-server v3 11/13] PVE/QemuMigrate: check for mapped resources on migration

2022-09-20 Thread Dominik Csapak
they can only be migrated to nodes where there exists a mapping and if the migration is done offline Signed-off-by: Dominik Csapak --- PVE/QemuMigrate.pm | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index d52dc8d..54530

[pve-devel] [PATCH cluster v3 1/1] add nodes/hardware-map.conf

2022-09-20 Thread Dominik Csapak
to PVE/Cluster.pm and status.c Signed-off-by: Dominik Csapak --- data/PVE/Cluster.pm | 1 + data/src/status.c | 1 + 2 files changed, 2 insertions(+) diff --git a/data/PVE/Cluster.pm b/data/PVE/Cluster.pm index abcc46d..5dfc6b2 100644 --- a/data/PVE/Cluster.pm +++ b/data/PVE/Cluster.pm @@ -76

[pve-devel] [PATCH common v3 1/3] SysFSTools: make mdev cleanup independent of pciid

2022-09-20 Thread Dominik Csapak
mediated devices also appear under /sys/bus/mdev/devices with their uuid, independent of the pci device, so we can use that instead Signed-off-by: Dominik Csapak --- src/PVE/SysFSTools.pm | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/PVE/SysFSTools.pm b/src/PVE/Sy

[pve-devel] [PATCH qemu-server v3 06/13] PVE/QemuServer: allow mapped pci deviced in config

2022-09-20 Thread Dominik Csapak
and get the correct pci device during parsing Signed-off-by: Dominik Csapak --- PVE/QemuServer/PCI.pm | 24 ++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/PVE/QemuServer/PCI.pm b/PVE/QemuServer/PCI.pm index b5284ef..08244c1 100644 --- a/PVE/QemuServer/PCI

[pve-devel] [PATCH manager v3 05/13] ui: form: add USBMapSelector

2022-09-20 Thread Dominik Csapak
similar to PCIMapSelector Signed-off-by: Dominik Csapak --- www/manager6/Makefile | 1 + www/manager6/form/USBMapSelector.js | 73 + 2 files changed, 74 insertions(+) create mode 100644 www/manager6/form/USBMapSelector.js diff --git a/www/manager6/Mak

[pve-devel] [PATCH qemu-server v3 09/13] PVE/QemuServer: extend 'check_local_resources' for mapped resources

2022-09-20 Thread Dominik Csapak
by adding them to their own list, saving the nodes where they are not allowed, and return those on 'wantarray' Signed-off-by: Dominik Csapak --- PVE/QemuServer.pm| 36 ++-- test/MigrationTest/Shared.pm | 7 +++ 2 files changed, 41 insertions(+), 2

[pve-devel] [PATCH manager v3 04/13] ui: form: add PCIMapSelector

2022-09-20 Thread Dominik Csapak
akin to the PCISelector, but uses the api for mapped devices Signed-off-by: Dominik Csapak --- www/manager6/Makefile | 1 + www/manager6/form/PCIMapSelector.js | 102 2 files changed, 103 insertions(+) create mode 100644 www/manager6/form/PCIMapSelec

[pve-devel] [PATCH manager v3 08/13] ui: form: add MultiPCISelector

2022-09-20 Thread Dominik Csapak
this is a grid field for selecting multiple pci devices at once, like we need for the mapped pci ui. There we want to be able to select multiple devices such that one gets selected automatically we can select a whole slot here, but that disables selecting the individual functions of that device. w

[pve-devel] [PATCH qemu-server v3 12/13] fix #3574: enable multi pci device mapping from config

2022-09-20 Thread Dominik Csapak
The hardware config now supports multiple devices as a semicolon seperated list. With this, instead of only having one device in a pci mapping, we now have a list of which we can choose from on vm start. This way one can dynamically start vms with a pool of (identical) pci devices without having to

[pve-devel] [PATCH manager v3 11/13] ui: add dc/HardwareView: a CRUD interface for hardware mapping

2022-09-20 Thread Dominik Csapak
it's possible to add/edit/remove mappings here, with a cluster wide view on the mappings and validity. to do that, we have to to an api call for each node, since we don't have the pci status synced across them. Signed-off-by: Dominik Csapak --- www/manager6/Makefile | 1 + www/manag

[pve-devel] [PATCH qemu-server v3 05/13] PVE/QemuServer: allow mapped usb devices in config

2022-09-20 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- PVE/QemuServer.pm | 2 ++ PVE/QemuServer/USB.pm | 21 - 2 files changed, 22 insertions(+), 1 deletion(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 0a573de..be38e10 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@

[pve-devel] [PATCH qemu-server v3 03/13] PCI: refactor print_pci_device

2022-09-20 Thread Dominik Csapak
into a private sub. This makes the 'print_hostpci_devices' function more easier to read Signed-off-by: Dominik Csapak --- PVE/QemuServer/PCI.pm | 57 +-- 1 file changed, 33 insertions(+), 24 deletions(-) diff --git a/PVE/QemuServer/PCI.pm b/PVE/QemuServer

[pve-devel] [PATCH qemu-server v3 13/13] add tests for mapped pci devices

2022-09-20 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- test/cfg2cmd/q35-linux-hostpci-mapping.conf | 17 + .../q35-linux-hostpci-mapping.conf.cmd| 36 + test/run_config2command_tests.pl | 76 +++ 3 files changed, 129 insertions(+) create mode 100644 test/cfg2cmd/q35

[pve-devel] [PATCH qemu-server v3 01/13] cleanup pci devices in more situations

2022-09-20 Thread Dominik Csapak
if the preparing of pci devices or the start of the vm fails, we need to cleanup the pci devices (reservations *and* mdevs), or else it might happen that there are leftovers which must be manually removed. to include also mdevs now, refactor the cleanup code from 'vm_stop_cleanup' into it's own fu

[pve-devel] [PATCH common v3 3/3] HardwareMap: add support for multiple pci device paths per mapping

2022-09-20 Thread Dominik Csapak
With this, we can now tell qemu-server to choose the first avaiable devices, which makes using vGPUs and SR-IOV capable devices much easier to use, since the user does not have to hardcode the device, but can give a list of identical ones, and qemu-server chooses dynamically. note that we require

[pve-devel] [PATCH qemu-server v3 04/13] PCI: reuse parsed info from print_hostpci_devices

2022-09-20 Thread Dominik Csapak
instead of parsing the config again when trying to reserver/prepare the pci devices. also split the preparing into non-mdev devices and mdev devices, this will come in handy later. Signed-off-by: Dominik Csapak --- PVE/QemuServer.pm | 20 +--- PVE/QemuServer/PCI.pm | 8 +

[pve-devel] [PATCH qemu-server v3 11/13] PVE/QemuMigrate: check for mapped resources on migration

2022-09-20 Thread Dominik Csapak
they can only be migrated to nodes where there exists a mapping and if the migration is done offline Signed-off-by: Dominik Csapak --- PVE/QemuMigrate.pm | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm index d52dc8d..54530

[pve-devel] [PATCH manager v3 03/13] ui: form/USBSelector: make it more flexible with nodename

2022-09-20 Thread Dominik Csapak
similar to the pciselector, make it accept a plain nodename, or no node at all and provide a setNodename function to keep backwards compatibility, also check pveSelNode for the nodename Signed-off-by: Dominik Csapak --- www/manager6/form/USBSelector.js | 32 1 f

[pve-devel] [PATCH qemu-server v3 07/13] PVE/API2/Qemu: add permission checks for mapped usb devices

2022-09-20 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- PVE/API2/Qemu.pm | 40 +--- PVE/QemuServer.pm | 2 ++ 2 files changed, 39 insertions(+), 3 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 3ec31c2..7afd7a4 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qem

[pve-devel] [PATCH manager v3 02/13] PVE/API2/Cluster: add Hardware mapping list api call

2022-09-20 Thread Dominik Csapak
this is a cluster wide api call that returns the list of mappings in a manner that is easy to consume by the ui (as a tree) it also automatically includes the validity of mappings on the node where it is called. for a consumer of this api call to get a complete picture, it is necessary to do an a

[pve-devel] [PATCH manager v3 12/13] ui: window/Migrate: allow mapped devices

2022-09-20 Thread Dominik Csapak
if the migration is an offline migration and when the mapping on the target node exists, otherwise not this does not change the behaviour for 'raw' devices in the config those can still be forced to be migrated, like before Signed-off-by: Dominik Csapak --- www/manager6/window/Migrate.js | 37 +

[pve-devel] [PATCH manager v3 06/13] ui: qemu/PCIEdit: rework panel to add a mapped configuration

2022-09-20 Thread Dominik Csapak
reworks the panel to use a controller, so that we can easily add the selector for mapped pci devices shows now a selection between 'raw' and 'mapped' devices, where 'raw' ones work like before, and 'mapped' ones take the values form the hardware map config Signed-off-by: Dominik Csapak --- www/

[pve-devel] [PATCH qemu-server v3 13/13] add tests for mapped pci devices

2022-09-20 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- test/cfg2cmd/q35-linux-hostpci-mapping.conf | 17 + .../q35-linux-hostpci-mapping.conf.cmd| 36 + test/run_config2command_tests.pl | 76 +++ 3 files changed, 129 insertions(+) create mode 100644 test/cfg2cmd/q35

[pve-devel] [PATCH manager v3 10/13] ui: add window/USBEdit: edit window for usb mappings

2022-09-20 Thread Dominik Csapak
allows to add a single host mapping for usb entries Signed-off-by: Dominik Csapak --- www/manager6/Makefile | 1 + www/manager6/window/USBEdit.js | 248 + 2 files changed, 249 insertions(+) create mode 100644 www/manager6/window/USBEdit.js diff --git

[pve-devel] [PATCH manager v3 01/13] PVE/API2/Hardware: add Mapping.pm

2022-09-20 Thread Dominik Csapak
adds the basic api calls to list/get/create/update/delete device mappings these api calls are only per node, so it only affects the node specific mapping (thought consistency checks are done for the whole config, e.g if an id exists already on another node with a different type) Signed-off-by: Do

[pve-devel] [PATCH common v3 2/3] add PVE/HardwareMap

2022-09-20 Thread Dominik Csapak
this adds functionality for the hardwaremap config (as json) the format of the config is like this: { usb => { name => { nodename1 => { /* mapping object */ }, nodename2 => { /* mapping object */ } } }, pci => { /* same as above */ },

[pve-devel] [PATCH manager v3 09/13] ui: add window/PCIEdit: edit window for pci mappings

2022-09-20 Thread Dominik Csapak
allows to add a single host mapping for pci entries Signed-off-by: Dominik Csapak --- www/manager6/Makefile| 1 + www/manager6/form/PCISelector.js | 17 +- www/manager6/window/PCIEdit.js | 283 +++ 3 files changed, 300 insertions(+), 1 deletion(-) cr

[pve-devel] [PATCH manager v3 07/13] ui: qemu/USBEdit: add 'mapped' device case

2022-09-20 Thread Dominik Csapak
to be able to select 'mapped' usb devices Signed-off-by: Dominik Csapak --- www/manager6/qemu/USBEdit.js | 36 +++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/www/manager6/qemu/USBEdit.js b/www/manager6/qemu/USBEdit.js index a2204584..1b017bc9 1

[pve-devel] [PATCH qemu-server v3 08/13] PVE/API2/Qemu: add permission checks for mapped pci devices

2022-09-20 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- PVE/API2/Qemu.pm | 54 ++-- 1 file changed, 52 insertions(+), 2 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 7afd7a4..d6d393f 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -26,6 +26,7 @

[pve-devel] [PATCH manager v3 13/13] ui: improve permission handling for hardware

2022-09-20 Thread Dominik Csapak
qemu/HardwareView: with the new Hardware privileges, we want to adapt a few places where we now allow to show the add/edit window with those permissions. form/{PCI,USB}Selector: increase the minHeight property of the PCI/USBSelector, so that the user can see the error message if he has not enoug

[pve-devel] [PATCH qemu-server v3 10/13] PVE/API2/Qemu: migrate preconditions: use new check_local_resources info

2022-09-20 Thread Dominik Csapak
restrict the nodes also for mapped devices, and return them in their own property Signed-off-by: Dominik Csapak --- PVE/API2/Qemu.pm | 15 +++ 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index d6d393f..cc726bd 100644 --- a/PVE/AP

Re: [pve-devel] [PATCH V4 manager 1/2] fix #2822: add lvm, lvmthin & zfs storage for all cluster nodes

2022-09-20 Thread Dominik Csapak
first, the patch does not apply here: Applying: fix #2822: add lvm, lvmthin & zfs storage for all cluster nodes error: corrupt patch at line 335 Patch failed at 0001 fix #2822: add lvm, lvmthin & zfs storage for all cluster nodes also, some comments/nits inline (aside from those, it lo

[pve-devel] [PATCH qemu-server] fix net regex in qm cleanup

2022-09-20 Thread Dominik Csapak
the '+' needs to be in the brackets, otherwise '$1' is not the right id Signed-off-by: Dominik Csapak --- PVE/CLI/qm.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/CLI/qm.pm b/PVE/CLI/qm.pm index 6a2e161..ca5d25f 100755 --- a/PVE/CLI/qm.pm +++ b/PVE/CLI/qm.pm @@ -809,

Re: [pve-devel] [PATCH many v3] add cluster-wide hardware device mapping

2022-09-20 Thread DERUMIER, Alexandre
Hi Dominik, I will try to test it this week. Le mardi 20 septembre 2022 à 14:50 +0200, Dominik Csapak a écrit : > this series aims to add a cluster-wide device mapping for pci and usb > devices. > so that an admin can configure a device to be availble for migration > and > configuring for uses th