Re: [pve-devel] [POC v8 storage 7/8] add backup provider example

2025-04-03 Thread Wolfgang Bumiller
On Thu, Apr 03, 2025 at 02:30:57PM +0200, Wolfgang Bumiller wrote: > From: Fiona Ebner > > The example uses a simple directory structure to save the backups, > grouped by guest ID. VM backups are saved as configuration files and > qcow2 images, with backing files when doing incremental backups. >

[pve-devel] [PATCH qemu-server v15 4/12] control: add virtiofsd as runtime dependency for qemu-server

2025-04-03 Thread Markus Frank
Reviewed-by: Fiona Ebner Reviewed-by: Laurențiu Leahu-Vlăducu Tested-by: Laurențiu Leahu-Vlăducu Tested-by: Daniel Kral Signed-off-by: Markus Frank --- no changes in v15 debian/control | 1 + 1 file changed, 1 insertion(+) diff --git a/debian/control b/debian/control index 81f0fad6..eda357a

[pve-devel] [PATCH v8 storage 6/8] extract backup config: delegate to backup provider for storages that support it

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner Signed-off-by: Fiona Ebner --- No changes to v7. src/PVE/Storage.pm | 11 +++ 1 file changed, 11 insertions(+) diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm index 10a4abc..7174f0f 100755 --- a/src/PVE/Storage.pm +++ b/src/PVE/Storage.pm @@ -1759,6 +1759,17 @@

[pve-devel] [PATCH v8 qemu 05/10] PVE backup: get device info: allow caller to specify filter for which devices use fleecing

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner For providing snapshot-access to external backup providers, EFI and TPM also need an associated fleecing image. The new caller will thus need a different filter. Signed-off-by: Fiona Ebner Reviewed-by: Wolfgang Bumiller --- No changes to v7. pve-backup.c | 9 + 1 fi

Re: [pve-devel] [PATCH ha-manager 05/15] rules: add colocation rule plugin

2025-04-03 Thread Fabian Grünbichler
On March 25, 2025 4:12 pm, Daniel Kral wrote: > Add the colocation rule plugin to allow users to specify inter-service > affinity constraints. > > These colocation rules can either be positive (keeping services > together) or negative (keeping service separate). Their strictness can > also be spec

[pve-devel] applied: [PATCH http-server] fix #6230: increase allowed post size

2025-04-03 Thread Thomas Lamprecht
Am 03.04.25 um 10:27 schrieb Dominik Csapak: > In some situations, e.g. having a large resource mapping, the UI can > generate a request that is bigger than the current limit of 64KiB. > > Our files in pmxcfs can grow up to 1 MiB, so theoretically, a single > mapping can grow to that size. In prac

Re: [pve-devel] [PATCH cluster/guest-common/docs/qemu-server/manager v15 0/12] virtiofs

2025-04-03 Thread Lukas Wagner
On 2025-04-03 12:34, Markus Frank wrote: > Virtio-fs is a shared file system that enables sharing a directory > between host and guest VMs. It takes advantage of the locality of > virtual machines and the hypervisor to get a higher throughput than > the 9p remote file system protocol. > > buil

[pve-devel] applied: [PATCH qemu-server v5 3/4] config: add AMD SEV-SNP support.

2025-04-03 Thread Thomas Lamprecht
Am 31.03.25 um 15:59 schrieb Philipp Giersfeld: > This patch is for enabling AMD SEV-SNP support. > > Where applicable, it extends support for existing SEV(-ES) variables > to SEV-SNP. This means that it retains no-debug and kernel-hashes > options, but the no-key-sharing option is removed. > > T

[pve-devel] applied: [PATCH qemu-server v5 2/4] Convert policy calculation

2025-04-03 Thread Thomas Lamprecht
Am 31.03.25 um 15:59 schrieb Philipp Giersfeld: > Convert policy calculation to use shift operators and OR operation > instead of binary numbers and addition. > > Signed-off-by: Philipp Giersfeld > Reviewed-by: Fiona Ebner > --- > > no changes since last version > > PVE/QemuServer/CPUConfig.

[pve-devel] [PATCH v8 qemu 03/10] PVE backup: factor out helper to initialize backup state stat struct

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner Suggested-by: Wolfgang Bumiller Signed-off-by: Fiona Ebner Reviewed-by: Wolfgang Bumiller --- No changes to v7. pve-backup.c | 62 1 file changed, 38 insertions(+), 24 deletions(-) diff --git a/pve-backup.c b/pve-backup.

[pve-devel] [PATCH manager] api: implement node-independent bulk actions

2025-04-03 Thread Dominik Csapak
To achieve this, start a worker task and use our generic api client to start the tasks on the relevant nodes. The client always points to 'localhost' so we let the pveproxy worry about the proxying etc. We reuse some logic from the startall/stopall/etc. calls, like getting the ordered guest info l

[pve-devel] applied: [PATCH edk2-firmware v5 1/4] Add OVMF targets for AMD SEV-ES and SEV-SNP

2025-04-03 Thread Thomas Lamprecht
Am 31.03.25 um 15:59 schrieb Philipp Giersfeld: > AMD SEV-SNP boots with a single volatile firmware image OVMF.fd via the > -bios option. > > This requires building the `OvmfPkg/OvmfPkgX64.dsc` target. > Also, SEV-ES and SEV-SNP do not support SMM [1,2]. > > Therefore, introduce a new target buil

[pve-devel] applied-series: [PATCH guest-common/qemu-server/manager/docs v7] implement experimental vgpu live migration

2025-04-03 Thread Thomas Lamprecht
Am 11.03.25 um 14:20 schrieb Dominik Csapak: > pve-guest-common: > > Dominik Csapak (2): > mapping: pci: check the mdev configuration on the device too > mapping: pci: add 'live-migration-capable' flag to mappings > > src/PVE/Mapping/PCI.pm | 16 +++- > 1 file changed, 15 inserti

[pve-devel] partially-applied-series: [PATCH v8 storage 0/9] backup provider API

2025-04-03 Thread Thomas Lamprecht
Am 03.04.25 um 14:30 schrieb Wolfgang Bumiller: > v7: > https://lore.proxmox.com/pve-devel/20250401173435.221892-1-f.eb...@proxmox.com/ > > Changes in v8: > - storage: replace backup_vm_available_bitmaps() with > backup_vm_query_incremental() > - qemu: add an explicit optional 'bitmap-mode' par

Re: [pve-devel] [PATCH storage v4 09/27] plugin: introduce new_backup_provider() method

2025-04-03 Thread Andreas Rogge
Am 03.04.25 um 09:24 schrieb Wolfgang Bumiller: On Wed, Apr 02, 2025 at 06:16:57PM +0200, Andreas Rogge wrote: Am 02.04.25 um 10:30 schrieb Wolfgang Bumiller: On Tue, Apr 01, 2025 at 08:21:30PM +0200, Thomas Lamprecht wrote: But I do wonder if - to reduce space-requirements for backing up runn

Re: [pve-devel] [PATCH qemu-server] config: add system and service credentials support

2025-04-03 Thread Lukas Wagner
On 2025-04-03 11:04, Thomas Lamprecht wrote: > Am 03.04.25 um 10:34 schrieb Maximiliano Sandoval: >> >> As per systemd-exec's man page, in total one can pass up to 1MB in >> system credentials. A VM config file is certainly not the vehicle for >> such an amount of data and I am also not fully comf

Re: [pve-devel] [RFC cluster/ha-manager 00/16] HA colocation rules

2025-04-03 Thread Fabian Grünbichler
On April 1, 2025 11:39 am, Daniel Kral wrote: > On 4/1/25 03:50, DERUMIER, Alexandre wrote: >> Small feature request from students && customers: they are a lot >> asking to be able to use vm tags in the colocation/affinity > > Good idea! We were thinking about this too and I forgot to add it to t

[pve-devel] [PATCH manager v15 08/12] api: add resource map api endpoints for directories

2025-04-03 Thread Markus Frank
Reviewed-by: Fiona Ebner Reviewed-by: Laurențiu Leahu-Vlăducu Tested-by: Laurențiu Leahu-Vlăducu Tested-by: Daniel Kral Signed-off-by: Markus Frank --- no changes in v15 PVE/API2/Cluster/Mapping.pm | 7 + PVE/API2/Cluster/Mapping/Dir.pm | 308 ++ PVE/API

[pve-devel] [PATCH follow-up qemu-server] api: migrate preconditions: add schema description for 'mapped-resource-info'

2025-04-03 Thread Dominik Csapak
Since it's an object that has the config-properties as keys (e.g. hostpci0), we have to add all possible values (hostpciX/usbX) to it as optional, like we do for e.g. config GET api calls. The actual info per property is not much currently, just the name of the mapping and for PCI devices if they'

Re: [pve-devel] [PATCH v1 pve-storage 2/8] pluginbase: add high-level plugin API description

2025-04-03 Thread Max Carrara
On Thu Apr 3, 2025 at 9:12 AM CEST, Fabian Grünbichler wrote: > On April 2, 2025 6:31 pm, Max Carrara wrote: > > On Mon Mar 31, 2025 at 5:13 PM CEST, Fabian Grünbichler wrote: > >> On March 26, 2025 3:20 pm, Max Carrara wrote: > >> > Add a short paragraph in DESCRIPTION serving as an introduction a

Re: [pve-devel] [PATCH v1 pve-storage 1/8] pluginbase: introduce PVE::Storage::PluginBase with doc scaffold

2025-04-03 Thread Max Carrara
On Thu Apr 3, 2025 at 9:12 AM CEST, Fabian Grünbichler wrote: > On April 2, 2025 6:31 pm, Max Carrara wrote: > > On Mon Mar 31, 2025 at 5:13 PM CEST, Fabian Grünbichler wrote: > >> On March 26, 2025 3:20 pm, Max Carrara wrote: > >> > +=head2 HOOKS > >> > + > >> > +=cut > >> > + > >> > +# called dur

Re: [pve-devel] [PATCH v1 pve-storage 6/8] pluginbase: document image operation methods

2025-04-03 Thread Max Carrara
On Thu Apr 3, 2025 at 9:23 AM CEST, Fabian Grünbichler wrote: > On April 2, 2025 6:32 pm, Max Carrara wrote: > > On Mon Mar 31, 2025 at 5:12 PM CEST, Fabian Grünbichler wrote: > >> On March 26, 2025 3:20 pm, Max Carrara wrote: > >> > Add documentation for the following methods: > >> > - list_images

Re: [pve-devel] [PATCH cluster/docs/manager/network/proxmox{, -ve-rs, -firewall, -perl-rs} 00/52] Add SDN Fabrics

2025-04-03 Thread Stefan Hanreich
On 4/3/25 15:44, Friedrich Weber wrote: >>> - when removing a fabric, the IP addresses defined on the interfaces >>> remain until the next reboot. I guess the reason is that ifupdown2 >>> doesn't remove IP addresses when the corresponding stanza vanishes. Not >>> sure if this can be easily fixed

Re: [pve-devel] [PATCH-SERIES qemu/storage/qemu-server/container/manager v7 00/37] backup provider API

2025-04-03 Thread Friedrich Weber
On 03/04/2025 09:52, Wolfgang Bumiller wrote: > On Wed, Apr 02, 2025 at 05:15:24PM +0200, Friedrich Weber wrote: >> Hi, I tried v7 very quickly with the backup-provider-dir-example >> storage. Some minor things I noticed below, IMO nothing that can't be >> fixed in follow-ups. >> >> Note that I did

[pve-devel] [PATCH v8 qemu-server 01/11] backup: keep track of block-node size for fleecing

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner For fleecing, the size needs to match exactly what QEMU sees. In particular, EFI disks might be attached with a 'size=' option, meaning that size can be different from the volume's size. Commit 36377acf ("backup: disk info: also keep track of size") introduced size tracking and

[pve-devel] [PATCH v8 qemu 04/10] PVE backup: add target ID in backup state

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner In preparation for allowing multiple backup providers and potentially multiple targets for a given provider. Each backup target can then have its own dirty bitmap and there can be additional checks that the current backup state is actually associated to the expected target. Sig

Re: [pve-devel] [PATCH cluster/docs/manager/network/proxmox{, -ve-rs, -firewall, -perl-rs} 00/52] Add SDN Fabrics

2025-04-03 Thread Friedrich Weber
On 03/04/2025 12:21, Gabriel Goller wrote: > On 03.04.2025 10:30, Friedrich Weber wrote: >> On 28/03/2025 18:12, Gabriel Goller wrote: >>> This series allows the user to add fabrics such as OpenFabric and >>> OSPF over >>> their clusters. >>> >>> Overview >>> >>> >>> This series allows the

Re: [pve-devel] [PATCH-SERIES v2 qemu-server/storage/guest-common 0/7] slightly improve image format handling

2025-04-03 Thread Fabian Grünbichler
for the whole series: Acked-by: Fabian Grünbichler Reviewed-by: Fabian Grünbichler On March 5, 2025 11:49 am, Fiona Ebner wrote: > v1: > https://lore.proxmox.com/pve-devel/20250207125514.42668-1-f.eb...@proxmox.com/ > > Changes in v2: > * different approach, use existing format > * introduce

[pve-devel] [PATCH v8 qemu-server 09/11] backup: future-proof checks for QEMU feature support

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner The features returned by the 'query-proxmox-support' QMP command are booleans, so just checking for definedness is not enough in principle. In practice, a feature is currently always true if defined. Still, fix the checks, should the need to disable support for a feature ever ar

Re: [pve-devel] [PATCH qemu-server v7 6/9] api: enable live migration for marked mapped pci devices

2025-04-03 Thread Thomas Lamprecht
Am 11.03.25 um 14:20 schrieb Dominik Csapak: > They have to be marked as 'live-migration-capable' in the mapping > config, and the driver and qemu must support it. > > For the gui checks, we now return the whole object of the mapped > resources, which includes info like the name and if it's marked

Re: [pve-devel] [PATCH v1 pve-storage 1/8] pluginbase: introduce PVE::Storage::PluginBase with doc scaffold

2025-04-03 Thread Fabian Grünbichler
On April 2, 2025 6:31 pm, Max Carrara wrote: > On Mon Mar 31, 2025 at 5:13 PM CEST, Fabian Grünbichler wrote: >> On March 26, 2025 3:20 pm, Max Carrara wrote: >> > +=head2 HOOKS >> > + >> > +=cut >> > + >> > +# called during addition of storage (before the new storage config got >> > written) >> >

[pve-devel] [PATCH v8 qemu 09/10] PVE backup: backup-access api: indicate situation where a bitmap was recreated

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner The backup-access api keeps track of what bitmap names got used for which devices and thus knows when a bitmap went missing. Propagate this information to the QMP user with a new 'missing-recreated' variant for the taken bitmap action. Signed-off-by: Fiona Ebner Reviewed-by: W

[pve-devel] [PATCH v8 qemu 08/10] PVE backup: implement bitmap support for external backup access

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner There can be one dirty bitmap for each backup target ID for each device (which are tracked in the backup_access_bitmaps hash table). The QMP user can specify the ID of the bitmap it likes to use. This ID is then compared to the current one for the given target and device. If the

[pve-devel] [POC v8 storage 8/8] Borg example plugin

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner Archive names start with the guest type and ID and then the same timestamp format as PBS. Container archives have the following structure: guest.config firewall.config filesystem/ # containing the whole filesystem structure VM archives have the following structure guest.config

[pve-devel] [PATCH v8 manager 1/2] ui: backup: also check for backup subtype to classify archive

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner In anticipation of future storage plugins that might not have PBS-specific formats or adhere to the vzdump naming scheme for backups. Signed-off-by: Fiona Ebner --- No changes to v7. www/manager6/Utils.js | 10 ++ www/manager6/grid/BackupView.js| 4

[pve-devel] [POC v8 storage 7/8] add backup provider example

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner The example uses a simple directory structure to save the backups, grouped by guest ID. VM backups are saved as configuration files and qcow2 images, with backing files when doing incremental backups. Container backups are saved as configuration files and a tar file or squashfs

[pve-devel] [PATCH v8 container 5/7] create: factor out compression option helper

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner In preparation to re-use it for checking potentially untrusted archives. Signed-off-by: Fiona Ebner --- No changes to v7. src/PVE/LXC/Create.pm | 51 +-- 1 file changed, 30 insertions(+), 21 deletions(-) diff --git a/src/PVE/LXC/Creat

[pve-devel] [PATCH v8 qemu-server 04/11] backup: implement backup for external providers

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner The state of the VM's disk images at the time the backup is started is preserved via a snapshot-access block node. Old data is moved to the fleecing image when new guest writes come in. The snapshot-access block node, as well as the associated bitmap in case of incremental backu

[pve-devel] [PATCH v8 manager 2/2] backup: implement backup for external providers

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner Call job_{init,cleanup}() and backup_{init,cleanup}() methods so that backup providers can prepare and clean up for the whole backup job and for individual guest backups. It is necessary to adapt some log messages and special case some things like is already done for PBS, e.g.

[pve-devel] [PATCH v8 container 2/7] backup: implement backup for external providers

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner The filesystem structure is made available as a directory in a consistent manner (with details depending on the vzdump backup mode) just like for regular backup via tar. The backup_container() method of the backup provider is executed in a user namespace with the container's ID

[pve-devel] [PATCH v8 qemu-server 03/11] backup: allow adding fleecing images also for EFI and TPM

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner For the external backup API, it will be necessary to add a fleecing image even for small disks like EFI and TPM, because there is no other place the old data could be copied to when a new guest write comes in. Signed-off-by: Fiona Ebner --- No changes to v7. PVE/VZDump/QemuS

[pve-devel] [PATCH v8 storage 4/8] config api/plugins: let plugins define sensitive properties themselves

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner Hard-coding a list of sensitive properties means that custom plugins cannot define their own sensitive properties for the on_add/on_update hooks. Have plugins declare the list of their sensitive properties in the plugin data. For backwards compatibility, return the previously h

[pve-devel] [PATCH v8 storage 5/8] plugin api: bump api version and age

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner Changes for version 11: * Allow declaring storage features via plugin data. * Introduce new_backup_provider() plugin method. * Allow declaring sensitive properties via plugin data. See the api changelog file for details. Signed-off-by: Fiona Ebner --- No changes to v7. Api

[pve-devel] [PATCH v8 storage 1/8] add storage_has_feature() helper function

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner Which looks up whether a storage supports a given feature in its 'plugindata'. This is intentionally kept simple and not implemented as a plugin method for now. Should it ever become more complex requiring plugins to override the default implementation, it can later be changed t

[pve-devel] [PATCH v8 storage 0/9] backup provider API

2025-04-03 Thread Wolfgang Bumiller
v7: https://lore.proxmox.com/pve-devel/20250401173435.221892-1-f.eb...@proxmox.com/ I picked this up since Fiona is currently not available. This contains 1 "major" change (and incorporates some minor doc feedback from Max (sans the `=head` rewrite, feel free to send follow-up patches for this @

[pve-devel] [PATCH v8 qemu 01/10] PVE backup: clean up directly in setup_snapshot_access() when it fails

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner The only thing that might need to be cleaned up after setup_snapshot_access() failed is dropping the cbw filter. Do so in the single branch it matters inside setup_snapshot_access() itself. This avoids the need that callers of setup_snapshot_access() use cleanup_snapshot_access(

[pve-devel] [PATCH v8 storage 3/8] plugin: introduce new_backup_provider() method

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner The new_backup_provider() method can be used by storage plugins for external backup providers. If the method returns a provider, Proxmox VE will use callbacks to that provider for backups and restore instead of using its usual backup/restore mechanisms. The backup provider API

[pve-devel] [PATCH v8 container 3/7] backup: implement restore for external providers

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner First, the provider is asked about what restore mechanism to use. Currently, 'directory' and 'tar' are possible. The 'directory' mechanism is for restoring from a directory containing the container's full filesystem structure, which is restored by piping from a privileged tar cf

[pve-devel] [PATCH v8 container 7/7] api: add early check against restoring privileged container from external source

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner While restore_external_archive() already has a check, that happens after an existing container is destroyed. Signed-off-by: Fiona Ebner --- No changes to v7. src/PVE/API2/LXC.pm | 14 ++ 1 file changed, 14 insertions(+) diff --git a/src/PVE/API2/LXC.pm b/src/PVE

[pve-devel] [PATCH v8 qemu-server 07/11] image convert: allow caller to specify the format of the source path

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner In preparation for the restore API for backup providers that doesn't want detection based on the file extension but always requires raw. Signed-off-by: Fiona Ebner --- No changes to v7. PVE/QemuServer.pm | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git

[pve-devel] [PATCH v8 qemu-server 05/11] test: qemu img convert: add test cases for snapshots

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner Signed-off-by: Fiona Ebner --- No changes to v7. test/run_qemu_img_convert_tests.pl | 18 ++ 1 file changed, 18 insertions(+) diff --git a/test/run_qemu_img_convert_tests.pl b/test/run_qemu_img_convert_tests.pl index 20ff387..29c188d 100755 --- a/test/run_qe

[pve-devel] [PATCH v8 qemu 06/10] PVE backup: implement backup access setup and teardown API for external providers

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner For external backup providers, the state of the VM's disk images at the time the backup is started is preserved via a snapshot-access block node. Old data is moved to the fleecing image when new guest writes come in. The snapshot-access block node, as well as the associated bitm

[pve-devel] [PATCH v8 qemu-server 02/11] backup: fleecing: use exact size when allocating non-raw fleecing images

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner A non-1KiB aligned source image could cause issues when used with qcow2 fleecing images, e.g. for an image with size 4.5 KiB: > Size mismatch for 'drive-tpmstate0-backup-fleecing' - sector count 10 != 9 Raw images are attached to QEMU with an explicit 'size' argument, so roundi

[pve-devel] [PATCH v8 qemu 07/10] PVE backup: factor out get_single_device_info() helper

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner Signed-off-by: Fiona Ebner [WB: free di and di->device_name on error] Sigend-off-by: Wolfgang Bumiller --- Changes in v8: described in the trailers above ^ pve-backup.c | 90 +++- 1 file changed, 53 insertions(+), 37 deletions(

[pve-devel] [PATCH v8 qemu 02/10] PVE backup: factor out helper to clear backup state's bitmap list

2025-04-03 Thread Wolfgang Bumiller
From: Fiona Ebner Suggested-by: Wolfgang Bumiller Signed-off-by: Fiona Ebner Reviewed-by: Wolfgang Bumiller --- No changes to v7. pve-backup.c | 28 ++-- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/pve-backup.c b/pve-backup.c index 2408f182bc..9156

Re: [pve-devel] [PATCH ha-manager 02/15] tools: add hash set helper subroutines

2025-04-03 Thread Fabian Grünbichler
On March 25, 2025 6:53 pm, Thomas Lamprecht wrote: > Am 25.03.25 um 16:12 schrieb Daniel Kral: >> Implement helper subroutines, which implement basic set operations done >> on hash sets, i.e. hashes with elements set to a true value, e.g. 1. >> >> These will be used for various tasks in the HA Man

Re: [pve-devel] [PATCH cluster/guest-common/docs/qemu-server/manager v15 0/12] virtiofs

2025-04-03 Thread Markus Frank
I forgot one v15 change. On 2025-04-03 12:34, Markus Frank wrote: Virtio-fs is a shared file system that enables sharing a directory between host and guest VMs. It takes advantage of the locality of virtual machines and the hypervisor to get a higher throughput than the 9p remote file system pr

Re: [pve-devel] [PATCH manager v4] fix #1926 ui: vm console: autodetect novnc or xtermjs

2025-04-03 Thread Thomas Lamprecht
Am 26.03.25 um 13:04 schrieb Aaron Lauterer: > I did not find if we already have the full VM config already. AFAICT we > go from `qemu/Config.js` -> `VNCConsole.js`. > > Only the status of the VM. As I mentioned in the comment below the > commit msg, the backend does check against the wrong conf

[pve-devel] [PATCH manager v15 10/12] ui: add resource mapping view for directories

2025-04-03 Thread Markus Frank
Signed-off-by: Markus Frank --- v15: * removed announce-submounts www/manager6/Makefile | 1 + www/manager6/dc/Config.js | 10 + www/manager6/dc/DirMapView.js | 38 +++ 3 files changed, 49 insertions(+) create mode 100644 www/manager6/dc/DirM

[pve-devel] [PATCH cluster/guest-common/docs/qemu-server/manager v15 0/12] virtiofs

2025-04-03 Thread Markus Frank
Virtio-fs is a shared file system that enables sharing a directory between host and guest VMs. It takes advantage of the locality of virtual machines and the hypervisor to get a higher throughput than the 9p remote file system protocol. build-order: 1. cluster 2. guest-common 3. docs 4. qemu-serve

[pve-devel] [PATCH qemu-server v15 5/7] fix #1027: virtio-fs support

2025-04-03 Thread Markus Frank
add support for sharing directories with a guest vm. virtio-fs needs virtiofsd to be started. In order to start virtiofsd as a process (despite being a daemon it is does not run in the background), a double-fork is used. virtiofsd should close itself together with QEMU. There are the parameters

[pve-devel] [PATCH manager v15 09/12] ui: add edit window for dir mappings

2025-04-03 Thread Markus Frank
Signed-off-by: Markus Frank --- v15: * removed announce-submounts www/manager6/Makefile | 1 + www/manager6/window/DirMapEdit.js | 202 ++ 2 files changed, 203 insertions(+) create mode 100644 www/manager6/window/DirMapEdit.js diff --git a/www/manager6

[pve-devel] [PATCH cluster v15 1/12] add mapping/dir.cfg for resource mapping

2025-04-03 Thread Markus Frank
Add it to both the perl side (PVE/Cluster.pm) and pmxcfs side (status.c). This dir.cfg is used to map directory IDs to paths on selected hosts. Reviewed-by: Fiona Ebner Reviewed-by: Laurențiu Leahu-Vlăducu Tested-by: Laurențiu Leahu-Vlăducu Tested-by: Daniel Kral Signed-off-by: Markus Frank -

[pve-devel] [PATCH guest-common v15 2/12] add dir mapping section config

2025-04-03 Thread Markus Frank
Adds a config file for directories by using a 'map' property string for each node mapping. example config: ``` some-dir-id map node=node1,path=/path/to/share/ map node=node2,path=/different/location/ ``` Signed-off-by: Markus Frank --- v15: * removed announce-submounts option alt

[pve-devel] [PATCH manager v15 12/12] ui: add options to add virtio-fs to qemu config

2025-04-03 Thread Markus Frank
Signed-off-by: Markus Frank --- v15: * moved all options except dirid to an advanced tab * improved field labels www/manager6/Makefile | 1 + www/manager6/Utils.js | 1 + www/manager6/qemu/HardwareView.js | 19 www/manager6/qemu/VirtiofsEdit.js | 138 +++

[pve-devel] [PATCH docs v15 3/12] add doc section for the shared filesystem virtio-fs

2025-04-03 Thread Markus Frank
Signed-off-by: Markus Frank --- v15: * added fstab entry example in the docs * added hyperlinks for websites * removed announce-submounts part qm.adoc | 102 ++-- 1 file changed, 100 insertions(+), 2 deletions(-) diff --git a/qm.adoc b/qm.adoc

[pve-devel] [PATCH manager v15 11/12] ui: form: add selector for directory mappings

2025-04-03 Thread Markus Frank
Reviewed-by: Laurențiu Leahu-Vlăducu Tested-by: Laurențiu Leahu-Vlăducu Tested-by: Daniel Kral Signed-off-by: Markus Frank --- no changes in v15 www/manager6/Makefile | 1 + www/manager6/form/DirMapSelector.js | 63 + 2 files changed, 64 insertions(+

Re: [pve-devel] [PATCH storage v4 09/27] plugin: introduce new_backup_provider() method

2025-04-03 Thread Wolfgang Bumiller
On Wed, Apr 02, 2025 at 06:16:57PM +0200, Andreas Rogge wrote: > Am 02.04.25 um 10:30 schrieb Wolfgang Bumiller: > > On Tue, Apr 01, 2025 at 08:21:30PM +0200, Thomas Lamprecht wrote: > > > > This sounds pretty inefficient - especially when > > > > comparing with qmrestore's ability to just read rea

Re: [pve-devel] [PATCH guest-common v7 1/2] mapping: pci: check the mdev configuration on the device too

2025-04-03 Thread Thomas Lamprecht
Am 11.03.25 um 14:20 schrieb Dominik Csapak: > but that lives int he 'global' part of the mapping config, not in a > specific mapping. To check that, add it to the $configured_props from > there. > > this requires all call sites to be adapted otherwise the check will > always fail for devices that

Re: [pve-devel] [PATCH guest-common v7 1/2] mapping: pci: check the mdev configuration on the device too

2025-04-03 Thread Thomas Lamprecht
Am 03.04.25 um 11:43 schrieb Dominik Csapak: > On 4/3/25 11:40, Thomas Lamprecht wrote: >> Am 11.03.25 um 14:20 schrieb Dominik Csapak: >>> but that lives int he 'global' part of the mapping config, not in a >>> specific mapping. To check that, add it to the $configured_props from >>> there. >>> >>

Re: [pve-devel] [PATCH pve-manager 5/7] fabrics: add NodeEdit components

2025-04-03 Thread Christoph Heiss
On Fri Mar 28, 2025 at 6:13 PM CET, Gabriel Goller wrote: [..] > diff --git a/www/manager6/sdn/fabrics/openfabric/NodeEdit.js > b/www/manager6/sdn/fabrics/openfabric/NodeEdit.js > new file mode 100644 > index ..f2d204c22542 > --- /dev/null > +++ b/www/manager6/sdn/fabrics/openfabric/No

Re: [pve-devel] [RFC PATCH http-server] fix #6230: increase allowed post size

2025-04-03 Thread Dominik Csapak
sent a new version: https://lore.proxmox.com/pve-devel/20250403082759.2506153-1-d.csa...@proxmox.com/ ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH qemu-server] config: add system and service credentials support

2025-04-03 Thread Maximiliano Sandoval
Thomas Lamprecht writes: > Am 03.04.25 um 09:49 schrieb Thomas Lamprecht: >> Am 02.04.25 um 16:36 schrieb Maximiliano Sandoval: >>> Allows to pass system and service credentials to a VM. See [1] for a >>> description of credentials. This can be potentially used to provision a >>> VM as per [2].

Re: [pve-devel] [PATCH qemu-server] config: add system and service credentials support

2025-04-03 Thread Maximiliano Sandoval
Thomas Lamprecht writes: > Am 02.04.25 um 16:36 schrieb Maximiliano Sandoval: >> Allows to pass system and service credentials to a VM. See [1] for a >> description of credentials. This can be potentially used to provision a >> VM as per [2]. Values can be passed either as plain text or as a bas

Re: [pve-devel] [PATCH storage v4 09/27] plugin: introduce new_backup_provider() method

2025-04-03 Thread Andreas Rogge
Am 02.04.25 um 10:33 schrieb Fiona Ebner: Am 01.04.25 um 20:21 schrieb Thomas Lamprecht: Am 01.04.25 um 18:02 schrieb Andreas Rogge: Um... that has nothing to do with what you provided when we take the backup. Is there a reason PVE cannot provide a writeable block device to restore to? Good i

Re: [pve-devel] [PATCH cluster/docs/manager/network/proxmox{, -ve-rs, -firewall, -perl-rs} 00/52] Add SDN Fabrics

2025-04-03 Thread Friedrich Weber
On 28/03/2025 18:12, Gabriel Goller wrote: > This series allows the user to add fabrics such as OpenFabric and OSPF over > their clusters. > > Overview > > > This series allows the user to create routed networks ('fabrics') across their > clusters, which can be used as the underlay netwo

[pve-devel] [PATCH http-server] fix #6230: increase allowed post size

2025-04-03 Thread Dominik Csapak
In some situations, e.g. having a large resource mapping, the UI can generate a request that is bigger than the current limit of 64KiB. Our files in pmxcfs can grow up to 1 MiB, so theoretically, a single mapping can grow to that size. In practice, a single entry will have much less. In #6230, a u

Re: [pve-devel] [PATCH-SERIES qemu/storage/qemu-server/container/manager v7 00/37] backup provider API

2025-04-03 Thread Wolfgang Bumiller
On Wed, Apr 02, 2025 at 05:15:24PM +0200, Friedrich Weber wrote: > Hi, I tried v7 very quickly with the backup-provider-dir-example > storage. Some minor things I noticed below, IMO nothing that can't be > fixed in follow-ups. > > Note that I didn't look too closely how the backup provider API rea

Re: [pve-devel] [PATCH qemu-server] config: add system and service credentials support

2025-04-03 Thread Thomas Lamprecht
Am 02.04.25 um 16:36 schrieb Maximiliano Sandoval: > Allows to pass system and service credentials to a VM. See [1] for a > description of credentials. This can be potentially used to provision a > VM as per [2]. Values can be passed either as plain text or as a base64 > encoded string when the bas

Re: [pve-devel] [PATCH v1 pve-storage 6/8] pluginbase: document image operation methods

2025-04-03 Thread Fabian Grünbichler
On April 2, 2025 6:32 pm, Max Carrara wrote: > On Mon Mar 31, 2025 at 5:12 PM CEST, Fabian Grünbichler wrote: >> On March 26, 2025 3:20 pm, Max Carrara wrote: >> > Add documentation for the following methods: >> > - list_images >> > - create_base >> > - clone_image >> > - alloc_image >> > - free_im

Re: [pve-devel] [PATCH v1 pve-storage 2/8] pluginbase: add high-level plugin API description

2025-04-03 Thread Fabian Grünbichler
On April 2, 2025 6:31 pm, Max Carrara wrote: > On Mon Mar 31, 2025 at 5:13 PM CEST, Fabian Grünbichler wrote: >> On March 26, 2025 3:20 pm, Max Carrara wrote: >> > Add a short paragraph in DESCRIPTION serving as an introduction as >> > well as the GENERAL PARAMETERS and CACHING EXPENSIVE OPERATIONS