[pve-devel] [PATCH v2 qemu-server] fix #4957: add vendor and product information passthrough for SCSI-Disks

2023-11-08 Thread Hannes Duerr
adds vendor and product information for SCSI devices to the json schema and checks in the VM create/update API call if it is possible to add these to QEMU as a device option Signed-off-by: Hannes Duerr --- changes in v2: - when calling the API to create/update a VM, check whether the devices ar

[pve-devel] [PATCH cluster/guest-common/docs/qemu-server/manager v8 0/7] virtiofs

2023-11-08 Thread Markus Frank
build-order: 1. cluster 2. guest-common 3. docs 4. qemu-server 5. manager I did not get virtiofsd to run with run_command without creating zombie processes after stutdown. So I replaced run_command with exec for now. Maybe someone can find out why this happens. cluster: Markus Frank (1): add

[pve-devel] [PATCH docs v8 3/7] added shared filesystem doc for virtio-fs

2023-11-08 Thread Markus Frank
Signed-off-by: Markus Frank --- qm.adoc | 84 +++-- 1 file changed, 82 insertions(+), 2 deletions(-) diff --git a/qm.adoc b/qm.adoc index c4f1024..571c42e 100644 --- a/qm.adoc +++ b/qm.adoc @@ -996,6 +996,85 @@ recommended to always use a limit

[pve-devel] [PATCH cluster v8 1/7] add mapping/dir.cfg for resource mapping

2023-11-08 Thread Markus Frank
Add it to both, the perl side (PVE/Cluster.pm) and pmxcfs side (status.c). Signed-off-by: Markus Frank --- src/PVE/Cluster.pm | 1 + src/pmxcfs/status.c | 1 + 2 files changed, 2 insertions(+) diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm index cfa2583..39bdfa1 100644 --- a/src/PVE/Clus

[pve-devel] [PATCH qemu-server v8 4/7] feature #1027: virtio-fs support

2023-11-08 Thread Markus Frank
add support for sharing directories with a guest vm virtio-fs needs virtiofsd to be started. In order to start virtiofsd as a process (despite being a daemon it is does not run in the background), a double-fork is used. virtiofsd should close itself together with qemu. There are the parameters

[pve-devel] [PATCH qemu-server v8 5/7] Permission check for virtiofs directory access

2023-11-08 Thread Markus Frank
Signed-off-by: Markus Frank --- PVE/API2/Qemu.pm | 39 ++- PVE/QemuServer.pm | 5 - 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index c8a87f3..1c5eb4c 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qem

[pve-devel] [PATCH qemu-server v8 6/7] check_local_resources: virtiofs

2023-11-08 Thread Markus Frank
add dir mapping checks to check_local_resources Signed-off-by: Markus Frank --- PVE/QemuServer.pm| 10 +- test/MigrationTest/Shared.pm | 7 +++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index f66f26e..b5c2c14 100

[pve-devel] [PATCH guest-common v8 2/7] add Dir mapping config

2023-11-08 Thread Markus Frank
Adds a config file for directories by using a 'map' array propertystring for each node mapping. Next to node & path, there is the optional submounts parameter in the map array. Additionally there are the default settings for xattr & acl. example config: ``` some-dir-id map node=node1,path

[pve-devel] [PATCH manager v8 7/7] api: add resource map api endpoints for directories

2023-11-08 Thread Markus Frank
Signed-off-by: Markus Frank --- PVE/API2/Cluster/Mapping.pm | 7 + PVE/API2/Cluster/Mapping/Dir.pm | 309 ++ PVE/API2/Cluster/Mapping/Makefile | 3 +- 3 files changed, 318 insertions(+), 1 deletion(-) create mode 100644 PVE/API2/Cluster/Mapping/Dir.pm dif

Re: [pve-devel] applied: [PATCH installer v4 0/6] fix #4829: wire up `arc_max` ZFS option to GUI/TUI for PVE

2023-11-08 Thread Christoph Heiss
On Tue, Nov 07, 2023 at 04:50:31PM +0100, Thomas Lamprecht wrote: > > Am 07/11/2023 um 13:20 schrieb Christoph Heiss: > > run env: remove debug print > > install: use correct variable names in zfs_setup_module_conf() > > proxinstall: expose `arc_max` ZFS option for PVE installations > > t

[pve-devel] [PATCH docs] notifications: update docs to for matcher-based notifications

2023-11-08 Thread Lukas Wagner
Target groups and filters have been replaced by notification matchers. The matcher can match on certain notification properties and route the notification to a target in case of a match. This patch updates the docs to reflect these changes. Signed-off-by: Lukas Wagner --- notifications.adoc | 2

Re: [pve-devel] [PATCH v2 qemu-server] fix #4957: add vendor and product information passthrough for SCSI-Disks

2023-11-08 Thread Fiona Ebner
Am 08.11.23 um 09:51 schrieb Hannes Duerr: > adds vendor and product information for SCSI devices to the json schema and > checks in the VM create/update API call if it is possible to add these to > QEMU as a device option > > Signed-off-by: Hannes Duerr > --- > > changes in v2: > - when callin

[pve-devel] [PATCH access-control] perms: fix wrong /pools entry in default set of ACL paths

2023-11-08 Thread Fabian Grünbichler
/pools is not an allowed ACL path, so this would add a bogus entry into the effective permissions in case something got propagated from /. Signed-off-by: Fabian Grünbichler --- stumbled upon this while working on unrelated pool stuff.. src/PVE/RPCEnvironment.pm | 2 +- 1 file changed, 1 inserti

Re: [pve-devel] [PATCH v3 container 1/1] Add device passthrough

2023-11-08 Thread Dominik Csapak
did not properly review this, but what caught my attention was that you don't define any permissions for this new property? by default new options in pve-container only need 'VM.Config.Options' but IMHO this should be root only for now? (unless we can use mappings where we can use those permissio

[pve-devel] [PATCH firewall/network 0/2] SDN: Create firewall aliases for SDN subnets

2023-11-08 Thread Stefan Lendl
Creates a cluster-way firewall alias when creating an SDN subnet Including Firewall from pve-network introduces a dependency cycle which the patch to Firewall.pm elimiates. pve-firewall: Stefan Lendl (1): Manually construct guest config path src/PVE/Firewall.pm | 34 +++--

[pve-devel] [PATCH pve-network 2/2] Create a cluster-wide firewall for SDN subnets

2023-11-08 Thread Stefan Lendl
Upon creation of a subnet, we create a cluster-wide firewall alias. Signed-off-by: Stefan Lendl --- Notes: Creates the alias directly when the Subnet is created. Other SDN objects are created upon 'Apply': commit_config(). Although, IPAM creates the subnet right away as well.

[pve-devel] [PATCH pve-firewall 1/2] Manually construct guest config path

2023-11-08 Thread Stefan Lendl
Remove require QemuConfig from Firewall.pm We only use it to construct the guest config paths. Fixes circular include when accessing Firewall::Aliases from pve-network. Signed-off-by: Stefan Lendl --- src/PVE/Firewall.pm | 34 +++--- 1 file changed, 7 insertions(+), 2

[pve-devel] applied: [PATCH access-control] perms: fix wrong /pools entry in default set of ACL paths

2023-11-08 Thread Thomas Lamprecht
Am 08/11/2023 um 11:29 schrieb Fabian Grünbichler: > /pools is not an allowed ACL path, so this would add a bogus entry into the > effective permissions in case something got propagated from /. > > Signed-off-by: Fabian Grünbichler > --- > stumbled upon this while working on unrelated pool stuff.

[pve-devel] [PATCH manager] api: osd: destroy: remove mclock max iops settings

2023-11-08 Thread Aaron Lauterer
Ceph does a quick benchmark when creating a new OSD and stores the osd_mclock_max_capacity_iops_{ssd,hdd} settings in the config DB. When destroying the OSD, Ceph does not automatically remove these settings. Keeping them can be problematic if a new OSD with potentially more performance is added a

[pve-devel] applied: [PATCH access-control] acl: add missing SDN ACL paths to allowed list

2023-11-08 Thread Thomas Lamprecht
Am 08/11/2023 um 07:55 schrieb Fabian Grünbichler: > else it's not actually possible to define ACLs on them, which means they are > effectively root only instead of allowing their intended permission scheme. > > Signed-off-by: Fabian Grünbichler > --- > src/PVE/AccessControl.pm | 6 ++ > 1 f

Re: [pve-devel] [PATCH manager 1/2] ui: BulkActions: rework filters and include tags

2023-11-08 Thread Dominik Csapak
On 11/6/23 17:01, Thomas Lamprecht wrote: for the commit subject please: s/BulkActions/bulk actions/ Am 30/10/2023 um 13:58 schrieb Dominik Csapak: This moves the filters out of the grid header for the BulkActions and puts them into their own fieldset above the grid. With that, we can easily in

Re: [pve-devel] [PATCH manager 1/2] ui: BulkActions: rework filters and include tags

2023-11-08 Thread Thomas Lamprecht
Am 08/11/2023 um 13:14 schrieb Dominik Csapak: > On 11/6/23 17:01, Thomas Lamprecht wrote: >> - the CT/VM ID filter is a bit odd, especially if tuned to match all, >>not only parts of the VMID (which would not be *that* much better >>either IMO), when I want to migrate/start/stop a single V

Re: [pve-devel] [PATCH manager 1/2] ui: BulkActions: rework filters and include tags

2023-11-08 Thread Dominik Csapak
On 11/8/23 13:38, Thomas Lamprecht wrote: Am 08/11/2023 um 13:14 schrieb Dominik Csapak: On 11/6/23 17:01, Thomas Lamprecht wrote: - the CT/VM ID filter is a bit odd, especially if tuned to match all, not only parts of the VMID (which would not be *that* much better either IMO), when I

[pve-devel] [PATCH i18n] german translation for the bulk action log messages

2023-11-08 Thread Folke Gleumes
--- de.po | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/de.po b/de.po index 3e2ff91..43f5d0c 100644 --- a/de.po +++ b/de.po @@ -1138,17 +1138,17 @@ msgstr "Massenstart" #: pve-manager/www/manager6/Utils.js:1970 #, fuzzy msgid "Bulk migrate VMs and Containers" -msgstr

[pve-devel] applied: [PATCH i18n] german translation for the bulk action log messages

2023-11-08 Thread Thomas Lamprecht
Am 08/11/2023 um 14:51 schrieb Folke Gleumes: > --- > de.po | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > applied, but you obviously did not test this, as leaving the `# fuzzy` comments in will make gettext ignore those translations completely. I really expect some basic test

Re: [pve-devel] [PATCH v2 qemu-server] fix #4957: add vendor and product information passthrough for SCSI-Disks

2023-11-08 Thread Hannes Dürr
On 11/8/23 11:04, Fiona Ebner wrote: Am 08.11.23 um 09:51 schrieb Hannes Duerr: adds vendor and product information for SCSI devices to the json schema and checks in the VM create/update API call if it is possible to add these to QEMU as a device option Signed-off-by: Hannes Duerr --- chang

Re: [pve-devel] [PATCH pve-firewall 1/2] Manually construct guest config path

2023-11-08 Thread Thomas Lamprecht
Am 08/11/2023 um 12:35 schrieb Stefan Lendl: > Remove require QemuConfig from Firewall.pm > We only use it to construct the guest config paths. > Fixes circular include when accessing Firewall::Aliases from > pve-network. > This won't work as now cfs_read_file only works by luck, if at all, as th

Re: [pve-devel] [RFC pve-network] do not remove DHCP mapping on stop

2023-11-08 Thread DERUMIER, Alexandre
hi, I'm back from Holiday, and I'll finally time to work on dhcp. I wonder if we couldn't add a property on subnet or dhcp, where user could choose between ephemeral ip (create a vm start / delete at vm stop), or reserved ip (reserved a vm|nic create, deleted a vm|nic delete) This should mat

Re: [pve-devel] [PATCH pve-network 2/2] Create a cluster-wide firewall for SDN subnets

2023-11-08 Thread Thomas Lamprecht
Am 08/11/2023 um 12:35 schrieb Stefan Lendl: > Upon creation of a subnet, we create a cluster-wide firewall alias. > > Signed-off-by: Stefan Lendl > --- > > Notes: > Creates the alias directly when the Subnet is created. > > Other SDN objects are created upon 'Apply': commit_config(

Re: [pve-devel] [RFC pve-network] do not remove DHCP mapping on stop

2023-11-08 Thread Stefan Hanreich
On 11/8/23 15:32, DERUMIER, Alexandre wrote: > hi, > I'm back from Holiday, and I'll finally time to work on dhcp. > Welcome back! It's also my first day after holiday today. > I wonder if we couldn't add a property on subnet or dhcp, > where user could choose between ephemeral ip (create a vm st

Re: [pve-devel] [PATCH v2 qemu-server] fix #4957: add vendor and product information passthrough for SCSI-Disks

2023-11-08 Thread Thomas Lamprecht
Am 08/11/2023 um 15:28 schrieb Hannes Dürr: > On 11/8/23 11:04, Fiona Ebner wrote: >> Am 08.11.23 um 09:51 schrieb Hannes Duerr: >>> + if ($opt =~ m/scsi/) { >>> + >>> PVE::QemuServer::check_scsi_feature_compatibility($opt, $created_opts, >>> $conf, $storec

[pve-devel] [PATCH v4 debcargo-conf 01/11] cherry-pick chumsky 0.9.2 from debian unstable

2023-11-08 Thread Lukas Wagner
Signed-off-by: Lukas Wagner --- src/chumsky/debian/changelog | 5 +++ src/chumsky/debian/copyright | 39 + src/chumsky/debian/copyright.debcargo.hint | 51 ++ src/chumsky/debian/debcargo.toml | 2 + 4 files changed, 97 in

[pve-devel] [PATCH v4 many 00/11] notifications: add SMTP endpoint

2023-11-08 Thread Lukas Wagner
This patch series adds support for a new notification endpoint type, smtp. As the name suggests, this new endpoint allows PVE to talk to SMTP server directly, without using the system's MTA (postfix). On the Rust side, these patches add a new dependency to the `lettre` crate for SMTP communication

[pve-devel] [PATCH v4 pve-docs 11/11] notifications: document 'comment' option for targets/matchers

2023-11-08 Thread Lukas Wagner
Signed-off-by: Lukas Wagner --- notifications.adoc | 3 +++ 1 file changed, 3 insertions(+) diff --git a/notifications.adoc b/notifications.adoc index acbdfae..e8ed51b 100644 --- a/notifications.adoc +++ b/notifications.adoc @@ -67,6 +67,7 @@ accomodate multiple recipients. set, the plugin will

[pve-devel] [PATCH v4 pve-manager 08/11] notify: add API routes for smtp endpoints

2023-11-08 Thread Lukas Wagner
The Perl part of the API methods primarily defines the API schema, checks for any needed privileges and then calls the actual Rust implementation exposed via perlmod. Any errors returned by the Rust code are translated into PVE::Exception, so that the API call fails with the correct HTTP error code

[pve-devel] [PATCH v4 proxmox-perl-rs 07/11] notify: add bindings for smtp API calls

2023-11-08 Thread Lukas Wagner
Signed-off-by: Lukas Wagner --- common/src/notify.rs | 106 +++ 1 file changed, 106 insertions(+) diff --git a/common/src/notify.rs b/common/src/notify.rs index 4fbd705..8a6d76e 100644 --- a/common/src/notify.rs +++ b/common/src/notify.rs @@ -15,6 +15,10 @

[pve-devel] [PATCH v4 proxmox 06/11] notify: add api for smtp endpoints

2023-11-08 Thread Lukas Wagner
Signed-off-by: Lukas Wagner --- proxmox-notify/src/api/mod.rs| 33 +++ proxmox-notify/src/api/smtp.rs | 356 +++ proxmox-notify/src/endpoints/smtp.rs | 8 - 3 files changed, 389 insertions(+), 8 deletions(-) create mode 100644 proxmox-notify/src/api/smtp.

[pve-devel] [PATCH v4 proxmox 03/11] sys: email: add `forward`

2023-11-08 Thread Lukas Wagner
This new function forwards an email to new recipients. Signed-off-by: Lukas Wagner --- proxmox-sys/src/email.rs | 52 +++- 1 file changed, 51 insertions(+), 1 deletion(-) diff --git a/proxmox-sys/src/email.rs b/proxmox-sys/src/email.rs index 8b3a1b6..c94f634

[pve-devel] [PATCH v4 proxmox 04/11] notify: add mechanisms for email message forwarding

2023-11-08 Thread Lukas Wagner
As preparation for the integration of `proxmox-mail-foward` into the notification system, this commit makes a few changes that allow us to forward raw email messages (as passed from postfix). For mail-based notification targets, the email will be forwarded as-is, including all headers. The only th

[pve-devel] [PATCH v4 debcargo-conf 02/11] update lettre to 0.11.1

2023-11-08 Thread Lukas Wagner
Signed-off-by: Lukas Wagner --- src/lettre/debian/changelog | 10 +++ .../debian/patches/downgrade_fastrand.patch | 13 .../debian/patches/downgrade_idna.patch | 13 src/lettre/debian/patches/downgrade_url.patch | 13 .../patches/remove_unused_features.pat

[pve-devel] [PATCH v4 pve-docs 10/11] notifications: document SMTP endpoints

2023-11-08 Thread Lukas Wagner
Signed-off-by: Lukas Wagner --- notifications.adoc | 47 ++ 1 file changed, 47 insertions(+) diff --git a/notifications.adoc b/notifications.adoc index 764ec72..acbdfae 100644 --- a/notifications.adoc +++ b/notifications.adoc @@ -67,6 +67,7 @@ accomoda

[pve-devel] [PATCH v4 proxmox-widget-toolkit 09/11] panel: notification: add gui for SMTP endpoints

2023-11-08 Thread Lukas Wagner
This new endpoint configuration panel is embedded in the existing EndpointEditBase dialog window. This commit also factors out some of the non-trivial common form elements that are shared between the new panel and the already existing SendmailEditPanel into a separate panel EmailRecipientPanel. Si

[pve-devel] [PATCH v4 proxmox 05/11] notify: add 'smtp' endpoint

2023-11-08 Thread Lukas Wagner
This commit adds a new endpoint type, namely 'smtp'. This endpoint uses the `lettre` crate to directly send emails to SMTP relays. The `lettre` crate was chosen since it is by far the most popular SMTP implementation for Rust that looks like it is well maintained. Also, it includes async support (

Re: [pve-devel] [RFC pve-network] do not remove DHCP mapping on stop

2023-11-08 Thread DERUMIER, Alexandre
Hi Stefan ! > I wonder if we couldn't add a property on subnet or dhcp, > where user could choose between ephemeral ip (create a vm start / > delete at vm stop), > > or reserved ip > > (reserved a vm|nic create,  deleted a vm|nic delete) > >>That sounds like something we could implement. I've

[pve-devel] [PATCH manager v2 3/3] ui: bulk actions: add clear filters button

2023-11-08 Thread Dominik Csapak
to be able to clear all of them at once Signed-off-by: Dominik Csapak --- no changes www/manager6/window/BulkAction.js | 26 ++ 1 file changed, 26 insertions(+) diff --git a/www/manager6/window/BulkAction.js b/www/manager6/window/BulkAction.js index cc561bd6..d3650f83 1

[pve-devel] [PATCH manager v2 1/3] ui: bulk actions: reorder fields

2023-11-08 Thread Dominik Csapak
to use less vertical space also remove the local-storage warning since this is not that helpful Signed-off-by: Dominik Csapak --- new in v2 www/manager6/window/BulkAction.js | 75 +-- 1 file changed, 41 insertions(+), 34 deletions(-) diff --git a/www/manager6/window

[pve-devel] [PATCH manager v2 2/3] ui: bulk actions: rework filters and include tags

2023-11-08 Thread Dominik Csapak
This moves the filters out of the grid header for the BulkActions and puts them into their own fieldset above the grid. With that, we can easily include a tags filter (one include and one exclude list). The filter fieldset is collapsible and shows the active filters in parenthesis. aside from that

[pve-devel] [PATCH manager 2/2] ui: add tooltips to non-full tags globally

2023-11-08 Thread Dominik Csapak
by using the delegate function of ExtJS' tooltips on the global Workspace element and using the proper css selectors this way, we can limit the tooltips to the non-full ones (in contrast to using data-qtip on the element, which would always be show, even for tags with the 'full' style) Signed-off

[pve-devel] [PATCH manager 1/2] ui: resource tree: limit tooltip to icon and text

2023-11-08 Thread Dominik Csapak
and exclude the tags for that, since we want the tags to have their own tooltips Signed-off-by: Dominik Csapak --- not really sure if we want to do this, since creating a custom tree column type just for that seems overkill. also we have to touch private properties of that here to change it which

Re: [pve-devel] [PATCH v4 many 00/11] notifications: add SMTP endpoint

2023-11-08 Thread Dietmar Maurer
> This patch series adds support for a new notification endpoint type, > smtp. As the name suggests, this new endpoint allows PVE to talk > to SMTP server directly, without using the system's MTA (postfix). Isn't this totally unreliable? What if the server responds with a temporary error code? (A

Re: [pve-devel] [RFC pve-network 2/6] always generate dnsmasq ethers file

2023-11-08 Thread DERUMIER, Alexandre
Personnaly, I really think that we shouldn't generate the whole dhcp config (reading the full ipam db), each time we allocate a single ip. With external ipam, that mean 1api call for each subnet, it can be really slow. (for example, I have 400 subnets in productions) and this is only done on loc

Re: [pve-devel] [RFC qemu-server 6/6] DHCP mappings on vNIC add/remove

2023-11-08 Thread DERUMIER, Alexandre
From my previous mail, Here I'll allocate only in ipam, but not generate dhcp config. Message initial De: Stefan Lendl Répondre à: Proxmox VE development discussion À: pve-devel@lists.proxmox.com Objet: [pve-devel] [RFC qemu-server 6/6] DHCP mappings on vNIC add/remove Date

Re: [pve-devel] [WIP v2 pve-network 05/10] dhcp: add DHCP plugin for dnsmasq

2023-11-08 Thread DERUMIER, Alexandre
>>/etc/default/dnsmasq. >>This file specifies the configuration directory for the dnsmasq >>instance (/etc/dnsmasq.d/). It also sets the configuration >>file to /dev/null so the default configuration from the package has >>no influence on the dnsmasq configuration. >> >>/etc/dnsmasq.d//00-default.c

[pve-devel] [WIP pve-network 2/3] dhcp : add|del_ip_mapping: only add|del dhcp reservervation

2023-11-08 Thread Alexandre Derumier
don't try to add|del ip from ipam here Signed-off-by: Alexandre Derumier --- src/PVE/Network/SDN/Dhcp.pm | 75 + 1 file changed, 18 insertions(+), 57 deletions(-) diff --git a/src/PVE/Network/SDN/Dhcp.pm b/src/PVE/Network/SDN/Dhcp.pm index e4c4078..1c32fec 10

[pve-devel] [WIP pve-network 1/3] define dhcpplugin in zone

2023-11-08 Thread Alexandre Derumier
simple: zone1 ipam pve dhcp dnsmasq simple: zone2 ipam pve dhcp dnsmasq This generate 1 dhcp by zone/vrf. Don't use dhcp.cfg anymore It's reuse node filtering from zone. same subnets in 2 differents zones can't use same dhcp server Signed-off-by: Alexandre Derum

[pve-devel] [WIP pve-network 3/3] vnet|subnet: add_next_free_ip : implement dhcprange ipam search

2023-11-08 Thread Alexandre Derumier
Signed-off-by: Alexandre Derumier --- src/PVE/Network/SDN/Ipams/PVEPlugin.pm | 12 ++-- src/PVE/Network/SDN/Ipams/Plugin.pm| 7 +++ src/PVE/Network/SDN/Subnets.pm | 22 +++--- src/PVE/Network/SDN/Vnets.pm | 4 ++-- 4 files changed, 34 insertions

[pve-devel] [WIP pve-network 0/3] dhcp changes

2023-11-08 Thread Alexandre Derumier
Here my current work, based on wip2 Stefan Hanreich It's not yet completed, I'll try to send a more complete serie next week - patch1: remove need of dhcp.cfg, and declare dhcp plugin type in zone directly. each zone is a dhcpserver , that's make sense, as 2 zones can have same subnets in dif