Re: [pve-devel] applied-series: [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Dominik Csapak
On 11/18/24 22:15, Thomas Lamprecht wrote: Am 18.11.24 um 16:29 schrieb Dominik Csapak: This series enables importing ova/ovf from directory based storages, inclusive upload/download via the webui (ova only). It also improves the ovf importer by parsing the ostype, nics, bootorder (and firmware

[pve-devel] applied: [PATCH proxmox-firewall v4 2/9] config: tests: add support for loading sdn and ipam config

2024-11-18 Thread Thomas Lamprecht
Am 15.11.24 um 13:09 schrieb Stefan Hanreich: > Also add example SDN configuration files that get automatically > loaded, which can be used for future tests. > > Signed-off-by: Stefan Hanreich > Reviewed-by: Wolfgang Bumiller > Tested-by: Gabriel Goller > Tested-by: Hannes Dürr > --- > proxmo

[pve-devel] applied: [PATCH proxmox-firewall v4 08/18] use std::mem::take over drain()

2024-11-18 Thread Thomas Lamprecht
Am 15.11.24 um 13:10 schrieb Stefan Hanreich: > This is more efficient than draining and collecting the Vec. It also > fixes the respective clippy lint. It's well deserved to comment this, but I find it slightly interestingly that one of the more trivial changes gets the honor of an actual message

[pve-devel] applied: [pve-cluster/pve-network] move macs.db from priv/ to sdn/

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 21:55 schrieb Stoiko Ivanov: > while looking through the changes in pve-network, and testing the > port-isolation series I saw the move of priv/ipam.db to > sdn/pve-ipam-state.json: > > https://git.proxmox.com/?p=pve-network.git;a=commitdiff;h=0f48bc6561f2fd901f2665387b4954c8105614e

Re: [pve-devel] [PATCH qemu-server v7 4/5] api: create: add 'import-working-storage' parameter

2024-11-18 Thread Thomas Lamprecht
I now looked into your diff. Am 18.11.24 um 18:24 schrieb Aaron Lauterer: > On 2024-11-18 16:29, Dominik Csapak wrote: >> +if (!$extraction_scfg->{content}->{images} || >> !$extraction_scfg->{path}) { > > I think the if condition here is grouped wrong. > > As it is, once if on

[pve-devel] applied: [PATCH manager v3] fix #5810: ui: show confirmation/warning dialog for sdn apply

2024-11-18 Thread Thomas Lamprecht
Am 12.11.24 um 13:02 schrieb Timothy Nicholson: > Signed-off-by: Timothy Nicholson > --- > > changes since v2 [0]: > - changed icon > - modified confirmation message > > A conditional confirmation message would definitely be nicer, I can > send a patch for that later on today as v4. > > [0]:

[pve-devel] applied: [PATCH manager] fix #5879: ui: add tooltip to security group comments

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 12:44 schrieb Timothy Nicholson: > This patch adds a tooltip in the security group selector for group > comments that are too long to fit within the column width. > > Signed-off-by: Timothy Nicholson > --- > www/manager6/form/SecurityGroupSelector.js | 8 +++- > 1 file changed

[pve-devel] applied: [PATCH installer v2 0/5] fix #5579: allow specifying optional first-boot script

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 13:38 schrieb Christoph Heiss: > This implements #5579 [0] as proposed by Thomas [1]. > > Adds a new (optional) section to the auto-installer answer file called > `first-boot`, which can be used to the configure a script/executable > file to run on the first boot after the installat

[pve-devel] applied-series: [PATCH qemu-server/docs/manager v13 0/5] AMD SEV

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 12:16 schrieb Markus Frank: > changes v13: > * renamed amd_sev to amd-sev > * removed already applied patches > * add systemd service file for query-machine-capabilities > * removed now unused $efidisk0 parameter from the get_amd_sev_object > function > > changes v12: > * new pve-

[pve-devel] applied-series: [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 16:29 schrieb Dominik Csapak: > This series enables importing ova/ovf from directory based storages, > inclusive upload/download via the webui (ova only). > > It also improves the ovf importer by parsing the ostype, nics, bootorder > (and firmware from vmware exported files). > > W

Re: [pve-devel] [PATCH pve-firewall v4 6/9] api: load sdn ipsets

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 10:02 schrieb Stefan Hanreich: > Depends on if you consider a database of all assigned IPs inside the > cluster as sensitive information, iirc we erred on the side of caution > in this case and stored it in /etc/pve/priv. We briefly talked off-list about that, but I think it might b

[pve-devel] [PATCH cluster 1/1] cfs: add 'sdn/mac-cache.json' to observed files

2024-11-18 Thread Stoiko Ivanov
follows commit: d8ef05c (cfs: add 'sdn/pve-ipam-state.json' to observed files) with the same motivation - the data in the macs.db file is a cache, to prevent unneccessary lookups to external IPAM modules - is not private in the sense of secrets for external resources. Signed-off-by: Stoiko Ivanov

[pve-devel] [pve-cluster/pve-network] move macs.db from priv/ to sdn/

2024-11-18 Thread Stoiko Ivanov
while looking through the changes in pve-network, and testing the port-isolation series I saw the move of priv/ipam.db to sdn/pve-ipam-state.json: https://git.proxmox.com/?p=pve-network.git;a=commitdiff;h=0f48bc6561f2fd901f2665387b4954c8105614e0 Following the argumentation the mac-cache file shou

[pve-devel] [pve-network 1/1] ipam: move mac-cache.db to unprivileged sdn/ subdirectory

2024-11-18 Thread Stoiko Ivanov
follows commit: 0f48bc6 ("ipam: move state file of PVE plugin over to common sdn directory") as far as reasoning goes, and also closely code-wise (if only to make the clean-up with PVE 9.0 a bit more straight-forward): files in priv/ are sensitive in the sense that access there can be used to hijac

Re: [pve-devel] partially-applied: [PATCH-SERIES v4 docs/qemu-server] more robust handling of fleecing images

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 09:52 schrieb Fiona Ebner: > Regarding the patch "schema: add fleecing-images config property", > Fabian off-list suggested using a config section "special:fleecing" > instead of a property, so that it is truly internal-only. If we go for > that, the commit should be reverted. Which

Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 19:03 schrieb Filip Schauer: > On 18/11/2024 18:53, Thomas Lamprecht wrote: >> diff --git a/src/PVE/GuestImport/OVF.pm b/src/PVE/GuestImport/OVF.pm >> index 4af58ed..ed6fcea 100644 --- a/src/PVE/GuestImport/OVF.pm +++ >> b/src/PVE/GuestImport/OVF.pm @@ -191,6 +191,7 @@ sub parse_ov

Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 19:03 schrieb Filip Schauer: > On 18/11/2024 18:53, Thomas Lamprecht wrote: >> diff --git a/src/PVE/GuestImport/OVF.pm b/src/PVE/GuestImport/OVF.pm >> index 4af58ed..ed6fcea 100644 --- a/src/PVE/GuestImport/OVF.pm +++ >> b/src/PVE/GuestImport/OVF.pm @@ -191,6 +191,7 @@ sub parse_ov

Re: [pve-devel] applied: [PATCH manager v6 1/9] ui: fix special 'import' icon for non-esxi storages

2024-11-18 Thread Dominik Csapak
On 11/17/24 17:21, Thomas Lamprecht wrote: Am 15.11.24 um 16:17 schrieb Dominik Csapak: we only want to show that icon in the tree when the storage is solely used for importing, not when it's just one of several content types. Signed-off-by: Dominik Csapak --- www/manager6/Utils.js | 2 +-

Re: [pve-devel] [PATCH pve-firewall v5 1/5] api: add protected flag to endpoints

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 12:41 schrieb Stefan Hanreich: > In preparation for loading the SDN configuration during > load_clusterfw_conf. Since we read /etc/pve/priv/ipam.db there, we > require the protected flag to be set. > should be obsolete now due to https://git.proxmox.com/?p=pve-network.git;a=commit

Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 19:03 schrieb Filip Schauer: > On 18/11/2024 18:53, Thomas Lamprecht wrote: >> diff --git a/src/PVE/GuestImport/OVF.pm b/src/PVE/GuestImport/OVF.pm >> index 4af58ed..ed6fcea 100644 --- a/src/PVE/GuestImport/OVF.pm +++ >> b/src/PVE/GuestImport/OVF.pm @@ -191,6 +191,7 @@ sub parse_ov

Re: [pve-devel] [PATCH pve-docs v2 4/4] sdn: add documentation for isolated ports option

2024-11-18 Thread Stoiko Ivanov
On Tue, 12 Nov 2024 16:54:25 +0100 Stefan Hanreich wrote: > Signed-off-by: Stefan Hanreich > --- > pvesdn.adoc | 5 + > 1 file changed, 5 insertions(+) > > diff --git a/pvesdn.adoc b/pvesdn.adoc > index 39de80f..b1f2578 100644 > --- a/pvesdn.adoc > +++ b/pvesdn.adoc > @@ -383,6 +383,11 @@

[pve-devel] applied: [PATCH pve-network v2 3/4] vnets : add ports isolation

2024-11-18 Thread Thomas Lamprecht
Am 12.11.24 um 16:54 schrieb Stefan Hanreich: > From: Alexandre Derumier via pve-devel > > Add support for bridge ports isolation > https://github.com/torvalds/linux/commit/7d850abd5f4edb1b1ca4b4141a4453305736f564 > > This allow to drop traffic between all ports having isolation enabled > on the

Re: [pve-devel] [PATCH pve-common v2 1/4] tap_plug: add support for bridge port isolation

2024-11-18 Thread Stoiko Ivanov
saw this when looking through our git repos and thought I'll give it a spin (as afaict only the manager and docs-patches are not applied yet) It works, and does what it says it does. small suggestions for the docs-patch will be sent as reply to the docs-patch directly. w/ or w/o the doc-suggestio

Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Filip Schauer
On 18/11/2024 18:53, Thomas Lamprecht wrote: diff --git a/src/PVE/GuestImport/OVF.pm b/src/PVE/GuestImport/OVF.pm index 4af58ed..ed6fcea 100644 --- a/src/PVE/GuestImport/OVF.pm +++ b/src/PVE/GuestImport/OVF.pm @@ -191,6 +191,7 @@ sub parse_ovf { # 'ovf' is the default namespace so it will prepe

Re: [pve-devel] [PATCH pve-firewall v4 6/9] api: load sdn ipsets

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 12:38 schrieb Thomas Lamprecht: > I'll take a short look if it's easily possible to add a sane migration path > at pmxcfs level, handling this transparently, otherwise we'll have to add > some compat handling at higher levels. doing this directly in pmxcfs is not trivial as our link

Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 18:53 schrieb Thomas Lamprecht: > diff --git a/src/PVE/GuestImport/OVF.pm b/src/PVE/GuestImport/OVF.pm > index 4af58ed..ed6fcea 100644 > --- a/src/PVE/GuestImport/OVF.pm > +++ b/src/PVE/GuestImport/OVF.pm > @@ -191,6 +191,7 @@ sub parse_ovf { > # 'ovf' is the default namespace s

[pve-devel] [PATCH pve-firewall v5 03/10] firewall: move to arrow syntax for calling functions

2024-11-18 Thread Stefan Hanreich
Signed-off-by: Stefan Hanreich Reviewed-by: Wolfgang Bumiller --- src/PVE/Firewall.pm | 78 ++--- 1 file changed, 39 insertions(+), 39 deletions(-) diff --git a/src/PVE/Firewall.pm b/src/PVE/Firewall.pm index b1e5af6..65b8c73 100644 --- a/src/PVE/Firewall

Re: [pve-devel] [PATCH qemu-server v7 4/5] api: create: add 'import-working-storage' parameter

2024-11-18 Thread Aaron Lauterer
On 2024-11-18 16:29, Dominik Csapak wrote: this is to override the target extraction storage for the option disk extraction for 'import-from'. This way if the storage does not supports the content type 'images', one can give an alternative one. Signed-off-by: Dominik Csapak --- changes fr

[pve-devel] [PATCH pve-docs v6 4/4] sdn: add documentation for firewall integration

2024-11-18 Thread Stefan Hanreich
Signed-off-by: Stefan Hanreich --- pvesdn.adoc | 92 + 1 file changed, 92 insertions(+) diff --git a/pvesdn.adoc b/pvesdn.adoc index 39de80f..c187365 100644 --- a/pvesdn.adoc +++ b/pvesdn.adoc @@ -702,6 +702,98 @@ For more information please co

[pve-devel] [PATCH pve-firewall v6 1/4] add support for loading sdn firewall configuration

2024-11-18 Thread Stefan Hanreich
This also includes support for parsing rules referencing IPSets in the new SDN scope and generating those IPSets in the firewall. We always load the new configuration, since loading the configuration always includes validating the loaded rules. Validation fails without including the SDN ipsets, lea

[pve-devel] [PATCH widget-toolkit v3 2/3] node tasks: align view task button label to other uses

2024-11-18 Thread Aaron Lauterer
We use "View Task" in the tooltip for the actionbutton per row. Aligning the label for consistency and making it clear what one opens/views. Signed-off-by: Aaron Lauterer --- changes since v2: * new patch src/node/Tasks.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/

Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 18:49 schrieb Thomas Lamprecht: > Am 18.11.24 um 18:44 schrieb Filip Schauer: >> I tried importing the GNS3 OVA files found here: >> https://www.gns3.com/software/download-vm >> >> I got the same XPath error with the VirtualBox file. > > Can you try something like: > > diff --git a

[pve-devel] [PATCH pve-firewall v5 02/10] api: add vnet endpoints

2024-11-18 Thread Stefan Hanreich
Signed-off-by: Stefan Hanreich --- src/PVE/API2/Firewall/Helpers.pm | 14 +++ src/PVE/API2/Firewall/Makefile | 1 + src/PVE/API2/Firewall/Rules.pm | 84 + src/PVE/API2/Firewall/Vnet.pm| 156 +++ src/PVE/Firewall.pm | 10 ++ 5 f

[pve-devel] [PATCH pve-firewall v5 01/10] sdn: add vnet firewall configuration

2024-11-18 Thread Stefan Hanreich
Signed-off-by: Stefan Hanreich Reviewed-by: Wolfgang Bumiller Tested-by: Hannes Dürr --- src/PVE/Firewall.pm | 127 ++-- src/PVE/Firewall/Helpers.pm | 12 2 files changed, 132 insertions(+), 7 deletions(-) diff --git a/src/PVE/Firewall.pm b/src/PVE

Re: [pve-devel] [PATCH qemu-server v7 4/5] api: create: add 'import-working-storage' parameter

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 18:39 schrieb Aaron Lauterer: > lore.proxmox.com seems to show the diff wrong... > I see them "wrong" too in my inbox, seems your mail user agent messes with them? ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.pro

[pve-devel] [PATCH docs/firewall/manager/network v5 00/10] add forward chain firewalling for hosts and vnets

2024-11-18 Thread Stefan Hanreich
## Introduction This patch series introduces a new direction for firewall rules: forward. Additionally this patch series introduces defining firewall rules on a vnet level. ## Use Cases For hosts: * hosts utilizing NAT can define firewall rules for NATed traffic * hosts utilizing EVPN zones can

[pve-devel] [PATCH pve-manager v5 07/10] sdn: add firewall panel

2024-11-18 Thread Stefan Hanreich
Expose the ability to create vnet-level firewalls in the PVE UI Signed-off-by: Stefan Hanreich Tested-by: Hannes Dürr --- www/manager6/Makefile| 2 + www/manager6/dc/Config.js| 8 +++ www/manager6/sdn/FirewallPanel.js| 50 ++ www/manager6/sdn/Fi

Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 18:44 schrieb Filip Schauer: > I tried importing the GNS3 OVA files found here: > https://www.gns3.com/software/download-vm > > I got the same XPath error with the VirtualBox file. Can you try something like: diff --git a/src/PVE/GuestImport/OVF.pm b/src/PVE/GuestImport/OVF.pm ind

Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 18:35 schrieb Aaron Lauterer: >> Upon clicking "Import" in the web UI the "Import Guest" dialog opens >> along with an error message stating: >> XPath error : Undefined namespace prefix at /usr/share/perl5/PVE/ >> GuestImport/OVF.pm line 239. (500) > I untar'ed that OVA and inspecte

Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Filip Schauer
I tried importing the GNS3 OVA files found here: https://www.gns3.com/software/download-vm I got the same XPath error with the VirtualBox file. The ESXi and Workstation & Fusion OVA files were imported successfully and they boot aswell. On 18/11/2024 18:14, Filip Schauer wrote: Hello, I trie

[pve-devel] [PATCH pve-network v5 09/10] firewall: add endpoints for vnet-level firewall

2024-11-18 Thread Stefan Hanreich
Signed-off-by: Stefan Hanreich Reviewed-by: Wolfgang Bumiller --- src/PVE/API2/Network/SDN/Vnets.pm | 6 ++ 1 file changed, 6 insertions(+) diff --git a/src/PVE/API2/Network/SDN/Vnets.pm b/src/PVE/API2/Network/SDN/Vnets.pm index 05915f6..e48b048 100644 --- a/src/PVE/API2/Network/SDN/Vnets.

[pve-devel] [PATCH pve-manager v5 04/10] firewall: add forward direction to rule panel

2024-11-18 Thread Stefan Hanreich
Enables us to use the new forward direction as an option when creating or editing firewall rules. By introducing firewall_type we can switch between the available directions depending on which ruleset is being edited. Signed-off-by: Stefan Hanreich Tested-by: Hannes Dürr --- www/manager6/dc/Con

[pve-devel] [PATCH pve-docs v5 10/10] firewall: add documentation for forward direction and vnet zone

2024-11-18 Thread Stefan Hanreich
Added a new direction section, mostly so I can write about the forward direction and explain its use cases. Signed-off-by: Stefan Hanreich --- Makefile | 1 + gen-pve-firewall-vnet-opts.pl | 12 + pve-firewall-vnet-opts.adoc | 8 +++ pve-firewall.adoc | 9

[pve-devel] [PATCH pve-manager v5 08/10] firewall: rules: show warning when creating forward rules

2024-11-18 Thread Stefan Hanreich
Since forward rules only take effect when the nftables firewall is enabled, show a warning to users that informs them of this. Signed-off-by: Stefan Hanreich --- www/manager6/grid/FirewallRules.js | 22 ++ 1 file changed, 22 insertions(+) diff --git a/www/manager6/grid/Firew

[pve-devel] [PATCH pve-manager v5 05/10] firewall: add vnet to firewall options component

2024-11-18 Thread Stefan Hanreich
Add the configuration options for vnet-level firewalls to the options component. Additionally add the new policy_forward configuration option to the datacenter-level firewall as well. Signed-off-by: Stefan Hanreich Tested-by: Hannes Dürr --- www/manager6/grid/FirewallOptions.js | 38 +++

[pve-devel] [PATCH pve-manager v5 06/10] firewall: make base_url dynamically configurable in options component

2024-11-18 Thread Stefan Hanreich
This adds the ability to dynamically configure and change the base_url for the firewall options. This is needed for the SDN firewall dialog, that updates the firewall components based on the selected vnet. This avoids having to reinstantiate the component every time the user selects a new vnet. Si

[pve-devel] [PATCH widget-toolkit v3 1/3] node tasks: add action column with button to view task

2024-11-18 Thread Aaron Lauterer
This is primarily aimed to make it more discoverable that one can open tasks directly, instead of selecting and clicking the view button at the top. The chevron-right icon was chosen, to be more in alignmend with other places where we use it. For now, mainly PBS. Signed-off-by: Aaron Lauterer --

Re: [pve-devel] [PATCH qemu-server v7 4/5] api: create: add 'import-working-storage' parameter

2024-11-18 Thread Aaron Lauterer
lore.proxmox.com seems to show the diff wrong... On 2024-11-18 18:24, Aaron Lauterer wrote: On  2024-11-18  16:29, Dominik Csapak wrote: this is to override the target extraction storage for the option disk extraction for 'import-from'. This way if the storage does not supports the content

[pve-devel] [PATCH pve-manager v6 3/4] firewall: add sdn scope to IPRefSelector

2024-11-18 Thread Stefan Hanreich
Signed-off-by: Stefan Hanreich Tested-by: Gabriel Goller Tested-by: Hannes Dürr --- www/manager6/form/IPRefSelector.js | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/www/manager6/form/IPRefSelector.js b/www/manager6/form/IPRefSelector.js index d41cde5f5..16078e428 1

[pve-devel] [PATCH docs/firewall/manager v6 0/4] autogenerate ipsets for sdn objects

2024-11-18 Thread Stefan Hanreich
This patch series adds support for autogenerating ipsets for SDN objects. It autogenerates ipsets for every VNet as follows: * ipset containing all IP ranges of the VNet * ipset containing all gateways of the VNet * ipset containing all IP ranges of the subnet - except gateways * ipset containing

[pve-devel] [PATCH pve-firewall v6 2/4] ipsets: return sdn ipsets from api

2024-11-18 Thread Stefan Hanreich
In order for the new SDN ipsets to show up we need to adapt the existing API endpoints so they read the SDN configuration. We reload the SDN configuration explicitly, in order to return only the IPSets the user is allowed to see. Signed-off-by: Stefan Hanreich --- src/PVE/API2/Firewall/Cluster.p

Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Aaron Lauterer
On 2024-11-18 18:14, Filip Schauer wrote: Hello, I tried to quickly test the OVA import with a CentOS 7 OVA file that I downloaded from: https:// dlconusc1.linuxvmimages.com/046389e06777452db2ccf9a32efa3760:dldatac/ VirtualBox/C/7/CentOS_7.9.2009_VBM.zip 6785cb04dc5e7ab5a28ec139f631dfc8

Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Filip Schauer
Hello, I tried to quickly test the OVA import with a CentOS 7 OVA file that I downloaded from: https://dlconusc1.linuxvmimages.com/046389e06777452db2ccf9a32efa3760:dldatac/VirtualBox/C/7/CentOS_7.9.2009_VBM.zip 6785cb04dc5e7ab5a28ec139f631dfc8aeb8a6278c5fc66e1e7030d3f5a83b18 /var/lib/vz/import/

Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Aaron Lauterer
Gave this series a test. GNS3 OVA, the following variants: OVA location -> Target Storage local -> local: worked local -> zfs:worked local -> lvm-thin: worked local -> rbd:worked cephfs -> any: did not work. got the following error: --- scsi0: storage se

[pve-devel] [PATCH storage v7 11/11] plugin: file_size_info: warn on parent images with unusual path

2024-11-18 Thread Dominik Csapak
If the base image (parent) of an image contains e.g. whitespace in it's path, the current untainting would not match and it would seem there was no parent. Since untrusted files are not allowed to have backing parts, just warn, when encountering this case to keep backwards compatibility. Signed-o

Re: [pve-devel] [PATCH pve-firewall v5 2/5] add support for loading sdn firewall configuration

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 17:11 schrieb Stefan Hanreich: > On 11/18/24 17:09, Thomas Lamprecht wrote: >> Am 18.11.24 um 15:24 schrieb Stefan Hanreich: >>> I think we should *always* load the whole configuration here and filter >>> which IPSets we output in the API methods instead (by invoking >>> load_sdn_con

Re: [pve-devel] [PATCH pve-firewall v5 2/5] add support for loading sdn firewall configuration

2024-11-18 Thread Stefan Hanreich
On 11/18/24 17:09, Thomas Lamprecht wrote: > Am 18.11.24 um 15:24 schrieb Stefan Hanreich: >> I think we should *always* load the whole configuration here and filter >> which IPSets we output in the API methods instead (by invoking >> load_sdn_conf there explicitly and updating the cluster_conf has

Re: [pve-devel] [PATCH pve-firewall v5 2/5] add support for loading sdn firewall configuration

2024-11-18 Thread Thomas Lamprecht
Am 18.11.24 um 15:24 schrieb Stefan Hanreich: > I think we should *always* load the whole configuration here and filter > which IPSets we output in the API methods instead (by invoking > load_sdn_conf there explicitly and updating the cluster_conf hash). sounds good to me, are you already working

[pve-devel] [PATCH qemu-server v7 3/5] api: create: implement extracting disks when needed for import-from

2024-11-18 Thread Dominik Csapak
when 'import-from' contains a disk image that needs extraction (currently only from an 'ova' archive), do that in 'create_disks' and overwrite the '$source' volid. Collect the names into a 'delete_sources' list, that we use later to clean it up again (either when we're finished with importing or i

[pve-devel] [PATCH storage v7 08/11] api: allow ova upload/download

2024-11-18 Thread Dominik Csapak
introducing a separate regex that only contains ova, since upload/downloading ovfs does not make sense (since the disks are then missing). Signed-off-by: Dominik Csapak --- changes from v6: * fix indentation/style * omit check for '.ova' extension * use assert_file_validity callback for download

[pve-devel] [PATCH storage v7 05/11] ovf: implement parsing out firmware type

2024-11-18 Thread Dominik Csapak
it seems there is no part of the ovf standard that handles which type of bios there is (at least i could not find it). Every ovf/ova i tested either has no info about it, or has it in a vmware specific property which we parse here. Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner --- src/

[pve-devel] [PATCH qemu-server v7 1/5] disk import: add additional safeguards for imported image files

2024-11-18 Thread Dominik Csapak
From: Fabian Grünbichler creating non-raw disk images with arbitrary content is only possible with raw access to the storage, but checking for references to external files doesn't hurt, in case for non pve-managed volumes. Signed-off-by: Fabian Grünbichler [ DC: removed problematic checks for

Re: [pve-devel] [PATCH storage/qemu-server/manager v6] implement ova/ovf import for file based storages

2024-11-18 Thread Dominik Csapak
sent a v7: https://lore.proxmox.com/pve-devel/20241118152928.858590-1-d.csa...@proxmox.com/ ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager v7 7/9] ui: guest import: change icon/text for non-esxi import storage

2024-11-18 Thread Dominik Csapak
since 'virtual guests' only make sense for a hypervisor, not e.g. a directory for OVAs also change the icon from 'desktop' to 'cloud-download' in the non-esxi case Signed-off-by: Dominik Csapak --- www/manager6/storage/Browser.js | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff

[pve-devel] [PATCH qemu-server v7 2/5] use OVF from Storage

2024-11-18 Thread Dominik Csapak
and delete it here (incl tests; they live in pve-storage now). Signed-off-by: Dominik Csapak --- changes from v6: * group imports correctly PVE/CLI/qm.pm | 4 +- PVE/QemuServer/Makefile | 1 - PVE/QemuServer/OVF.pm

[pve-devel] [PATCH storage v7 07/11] ovf: implement parsing nics

2024-11-18 Thread Dominik Csapak
by iterating over the relevant parts and trying to parse out the 'ResourceSubType'. The content of that is not standardized, but I only ever found examples that are compatible with vmware, meaning it's either 'e1000', 'e1000e' or 'vmxnet3' (in various capitalizations; thus the `lc()`) As a fallbac

[pve-devel] [PATCH qemu-server v7 4/5] api: create: add 'import-working-storage' parameter

2024-11-18 Thread Dominik Csapak
this is to override the target extraction storage for the option disk extraction for 'import-from'. This way if the storage does not supports the content type 'images', one can give an alternative one. Signed-off-by: Dominik Csapak --- changes from v6: * rename 'import-extraction-storage' to 'im

[pve-devel] [PATCH manager v7 9/9] ui: import: adapt live import help text to ova

2024-11-18 Thread Dominik Csapak
For ova import, the help text for live-import is not really fitting, as there is no 'old' vm that needs to be stopped, etc. So show a more appropriate one if we import an ova Signed-off-by: Dominik Csapak --- new in v7 www/manager6/window/GuestImport.js | 4 +++- 1 file changed, 3 insertions(+)

[pve-devel] [PATCH manager v7 1/9] ui: guest import: add ova-needs-extracting warning text

2024-11-18 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- changes from v6: * reworded warning www/manager6/window/GuestImport.js | 1 + 1 file changed, 1 insertion(+) diff --git a/www/manager6/window/GuestImport.js b/www/manager6/window/GuestImport.js index 2577ece2..1a08195e 100644 --- a/www/manager6/window/GuestImpo

[pve-devel] [PATCH storage v7 09/11] plugin: enable import for nfs/btrfs/cifs/cephfs/glusterfs

2024-11-18 Thread Dominik Csapak
and reuse the DirPlugin implementation Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner --- src/PVE/Storage/BTRFSPlugin.pm | 5 + src/PVE/Storage/CIFSPlugin.pm | 6 +- src/PVE/Storage/CephFSPlugin.pm| 6 +- src/PVE/Storage/GlusterfsPlugin.pm | 6 +- src/PVE/St

[pve-devel] [PATCH storage v7 10/11] add 'import' content type to 'check_volume_access'

2024-11-18 Thread Dominik Csapak
in the same branch as 'vztmpl' and 'iso' Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner --- src/PVE/Storage.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm index de730b7..1e12b51 100755 --- a/src/PVE/Storage.pm +++ b/src/PVE

[pve-devel] [PATCH manager v7 3/9] ui: enable upload/download/remove buttons for 'import' type storages

2024-11-18 Thread Dominik Csapak
but only for non esxi ones, since that does not allow uploading/downloading there Signed-off-by: Dominik Csapak --- www/manager6/storage/Browser.js| 9 +++-- www/manager6/window/UploadToStorage.js | 1 + 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/www/manager6/stor

[pve-devel] [PATCH storage v7 02/11] plugin: dir: handle ova files for import

2024-11-18 Thread Dominik Csapak
since we want to handle ova files (which are only ovf+images bundled in a tar file) for import, add code that handles that. we introduce a valid volname for files contained in ovas like this: storage:import/archive.ova/disk-1.vmdk by basically treating the last part of the path as the name for

[pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

2024-11-18 Thread Dominik Csapak
This series enables importing ova/ovf from directory based storages, inclusive upload/download via the webui (ova only). It also improves the ovf importer by parsing the ostype, nics, bootorder (and firmware from vmware exported files). We now extract the images into either a given target storage

[pve-devel] [PATCH manager v7 6/9] ui: guest import: add storage selector for ova extraction storage

2024-11-18 Thread Dominik Csapak
but only when we detect the 'ova-needs-extraction' warning. This can be used to select the storage where the disks contained in an OVA will be extracted to temporarily. Signed-off-by: Dominik Csapak --- changes from v6: * adapted to new parameter * reworded import working storage label www/mana

[pve-devel] [PATCH qemu-server v7 5/5] api: check untrusted image files for import content type

2024-11-18 Thread Dominik Csapak
check to be imported files for external references if they are of content type 'import'. Signed-off-by: Dominik Csapak --- changes from v6: * remove unused $scfg assignment PVE/API2/Qemu.pm | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2

[pve-devel] [PATCH manager v7 4/9] ui: disable 'import' button for non importable formats

2024-11-18 Thread Dominik Csapak
importable formats are currently ova/ovf/vmx Signed-off-by: Dominik Csapak --- www/manager6/storage/Browser.js | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/www/manager6/storage/Browser.js b/www/manager6/storage/Browser.js index 934ce706..822257e7 100644 --- a/www/ma

[pve-devel] [PATCH manager v7 2/9] ui: enable import content type for relevant storages

2024-11-18 Thread Dominik Csapak
Signed-off-by: Dominik Csapak --- www/manager6/Utils.js| 1 + www/manager6/form/ContentTypeSelector.js | 2 +- www/manager6/storage/CephFSEdit.js | 2 +- www/manager6/storage/GlusterFsEdit.js| 2 +- 4 files changed, 4 insertions(+), 3 deletions(-) diff --git a/www/m

[pve-devel] [PATCH manager v7 5/9] ui: import: improve rendering of volume names

2024-11-18 Thread Dominik Csapak
in directory storages, we don't need the 'import/' part of the volumes, as that is implied in dir based storages Signed-off-by: Dominik Csapak --- www/manager6/Utils.js | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js index

[pve-devel] [PATCH manager v7 8/9] ui: import: show size for dir-based storages

2024-11-18 Thread Dominik Csapak
since there we already have the size information Signed-off-by: Dominik Csapak --- www/manager6/storage/Browser.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/www/manager6/storage/Browser.js b/www/manager6/storage/Browser.js index 763abc70..c0b66acc 100644 --- a/www/manag

[pve-devel] [PATCH storage v7 01/11] plugin: dir: implement import content type

2024-11-18 Thread Dominik Csapak
in DirPlugin and not Plugin (because of cyclic dependency of Plugin -> OVF -> Storage -> Plugin otherwise) only ovf is currently supported (though ova will be shown in import listing), expects the files to not be in a subdir, and adjacent to the ovf file. listed will be all ovf/qcow2/raw/vmdk fil

[pve-devel] [PATCH storage v7 04/11] ovf: implement parsing the ostype

2024-11-18 Thread Dominik Csapak
use the standards info about the ostypes to map to our own (see comment for link to the relevant part of the dmtf schema) every type that is not listed we map to 'other', so no need to have it in a list. Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner --- src/PVE/GuestImport/OVF.pm | 69

[pve-devel] [PATCH storage v7 06/11] ovf: implement rudimentary boot order

2024-11-18 Thread Dominik Csapak
simply add all parsed disks to the boot order in the order we encounter them (similar to the esxi plugin). Signed-off-by: Dominik Csapak Reviewed-by: Fiona Ebner --- src/PVE/GuestImport/OVF.pm | 6 +- src/test/run_ovf_tests.pl | 3 +++ 2 files changed, 8 insertions(+), 1 deletion(-) diff

[pve-devel] [PATCH storage v7 03/11] ovf: improve and simplify path checking code

2024-11-18 Thread Dominik Csapak
moves the filepath code a bit more closer to where it's actually used checks the contained path before trying to find it's absolute path properly add error handling to realpath instead of checking the combined ovf_path + filepath, just make sure filepath can't point to anythign besides a file in t

Re: [pve-devel] [PATCH storage/qemu-server/manager v6] implement ova/ovf import for file based storages

2024-11-18 Thread Dominik Csapak
On 11/18/24 14:06, Lukas Wagner wrote: On Fri Nov 15, 2024 at 4:17 PM CET, Dominik Csapak wrote: This series enables importing ova/ovf from directory based storages, inclusive upload/download via the webui (ova only). It also improves the ovf importer by parsing the ostype, nics, bootorder (and

Re: [pve-devel] [PATCH storage/qemu-server/manager v6] implement ova/ovf import for file based storages

2024-11-18 Thread Daniel Herzig
Daniel Herzig writes: > I've just tested this series with the following images: > > + GNS3 with VMware ESXi image from https://www.gns3.com/software/download-vm, > unzipped and uploaded to local dir storage. > + Ubuntu Noble from > https://cloud-images.ubuntu.com/noble/current/noble-server-cloud

Re: [pve-devel] [PATCH pve-firewall v5 2/5] add support for loading sdn firewall configuration

2024-11-18 Thread Stefan Hanreich
On 11/18/24 12:41, Stefan Hanreich wrote: > +sub load_sdn_conf { > +my $rpcenv = eval { PVE::RPCEnvironment::get() }; After some additional consideration and testing, I think it is a bad idea to have the permission filtering in the core firewall code. Particularly because loading and validat

Re: [pve-devel] [PATCH storage/qemu-server/manager v6] implement ova/ovf import for file based storages

2024-11-18 Thread Daniel Herzig
I've just tested this series with the following images: + GNS3 with VMware ESXi image from https://www.gns3.com/software/download-vm, unzipped and uploaded to local dir storage. + Ubuntu Noble from https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.ova, downloaded straight

Re: [pve-devel] [PATCH qemu-server v6 6/6] api: check untrusted image files for import content type

2024-11-18 Thread Fiona Ebner
Am 15.11.24 um 16:17 schrieb Dominik Csapak: > @@ -468,6 +470,13 @@ my sub create_disks : prototype($$$) { > $live_import_mapping->{$ds}->{'delete-after-finish'} = > $source > if $needs_extraction; > } else { > +

Re: [pve-devel] [PATCH manager v13 5/5] ui: add AMD SEV configuration to Options

2024-11-18 Thread Shannon Sterz
On Mon Nov 18, 2024 at 12:17 PM CET, Markus Frank wrote: > By adding a new input panel with an AMD SEV technology selection combo > box and checkboxes for the optional parameters in an advanced section, > the user can configure the amd_sev option via the WebUI's Options tab. > > Signed-off-by: Mark

Re: [pve-devel] [PATCH docs v13 4/5] add AMD SEV documentation

2024-11-18 Thread Shannon Sterz
On Mon Nov 18, 2024 at 12:16 PM CET, Markus Frank wrote: > add documentation for the "[PATCH qemu-server] config: QEMU AMD SEV > enable" patch. > > Signed-off-by: Markus Frank > --- > qm.adoc | 103 > 1 file changed, 103 insertions(+) > > d

Re: [pve-devel] [PATCH qemu-server v6 1/6] disk import: add additional safeguards for imported image files

2024-11-18 Thread Fiona Ebner
Am 15.11.24 um 16:17 schrieb Dominik Csapak: > From: Fabian Grünbichler > > creating non-raw disk images with arbitrary content is only possible with raw > access to the storage, but checking for references to external files doesn't > hurt, in case for non pve-managed volumes. > > Signed-off-by:

[pve-devel] [PATCH manager v3 3/3] ui: tasks: add actioncolumn to open task

2024-11-18 Thread Aaron Lauterer
Many (new) users don't realize that the task view can be used to open the task details. By adding a small action column we can have a button to open the task as well, next to double clicking it. Signed-off-by: Aaron Lauterer --- I placed the actioncolumn at the end. See the cover letter for the r

[pve-devel] applied: [PATCH proxmox-firewall v4 05/18] nftables: derive additional traits for nftables types

2024-11-18 Thread Thomas Lamprecht
Am 15.11.24 um 13:10 schrieb Stefan Hanreich: > Signed-off-by: Stefan Hanreich > Reviewed-by: Wolfgang Bumiller > --- > proxmox-nftables/src/types.rs | 8 > 1 file changed, 4 insertions(+), 4 deletions(-) > > applied, thanks! ___ pve-devel

Re: [pve-devel] [PATCH pve-firewall v4 6/9] api: load sdn ipsets

2024-11-18 Thread Thomas Lamprecht
Am 15.11.24 um 13:09 schrieb Stefan Hanreich: > Since the SDN configuration reads the IPAM config file, which resides does that mean the earlier patches already require this? They load the SDN config already FWICT; and if so, it would be great to either have that change in those patches or upfront

[pve-devel] [PATCH pve-firewall v5 2/5] add support for loading sdn firewall configuration

2024-11-18 Thread Stefan Hanreich
This also includes support for parsing rules referencing IPSets in the new SDN scope and generating those IPSets in the firewall. We always load the new configuration, since loading the configuration always includes validating the loaded rules. Validation fails without including the SDN ipsets, lea

[pve-devel] [PATCH pve-manager v5 4/5] firewall: add sdn scope to IPRefSelector

2024-11-18 Thread Stefan Hanreich
Signed-off-by: Stefan Hanreich Tested-by: Gabriel Goller Tested-by: Hannes Dürr --- www/manager6/form/IPRefSelector.js | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/www/manager6/form/IPRefSelector.js b/www/manager6/form/IPRefSelector.js index d41cde5f5..16078e428 1

Re: [pve-devel] [PATCH storage/qemu-server/manager v6] implement ova/ovf import for file based storages

2024-11-18 Thread Lukas Wagner
On Mon Nov 18, 2024 at 2:18 PM CET, Dominik Csapak wrote: > On 11/18/24 14:06, Lukas Wagner wrote: > > On Fri Nov 15, 2024 at 4:17 PM CET, Dominik Csapak wrote: > >> This series enables importing ova/ovf from directory based storages, > >> inclusive upload/download via the webui (ova only). > >> >

Re: [pve-devel] [PATCH qemu-server v6 4/6] api: create: implement extracting disks when needed for import-from

2024-11-18 Thread Dominik Csapak
On 11/18/24 14:31, Fiona Ebner wrote: Am 15.11.24 um 16:17 schrieb Dominik Csapak: @@ -416,6 +427,23 @@ my sub create_disks : prototype($$) { my ($source_storage, $source_volid) = PVE::Storage::parse_volume_id($source, 1); if ($source_storage) { # PVE-managed volu

Re: [pve-devel] [PATCH storage/qemu-server/manager v6] implement ova/ovf import for file based storages

2024-11-18 Thread Dominik Csapak
On 11/18/24 14:44, Dominik Csapak wrote: On 11/18/24 14:39, Lukas Wagner wrote: On Mon Nov 18, 2024 at 2:18 PM CET, Dominik Csapak wrote: On 11/18/24 14:06, Lukas Wagner wrote: On Fri Nov 15, 2024 at 4:17 PM CET, Dominik Csapak wrote: This series enables importing ova/ovf from directory based

  1   2   >