[pve-devel] [PATCH v2 container 2/2] close #3471: Support for NetworkManager on CentOS.

2021-06-16 Thread Denis Knauf
NetworkManager is used on RHEL8 and similar distributions like CentOS8, AlmaLinux8 and RockyLinux8. On RHEL7/CentOS7 it was not mandatory, but possible to use NetworkManager. The past config was used on CentOS6/7 withour NetworkManager, so tests were also testing this basic network-config. The pat

[pve-devel] [PATCH v2 container 1/2] CentOS6 is obsolete. Tests renamed to centos7.

2021-06-16 Thread Denis Knauf
Signed-off-by: Denis Knauf --- src/test/{test-centos6-001 => test-centos7-001}/config| 0 src/test/{test-centos6-001 => test-centos7-001}/etc/hosts.exp | 0 .../etc/init/power-status-changed.conf.exp| 0 .../etc/init/start-ttys.conf.exp

[pve-devel] applied: [RFC qemu-server] avoid setting lun number for drives when pvscsi controller is used

2021-06-16 Thread Thomas Lamprecht
On 13.04.21 10:24, Fabian Ebner wrote: > Reported in the community forum[0]. > > In QEMU's hw/scsi/vmw_pvscsi.c in the SCSIBusInfo struct, the max_lun property > is set to 0. This means that in our stack, one cannot have multiple disks and > use 'scsihw: pvscsi' currently, as kvm would fail with >

[pve-devel] applied: [PATCH container] pct exec: add friendly warning in case container not running

2021-06-16 Thread Thomas Lamprecht
On 14.06.21 15:07, Oguz Bektas wrote: > Signed-off-by: Oguz Bektas > --- > src/PVE/CLI/pct.pm | 2 ++ > 1 file changed, 2 insertions(+) > > applied, thanks! ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mai

Re: [pve-devel] [PATCH access-control v2] add missing paths in check_path

2021-06-16 Thread Thomas Lamprecht
On 16.06.21 15:55, Dominik Csapak wrote: > * /access/realm/ > * /access/groups/ > > were overlooked when fixing #1500 > > see: https://forum.proxmox.com/threads/are-group-acls-broken-in-v6-4.91000/ > > Signed-off-by: Dominik Csapak > --- > supersedes my earlier patch: > https://lists.proxmox.co

[pve-devel] applied: [PATCH access-control v2] add missing paths in check_path

2021-06-16 Thread Thomas Lamprecht
On 16.06.21 15:55, Dominik Csapak wrote: > * /access/realm/ > * /access/groups/ > > were overlooked when fixing #1500 > > see: https://forum.proxmox.com/threads/are-group-acls-broken-in-v6-4.91000/ > > Signed-off-by: Dominik Csapak > --- > supersedes my earlier patch: > https://lists.proxmox.co

Re: [pve-devel] [PATCH v4 container 0/2] post_clone hook for containers

2021-06-16 Thread Oguz Bektas
On Wed, Jun 16, 2021 at 03:48:42PM +0200, Thomas Lamprecht wrote: > On 16.06.21 15:24, Oguz Bektas wrote: > > fixes #3443 by implementing post_clone_hook for containers (which at the > > moment only generates a unique /etc/machine-id for the cloned > > container). this can be reused later for other

[pve-devel] [PATCH access-control v2] add missing paths in check_path

2021-06-16 Thread Dominik Csapak
* /access/realm/ * /access/groups/ were overlooked when fixing #1500 see: https://forum.proxmox.com/threads/are-group-acls-broken-in-v6-4.91000/ Signed-off-by: Dominik Csapak --- supersedes my earlier patch: https://lists.proxmox.com/pipermail/pve-devel/2021-June/048704.html changes: * also ad

[pve-devel] applied: [PATCH docs] faq: fix link

2021-06-16 Thread Thomas Lamprecht
On 16.06.21 08:22, Fabian Ebner wrote: > Signed-off-by: Fabian Ebner > --- > pve-faq.adoc | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > applied, nice catch, thanks! ___ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.p

[pve-devel] [PATCH access-control] fix missing /access/realm/ paths in check_path

2021-06-16 Thread Dominik Csapak
they were overlooked see: https://forum.proxmox.com/threads/are-group-acls-broken-in-v6-4.91000/ Signed-off-by: Dominik Csapak --- src/PVE/AccessControl.pm | 1 + 1 file changed, 1 insertion(+) diff --git a/src/PVE/AccessControl.pm b/src/PVE/AccessControl.pm index 75e..9a8d1fb 100644 --- a/

Re: [pve-devel] [PATCH v4 container 0/2] post_clone hook for containers

2021-06-16 Thread Thomas Lamprecht
On 16.06.21 15:24, Oguz Bektas wrote: > fixes #3443 by implementing post_clone_hook for containers (which at the > moment only generates a unique /etc/machine-id for the cloned > container). this can be reused later for other things we want to do post > clone. > what are the changes since v3? I d

[pve-devel] applied: [PATCH qemu-server] fix #3329: turn on cache=writeback for efidisks on rbd

2021-06-16 Thread Thomas Lamprecht
On 16.06.21 15:09, Dominik Csapak wrote: > on slower ceph clusters, the write pattern of the ovmf booting process > slows down the boot of the vm, so we turn on caching by default > > it seems no other storage (until now) behaves like this. if it does in > the future, we can still add them too, or

Re: [pve-devel] [PATCH qemu-server] fix #3329: turn on cache=writeback for efidisks on rbd

2021-06-16 Thread Thomas Lamprecht
hi, On 16.06.21 15:25, Maximilian Hill wrote: > On Wed, Jun 16, 2021 at 03:09:33PM +0200, Dominik Csapak wrote: >> on slower ceph clusters, the write pattern of the ovmf booting process >> slows down the boot of the vm, so we turn on caching by default >> >> +my $cache = ""; >> +if ($path

Re: [pve-devel] [PATCH qemu-server] fix #3329: turn on cache=writeback for efidisks on rbd

2021-06-16 Thread Maximilian Hill
Hello, On Wed, Jun 16, 2021 at 03:09:33PM +0200, Dominik Csapak wrote: > on slower ceph clusters, the write pattern of the ovmf booting process > slows down the boot of the vm, so we turn on caching by default > > + my $cache = ""; > + if ($path =~ m/^rbd:/) { > + $cache = ',cache

[pve-devel] [PATCH v4 container 2/2] run post_clone_hook in clone_vm

2021-06-16 Thread Oguz Bektas
also cleaned up the locking situation with this, as Fabian G. suggested. now we check if the 'create' lock is held before writing out the config file. use the 'create_and_lock_config' helper in the beginning to ensure that the target CTID is available, and that the target config is locked from the

[pve-devel] [PATCH v4 container 1/2] setup: add post_clone_hook for containers

2021-06-16 Thread Oguz Bektas
for now it only calls the new clear_machine_id function. this way when new containers are created they will have a unique /etc/machine-id. Signed-off-by: Oguz Bektas --- src/PVE/LXC/Setup.pm | 12 src/PVE/LXC/Setup/Base.pm | 31 +++ 2 files changed,

[pve-devel] [PATCH v4 container 0/2] post_clone hook for containers

2021-06-16 Thread Oguz Bektas
fixes #3443 by implementing post_clone_hook for containers (which at the moment only generates a unique /etc/machine-id for the cloned container). this can be reused later for other things we want to do post clone. Oguz Bektas (2): setup: add post_clone_hook for containers run post_clone_hoo

Re: [pve-devel] [PATCH v2 storage 1/5] postinst: move cifs credential files into subdirectory upon update

2021-06-16 Thread Thomas Lamprecht
On 16.06.21 09:26, Fabian Ebner wrote: > +case "$1" in > + configure) > +if test -n "$2"; then > + > +# TODO: remove once PVE 8.0 is released > +if dpkg --compare-versions "$2" 'lt' '7.0-3'; then > +for file in /etc/pve/priv/*.cred; do > +if [ -f "$f

[pve-devel] [PATCH qemu-server] fix #3329: turn on cache=writeback for efidisks on rbd

2021-06-16 Thread Dominik Csapak
on slower ceph clusters, the write pattern of the ovmf booting process slows down the boot of the vm, so we turn on caching by default it seems no other storage (until now) behaves like this. if it does in the future, we can still add them too, or add a 'cache' property for the efidisk Signed-off

Re: [pve-devel] [PATCH v2 manager] pve6to7: add check for pool permissions

2021-06-16 Thread Thomas Lamprecht
On 16.06.21 14:45, Fabian Grünbichler wrote: > On June 16, 2021 2:16 pm, Lorenz Stechauner wrote: >> the two checks make sure that: >> * no user defined role 'PVEPoolUser' exists >> * the user gets a hint for roles only containing Pool.Allocate and >> not Pool.Audit >> >> Signed-off-by: Lorenz

Re: [pve-devel] [PATCH v2 manager] pve6to7: add check for pool permissions

2021-06-16 Thread Fabian Grünbichler
On June 16, 2021 2:16 pm, Lorenz Stechauner wrote: > the two checks make sure that: > * no user defined role 'PVEPoolUser' exists > * the user gets a hint for roles only containing Pool.Allocate and > not Pool.Audit > > Signed-off-by: Lorenz Stechauner > --- > changes to v1: > * rebased on ma

[pve-devel] [PATCH v2 manager] pve6to7: add check for pool permissions

2021-06-16 Thread Lorenz Stechauner
the two checks make sure that: * no user defined role 'PVEPoolUser' exists * the user gets a hint for roles only containing Pool.Allocate and not Pool.Audit Signed-off-by: Lorenz Stechauner --- changes to v1: * rebased on master PVE/CLI/pve6to7.pm | 25 + 1 file chan

[pve-devel] applied-series: [PATCH-SERIES v2 manager/storage] Some breaking API changes/cleanups

2021-06-16 Thread Fabian Grünbichler
with two follow-ups: - pve-storage postinst: handle read-only postinst (warn, but don't fail the upgrade) - pve-manager: wrap vzdump.cron check in eval {} and warn on error On June 16, 2021 9:26 am, Fabian Ebner wrote: > Improved version of the not-yet applied patches and follow-ups, regarding

[pve-devel] [PATCH manager] ui: HD edit: Add multiple disks & tabs

2021-06-16 Thread Dominic Jäger
Enable adding multiple disks in VM create wizard. This is a first step for future import features. Split disk edit panel into multiple tabbed panels to make it less cluttered. This affects the create wizard & the HD edit windows in the VM hardware view. Signed-off-by: Dominic Jäger --- www/mana

[pve-devel] [PATCH manager] pve6to7: add check for pool permissions

2021-06-16 Thread Lorenz Stechauner
the two checks make sure that: * no user defined role 'PVEPoolUser' exists * the user gets a hint for roles only containing Pool.Allocate and not Pool.Audit --- PVE/CLI/pve6to7.pm | 25 + 1 file changed, 25 insertions(+) diff --git a/PVE/CLI/pve6to7.pm b/PVE/CLI/pve6to

[pve-devel] applied: [PATCH v9 common 2/2] tools: download_file_from_url: move check for existing file outside eval

2021-06-16 Thread Thomas Lamprecht
On 16.06.21 11:35, Lorenz Stechauner wrote: > it is not necessary to include this block in the eval which when it > fails tries to unlink $tmpdest, because in the check for the existing > file $tmpdest is not used. > --- > src/PVE/Tools.pm | 26 +- > 1 file changed, 13 inse

[pve-devel] applied: [PATCH] tools: download from url: add option to allow overriding existing files

2021-06-16 Thread Thomas Lamprecht
Signed-off-by: Thomas Lamprecht --- allows for slightly nicer UX and drops the confusing special case of "always allow overwriting if no checksum algorithm was passed". src/PVE/Tools.pm | 26 -- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/src/PVE/Tool

[pve-devel] applied: [PATCH v9 common 1/2] tools: download_file_from_url: adapt error messages to start at new line

2021-06-16 Thread Thomas Lamprecht
On 16.06.21 11:35, Lorenz Stechauner wrote: > the front end expects the error message to be the first part of the > last line. putting the new line at the beginning of the die message > does not work, either. > > https://lists.proxmox.com/pipermail/pve-devel/2021-June/048676.html > --- > src/PVE/

[pve-devel] [PATCH v9 manager 5/5] fix #1710: ui: storage: add download from url button

2021-06-16 Thread Lorenz Stechauner
uses the common function PVE::Tools::download_file_from_url to download a iso image or container template. note: Only users with permissions `Sys.Audit` and `Sys.Modify` on `/` are permitted to use the api endpoints due to security reasons. (it is possible to download files from internal networks

[pve-devel] [PATCH v9 storage 1/1] status: add download_url method

2021-06-16 Thread Lorenz Stechauner
uses common function PVE::Tools::download_file_from_url to download iso files. Only users with permissions `Sys.Audit` and `Sys.Modify` on `/` are permitted to perform this action. This restriction is due to the fact, that the download function is able to download files from internal networks (whi

[pve-devel] [PATCH v9 manager 1/5] api: nodes: add query_url_metadata method

2021-06-16 Thread Lorenz Stechauner
metadata is gained using a HEAD request. Due to the ability of this api endpoint to request files on internal networks (which would not be visible/accessible from outside) it is restricted to users with permissions `Sys.Audit` and `Sys.Modify` on `/`. Users with these permissions are able to alter

[pve-devel] [PATCH v9 manager 3/5] ui: add HashAlgorithmSelector

2021-06-16 Thread Lorenz Stechauner
Signed-off-by: Lorenz Stechauner --- www/manager6/Makefile | 1 + www/manager6/form/HashAlgorithmSelector.js | 16 2 files changed, 17 insertions(+) create mode 100644 www/manager6/form/HashAlgorithmSelector.js diff --git a/www/manager6/Makefile b/www/mana

[pve-devel] [PATCH v9 common 1/2] tools: download_file_from_url: adapt error messages to start at new line

2021-06-16 Thread Lorenz Stechauner
the front end expects the error message to be the first part of the last line. putting the new line at the beginning of the die message does not work, either. https://lists.proxmox.com/pipermail/pve-devel/2021-June/048676.html --- src/PVE/Tools.pm | 6 -- 1 file changed, 4 insertions(+), 2 de

[pve-devel] [PATCH-SERIES v9 manager/common/storage] fix #1710: add download from url button

2021-06-16 Thread Lorenz Stechauner
changes to v8: * explanation why 'print "\n"' is needed * move check for existing file outside eval block pve-manager: Lorenz Stechauner (5): api: nodes: add query_url_metadata method api: nodes: refactor aplinfo to use common download function ui: add HashAlgorithmSelector ui: Utils: cha

[pve-devel] [PATCH v9 manager 2/5] api: nodes: refactor aplinfo to use common download function

2021-06-16 Thread Lorenz Stechauner
a common function to download arbitrary files from urls has been defined as PVE::Tools::download_file_from_url and is now used. Signed-off-by: Lorenz Stechauner --- PVE/API2/Nodes.pm | 93 +-- 1 file changed, 17 insertions(+), 76 deletions(-) diff --g

[pve-devel] [PATCH v9 manager 4/5] ui: Utils: change download task format

2021-06-16 Thread Lorenz Stechauner
Signed-off-by: Lorenz Stechauner --- www/manager6/Utils.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js index d9567979..9fef29bf 100644 --- a/www/manager6/Utils.js +++ b/www/manager6/Utils.js @@ -1776,7 +1776,7 @@ Ext.define('PV

[pve-devel] [PATCH v9 common 2/2] tools: download_file_from_url: move check for existing file outside eval

2021-06-16 Thread Lorenz Stechauner
it is not necessary to include this block in the eval which when it fails tries to unlink $tmpdest, because in the check for the existing file $tmpdest is not used. --- src/PVE/Tools.pm | 26 +- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/src/PVE/Tools.p

Re: [pve-devel] [PATCH v8 common 2/2] tools: download_file_from_url: adapt error messages to start at new line

2021-06-16 Thread Thomas Lamprecht
On 16.06.21 10:46, Lorenz Stechauner wrote: >>> -    die "ERROR, checksum mismatch: got '$checksum_got' != expect >>> '$checksum_expected'\n"; >>> +    print "\n"; >> If, we could just include the "\n" in the start of the die, but we do not >> really >> add such newlines for separating er

Re: [pve-devel] [PATCH v8 common 2/2] tools: download_file_from_url: adapt error messages to start at new line

2021-06-16 Thread Lorenz Stechauner
On 15.06.21 16:23, Thomas Lamprecht wrote: On 15.06.21 16:08, Lorenz Stechauner wrote: the front end expects the error message to be the first part of the last line. --- src/PVE/Tools.pm | 10 ++ 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/PVE/Tools.pm b/src/PVE

[pve-devel] [RFC v2 storage 5/5] cephfs: update reminder for systemd_netmount removal

2021-06-16 Thread Fabian Ebner
Commit d9ece228fbbbf0eb575c8a067a70ed7023078f84 introduced the workaround with using systemd units and 25e222ca0d96a0da0d491e749b437d2822215e9e re-used the functionality for fuse-mounts too. The latter commit suggests to switch to using mount.fuse.ceph for the '_netdev' option, but it doesn't seem

[pve-devel] [PATCH v2 storage 2/5] config: mention that maxfiles is deprecated

2021-06-16 Thread Fabian Ebner
Don't add an explicit deprecation warning on parsing (yet), this already done in the pve6to7 script. Also, automatic conversion to 'prune-backups' happens when the section config is read, so over time fewer users should be affected. Postpone explicit warning/dropping the parameter to a future major

[pve-devel] [PATCH v2 manager 1/3] pve6to7: add checks for backup retention options

2021-06-16 Thread Fabian Ebner
Note that it's not possible to use read_vzdump_defaults() and storage_config(), because they auto-converts maxfiles already. Signed-off-by: Fabian Ebner --- New in v2. PVE/CLI/pve6to7.pm | 63 ++ 1 file changed, 63 insertions(+) diff --git a/PVE/CLI

[pve-devel] [PATCH v2 manager 3/3] ui: storage: update hint about prune-backups default

2021-06-16 Thread Fabian Ebner
and also remove the keepLastEmptyText, which too suggested the old default. Signed-off-by: Fabian Ebner --- New in v2. www/manager6/storage/Base.js | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/www/manager6/storage/Base.js b/www/manager6/storage/Base.js index 20ec3464.

[pve-devel] [PATCH v2 storage 4/5] cephfs: revert safe-guard check for Luminous

2021-06-16 Thread Fabian Ebner
It's necessary to be on Nautilus before upgrading to 7.x, so the check is no longer needed. See commit e54c3e334760491954bc42f3585a8b5b136d4b1d. It didn't cleanly revert, because there were cleanups made afterwards. Signed-off-by: Fabian Ebner --- New in v2. PVE/Storage/CephFSPlugin.pm | 5 +--

[pve-devel] [PATCH v2 storage 3/5] config: add backup content type to default local storage

2021-06-16 Thread Fabian Ebner
which is used if there is no ('dir'-type) 'local' entry. Storage configurations made by the installer also support backups for the 'local' storage, and the 'prune-backups' parameter is not really useful otherwise. Signed-off-by: Fabian Ebner --- New in v2. PVE/Storage/Plugin.pm | 9 -

[pve-devel] [PATCH v2 storage 1/5] postinst: move cifs credential files into subdirectory upon update

2021-06-16 Thread Fabian Ebner
and drop the compat code. Signed-off-by: Fabian Ebner --- Changes from v1: * make sure to create the /etc/pve/priv/storage directory which might not exist yet * always print info when a file was found * use variable for target path * use mv * removing the helper needs t

[pve-devel] [PATCH-SERIES v2 manager/storage] Some breaking API changes/cleanups

2021-06-16 Thread Fabian Ebner
Improved version of the not-yet applied patches and follow-ups, regarding backup retention and CIFS credentials, and path+RFC for CephFS at the end. Changes from v1: * improve CIFS credential renaming * mention that maxfiles is deprecated * add checks for pve6to7 * add patch+RFC f

[pve-devel] [PATCH v2 manager 2/3] pve6to7: add check for CIFS credentials

2021-06-16 Thread Fabian Ebner
Signed-off-by: Fabian Ebner --- New in v2. PVE/CLI/pve6to7.pm | 22 ++ 1 file changed, 22 insertions(+) diff --git a/PVE/CLI/pve6to7.pm b/PVE/CLI/pve6to7.pm index b8263dab..f486ac2d 100644 --- a/PVE/CLI/pve6to7.pm +++ b/PVE/CLI/pve6to7.pm @@ -574,6 +574,27 @@ sub check_back