[pve-devel] [RFC pve-qemu 1/9] Add -list-flags command line option

2021-10-28 Thread Stefan Reiter
Useful for retrieving flags from default CPU models. Signed-off-by: Stefan Reiter --- ...E-Add-list-flags-command-line-option.patch | 213 ++ debian/patches/series | 1 + 2 files changed, 214 insertions(+) create mode 100644 debian/patches/pve/0049

[pve-devel] [RFC qemu-server 2/9] api: add recognized-flags and supported-flags endpoints

2021-10-28 Thread Stefan Reiter
For supporting a GUI to easily create custom CPU models based on cluster CPU availability. Signed-off-by: Stefan Reiter --- PVE/API2/Qemu/CPU.pm | 86 1 file changed, 86 insertions(+) diff --git a/PVE/API2/Qemu/CPU.pm b/PVE/API2/Qemu/CPU.pm index

[pve-devel] [RFC manager 6/9] gui: CPUModelSelector: fix dirty state on default

2021-10-28 Thread Stefan Reiter
"originalValue" is set to "null" in case it's the default, but getValue() returns an empty string. This means that when editing a VM's CPU config when the model is "default", the form would always be marked dirty. Signed-off-by: Stefan Reiter --- w

[pve-devel] [RFC 0/9] Unfinished: Custom CPU type API and GUI

2021-10-28 Thread Stefan Reiter
and would have to stay up-to-date with supported CPU models in QEMU. pve-qemu: Stefan Reiter (1): Add -list-flags command line option ...E-Add-list-flags-command-line-option.patch | 213 ++ debian/patches/series | 1 + 2 files changed, 214 insertions(

[pve-devel] [RFC qemu-server 4/9] api: add /cpu/model/* get/create/delete/update endpoints

2021-10-28 Thread Stefan Reiter
onfig fails. Signed-off-by: Stefan Reiter --- PVE/API2/Qemu/CPU.pm| 174 +++- PVE/QemuServer/CPUConfig.pm | 17 +++- 2 files changed, 189 insertions(+), 2 deletions(-) diff --git a/PVE/API2/Qemu/CPU.pm b/PVE/API2/Qemu/CPU.pm index 9cc1d77..8a3dcfd 100644

[pve-devel] [RFC manager 8/9] gui: add basic custom CPU model editor

2021-10-28 Thread Stefan Reiter
sidebar, so add a new class that correctly aligns pve-itype-icon-* elements. Signed-off-by: Stefan Reiter --- www/css/ext6-pve.css | 4 + www/manager6/Makefile | 3 + www/manager6/dc/CPUTypeEdit.js| 81 ++ www/manager6/dc/CPUTypeView.js

[pve-devel] [RFC manager 9/9] Initial attempt at CPU flag editor for custom models

2021-10-28 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- I believe this one is the most outdated, it's original title in my working branch was "tmp", dating back to more than a year ago ;) www/manager6/dc/CPUTypeEdit.js | 8 ++ www/manager6/form/VMCPUFlagSelector.js | 125 +++

[pve-devel] [RFC qemu-server 3/9] api: add /cpu/model/* get endpoint

2021-10-28 Thread Stefan Reiter
case someone retrieved it to modify it with a PUT call afterwards. Signed-off-by: Stefan Reiter --- PVE/API2/Qemu/CPU.pm| 76 - PVE/QemuServer/CPUConfig.pm | 7 2 files changed, 82 insertions(+), 1 deletion(-) diff --git a/PVE/API2/Qemu/CPU.pm b

[pve-devel] [RFC manager 5/9] gui: VMCPUFlagSelector: fix unknownFlags behaviour

2021-10-28 Thread Stefan Reiter
...and typo in name. Signed-off-by: Stefan Reiter --- www/manager6/form/VMCPUFlagSelector.js | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/www/manager6/form/VMCPUFlagSelector.js b/www/manager6/form/VMCPUFlagSelector.js index ace3c531..a6d17930 100644 --- a

[pve-devel] [RFC manager 7/9] gui: CPUModelSelector: add 'allowCustom'

2021-10-28 Thread Stefan Reiter
When set to false, only default CPU models will be shown. Signed-off-by: Stefan Reiter --- www/manager6/form/CPUModelSelector.js | 16 1 file changed, 16 insertions(+) diff --git a/www/manager6/form/CPUModelSelector.js b/www/manager6/form/CPUModelSelector.js index a4d883a4

[pve-devel] [PATCH qemu] extra: update VNC 'display' patches to latest from list

2021-10-28 Thread Stefan Reiter
CONFIG_SPICE) in the QAPI JSON as that semantic was renamed in the meantime. Signed-off-by: Stefan Reiter --- See: https://lists.nongnu.org/archive/html/qemu-devel/2021-10/msg06676.html ...atch => 0003-qxl-fix-pre-save-logic.patch} | 0 ...upport-for-flag-argument-with-value.patch} |

[pve-devel] [PATCH qemu-server] vzdump: increase timeout for QMP 'cont' after backup start

2021-10-27 Thread Stefan Reiter
auf-7-0.97388/ https://forum.proxmox.com/threads/error-with-backup-when-backing-up-qmp-command-query-backup-failed-got-wrong-command-id.88017/page-3#post-416339 Signed-off-by: Stefan Reiter --- PVE/VZDump/QemuServer.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/VZDump/QemuSe

Re: [pve-devel] [PATCH qemu-server v2] use non SMM ovmf code file for i440fx machines

2021-10-21 Thread Stefan Reiter
Works as intended, both i440fx and Q35 run with a secure boot efidisk. Tested-by: Stefan Reiter Reviewed-by: Stefan Reiter On 10/11/21 2:10 PM, Dominik Csapak wrote: ovmf with SMM enabled will not boot on i440fx (hangs on graphics initialization), so load the non SMM variant. should be no

Re: [pve-devel] [PATCH 0/4] add meta info and bandaid for QEMU 6.1 and unpinned q35 machine backward compat

2021-10-21 Thread Stefan Reiter
On 10/21/21 11:47 AM, Thomas Lamprecht wrote: On 21.10.21 11:34, Stefan Reiter wrote: On 10/21/21 10:36 AM, Thomas Lamprecht wrote: First add a new meta property that is currently exclusively set on new VM creation and then read-only, initially add the creation time as UNIX epoch and the QEMU

Re: [pve-devel] [PATCH 4/4] cfg2cmd: switch off ACPI hotplug on bridges for q35 VMs

2021-10-21 Thread Stefan Reiter
On 10/21/21 10:36 AM, Thomas Lamprecht wrote: See commit 17858a1695 (hw/acpi/ich9: Set ACPI PCI hot-plug as default on Q35)[0] in upstream QEMU repository for details about why the change was made. As that change affects systemds predictable interface naming[1], e.g., by going from a previously

Re: [pve-devel] [PATCH 3/4] tests: cfg2cmd: add a few q35 related tests

2021-10-21 Thread Stefan Reiter
On 10/21/21 10:36 AM, Thomas Lamprecht wrote: Signed-off-by: Thomas Lamprecht --- test/cfg2cmd/q35-simple-6.0.conf| 13 ++ test/cfg2cmd/q35-simple-6.0.conf.cmd| 28 + test/cfg2cmd/q35-simple-6.1.conf| 14 +++ test/cfg2cmd/q

Re: [pve-devel] [PATCH 0/4] add meta info and bandaid for QEMU 6.1 and unpinned q35 machine backward compat

2021-10-21 Thread Stefan Reiter
On 10/21/21 10:36 AM, Thomas Lamprecht wrote: First add a new meta property that is currently exclusively set on new VM creation and then read-only, initially add the creation time as UNIX epoch and the QEMU version that was installed during installation (thought about using the one on first star

[pve-devel] [PATCH qemu-server 2/2] swtpm: wait for pidfile

2021-10-14 Thread Stefan Reiter
the timeout duration should be more than enough. Also add a missing 'usleep' import, 'usleep' was used before but never imported, apparently the other case never got triggered... Signed-off-by: Stefan Reiter --- Triggered during rollback testing, potentially due to extra lo

[pve-devel] [PATCH qemu-server 1/2] snapshot: fix tpmstate with rbd

2021-10-14 Thread Stefan Reiter
QEMU doesn't know about the tpmstate, so 'do_snapshots_with_qemu' should never return true in that case. Note that inconsistencies related to snapshot timing do not matter much, as the actual TPM data is exported together with other device state by QEMU anyway. Signed-off-by

[pve-devel] [PATCH pve-qemu] add fixup patch for qxl migration logic

2021-10-13 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- Applies on top of the 6.1 rebase. .../extra/0006-qxl-fix-pre-save-logic.patch | 34 +++ debian/patches/series | 1 + 2 files changed, 35 insertions(+) create mode 100644 debian/patches/extra/0006-qxl-fix-pre-save

[pve-devel] [PATCH manager 2/3] ui: SystemEdit: promote advanced settings to normal

2021-10-11 Thread Stefan Reiter
and reorder the panel a bit, as otherwise I found it *very* cluttered. Signed-off-by: Stefan Reiter --- www/manager6/qemu/SystemEdit.js | 50 +++-- 1 file changed, 22 insertions(+), 28 deletions(-) diff --git a/www/manager6/qemu/SystemEdit.js b/www/manager6/qemu

[pve-devel] [PATCH qemu-server 1/3] ostype: support Windows 11/Server 2022

2021-10-11 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- PVE/QemuServer.pm | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index eb29fc2..6fc28e8 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -386,7 +386,7 @@ my $confdesc = { ostype

[pve-devel] [PATCH 0/3] Windows 11 OS type and UI improvements

2021-10-11 Thread Stefan Reiter
Add a specific 'win11' ostype for Windows 11 and Server 2022, with currently no change on the backend, but enable q35, secure boot and a TPM in the wizard by default if selected. qemu-server: Stefan Reiter (1): ostype: support Windows 11/Server 2022 PVE/QemuServer.pm | 3 ++- 1 fi

[pve-devel] [PATCH manager 3/3] ui: add 'win11' ostype and set defaults in wizard

2021-10-11 Thread Stefan Reiter
Windows 11 requires a TPM and secure boot, so try to auto-enable both. Signed-off-by: Stefan Reiter --- www/manager6/Utils.js | 1 + www/manager6/qemu/SystemEdit.js | 16 2 files changed, 17 insertions(+) diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js

[pve-devel] [PATCH docs 1/2] qm: add note about secure boot and new efidisk behaviour

2021-10-06 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- qm.adoc | 30 -- 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/qm.adoc b/qm.adoc index 80734e3..93576c7 100644 --- a/qm.adoc +++ b/qm.adoc @@ -733,10 +733,13 @@ the operating system. By default QEMU uses *SeaBIOS* for

[pve-devel] [PATCH docs 2/2] qm: add section about TPM

2021-10-06 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- qm.adoc | 30 ++ 1 file changed, 30 insertions(+) diff --git a/qm.adoc b/qm.adoc index 93576c7..b9f4269 100644 --- a/qm.adoc +++ b/qm.adoc @@ -775,6 +775,36 @@ you need to set the client resolution in the OVMF menu (which you can

[pve-devel] [PATCH qemu-server] ovmf: support secure boot with 4m and 4m-ms efidisk types

2021-10-05 Thread Stefan Reiter
Provide support for secure boot by using the new "4m" and "4m-ms" variants of the OVMF code/vars templates. This is specified on the efidisk via the 'efitype' and 'ms-keys' parameters. Signed-off-by: Stefan Reiter --- Should depend on updated pve-edk2-

[pve-devel] [PATCH v3 0/3] Initial TPM support for VMs

2021-10-04 Thread Stefan Reiter
similar to an efidisk - this eliminates any concerns about storing on pmxcfs * always allocate the state as 4 MiB (on directory storage it might auto-shrink) * fixes migration, since source and destination are now different * add GUI patch storage: Stefan Reiter (1): import: don't check

[pve-devel] [PATCH v3 qemu-server 2/3] fix #3075: add TPM v1.2 and v2.0 support via swtpm

2021-10-04 Thread Stefan Reiter
to avoid it ever being treated as a regular drive from the rest of the stack in case it gets left over after a backup for some reason (shouldn't happen). Signed-off-by: Stefan Reiter --- PVE/API2/Qemu.pm | 5 ++ PVE/QemuMigrate.pm | 14 +++- PVE/QemuServer.pm|

[pve-devel] [PATCH v3 storage 1/3] import: don't check for 1K aligned size

2021-10-04 Thread Stefan Reiter
TPM state disks on directory storages may have completely unaligned sizes, this check doesn't make sense for them. This appears to just be a (weak) safeguard and not serve an actual functional purpose, so simply get rid of it to allow migration of TPM state. Signed-off-by: Stefan R

[pve-devel] [PATCH v3 manager 3/3] ui: add support for adding TPM devices

2021-10-04 Thread Stefan Reiter
Inspired by HDEfi for efidisks. Extends the DiskStorageSelector to allow hiding the format, since tpmstate can only be stored in 'raw' format (even on directory storages). Signed-off-by: Stefan Reiter --- www/manager6/Makefile| 1 + www/manager

[pve-devel] [RFC v2 qemu-server 3/4] fix #3075: add TPM v1.2 and v2.0 support via swtpm

2021-09-23 Thread Stefan Reiter
27;t work. swtpm itself has no method of moving a backing storage at runtime. Signed-off-by: Stefan Reiter --- PVE/API2/Qemu.pm| 5 ++ PVE/QemuMigrate.pm | 14 - PVE/QemuServer.pm | 135 +--- PVE/QemuServer/Drive.pm | 63 +++

[pve-devel] [RFC v2 edk2-firmware 1/4] enable TPM and TPM2 support

2021-09-23 Thread Stefan Reiter
Necessary for an OS to use a TPM attached to a OVMF VM. Signed-off-by: Stefan Reiter --- debian/rules | 4 1 file changed, 4 insertions(+) diff --git a/debian/rules b/debian/rules index fb85b29..ece77be 100755 --- a/debian/rules +++ b/debian/rules @@ -54,6 +54,8 @@ build-ovmf: setup-build

[pve-devel] [RFC v2 manager 4/4] ui: add support for adding TPM devices

2021-09-23 Thread Stefan Reiter
Inspired by HDEfi for efidisks. Extends the DiskStorageSelector to allow hiding the format, since tpmstate can only be stored in 'raw' format (even on directory storages). Signed-off-by: Stefan Reiter --- www/manager6/Makefile| 1 + www/manager

[pve-devel] [RFC v2 0/4] Initial TPM support for VMs

2021-09-23 Thread Stefan Reiter
ate the state as 4 MiB (on directory storage it might auto-shrink) * fixes migration, since source and destination are now different * add GUI patch edk2-firmware: Stefan Reiter (1): enable TPM and TPM2 support debian/rules | 4 ++++ 1 file changed, 4 insertions(+) storage: Stefan Reiter (1

[pve-devel] [RFC v2 storage 2/4] import: don't check for 1K aligned size

2021-09-23 Thread Stefan Reiter
TPM state disks on directory storages may have completely unaligned sizes, this check doesn't make sense for them. This appears to just be a (weak) safeguard and not serve an actual functional purpose, so simply get rid of it to allow migration of TPM state. Signed-off-by: Stefan R

[pve-devel] [PATCH manager] fix #3629: ui: rng: increase default ratelimit to 2048 B/s

2021-09-20 Thread Stefan Reiter
FreeBSD 13 VMs will bug out and consume loads of idle CPU time if the ratelimit is 1024 B/s or lower. Increasing to 2048 fixes the issue, confirmed experimentally, and doesn't seem much worse as a default. Signed-off-by: Stefan Reiter --- On the one hand this *really* sounds like a FreeBS

Re: [pve-devel] [PATCH qemu] drop patch force-disabling smm

2021-08-24 Thread Stefan Reiter
igration should always work. Tested-by: Stefan Reiter If it breaks that we'd need to fallback to the old "false" with a QEMU machine version check and we could only drop it then in PVE 8.0, so hopefully it Just Works™ to safe that hassle.. __

Re: [pve-devel] VM Incremental backup for 3-party solution (Qemu)

2021-08-19 Thread Stefan Reiter
On 8/19/21 3:21 PM, Radosław Korzeniewski wrote: Hello, I'm working on implementing VM incremental backup for a 3-party solution based on Qemu dirty bitmap which is available in Proxmox. Unfortunately I'm unable to make it working, so I decided to ask for some help. First I'm adding a dirty bitm

Re: [pve-devel] [RFC qemu-server 2/2] fix #3075: add TPM v1.2 and v2.0 support via swtpm

2021-08-10 Thread Stefan Reiter
On 09/08/2021 20:17, Nick Chevsky wrote: Hi Stefan, Thank you for your work on this; I've been testing it locally for a few weeks and have since contributed improved Debian packaging and other fixes upstream [3]. Please see my comment below the quoted code: Thanks for your upstream work! I've

Re: [pve-devel] [PATCH v2 qemu-server 3/3] restore: remove efidisk from config if not backed up

2021-08-02 Thread Stefan Reiter
On 7/23/21 11:02 AM, Thomas Lamprecht wrote: On 08.07.21 13:46, Stefan Reiter wrote: If it doesn't exist, there's no need to keep it around at all. Such a backup might easily be created if an efidisk is configured with BIOS set to anything but 'OVMF'. Unbreaks live-restor

[pve-devel] [RFC qemu-server 2/2] fix #3075: add TPM v1.2 and v2.0 support via swtpm

2021-07-15 Thread Stefan Reiter
7;tpm' config option is removed or the version changed, effectively clearing any stored keys/data. Signed-off-by: Stefan Reiter --- PVE/QemuServer.pm | 134 +- 1 file changed, 133 insertions(+), 1 deletion(-) diff --git a/PVE/QemuServer.pm b/PVE/Qe

[pve-devel] [RFC 0/2] Initial TPM support for VMs

2021-07-15 Thread Stefan Reiter
pshots don't include TPM data * migration may hickup, since destination and target are effectively "the same" * needs improved clearing logic, or potentially none at all (would be a benefit for the "efidisk-like" variant) edk2-firmware: Stefan Reiter (1): enable TPM

[pve-devel] [RFC edk2-firmware 1/2] enable TPM and TPM2 support

2021-07-15 Thread Stefan Reiter
Necessary for an OS to use a TPM attached to a OVMF VM. Signed-off-by: Stefan Reiter --- debian/rules | 4 1 file changed, 4 insertions(+) diff --git a/debian/rules b/debian/rules index fb85b29..ece77be 100755 --- a/debian/rules +++ b/debian/rules @@ -54,6 +54,8 @@ build-ovmf: setup-build

[pve-devel] [PATCH v2 qemu-server 3/3] restore: remove efidisk from config if not backed up

2021-07-08 Thread Stefan Reiter
If it doesn't exist, there's no need to keep it around at all. Such a backup might easily be created if an efidisk is configured with BIOS set to anything but 'OVMF'. Unbreaks live-restore for such cases too. Signed-off-by: Stefan Reiter --- v1->v2: * Actually remove e

[pve-devel] [PATCH qemu-server 2/3] cfg2cmd: only warn on non-existant ISO image

2021-07-08 Thread Stefan Reiter
...instead of hard failing. Requires a RPCEnvironment to issue the warnings, so init a dummy one in run_config2command_tests. Signed-off-by: Stefan Reiter --- PVE/QemuServer.pm| 10 ++ test/run_config2command_tests.pl | 2 ++ 2 files changed, 12 insertions(+) diff

[pve-devel] [PATCH qemu-server 1/3] live-restore: fail early if target storage doesn't exist

2021-07-08 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- PVE/QemuServer.pm | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index b0fe257..0d8affa 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -6381,13 +6381,13 @@ sub

[pve-devel] [PATCH qemu-server 3/3] live-restore: ignore missing efidisk

2021-07-08 Thread Stefan Reiter
up in general. Signed-off-by: Stefan Reiter --- PVE/QemuServer.pm | 7 +++ 1 file changed, 7 insertions(+) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 4082e69..a86a2a4 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -6471,6 +6471,13 @@ sub restore_proxmox_backup_ar

[pve-devel] [PATCH manager] ui: remove 'line-height' from datacenter/summary to fix FF

2021-07-05 Thread Stefan Reiter
Firefox doesn't correctly break to the next line if the line-height is set as is (with certain fonts, including the default on debian it seems). Simply remove it, as it isn't necessary. Suggested-by: Dominik Csapak Signed-off-by: Stefan Reiter --- www/manager6/dc/Guests.js | 1

[pve-devel] [PATCH qemu-server] api: always add new CD drives to bootorder

2021-07-05 Thread Stefan Reiter
: Stefan Reiter --- PVE/API2/Qemu.pm | 8 1 file changed, 8 insertions(+) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 1e540f5..b750db0 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -1357,6 +1357,14 @@ my $update_vm_api = sub { if defined($conf

Re: [pve-devel] applied: [PATCH qemu-server] live-restore: preload efidisk before starting VM

2021-07-01 Thread Stefan Reiter
On 7/1/21 12:57 PM, Thomas Lamprecht wrote: On 01.07.21 11:37, Stefan Reiter wrote: The efidisk never got restored correctly before, since we don't use the generic print_drive_commandline_full for it, and as such it didn't get a backing image attached. This not only causes the efidi

[pve-devel] [PATCH qemu-server] live-restore: preload efidisk before starting VM

2021-07-01 Thread Stefan Reiter
t from the live restore entirely. Signed-off-by: Stefan Reiter --- best viewed with -w I guess in QEMU < 6.0 this didn't cause an error, since I definitely tested with EFI VMs before, however, it would have never actually restored the data and instead always created a new one.

[pve-devel] [PATCH qemu-server] cfg2cmd/drive: don't use io_uring for krbd with wb/wt cache

2021-06-30 Thread Stefan Reiter
=threads as it used to. Signed-off-by: Stefan Reiter --- I tried my usual QEMU debugging routine, but couldn't figure anything specific out - it's not related to EFI, happens on SeaBIOS too, but looking at GDB just appears that it's waiting for the io_uring fd to become ready (in

[pve-devel] [PATCH installer] zfs: allow RAID0 with different-sized disks

2021-06-24 Thread Stefan Reiter
zfs_mirror_size_check is already called in get_zfs_raid_setup when necessary, so don't call it unconditionally, as this will cause a false-positive error on RAID0 devices (where size mismatches are fine). Signed-off-by: Stefan Reiter --- proxinstall | 2 -- 1 file changed, 2 deletions(-)

Re: [pve-devel] [PATCH qemu-server 1/2] use KillMode 'process' for systemd scope

2021-06-22 Thread Stefan Reiter
On 22/06/2021 08:02, Thomas Lamprecht wrote: On 21.06.21 18:35, Stefan Reiter wrote: KillMode 'none' is deprecated, and systemd loudly complains about that in the journal. To avoid the warning, but keep the behaviour the same, use KillMode 'process'. This mode does tw

[pve-devel] [PATCH 0/2] Work around systemd warning about KillMode 'none'

2021-06-21 Thread Stefan Reiter
nt/101.scope:3: Unit configured to use KillMode=none. [...] I still get the same warning about a 'ceph-volume@.service' in my logs, after upgrading to pacific, though I'm not sure if that's something we need to/can fix, or something left over from the upgrade - any ceph expertise we

[pve-devel] [PATCH qemu-server 1/2] use KillMode 'process' for systemd scope

2021-06-21 Thread Stefan Reiter
lse, or TimeoutStopUSec to infinity - for safety, we do both In my testing, this replicated the previous behaviour exactly, but without using the deprecated 'none' mode. Signed-off-by: Stefan Reiter --- Depends on updated pve-common from patch 2. PVE/QemuServer.pm | 4 +++- 1 file

[pve-devel] [PATCH common 2/2] systemd: allow SendSIGKILL and TimeoutStopUSec dbus properties

2021-06-21 Thread Stefan Reiter
Used in qemu-server for avoiding KillMode 'none'. SendSIGKILL is a boolean, so we need to use dbus_boolean to serialize it. Signed-off-by: Stefan Reiter --- src/PVE/Systemd.pm | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/src/PVE/Systemd.pm b/src/PVE/System

Re: [pve-devel] [PATCH v2 qemu-server 2/2] cfg2cmd: make io_uring default

2021-06-21 Thread Stefan Reiter
similar for a version that'll probably never leave staging seemed unnecessary. For testing, build pve-qemu-kvm from git, as io_uring is already enabled there. On 6/21/21 5:33 PM, Stefan Reiter wrote: The 'aio' setting is not visible to the guest, and so can be changed during migratio

[pve-devel] [PATCH v2 qemu-server 2/2] cfg2cmd: make io_uring default

2021-06-21 Thread Stefan Reiter
The 'aio' setting is not visible to the guest, and so can be changed during migrations or snapshots without issue. It is thus only dependendent on the actual QEMU version being >= 6.0, not machine version. Signed-off-by: Stefan Reiter --- new in v2 tested migration from and to old

[pve-devel] [PATCH v2 qemu-server 1/2] enable io-uring support

2021-06-21 Thread Stefan Reiter
Note that the value in this enum directly represents the value passed to QEMU, so we need to use the underscore. Signed-off-by: Stefan Reiter --- same as v1, par the commit message PVE/QemuServer/Drive.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/QemuServer

Re: [pve-devel] [PATCH manager] ui: file-restore: start the file-restore on the selected node

2021-06-21 Thread Stefan Reiter
On 6/18/21 5:17 PM, Thomas Lamprecht wrote: On 10.06.21 13:28, Dominik Csapak wrote: On 6/10/21 11:29, Stefan Reiter wrote: On 6/10/21 10:40 AM, Dominik Csapak wrote: On 6/10/21 10:23, Stefan Reiter wrote: On 6/10/21 9:37 AM, Dominik Csapak wrote: and not the node where the browser connects

Re: [pve-devel] applied: [RFC qemu-server] avoid setting lun number for drives when pvscsi controller is used

2021-06-17 Thread Stefan Reiter
On 16/06/2021 20:27, Thomas Lamprecht wrote: On 13.04.21 10:24, Fabian Ebner wrote: Reported in the community forum[0]. In QEMU's hw/scsi/vmw_pvscsi.c in the SCSIBusInfo struct, the max_lun property is set to 0. This means that in our stack, one cannot have multiple disks and use 'scsihw: pvscs

Re: [pve-devel] [RFC qemu-server] avoid setting lun number for drives when pvscsi controller is used

2021-06-15 Thread Stefan Reiter
On 6/15/21 10:04 AM, Fabian Ebner wrote: Am 14.06.21 um 11:15 schrieb Stefan Reiter: On 6/14/21 8:29 AM, Fabian Ebner wrote: Any feedback for this? Looks good to me and works as advertised: Reviewed-by: Stefan Reiter Tested-by: Stefan Reiter Thanks for the review! The `$unit

Re: [pve-devel] [RFC qemu-server] avoid setting lun number for drives when pvscsi controller is used

2021-06-14 Thread Stefan Reiter
On 6/14/21 8:29 AM, Fabian Ebner wrote: Any feedback for this? Looks good to me and works as advertised: Reviewed-by: Stefan Reiter Tested-by: Stefan Reiter The `$unit = $drive->{index} % maxdev` logic doesn't seem useful here though, I think it's a copy/paste from the IDE/SA

Re: [pve-devel] [PATCH manager] ui: file-restore: start the file-restore on the selected node

2021-06-10 Thread Stefan Reiter
On 6/10/21 10:40 AM, Dominik Csapak wrote: On 6/10/21 10:23, Stefan Reiter wrote: On 6/10/21 9:37 AM, Dominik Csapak wrote: and not the node where the browser connects. there are at least two good reasons for this: * it is confusing, since the user would expect it to start where    the ui is

Re: [pve-devel] [PATCH manager] ui: file-restore: start the file-restore on the selected node

2021-06-10 Thread Stefan Reiter
On 6/10/21 9:37 AM, Dominik Csapak wrote: and not the node where the browser connects. there are at least two good reasons for this: * it is confusing, since the user would expect it to start where the ui is pointint to * the storage may not be available on the node the browser connects to,

[pve-devel] [PATCH v2 proxmox-backup 6/9] backup: remove AsyncIndexReader

2021-06-07 Thread Stefan Reiter
superseded by CachedChunkReader, with less code and more speed Signed-off-by: Stefan Reiter --- src/backup.rs| 3 - src/backup/async_index_reader.rs | 215 --- 2 files changed, 218 deletions(-) delete mode 100644 src/backup

[pve-devel] [PATCH v2 proxmox-backup 5/9] replace AsyncIndexReader with SeekableCachedChunkReader

2021-06-07 Thread Stefan Reiter
admin/datastore reads linearly only, so no need for cache (capacity of 1 basically means no cache except for the currently active chunk). mount can do random access too, so cache last 8 chunks for possibly a mild performance improvement. Signed-off-by: Stefan Reiter --- src/api2/admin

[pve-devel] [PATCH v2 proxmox-backup 2/9] tools: add AsyncLruCache as a wrapper around sync LruCache

2021-06-07 Thread Stefan Reiter
Supports concurrent 'access' calls to the same key via a BroadcastFuture. These are stored in a seperate HashMap, the LruCache underneath is only modified once a valid value has been retrieved. Signed-off-by: Stefan Reiter --- src/tools.rs | 1 + src/tools/async_lr

[pve-devel] [PATCH v2 proxmox-backup 3/9] backup: add CachedChunkReader utilizing AsyncLruCache

2021-06-07 Thread Stefan Reiter
Provides a fast arbitrary read implementation with full async and concurrency support. Signed-off-by: Stefan Reiter --- This is technically all that's needed for proxmox-backup-qemu to build and function as intended, but I decided to also use this IMHO cleaner implementation to replac

[pve-devel] [PATCH v2 proxmox-backup-qemu 9/9] access: use CachedChunkReader

2021-06-07 Thread Stefan Reiter
read_image_at, providing real concurrency without lock contention Signed-off-by: Stefan Reiter --- src/restore.rs | 30 -- 1 file changed, 8 insertions(+), 22 deletions(-) diff --git a/src/restore.rs b/src/restore.rs index 0790d7f..33959d9 100644 --- a/src/restore.rs +++ b

[pve-devel] [PATCH v2 proxmox-backup-qemu 8/9] add shared_cache module

2021-06-07 Thread Stefan Reiter
Provides a shared AsyncLruCache of 256MB (w/ 4MB chunks) that can be used by multiple readers at the same time. It is dropped once no more readers exist, so the memory gets freed if all QEMU block/pbs instances disappear. Signed-off-by: Stefan Reiter --- v2: * adapt to dropping ChunkCache type

[pve-devel] [PATCH v2 0/9] Improve live-restore speed and replace AsyncIndexReader

2021-06-07 Thread Stefan Reiter
25 30 35 40 iteration 1.0.3-1A patchedB [0] see: https://lists.proxmox.com/pipermail/pbs-devel/2021-April/002932.html proxmox-backup: Stefan Reiter (7): tools/BroadcastFuture: add testcase for better understand

[pve-devel] [PATCH v2 proxmox-backup 4/9] backup: add AsyncRead/Seek to CachedChunkReader

2021-06-07 Thread Stefan Reiter
actual request concurrency is not supported in these traits). Signed-off-by: Stefan Reiter --- v2: * drop 'seek_to_pos' from struct and implement error handling directly in start_seek * simplify future handling in poll_read with Option::get_or_insert_with src/backup/cached_chunk

[pve-devel] [PATCH v2 proxmox-backup 7/9] tools/lru_cache: make minimum capacity 1

2021-06-07 Thread Stefan Reiter
Setting this to 0 is not just useless, but breaks the logic horribly enough to cause random segfaults - better forbid this, to avoid someone else having to debug it again ;) Signed-off-by: Stefan Reiter --- src/tools/lru_cache.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/src/tools

[pve-devel] [PATCH v2 proxmox-backup 1/9] tools/BroadcastFuture: add testcase for better understanding

2021-06-07 Thread Stefan Reiter
Explicitly test that data will stay available and can be retrieved immediately via listen(), even if the future producing the data and notifying the consumers was already run in the past. Wasn't broken or anything, but helps with understanding IMO. Signed-off-by: Stefan Reiter --- W

[pve-devel] [PATCH qemu-server 2/2] enable io-uring support

2021-06-07 Thread Stefan Reiter
Note that the value in this enum directly represents the value passed to QEMU, so we need to use the underscore. Off by default, updated QEMU required, but no compatibility issues, as this is a new value that wasn't valid before, and setting it is the users choice. Signed-off-by: Stefan R

[pve-devel] [PATCH 0/2] Enable io-uring support in QEMU

2021-06-07 Thread Stefan Reiter
in Debian bullseye, but there seems to be some dependency issues with our pve-kernel-libc-dev package. For now, testing was conducted by building liburing from source, i.e. clone 'https://github.com/axboe/liburing' and run 'make-debs.sh'. pve-qemu: Stefan Reiter (1): enable i

[pve-devel] [PATCH pve-qemu 1/2] enable io-uring support in QEMU builds

2021-06-07 Thread Stefan Reiter
Signed-off-by: Stefan Reiter --- Oh if only all QEMU changes were this simple... debian/control | 1 + debian/rules | 1 + 2 files changed, 2 insertions(+) diff --git a/debian/control b/debian/control index 38c45ce..4da3409 100644 --- a/debian/control +++ b/debian/control @@ -27,6 +27,7

Re: [pve-devel] [PATCH qemu-server 2/6] drive: factor out read-only helper

2021-06-07 Thread Stefan Reiter
On 6/7/21 12:23 PM, Fabian Grünbichler wrote: On June 7, 2021 11:29 am, Stefan Reiter wrote: On 6/4/21 11:47 AM, Fabian Grünbichler wrote: we also need it for efidisks. Signed-off-by: Fabian Grünbichler --- PVE/QemuServer.pm | 8 ++-- PVE/QemuServer/Drive.pm | 10

Re: [pve-devel] [PATCH qemu-server 3/6] template: mark efidisk as read-only

2021-06-07 Thread Stefan Reiter
High-level: If you make the $read_only_str contain the entire ",readonly=on" stanza, you can just leave it blank by default and avoid updating all the test cases. On 6/4/21 11:47 AM, Fabian Grünbichler wrote: otherwise backups of templates using UEFI fail with storages like LVM thin, where the

Re: [pve-devel] [RFC qemu-server 6/6] template: start VM for VMA backup

2021-06-07 Thread Stefan Reiter
On 6/4/21 11:47 AM, Fabian Grünbichler wrote: since using 'vma create ..' no longer works with immutable/read-only base volumes. Why that? It shouldn't matter to vma if the base is read-only? Though I do see that in vma.c we don't tell QEMU that it should open the block backend read-only...

Re: [pve-devel] [PATCH qemu-server 2/6] drive: factor out read-only helper

2021-06-07 Thread Stefan Reiter
On 6/4/21 11:47 AM, Fabian Grünbichler wrote: we also need it for efidisks. Signed-off-by: Fabian Grünbichler --- PVE/QemuServer.pm | 8 ++-- PVE/QemuServer/Drive.pm | 10 ++ 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServe

Re: [pve-devel] [PATCH qemu-server 1/3] Revert "revert spice_ticket prefix change in 7827de4"

2021-06-07 Thread Stefan Reiter
For the three qemu-server changes: Reviewed-by: Stefan Reiter I'll also see if I can find some more stuff to cleanup in the code I often traverse :) On 6/4/21 3:49 PM, Fabian Ebner wrote: This reverts commit ff09c795edd12b1cc4604ee28a7cdd99cdd1afa8. We wanted to wait until PVE 7.0 fo

Re: [pve-devel] [pbs-devel] [PATCH proxmox-backup-qemu 8/9] add shared_cache module

2021-06-07 Thread Stefan Reiter
On 6/4/21 2:16 PM, Wolfgang Bumiller wrote: On Wed, Jun 02, 2021 at 04:38:32PM +0200, Stefan Reiter wrote: Provides a shared AsyncLruCache of 256MB (w/ 4MB chunks) that can be used by multiple readers at the same time. It is dropped once no more readers exist, so the memory gets freed if all

[pve-devel] [PATCH proxmox-backup 3/9] backup: add CachedChunkReader utilizing AsyncLruCache

2021-06-02 Thread Stefan Reiter
Provides a fast cache read implementation with full async and concurrency support. Signed-off-by: Stefan Reiter --- This is technically all that's needed for proxmox-backup-qemu to build and function as intended, but I decided to also use this IMHO cleaner implementation to replac

[pve-devel] [PATCH proxmox-backup-qemu 9/9] access: use CachedChunkReader

2021-06-02 Thread Stefan Reiter
read_image_at, providing real concurrency without lock contention Signed-off-by: Stefan Reiter --- src/restore.rs | 30 -- 1 file changed, 8 insertions(+), 22 deletions(-) diff --git a/src/restore.rs b/src/restore.rs index 0790d7f..33959d9 100644 --- a/src/restore.rs +++ b

[pve-devel] [PATCH proxmox-backup 4/9] backup: add AsyncRead/Seek to CachedChunkReader

2021-06-02 Thread Stefan Reiter
actual request concurrency is not supported in these traits). Signed-off-by: Stefan Reiter --- src/backup/cached_chunk_reader.rs | 116 +- 1 file changed, 114 insertions(+), 2 deletions(-) diff --git a/src/backup/cached_chunk_reader.rs b/src/backup

[pve-devel] [PATCH proxmox-backup 7/9] tools/lru_cache: make minimum capacity 1

2021-06-02 Thread Stefan Reiter
Setting this to 0 is not just useless, but breaks the logic horribly enough to cause random segfaults - better forbid this, to avoid someone else having to debug it again ;) Signed-off-by: Stefan Reiter --- src/tools/lru_cache.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/src/tools

[pve-devel] [PATCH proxmox-backup 6/9] backup: remove AsyncIndexReader

2021-06-02 Thread Stefan Reiter
superseded by CachedChunkReader, with less code and more speed Signed-off-by: Stefan Reiter --- src/backup.rs| 3 - src/backup/async_index_reader.rs | 215 --- 2 files changed, 218 deletions(-) delete mode 100644 src/backup

[pve-devel] [PATCH 0/9] Improve live-restore speed and replace AsyncIndexReader

2021-06-02 Thread Stefan Reiter
iteration 1.0.3-1A patchedB [0] see: https://lists.proxmox.com/pipermail/pbs-devel/2021-April/002932.html proxmox-backup: Stefan Reiter (7): tools/BroadcastFuture: add testcase for better understanding tools: add AsyncLruCache as a wrapper around sync LruCache backup: a

[pve-devel] [PATCH proxmox-backup 5/9] replace AsyncIndexReader with SeekableCachedChunkReader

2021-06-02 Thread Stefan Reiter
admin/datastore reads linearly only, so no need for cache (capacity of 1 basically means no cache except for the currently active chunk). mount can do random access too, so cache last 8 chunks for possibly a mild performance improvement. Signed-off-by: Stefan Reiter --- src/api2/admin

[pve-devel] [PATCH proxmox-backup-qemu 8/9] add shared_cache module

2021-06-02 Thread Stefan Reiter
Provides a shared AsyncLruCache of 256MB (w/ 4MB chunks) that can be used by multiple readers at the same time. It is dropped once no more readers exist, so the memory gets freed if all QEMU block/pbs instances disappear. Signed-off-by: Stefan Reiter --- src/lib.rs | 7 ++- src

[pve-devel] [PATCH proxmox-backup 2/9] tools: add AsyncLruCache as a wrapper around sync LruCache

2021-06-02 Thread Stefan Reiter
Supports concurrent 'access' calls to the same key via a BroadcastFuture. These are stored in a seperate HashMap, the LruCache underneath is only modified once a valid value has been retrieved. Signed-off-by: Stefan Reiter --- src/tools.rs | 1 + src/tools/async_lr

[pve-devel] [PATCH proxmox-backup 1/9] tools/BroadcastFuture: add testcase for better understanding

2021-06-02 Thread Stefan Reiter
Explicitly test that data will stay available and can be retrieved immediately via listen(), even if the future producing the data and notifying the consumers was already run in the past. Signed-off-by: Stefan Reiter --- Wasn't broken or anything, but helps with understanding IMO. src/

[pve-devel] [PATCH pve-qemu 3/3] update keycodemapdb for 6.0

2021-05-27 Thread Stefan Reiter
QEMU 6.0 requires the updated version to build correctly, as the keymap-gen tool gained some new parameters. Signed-off-by: Stefan Reiter --- keycodemapdb/data/keymaps.csv | 16 +- keycodemapdb/meson.build | 1 + keycodemapdb/tests/Makefile | 84 +++ keycodemapdb/tools/keymap

[pve-devel] [PATCH 0/3] Upgrade to QEMU 6.0

2021-05-27 Thread Stefan Reiter
, but oh well): https://lists.proxmox.com/pipermail/pve-devel/2021-May/048327.html pve-qemu: Stefan Reiter (3): udpate and rebase to QEMU v6.0.0 debian: ignore submodule checks in QEMU build update keycodemapdb for 6.0 debian/changelog | 6 + ...d-support-for

[pve-devel] [PATCH pve-qemu 2/3] debian: ignore submodule checks in QEMU build

2021-05-27 Thread Stefan Reiter
...we do those manually, and the build dir is not a git repo. Signed-off-by: Stefan Reiter --- debian/rules | 1 + 1 file changed, 1 insertion(+) diff --git a/debian/rules b/debian/rules index 349c610..1f623d8 100755 --- a/debian/rules +++ b/debian/rules @@ -41,6 +41,7 @@ ${BUILDDIR

[pve-devel] [PATCH qemu-server 1/2] cfg2cmd: use long form QEMU parameters to avoid warning in 6.0

2021-05-27 Thread Stefan Reiter
rm boolean option 'nowait' deprecated Please use wait=off instead kvm: -vnc unix:/var/run/qemu-server/100.vnc,password: warning: short-form boolean option 'password' deprecated Please use password=on instead The new syntax is backwards compatible to at least QEMU 4.0. Signed-o

  1   2   3   4   >