[pve-devel] applied: [PATCH] combo grid: load: rework auto-selection and validity logic

We do not want to trigger an autoSelect if there's a value set, even if it isn't found in the store, as that hides the fact that an (now) invalid valid is configured from the user, which can be confusing if something is not working, as when editing an object it seems like a valid value is selected.

[pve-devel] [PATCH v2 proxmox-backup 6/9] backup: remove AsyncIndexReader

superseded by CachedChunkReader, with less code and more speed Signed-off-by: Stefan Reiter --- src/backup.rs| 3 - src/backup/async_index_reader.rs | 215 --- 2 files changed, 218 deletions(-) delete mode 100644 src/backup/async_index_reader.rs

[pve-devel] [PATCH v2 proxmox-backup 5/9] replace AsyncIndexReader with SeekableCachedChunkReader

admin/datastore reads linearly only, so no need for cache (capacity of 1 basically means no cache except for the currently active chunk). mount can do random access too, so cache last 8 chunks for possibly a mild performance improvement. Signed-off-by: Stefan Reiter --- src/api2/admin/datastore.

[pve-devel] [PATCH v2 proxmox-backup 2/9] tools: add AsyncLruCache as a wrapper around sync LruCache

Supports concurrent 'access' calls to the same key via a BroadcastFuture. These are stored in a seperate HashMap, the LruCache underneath is only modified once a valid value has been retrieved. Signed-off-by: Stefan Reiter --- src/tools.rs | 1 + src/tools/async_lru_cache.rs |

[pve-devel] [PATCH v2 proxmox-backup 3/9] backup: add CachedChunkReader utilizing AsyncLruCache

Provides a fast arbitrary read implementation with full async and concurrency support. Signed-off-by: Stefan Reiter --- This is technically all that's needed for proxmox-backup-qemu to build and function as intended, but I decided to also use this IMHO cleaner implementation to replace the Async

[pve-devel] [PATCH v2 proxmox-backup-qemu 9/9] access: use CachedChunkReader

Use the new CachedChunkReader with the shared_cache implementation to provide a concurrency-safe async way of accessing data. This provides two benefits: * uses a shared LRU cache, which is very helpful for random-access like during a live-restore * does away with the global Mutex in read_image_

[pve-devel] [PATCH v2 proxmox-backup-qemu 8/9] add shared_cache module

Provides a shared AsyncLruCache of 256MB (w/ 4MB chunks) that can be used by multiple readers at the same time. It is dropped once no more readers exist, so the memory gets freed if all QEMU block/pbs instances disappear. Signed-off-by: Stefan Reiter --- v2: * adapt to dropping ChunkCache type a

[pve-devel] [PATCH v2 0/9] Improve live-restore speed and replace AsyncIndexReader

This series is the third/fourth attempt[0] at substantially improving live-restore performance. This time, a fully async- and concurrency safe LRU cache is implemented, and a new CachedChunkReader is used to provide lock-free reading from a remote chunk source. The big performance improvements come

[pve-devel] [PATCH v2 proxmox-backup 4/9] backup: add AsyncRead/Seek to CachedChunkReader

Implemented as a seperate struct SeekableCachedChunkReader that contains the original as an Arc, since the read_at future captures the CachedChunkReader, which would otherwise not work with the lifetimes required by AsyncRead. This is also the reason we cannot use a shared read buffer and have to a

[pve-devel] [PATCH v2 proxmox-backup 7/9] tools/lru_cache: make minimum capacity 1

Setting this to 0 is not just useless, but breaks the logic horribly enough to cause random segfaults - better forbid this, to avoid someone else having to debug it again ;) Signed-off-by: Stefan Reiter --- src/tools/lru_cache.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/src/tools/lru_

[pve-devel] [PATCH v2 proxmox-backup 1/9] tools/BroadcastFuture: add testcase for better understanding

Explicitly test that data will stay available and can be retrieved immediately via listen(), even if the future producing the data and notifying the consumers was already run in the past. Wasn't broken or anything, but helps with understanding IMO. Signed-off-by: Stefan Reiter --- Wasn't broken

[pve-devel] [PATCH qemu-server 2/2] enable io-uring support

Note that the value in this enum directly represents the value passed to QEMU, so we need to use the underscore. Off by default, updated QEMU required, but no compatibility issues, as this is a new value that wasn't valid before, and setting it is the users choice. Signed-off-by: Stefan Reiter -

[pve-devel] [PATCH 0/2] Enable io-uring support in QEMU

Ran my full test suite on it (QEMU 6.0, kernel 5.11.21) and it completed fine, including migrate, snapshot and backup. No benchmarks done yet, so any performance gains are only theoretical for now :) pve-qemu gains a new build-depends on 'liburing-dev' - this package is available in Debian bullsey

[pve-devel] [PATCH pve-qemu 1/2] enable io-uring support in QEMU builds

Signed-off-by: Stefan Reiter --- Oh if only all QEMU changes were this simple... debian/control | 1 + debian/rules | 1 + 2 files changed, 2 insertions(+) diff --git a/debian/control b/debian/control index 38c45ce..4da3409 100644 --- a/debian/control +++ b/debian/control @@ -27,6 +27,7 @@ B

[pve-devel] [PATCH manager] fix #3426: ui: parser: adapt parseSSHKey to accept "@..." suffix

also rearranged the regex groups to allow sk-ecdsa-sha2-... note: FIDO keys are now parsable, the backend only supports it with OpenSSH >= 8.2. This may be achieved with upgrading to PVE 7 or using a back port. --- www/manager6/Parser.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

Re: [pve-devel] [PATCH qemu-server 2/6] drive: factor out read-only helper

On 6/7/21 12:23 PM, Fabian Grünbichler wrote: On June 7, 2021 11:29 am, Stefan Reiter wrote: On 6/4/21 11:47 AM, Fabian Grünbichler wrote: we also need it for efidisks. Signed-off-by: Fabian Grünbichler --- PVE/QemuServer.pm | 8 ++-- PVE/QemuServer/Drive.pm | 10 ++

Re: [pve-devel] [PATCH manager] ui: ceph/Pools: defuse min_size warning

argh a typo slipped through... On 6/7/21 12:31, Dominik Csapak wrote: we should not warn for the ceph built-in default warnings for min_size the word 'warnings' should not be there... having the min_size half of the size (rounded up) is ok and even the default for ceph since there seems to

[pve-devel] [PATCH manager] ui: ceph/Pools: defuse min_size warning

we should not warn for the ceph built-in default warnings for min_size having the min_size half of the size (rounded up) is ok and even the default for ceph since there seems to be no 'quorum based' pg inconsistency recovery[0], only a copy from the authoritative osd, there is nothing wrong with s

Re: [pve-devel] [PATCH qemu-server 3/6] template: mark efidisk as read-only

On June 7, 2021 11:29 am, Stefan Reiter wrote: > High-level: If you make the $read_only_str contain the entire > ",readonly=on" stanza, you can just leave it blank by default and avoid > updating all the test cases. yeah, but once you add a few of those the resulting code gets rather messy/ugly

Re: [pve-devel] [RFC qemu-server 6/6] template: start VM for VMA backup

On June 7, 2021 11:29 am, Stefan Reiter wrote: > On 6/4/21 11:47 AM, Fabian Grünbichler wrote: >> since using 'vma create ..' no longer works with immutable/read-only >> base volumes. > > Why that? It shouldn't matter to vma if the base is read-only? Though I > do see that in vma.c we don't tell

Re: [pve-devel] [PATCH qemu-server 2/6] drive: factor out read-only helper

On June 7, 2021 11:29 am, Stefan Reiter wrote: > On 6/4/21 11:47 AM, Fabian Grünbichler wrote: >> we also need it for efidisks. >> >> Signed-off-by: Fabian Grünbichler >> --- >> PVE/QemuServer.pm | 8 ++-- >> PVE/QemuServer/Drive.pm | 10 ++ >> 2 files changed, 12 insertion

Re: [pve-devel] ceph create pool with min_size=1 not possible anymore with last gui wizard

Hi, On Mon, Jun 07, 2021 at 09:14:13AM +0200, Dominik Csapak wrote: > On 6/7/21 08:57, aderum...@odiso.com wrote: > > Le vendredi 04 juin 2021 à 15:23 +0200, Dominik Csapak a écrit : > > > On 6/4/21 04:47, aderum...@odiso.com  wrote: > > > > I was doing a training week

Re: [pve-devel] [PATCH qemu-server 3/6] template: mark efidisk as read-only

High-level: If you make the $read_only_str contain the entire ",readonly=on" stanza, you can just leave it blank by default and avoid updating all the test cases. On 6/4/21 11:47 AM, Fabian Grünbichler wrote: otherwise backups of templates using UEFI fail with storages like LVM thin, where the

Re: [pve-devel] [RFC qemu-server 6/6] template: start VM for VMA backup

On 6/4/21 11:47 AM, Fabian Grünbichler wrote: since using 'vma create ..' no longer works with immutable/read-only base volumes. Why that? It shouldn't matter to vma if the base is read-only? Though I do see that in vma.c we don't tell QEMU that it should open the block backend read-only...

Re: [pve-devel] [PATCH qemu-server 2/6] drive: factor out read-only helper

On 6/4/21 11:47 AM, Fabian Grünbichler wrote: we also need it for efidisks. Signed-off-by: Fabian Grünbichler --- PVE/QemuServer.pm | 8 ++-- PVE/QemuServer/Drive.pm | 10 ++ 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServe

Re: [pve-devel] [PATCH qemu-server 1/3] Revert "revert spice_ticket prefix change in 7827de4"

For the three qemu-server changes: Reviewed-by: Stefan Reiter I'll also see if I can find some more stuff to cleanup in the code I often traverse :) On 6/4/21 3:49 PM, Fabian Ebner wrote: This reverts commit ff09c795edd12b1cc4604ee28a7cdd99cdd1afa8. We wanted to wait until PVE 7.0 for the c

Re: [pve-devel] training week: students feedback/requests

Le vendredi 04 juin 2021 à 09:52 +0200, Thomas Lamprecht a écrit : > Hi, > > On 04.06.21 05:21, aderum...@odiso.com wrote: > > Hi, > > I just finish the last training week session, > > here some students feedback/requesst: > > thanks for your feedback! > > - add support for vm offlline vm migrati

Re: [pve-devel] [pbs-devel] [PATCH proxmox-backup-qemu 8/9] add shared_cache module

> On 06/07/2021 10:03 AM Stefan Reiter wrote: > > > On 6/4/21 2:16 PM, Wolfgang Bumiller wrote: > > On Wed, Jun 02, 2021 at 04:38:32PM +0200, Stefan Reiter wrote: > >> Provides a shared AsyncLruCache of 256MB (w/ 4MB chunks) that can be > >> used by multiple readers at the same time. It is dr

Re: [pve-devel] [pbs-devel] [PATCH proxmox-backup-qemu 8/9] add shared_cache module

On 6/4/21 2:16 PM, Wolfgang Bumiller wrote: On Wed, Jun 02, 2021 at 04:38:32PM +0200, Stefan Reiter wrote: Provides a shared AsyncLruCache of 256MB (w/ 4MB chunks) that can be used by multiple readers at the same time. It is dropped once no more readers exist, so the memory gets freed if all QEM

Re: [pve-devel] ceph create pool with min_size=1 not possible anymore with last gui wizard

On 6/7/21 08:57, aderum...@odiso.com wrote: Le vendredi 04 juin 2021 à 15:23 +0200, Dominik Csapak a écrit : On 6/4/21 04:47, aderum...@odiso.com  wrote: Hi, Hi, I was doing a training week with students, and I see that the new ceph wizard to create pool don't