--- Begin Message ---
I 100% concur. I am not suggesting any breaking changes; I was just wondering
if this work on the API unlocked any new optimizations to make the interactions
between the backup client, PBS, and storage more efficient. And also, bbgeek
has pinged me to check out the awesome work going on in this space :)
Between your and Dietmar's replies, I see the constraints and potential avenues
for improvement. Thanks for your reply!
Respectfully,
-Jonathan
> On Jul 29, 2024, at 4:15 AM, Fiona Ebner <f.eb...@proxmox.com> wrote:
>
> Hi,
>
> Am 26.07.24 um 21:47 schrieb Jonathan Nicklin via pve-devel:
>>
>> Hi Fiona,
>>
>> Would adding support for offloading incremental difference detection
>> to the underlying storage be feasible with the API updates? The QEMU
>> bitmap strategy works for all storage devices but is far from
>> optimal. If backup coordinated a storage snapshot, the underlying
>> storage could enumerate the differences (or generate a bitmap).
>>
>> This would allow PBS to connect directly to storage and retrieve
>> incremental differences, which could remove the PVE hosts from the
>> equation. This "storage-direct" approach for backup would improve
>> performance, reduce resources, and support incremental backups in all
>> cases (i.e., power failues, shutdowns, etc.). It would also eliminate
>> the dependency on QEMU bitmaps and the overhead of fleecing.
>>
>> Theoretically, this should be possible with any shared storage that
>> can enumerate incremental differences between snapshots: Ceph,
>> Blockbridge, iSCSi/ZFS?
>>
>> Thoughts?
>>
>
> The two big advantages of the current mechanism are:
>
> 1. it's completely storage-agnostic, so you can even use it with raw
> files on a directory storage. It follows in the same spirit as existing
> backup. Prohibiting backup for users when they use certain kinds of
> storages for VMs is not nice.
> 2. it's been battle-tested with PBS and works nicely.
>
> I don't see why your suggestion can't be implemented in principle.
> Feature requests for (non-incremental) "storage-snapshot" mode backup
> have been around since a while. It was not a priority for development
> yet and is totally different from the current "snapshot" backup mode, so
> will need to be developed from the ground up.
>
> That said, AFAICS, it's orthogonal to the series here. When an
> implementation like you outlined exists, it can just be added as a new
> backup mechanism for external providers (and PBS).
>
> See also the related discussion over at:
> https://bugzilla.proxmox.com/show_bug.cgi?id=3233#c19
>
> Best Regards,
> Fiona
>
--- End Message ---
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
