Am 03.04.25 um 09:24 schrieb Wolfgang Bumiller:
On Wed, Apr 02, 2025 at 06:16:57PM +0200, Andreas Rogge wrote:
Am 02.04.25 um 10:30 schrieb Wolfgang Bumiller:
On Tue, Apr 01, 2025 at 08:21:30PM +0200, Thomas Lamprecht wrote:
But I do wonder if - to reduce space-requirements for backing up running
VMs - at some point we might also add the ability for qemu to provide
some kind of queue containing the offset-length pairs of blocks which
have been stored in the temporary fleecing image. The provider could
consume this queue to keep the temporary storage at a minimum by doing
out-of-order backups.
As we have to be able to restore out-of-order anyway, backing up
out-of-order is not a problem at all.
However, offset-length in 512-byte offsets might be a bit too
fine-grained. If we, for example, have a deduplication blocksize of 256k
on a storage, offset and offset+length would preferably be divisible by
256k.
While the backup application could easily compute aligned offsets
itself, read-access to the whole computed region would be required.
This only makes sense if there are backup systems which can benefit from
it. (And it should be a simple-enough api extension to add this in the
future, from what I can tell)
I am not sure if this provides a benefit on our end. However, if it
makes the backup process more lightweight on the PVE side, it might
still be worth the effort.
I guess it makes sense for us to not expect/require random access, as
any feature like that already imposes limitations on how the data can be
stored. I'd expect different backup solutions to have different
limitations in that regard.
Exactly.
I *believe* `qemu-nbd` should be able to bind all the storage types we
want to restore to to /dev/nbdXY devices, which would give the provider
a bunch of block devices to write to in whichever way they want to, so
the provider would then only need to figure out how to receive the data
and forward that to the devices.
We'll need to try.
That's pretty much what I had hoped for.
Which is why the most flexible thing to do is to use a `qemu-img` call
and giving it the paths, or more precisely, the URLs to the disks.
I understand how this makes sense. However, if you don't have the data in a
format that qemu-img can consume, things become complicated.
It can also just read data from a stream (but without any smarter
protocol, this would make holes/unallocated blocks extremely
inefficient), where the provider would create that stream in whichever
way they want to.
As the only viable formats to stream into qemu-img are qcow2 and raw and
(to my understanding) these require the data to be in the right order
Bareos would have to re-order the data before streaming it into
qemu-img, which brings us back to staging the image - in which case we
don't need to stream anymore :)
--
Andreas Rogge andreas.ro...@bareos.com
Bareos GmbH & Co. KG Phone: +49 221-630693-86
http://www.bareos.com
Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Komplementär: Bareos Verwaltungs-GmbH
Geschäftsführer: Stephan Dühr, Jörg Steffens, Philipp Storz
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
