On 02/22/2017 03:45 AM, Peter Lieven wrote:
>
> Am 21.02.2017 um 22:13 schrieb John Snow:
>>
>> On 02/21/2017 07:43 AM, Peter Lieven wrote:
>>> Hi,
>>>
>>>
>>> is there anyone ever thought about implementing something like VMware
>>> CBT in Qemu?
>>>
>>>
>>> https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1020128
>>>
>>>
>>>
>>> Thanks,
>>> Peter
>>>
>>>
>> A bit outdated now, but:
>> http://wiki.qemu-project.org/Features/IncrementalBackup
>>
>> and also a summary I wrote not too far back (PDF):
>> https://drive.google.com/file/d/0B3CFr1TuHydWalVJaEdPaE5PbFE
>>
>> and I'm sure the Virtuozzo developers could chime in on this subject,
>> but basically we do have something similar in the works, as eblake says.
>
> Hi John, Hi Erik,
>
> thanks for your feedback. Are you both the ones working primary on this topic?
> If there is anything to review or help needed, please let me know.
>
I've been working on incremental backups; Fam and I now co-maintain
block/dirty-bitmap.c.
Vladimir Sementsov-Ogievskiy has been working on bitmap persistence and
migration from Virtuozzo; as well as the NBD specification amendment to
allow us to fleece images with dirty bitmaps.
(Check the wiki and the whitepaper I linked!)
Eric has been guiding the review process for the NBD side of things.
> My 2 cents:
> I thing I had in mind if there is no image fleecing available, but fetching
> the dirty bitmap
> from external would be a feauture to put a write lock on a block device.
> Write lock means, drain all pending writes and queue all further writes until
> unlock (as if they
> were throttled to zero). This could help fetch consistent backups from
> storage device (thinking of iSCSI SAN) without
> the help of the hypervisor to actually transfer data (no load in the frontend
> network or the host). What would further
> be needed is a write generation for each block, not just only a dirty bitmap.
>
> In this case something like this via QMP (and external software) should work:
> ---8<---
> gen = write generation of last backup (or 0 for full backup)
> do {
> nextgen = fetch current write generation (via QMP)
As Eric said, there's a lot of hostility to using QMP as a metadata
transmission protocol.
> dirtymap = send all block whose write generation is greater than 'gen'
> (via QMP)
> dirtycnt = 0
> foreach block in dirtymap {
> copy to backup via external software
> dirtycnt++
> }
> gen = nextgen
> } while (dirtycnt < X) <--- to achieve this a thorttling or similar
> might be needed
>
> fsfreeze (optional)
> write lock (via QMP)
> backupgen = fetch current write generation (via QMP)
> dirtymap = send all block whose write generation is greater than 'gen' (via
> QMP)
> foreach block in dirtymap {
> copy to backup via external software
> }
> unlock (via QMP)
> fsthaw (optional)
> --->8---
>
> As far as I understand CBT in VMware is not just only a dirty bitmap, but
> also a write generation tracking for blocks (size 64kb or whatever)
>
I think at the moment I'm worried about getting the basic features out
the door, but I'm not opposed to adding fancier features if there's
justification or demand for them.
> Peter
>
--js
