On Mon, Jan 20, 2014 at 10:55:58AM +0800, Fam Zheng wrote:
> On Fri, 01/17 17:25, Stefan Hajnoczi wrote:
> > On Mon, Jan 13, 2014 at 06:39:39PM +0800, Fam Zheng wrote:
> > > This implements incremental backup.
> > >
> > > A few new QMP commands related to dirty bitmap are added:
> > >
> > > dirty-bitmap-add *
> > > dirty-bitmap-disable *
> > > dirty-bitmap-remove
> > >
> > > (*: also supported as transactions)
> > >
> > > As their name implies, they manipulate a block device's dirty bitmap. This
> > > doesn't interfere with dirty bitmap used for migration, backup, mirror,
> > > etc,
> > > which don't have a name and are invisible to user. Only named bitmaps
> > > (created
> > > by dirty-bitmap-add) can be disabled/removed by user.
> > >
> > > They are added to support "user controlled write tracking", so as to
> > > determine
> > > the range of date for incremental backup.
> > >
> > > A new sync mode for drive-backup is introduced:
> > >
> > > drive-backup device=.. mode=.. sync=dirty-bitmap bitmap=bitmap0
> > >
> > > Which will scan dirty bitmap "bitmap0" and only copy all dirty sectors to
> > > target.
> > >
> > > Now, let's see the usage with a simple example:
> > >
> > > # Start the guest
> > > vm = VM()
> > > vm.start()
> > >
> > > # Fake some guest writes with "qemu-io", this is before creating dirty
> > > # bitmap, so it won't be copied
> > > vm.hmp('qemu-io ide0-hd0 "write -P 0xa 512k 1M"')
> > >
> > > # Create a dirty bitmap to track writes
> > > vm.qmp("dirty-bitmap-add", device="ide0-hd0", name="dirty-0")
> > >
> > > # Fake some more guest writes with "qemu-io", this will be copied
> > > vm.hmp('qemu-io ide0-hd0 "write -P 0xa 512M 1M"')
> > >
> > > # Now "disable" the first dirty bitmap, do the backup according to it,
> > > # at meantime continue to track dirty with a new dirty bitmap
> > > vm.qmp("transaction", actions=[
> > > {
> > > 'type': 'dirty-bitmap-disable', 'data': {
> > > 'device': 'ide0-hd0',
> > > 'name': 'dirty-0'
> > > }
> > > }, {
> > > 'type': 'dirty-bitmap-add', 'data': {
> > > 'device': 'ide0-hd0',
> > > 'name': 'dirty-1'
> > > }
> > > }, {
> > > 'type': 'drive-backup', 'data': {
> > > 'device': 'ide0-hd0',
> > > 'target': '/tmp/incre.qcow2',
> > > 'bitmap': 'dirty-0',
> > > 'sync': 'dirty-bitmap'
> > > }
> > > }
> > > ])
> > >
> > > # Once backup job started, the dirty bitmap can be removed (actually
> > > only
> > > # hidden from user since it is still referenced by block job
> > > vm.qmp("dirty-bitmap-remove", device="ide0-hd0", name="dirty-0")
> >
> > I'm interested in the lifecycle of a dirty bitmap (but haven't reviewed
> > the patches yet). In particular, what happens if a bitmap is added to
> > more than one drive? Is there a more elegant way to handle the disable,
> > drive-backup, remove step (we only need to explicitly disable because we
> > still need the bitmap name for the drive-backup command)? Also what
> > happens if we add the bitmap again after disabling?
>
> A same name on that device can't be used again unless it's removed. A bitmap
> is
> associated to (and only) one device, it can't be shared.
>
> >
> > No need to answer all these questions, but it suggests the interface
> > exposes a bit of complexity. Maybe it's possible to make it simpler and
> > easier to use?
> >
>
> At least the user has to explicitly start tracking, that's the
> dirty-bitmap-add
> step. Alternatively, we can have "disable, drive-backup, remove" step
> simplified as:
>
> drive-backup sync=dirty-bitmap bitmap=dirty0 reset-bitmap=true
>
> where backup job copy out the dirty bitmap (and clears it, as reset-bitmap is
> true), and backup with it atomically.
>
> Of course it doesn't have to actually copy the whole bitmap: it just makes the
> old one anonymous, create a new empty one and give it the same name. When
> backup is done, the old bitmap is removed.
>
> What do you think?
That simplifies usage. I can think of two options:
1. We want to continue incremental backup so drive-backup should
atomically swap the bitmap with a new one of the same name (as you
described).
2. We want to end incremental backup so the drive-backup command should
consume the bitmap and not create a new one.
Then users just need to add a new bitmap and later use drive-backup.
They don't need to manually manage the bitmap's lifecycle.
Stefan
