Re: [Qemu-devel] [PATCH RFC v2 4/8] block: add dirty-dirty bitmaps

2015-02-12 Thread Vladimir Sementsov-Ogievskiy
On 11.02.2015 00:30, John Snow wrote: I had hoped it wouldn't come to this :) On 01/27/2015 05:56 AM, Vladimir Sementsov-Ogievskiy wrote: A dirty-dirty bitmap is a dirty bitmap for BdrvDirtyBitmap. It tracks set/unset changes of BdrvDirtyBitmap. Signed-off-by: Vladimir Sementsov-Ogie

Re: [Qemu-devel] [PATCH RFC v2 5/8] block: add bdrv_dirty_bitmap_enabled()

2015-02-12 Thread Vladimir Sementsov-Ogievskiy
On 11.02.2015 00:30, John Snow wrote: On 01/27/2015 05:56 AM, Vladimir Sementsov-Ogievskiy wrote: Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 5 + include/block/block.h | 1 + 2 files changed, 6 insertions(+) diff --git a/block.c b/block.c index 8ab724b

Re: [Qemu-devel] [PATCH RFC v2 8/8] migration: add migration/dirty-bitmap.c

2015-02-13 Thread Vladimir Sementsov-Ogievskiy
tainers. On 01/27/2015 05:56 AM, Vladimir Sementsov-Ogievskiy wrote: Live migration of dirty bitmaps. Only named dirty bitmaps are migrated. If destination qemu is already containing a dirty bitmap with the same name as a migrated bitmap, then their granularities should be the same, otherwise th

Re: [Qemu-devel] [PATCH RFC v2 8/8] migration: add migration/dirty-bitmap.c

2015-02-13 Thread Vladimir Sementsov-Ogievskiy
+ +blk_mig_reset_dirty_cursor(); +dirty_phase(f, false); + +QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { +uint8_t flags = DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME | +DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME | +DIRTY_BITM

Re: [Qemu-devel] [PATCH v12 07/17] qmp: Add support of "dirty-bitmap" sync mode for drive-backup

2015-02-13 Thread Vladimir Sementsov-Ogievskiy
On 10.02.2015 04:35, John Snow wrote: .. @@ -278,28 +305,61 @@ static void coroutine_fn backup_run(void *opaque) qemu_coroutine_yield(); job->common.busy = true; } +} else if (job->sync_mode == MIRROR_SYNC_MODE_DIRTY_BITMAP) { +/* Dirty Bi

Re: [Qemu-devel] [PATCH RFC v2 8/8] migration: add migration/dirty-bitmap.c

2015-02-13 Thread Vladimir Sementsov-Ogievskiy
On 13.02.2015 20:32, John Snow wrote: On 02/13/2015 04:06 AM, Vladimir Sementsov-Ogievskiy wrote: + +blk_mig_reset_dirty_cursor(); +dirty_phase(f, false); + +QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) { +uint8_t f

Re: [Qemu-devel] [PATCH RFC v2 8/8] migration: add migration/dirty-bitmap.c

2015-02-16 Thread Vladimir Sementsov-Ogievskiy
On 13.02.2015 23:22, John Snow wrote: On 02/13/2015 03:19 AM, Vladimir Sementsov-Ogievskiy wrote: On 11.02.2015 00:33, John Snow wrote: Peter Maydell: What's the right way to license a file as copied from a previous version? See below, please; Max, Markus: ctrl+f "bdrv_get_device

Re: [Qemu-devel] [PATCH RFC v2 8/8] migration: add migration/dirty-bitmap.c

2015-02-17 Thread Vladimir Sementsov-Ogievskiy
On 16.02.2015 21:18, John Snow wrote: On 02/16/2015 07:06 AM, Vladimir Sementsov-Ogievskiy wrote: On 13.02.2015 23:22, John Snow wrote: On 02/13/2015 03:19 AM, Vladimir Sementsov-Ogievskiy wrote: On 11.02.2015 00:33, John Snow wrote: So in summary: using device names is probably fine

[Qemu-devel] RFC RFC

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
Hi Can anybody explain what the reasons, politics and consequences of adding RFC into patch header in this mailing list? I think, it is not worth to add this information to http://wiki.qemu.org/Contribute/SubmitAPatch -- Best regards, Vladimir

[Qemu-devel] [PATCH RFC v3 01/14] qmp: add query-block-dirty-bitmap

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
Adds qmp and hmp commands to query/info dirty bitmap. This is needed only for testing. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 41 blockdev.c| 24 + hmp-commands.hx | 2 ++ hmp.c | 32

[Qemu-devel] [PATCH RFC v3 08/14] migration: add migration/block-dirty-bitmap.c

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
migration will be done, otherwise the error will be generated. If destination qemu doesn't contain such bitmap it will be created. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/migration/block.h | 1 + migration/Makefile.objs| 2 +- migration/block-dirty-bitmap.c

[Qemu-devel] [PATCH RFC v3 02/14] hbitmap: serialization

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
in several steps is available. To save performance, every step writes only the last level of the bitmap. All other levels are restored by hbitmap_deserialize_finish() as a last step of restoring. So, HBitmap is inconsistent while restoring. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include

[Qemu-devel] [PATCH RFC v3 13/14] iotests: add dirty bitmap migration test

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/118 | 87 ++ tests/qemu-iotests/118.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files changed, 93 insertions(+) create mode 100755 tests/qemu-iotests/118 create mode

[Qemu-devel] [PATCH RFC v3 09/14] iotests: maintain several vms in test

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
The only problem with it is the same qmp socket name (which is vm._monitor_path) for all vms. And because of this second vm couldn't be lauched (vm.launch() fails because of socket is already in use). This patch adds a number of vm into vm._monitor_path Signed-off-by: Vladimir Sementsov-Ogie

[Qemu-devel] [PATCH RFC v3 03/14] block: BdrvDirtyBitmap serialization interface

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
Several functions to provide necessary access to BdrvDirtyBitmap for block-migration.c Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 36 include/block/block.h | 13 + 2 files changed, 49 insertions(+) diff --git a

[Qemu-devel] [PATCH RFC v3 06/14] block: add bdrv_next_dirty_bitmap()

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
Like bdrv_next() - bdrv_next_dirty_bitmap() is a function to provide access to private dirty bitmaps list. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 10 ++ include/block/block.h | 2 ++ 2 files changed, 12 insertions(+) diff --git a/block.c b/block.c

[Qemu-devel] [PATCH RFC v3 00/14] Dirty bitmaps migration

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
locatoions, no bdrv_read's, no bdrv_write's, only bitmaps are migrated. The patch set includes two my previous bug fixes, which are necessary for it. The patch set is based on Incremental backup series by John Snow. Vladimir Sementsov-Ogievskiy (14): qmp: add query-block-dirty-bitmap

Re: [Qemu-devel] RFC RFC

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
On 18.02.2015 17:01, Eric Blake wrote: On 02/18/2015 01:10 AM, Vladimir Sementsov-Ogievskiy wrote: Hi Can anybody explain what the reasons, politics and consequences of adding RFC into patch header in this mailing list? I think, it is not worth to add this information to http://wiki.qemu.org

[Qemu-devel] [PATCH RFC v3 04/14] block: tiny refactoring: minimize hbitmap_(set/reset) usage

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/block.c b/block.c index e95a5ae..a127fd2 100644 --- a/block.c +++ b/block.c @@ -5670,8 +5670,7 @@ void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap, void

[Qemu-devel] [PATCH RFC v3 07/14] qapi: add dirty-bitmaps migration capability

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/migration/migration.h | 1 + migration/migration.c | 9 + qapi-schema.json | 5 - 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/include/migration/migration.h b/include/migration/migration.h

[Qemu-devel] [PATCH RFC v3 12/14] qapi: add md5 checksum of last dirty bitmap level to query-block

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c| 1 + include/qemu/hbitmap.h | 8 qapi/block-core.json | 4 +++- util/hbitmap.c | 8 4 files changed, 20 insertions(+), 1 deletion(-) diff --git a/block.c b/block.c index 4cca55d..9532ccc 100644

[Qemu-devel] [PATCH RFC v3 10/14] iotests: add add_incoming_migration to VM class

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 6 ++ 1 file changed, 6 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index fa756b4..75640b2 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests

[Qemu-devel] [PATCH RFC v3 14/14] migration/qemu-file: make functions qemu_(get/put)_string public

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/migration/qemu-file.h | 17 + migration/block-dirty-bitmap.c | 35 --- migration/qemu-file.c | 18 ++ 3 files changed, 35 insertions(+), 35 deletions(-) diff --git a

[Qemu-devel] [PATCH RFC v3 05/14] block: add meta bitmaps

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It tracks changes (set/unset) of this BdrvDirtyBitmap. It is needed for live migration of block dirty bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 40 +++

[Qemu-devel] [PATCH RFC v3 11/14] iotests: add dirty bitmap migration test

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
The test starts two vms (vm_a, vm_b), create dirty bitmap in the first one, do several writes to corresponding device and then migrate vm_a to vm_b with dirty bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/117 | 88

Re: [Qemu-devel] RFC RFC

2015-02-18 Thread Vladimir Sementsov-Ogievskiy
On 18.02.2015 17:01, Eric Blake wrote: On 02/18/2015 01:10 AM, Vladimir Sementsov-Ogievskiy wrote: Hi Can anybody explain what the reasons, politics and consequences of adding RFC into patch header in this mailing list? I think, it is not worth to add this information to http://wiki.qemu.org

Re: [Qemu-devel] [PATCH RFC v3 05/14] block: add meta bitmaps

2015-02-19 Thread Vladimir Sementsov-Ogievskiy
On 19.02.2015 02:45, John Snow wrote: On 02/18/2015 09:00 AM, Vladimir Sementsov-Ogievskiy wrote: Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It tracks changes (set/unset) of this BdrvDirtyBitmap. It is needed for live migration of block dirty bitmaps. Signed-off-by

Re: [Qemu-devel] [PATCH RFC v3 08/14] migration: add migration/block-dirty-bitmap.c

2015-02-19 Thread Vladimir Sementsov-Ogievskiy
On 19.02.2015 02:47, John Snow wrote: On 02/18/2015 09:00 AM, Vladimir Sementsov-Ogievskiy wrote: Live migration of dirty bitmaps. Only named dirty bitmaps, associated with root nodes and non-root named nodes are migrated. If destination qemu is already containing a dirty bitmap with the

[Qemu-devel] [PATCH RFC v4 09/13] iotests: add add_incoming_migration to VM class

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 6 ++ 1 file changed, 6 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index fa756b4..75640b2 100644 --- a/tests/qemu-iotests/iotests.py +++ b

[Qemu-devel] [PATCH RFC v4 03/13] block: tiny refactoring: minimize hbitmap_(set/reset) usage

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/block.c b/block.c index e95a5ae..a127fd2 100644 --- a/block.c +++ b/block.c @@ -5670,8 +5670,7 @@ void bdrv_reset_dirty_bitmap

[Qemu-devel] [PATCH RFC v4 08/13] iotests: maintain several vms in test

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
ff-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index cf5faac..fa756b4 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotes

[Qemu-devel] [PATCH RFC v4 05/13] block: add bdrv_next_dirty_bitmap()

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
Like bdrv_next() - bdrv_next_dirty_bitmap() is a function to provide access to private dirty bitmaps list. Reviewed-by: John Snow Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 10 ++ include/block/block.h | 2 ++ 2 files changed, 12 insertions(+) diff --git

[Qemu-devel] [PATCH RFC v4 11/13] qapi: add md5 checksum of last dirty bitmap level to query-block

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c| 1 + include/qemu/hbitmap.h | 8 qapi/block-core.json | 4 +++- util/hbitmap.c | 8 4 files changed, 20 insertions(+), 1 deletion(-) diff --git a/block.c b/block.c index

[Qemu-devel] [PATCH RFC v4 04/13] block: add meta bitmaps

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It tracks changes (set/unset) of this BdrvDirtyBitmap. It is needed for live migration of block dirty bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c

[Qemu-devel] [PATCH RFC v4 06/13] qapi: add dirty-bitmaps migration capability

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/migration/migration.h | 1 + migration/migration.c | 9 + qapi-schema.json | 5 - 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/include/migration/migration.h b

[Qemu-devel] [PATCH RFC v4 10/13] iotests: add event_wait to VM class

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: John Snow Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 7 +++ 1 file changed, 7 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 75640b2..294b158 100644 --- a/tests/qemu-iotests/iotests.py +++ b

[Qemu-devel] [PATCH RFC v4 13/13] migration/qemu-file: make functions qemu_(get/put)_string public

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/migration/qemu-file.h | 17 + migration/block-dirty-bitmap.c | 35 --- migration/qemu-file.c | 18 ++ 3 files changed, 35 insertions(+), 35

[Qemu-devel] [PATCH RFC v4 02/13] block: BdrvDirtyBitmap serialization interface

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
Several functions to provide necessary access to BdrvDirtyBitmap for block-migration.c Reviewed-by: John Snow Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 36 include/block/block.h | 13 + 2 files changed, 49

[Qemu-devel] [PATCH RFC v4 01/13] hbitmap: serialization

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
in several steps is available. To save performance, every step writes only the last level of the bitmap. All other levels are restored by hbitmap_deserialize_finish() as a last step of restoring. So, HBitmap is inconsistent while restoring. Reviewed-by: John Snow Signed-off-by: Vladimir Sementsov

[Qemu-devel] [PATCH RFC v4 00/13] Dirty bitmaps migration

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
false when "dirty" is true block migration is actually skipped: no allocatoions, no bdrv_read's, no bdrv_write's, only bitmaps are migrated. The patch set includes two my previous bug fixes, which are necessary for it. The patch set is based on Incremental backup series by John

[Qemu-devel] [PATCH RFC v4 12/13] iotests: add dirty bitmap migration test

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
The test starts two vms (vm_a, vm_b), create dirty bitmap in the first one, do several writes to corresponding device and then migrate vm_a to vm_b with dirty bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/117 | 84

[Qemu-devel] [PATCH RFC v4 07/13] migration: add migration/block-dirty-bitmap.c

2015-02-27 Thread Vladimir Sementsov-Ogievskiy
migration will be done, otherwise the error will be generated. If destination qemu doesn't contain such bitmap it will be created. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/migration/block.h | 1 + migration/Makefile.objs| 2 +- migration/block-dirty-bitmap.c

[Qemu-devel] [PATCH RFC v2 3/8] block: BdrvDirtyBitmap serialization interface

2015-01-27 Thread Vladimir Sementsov-Ogievskiy
Several functions to provide necessary access to BdrvDirtyBitmap for block-migration.c Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 36 include/block/block.h | 12 2 files changed, 48 insertions(+) diff --git a

[Qemu-devel] [PATCH RFC v2 4/8] block: add dirty-dirty bitmaps

2015-01-27 Thread Vladimir Sementsov-Ogievskiy
A dirty-dirty bitmap is a dirty bitmap for BdrvDirtyBitmap. It tracks set/unset changes of BdrvDirtyBitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 44 include/block/block.h | 5 + 2 files changed, 49 insertions

[Qemu-devel] [PATCH RFC v2 6/8] block: add bdrv_next_dirty_bitmap()

2015-01-27 Thread Vladimir Sementsov-Ogievskiy
Like bdrv_next() - bdrv_next_dirty_bitmap() is a function to provide access to private dirty bitmaps list. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 10 ++ include/block/block.h | 2 ++ 2 files changed, 12 insertions(+) diff --git a/block.c b/block.c

[Qemu-devel] [PATCH RFC v2 7/8] migration: add dirty parameter

2015-01-27 Thread Vladimir Sementsov-Ogievskiy
Add dirty parameter to qmp-migrate command. If this parameter is true, migration/block.c will migrate dirty bitmaps. This parameter can be used without "blk" parameter to migrate only dirty bitmaps, skipping block migration. Signed-off-by: Vladimir Sementsov-Ogievskiy --- hmp-c

[Qemu-devel] [PATCH RFC v2 2/8] hbitmap: serialization

2015-01-27 Thread Vladimir Sementsov-Ogievskiy
in several steps is available. To save performance, every step writes only the last level of the bitmap. All other levels are restored by hbitmap_deserialize_finish() as a last step of restoring. So, HBitmap is inconsistent while restoring. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include

[Qemu-devel] [PATCH RFC v2 5/8] block: add bdrv_dirty_bitmap_enabled()

2015-01-27 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 5 + include/block/block.h | 1 + 2 files changed, 6 insertions(+) diff --git a/block.c b/block.c index 8ab724b..15fc621 100644 --- a/block.c +++ b/block.c @@ -5551,6 +5551,11 @@ uint64_t bdrv_dirty_bitmap_granularity

[Qemu-devel] [PATCH RFC v2 1/8] qmp: print dirty bitmap

2015-01-27 Thread Vladimir Sementsov-Ogievskiy
Adds qmp and hmp commands to print dirty bitmap. This is needed only for testing. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 33 + blockdev.c| 13 + hmp-commands.hx | 15 +++ hmp.c

[Qemu-devel] [PATCH RFC v2 8/8] migration: add migration/dirty-bitmap.c

2015-01-27 Thread Vladimir Sementsov-Ogievskiy
bit to corresponding value. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/migration/block.h | 1 + migration/Makefile.objs | 2 +- migration/dirty-bitmap.c | 606 ++ vl.c | 1 + 4 files changed, 609 insertions(+),

[Qemu-devel] [PATCH RFC v2 0/8] Dirty bitmaps migration

2015-01-27 Thread Vladimir Sementsov-Ogievskiy
lk" and "inc" parameters are false when "dirty" is true block migration is actually skipped: no allocatoions, no bdrv_read's, no bdrv_write's, only bitmaps are migrated. The patch set includes two my previous bug fixes, which are necessary for it. The patch set

Re: [Qemu-devel] [PATCH 0/8] block: persistent dirty bitmaps (RFC)

2015-01-27 Thread Vladimir Sementsov-Ogievskiy
ping Best regards, Vladimir On 13.01.2015 20:02, Vladimir Sementsov-Ogievskiy wrote: The bitmaps are saved into qcow2 file format. It provides both 'internal' and 'external' dirty bitmaps feature: - for qcow2 drives we can store bitmaps in the same file - for other

Re: [Qemu-devel] [PATCH RFC v2 1/8] qmp: print dirty bitmap

2015-01-27 Thread Vladimir Sementsov-Ogievskiy
Ok, I agree. It was just a simple way to test the other staff. I'll rewrite it in the following versions of my two series. Best regards, Vladimir On 27.01.2015 19:17, Eric Blake wrote: On 01/27/2015 03:56 AM, Vladimir Sementsov-Ogievskiy wrote: Adds qmp and hmp commands to print dirty b

Re: [Qemu-devel] [PATCH] block-migration: fix pending() return value

2015-01-29 Thread Vladimir Sementsov-Ogievskiy
v2 is already pushed http://git.qemu.org/?p=qemu.git;a=commit;h=04636dc410b163c2243e66c3813dd4900a50a4ed Best regards, Vladimir On 29.01.2015 10:58, Markus Armbruster wrote: Vladimir Sementsov-Ogievskiy writes: Hi there.. Can someone review my bug fix? I'll surely fix typos in descri

Re: [Qemu-devel] [PATCH v11 03/13] qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove

2015-01-29 Thread Vladimir Sementsov-Ogievskiy
s/{'command'/{ 'command'/g - to be consistent with other staff in qapi/block-core.json and the same fix for block-dirty-bitmap-enable and block-dirty-bitmap-disable Best regards, Vladimir On 12.01.2015 19:30, John Snow wrote: From: Fam Zheng The new command pair is added to manage user cr

Re: [Qemu-devel] [PATCH 7/8] qmp: print dirty bitmap

2015-01-30 Thread Vladimir Sementsov-Ogievskiy
-Ogievskiy wrote: Adds qmp and hmp commands to print dirty bitmap. This is needed only for testing persistent dirty bitmap feature. Signed-off-by: Vladimir Sementsov-Ogievskiy --- +++ b/block.c @@ -5445,6 +5445,39 @@ int bdrv_store_dirty_bitmap(BdrvDirtyBitmap *bitmap) return res; } +void

Re: [Qemu-devel] [PATCH v11 00/13] block: Incremental backup series

2015-01-30 Thread Vladimir Sementsov-Ogievskiy
Personally, I prefer the second one. Best regards, Vladimir On 12.01.2015 19:30, John Snow wrote: Welcome to version 11. I hope you are enjoying our regular newsletter. This patchset enables the in-memory part of the incremental backup feature. A patchset by Vladimir Sementsov-Ogievskiy enables t

Re: [Qemu-devel] [PATCH RFC v2 7/8] migration: add dirty parameter

2015-02-04 Thread Vladimir Sementsov-Ogievskiy
On 27.01.2015 19:20, Eric Blake wrote: On 01/27/2015 03:56 AM, Vladimir Sementsov-Ogievskiy wrote: Add dirty parameter to qmp-migrate command. If this parameter is true, migration/block.c will migrate dirty bitmaps. This parameter can be used without "blk" parameter to migrate

Re: [Qemu-devel] [PATCH 0/8] block: persistent dirty bitmaps (RFC)

2015-02-04 Thread Vladimir Sementsov-Ogievskiy
On 13.01.2015 20:02, Vladimir Sementsov-Ogievskiy wrote: The bitmaps are saved into qcow2 file format. It provides both 'internal' and 'external' dirty bitmaps feature: - for qcow2 drives we can store bitmaps in the same file - for other formats we can store bitmaps i

Re: [Qemu-devel] [PATCH 0/8] block: persistent dirty bitmaps (RFC)

2015-02-04 Thread Vladimir Sementsov-Ogievskiy
On 04.02.2015 18:20, John Snow wrote: On 02/04/2015 10:07 AM, Vladimir Sementsov-Ogievskiy wrote: On 13.01.2015 20:02, Vladimir Sementsov-Ogievskiy wrote: The bitmaps are saved into qcow2 file format. It provides both 'internal' and 'external' dirty bitmaps feature: -

Re: [Qemu-devel] [PATCH v11 12/13] qemu-iotests: Add tests for drive-backup sync=dirty-bitmap

2015-02-06 Thread Vladimir Sementsov-Ogievskiy
On 12.01.2015 19:31, John Snow wrote: From: Fam Zheng Signed-off-by: Fam Zheng Signed-off-by: John Snow --- tests/qemu-iotests/056| 33 ++--- tests/qemu-iotests/056.out| 4 ++-- tests/qemu-iotests/iotests.py | 8 3 files changed, 40 inse

Re: [Qemu-devel] [PATCH v10 00/13] block: Incremental backup series (RFC)

2014-12-23 Thread Vladimir Sementsov-Ogievskiy
On 23.12.2014 04:12, John Snow wrote: For convenience, this patchset is available on github: https://github.com/jnsnow/qemu/commits/dbm-backup - old version here.

[Qemu-devel] [PATCH v2 1/1] migration/block: fix pending() return value

2014-12-30 Thread Vladimir Sementsov-Ogievskiy
off-by: Vladimir Sementsov-Ogievskiy --- migration/block.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/migration/block.c b/migration/block.c index 74d9eb1..2e92605 100644 --- a/migration/block.c +++ b/migration/block.c @@ -765,8 +765,8 @@ static uint64_t block_save_pending(Q

[Qemu-devel] [PATCH v2 0/1] Fix block migration bug

2014-12-30 Thread Vladimir Sementsov-Ogievskiy
M' qemu-io check finishes successfully, but for './bug.sh 1G 1022M' it finishes with 'Pattern verification failed' status. The following patch fixes this bug. Vladimir Sementsov-Ogievskiy (1): migration/block: fix pending() return value migration/block.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 1.9.1

Re: [Qemu-devel] [PATCH v10 06/13] block: Add bdrv_copy_dirty_bitmap and bdrv_clear_dirty_bitmap

2014-12-30 Thread Vladimir Sementsov-Ogievskiy
I'm sorry if it was already discussed, but I think it is inconsistent to have "size" in sectors and "granularity" in bytes in one structure. I've misused these fields because of this in my current work. At least, I think there should be comments about this. Best regards, Vladimir On 23.12.201

Re: [Qemu-devel] [PATCH 9/9] block-migration: add named dirty bitmaps migration

2015-01-12 Thread Vladimir Sementsov-Ogievskiy
Best regards, Vladimir On 09.01.2015 01:36, Paolo Bonzini wrote: The bitmaps are transmitted many times in their entirety, but only the last copy actually means something. The others are lost. This means you should use the non-live interface (register_savevm). This will simplify the code a lo

Re: [Qemu-devel] [PATCH 9/9] block-migration: add named dirty bitmaps migration

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
On 12.01.2015 17:42, Paolo Bonzini wrote: On 12/01/2015 15:20, Vladimir Sementsov-Ogievskiy wrote: On 09.01.2015 01:36, Paolo Bonzini wrote: The bitmaps are transmitted many times in their entirety, but only the last copy actually means something. The others are lost. This means you should

Re: [Qemu-devel] [PATCH v10 06/13] block: Add bdrv_copy_dirty_bitmap and bdrv_clear_dirty_bitmap

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
Hmm, can't find it in v11 Best regards, Vladimir On 10.01.2015 06:25, John Snow wrote: On 12/30/2014 08:47 AM, Vladimir Sementsov-Ogievskiy wrote: I'm sorry if it was already discussed, but I think it is inconsistent to have "size" in sectors and "granularity&q

Re: [Qemu-devel] [PATCH 4/9] hbitmap: store / restore

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
Best regards, Vladimir On 09.01.2015 00:21, John Snow wrote: On 12/11/2014 09:17 AM, Vladimir Sementsov-Ogievskiy wrote: Functions to store / restore HBitmap. HBitmap should be saved to linear bitmap format independently of endianess. Because of restoring in several steps, every step

Re: [Qemu-devel] [PATCH v11 01/13] block: fix spoiling all dirty bitmaps by mirror and migration

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
it is already in master and can be dropped here Best regards, Vladimir On 12.01.2015 19:30, John Snow wrote: From: Vladimir Sementsov-Ogievskiy Mirror and migration use dirty bitmaps for their purposes, and since commit [block: per caller dirty bitmap] they use their own bitmaps, not the

Re: [Qemu-devel] [PATCH v10 06/13] block: Add bdrv_copy_dirty_bitmap and bdrv_clear_dirty_bitmap

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
Snow wrote: On 01/13/2015 04:54 AM, Vladimir Sementsov-Ogievskiy wrote: Hmm, can't find it in v11 Best regards, Vladimir I changed approach and instead of copying bitmaps to be used with the incremental backup, I lock them in place. So copy isn't used anymore in my set, so the na

Re: [Qemu-devel] [PATCH v11 13/13] block: BdrvDirtyBitmap miscellaneous fixup

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
Hmm. May be, update the size field to be uint64_t too? Negative size is not meaningful.. Best regards, Vladimir On 12.01.2015 19:31, John Snow wrote: (1) granularity type consistency: Update the granularity to be uint64_t in all places. This value is always in bytes. (2) Some documentati

[Qemu-devel] [PATCH 0/8] block: persistent dirty bitmaps (RFC)

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
'on'. Disabled bitmaps are not changing regardless of writes to corresponding drive. Examples: qemu -drive file=a.qcow2,id=disk -dirty-bitmap name=b,drive=disk qemu -drive file=a.raw,id=disk \ -dirty-bitmap name=b,drive=disk,file=b.qcow2,enabled=off Vladimir Sementsov-Ogie

[Qemu-devel] [PATCH 8/8] iotests: test internal persistent dirty bitmap

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
The test performs several vm reloads with checking and updating dirty bitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/115 | 96 ++ tests/qemu-iotests/115.out | 64 +++ tests/qemu-iotests/group

[Qemu-devel] [PATCH 2/8] hbitmap: store / restore

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
bitmap. All other levels are restored by hbitmap_restore_finish as a last step of restoring. So, HBitmap is inconsistent while restoring. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/qemu/hbitmap.h | 49 util/hbitmap.c | 87

[Qemu-devel] [PATCH 1/8] spec: add qcow2-dirty-bitmaps specification

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
Persistent dirty bitmaps will be saved into qcow2 files. It may be used as 'internal' bitmaps (for qcow2 drives) or as 'external' bitmaps for other drives (there may be qcow2 file with zero disk size but with several dirty bitmaps for other drives). Signed-off-by: Vladimir

[Qemu-devel] [PATCH 6/8] qemu: command line option for dirty bitmaps

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
will be loaded from that drive (internal dirty bitmap). granularity The granularity for the bitmap. Not necessary, the default value may be used. enabled on|off. Default is 'on'. Disabled bitmaps are not changing regardless of write

[Qemu-devel] [PATCH 4/8] block: store persistent dirty bitmaps

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
bdrv_dirty_bitmap_set_file() only once. bdrv_ref/bdrv_unref are used for BdrvDirtyBitmap.file to be sure that files will be closed and resources will be freed. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 43 +++ include/block/block.h | 3

[Qemu-devel] [PATCH 3/8] qcow2: add dirty-bitmaps feature

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
Adds dirty-bitmaps feature to qcow2 format as specified in docs/specs/qcow2.txt Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/Makefile.objs| 2 +- block/qcow2-dirty-bitmap.c | 514 + block/qcow2.c | 26 +++ block/qcow2

[Qemu-devel] [PATCH 5/8] block: add bdrv_load_dirty_bitmap

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
The funcion loads dirty bitmap from file, using underlying driver function. Note: the function doesn't change BdrvDirtyBitmap.file field. This field is only used by bdrv_store_dirty_bitmap() function and is ONLY written by bdrv_dirty_bitmap_set_file() function. Signed-off-by: Vladimir Seme

[Qemu-devel] [PATCH 7/8] qmp: print dirty bitmap

2015-01-13 Thread Vladimir Sementsov-Ogievskiy
Adds qmp and hmp commands to print dirty bitmap. This is needed only for testing persistent dirty bitmap feature. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 33 + blockdev.c| 13 + hmp-commands.hx | 15

Re: [Qemu-devel] [PATCH 4/9] hbitmap: store / restore

2015-01-14 Thread Vladimir Sementsov-Ogievskiy
Best regards, Vladimir On 13.01.2015 20:08, John Snow wrote: On 01/13/2015 07:59 AM, Vladimir Sementsov-Ogievskiy wrote: On 09.01.2015 00:21, John Snow wrote: On 12/11/2014 09:17 AM, Vladimir Sementsov-Ogievskiy wrote: +/** + * hbitmap_restore_finish + * @hb: HBitmap to operate on

Re: [Qemu-devel] [PATCH 5/9] block: BdrvDirtyBitmap store/restore interface

2015-01-14 Thread Vladimir Sementsov-Ogievskiy
As in previous patch, rename store/restore to serialize/deserialize... Hmm. In this case, isn't it be better to include serialization of granularity, name and name length in these functions? Best regards, Vladimir On 11.12.2014 17:17, Vladimir Sementsov-Ogievskiy wrote: Several functio

Re: [Qemu-devel] [PATCH 6/9] block-migration: tiny refactoring

2015-01-14 Thread Vladimir Sementsov-Ogievskiy
On 09.01.2015 00:23, John Snow wrote: On 12/11/2014 09:17 AM, Vladimir Sementsov-Ogievskiy wrote: Add blk_create and blk_free to remove code duplicates. Otherwise, duplicates will rise in the following patches because of BlkMigBlock sturcture extendin. Signed-off-by: Vladimir Sementsov

Re: [Qemu-devel] [PATCH 7/9] block-migration: remove not needed iothread lock

2015-01-16 Thread Vladimir Sementsov-Ogievskiy
On 09.01.2015 00:24, John Snow wrote: On 12/11/2014 09:17 AM, Vladimir Sementsov-Ogievskiy wrote: Instead of locking iothread, we can just swap these calls. So, if some write to our range occures before resetting the bitmap, then it will get into subsequent aio read, becouse it occures, in

Re: [Qemu-devel] [PATCH 7/9] block-migration: remove not needed iothread lock

2015-01-16 Thread Vladimir Sementsov-Ogievskiy
Best regards, Vladimir On 09.01.2015 01:28, Paolo Bonzini wrote: On 11/12/2014 15:17, Vladimir Sementsov-Ogievskiy wrote: -qemu_mutex_lock_iothread(); +bdrv_reset_dirty_bitmap(bs, bmds->dirty_bitmap, cur_sector, nr_sectors); + blk->aiocb = bdrv_aio_readv(bs, cur_sector

Re: [Qemu-devel] [PATCH 9/9] block-migration: add named dirty bitmaps migration

2015-01-17 Thread Vladimir Sementsov-Ogievskiy
Best regards, Vladimir On 09.01.2015 01:05, John Snow wrote: CCing migration maintainers, feedback otherwise in-line. On 12/11/2014 09:17 AM, Vladimir Sementsov-Ogievskiy wrote: Just migrate parts of dirty bitmaps, corresponding to the block being migrated. Also, skip block migration if it

Re: [Qemu-devel] [PATCH v6 02/10] qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove

2014-11-07 Thread Vladimir Sementsov-Ogievskiy
+if (!name || name[0] == '\0') { Isn't is better to move "name[0] == '\0'" check to bdrv_create_dirty_bitmap, near existed name checking? +if (granularity < 512 || is_power_of_2(granularity)) { +error_setg(errp, "Granularity must be power of 2 " +

Re: [Qemu-devel] [PATCH v6 05/10] block: Add bdrv_copy_dirty_bitmap and bdrv_reset_dirty_bitmap

2014-11-07 Thread Vladimir Sementsov-Ogievskiy
from [PATCH v6 02/10] +void qmp_block_dirty_bitmap_remove(const char *device, const char *name, + Error **errp) +{ +BlockDriverState *bs; +BdrvDirtyBitmap *bitmap; + +bs = bdrv_find(device); +if (!bs) { +error_set(errp, QERR_DEVICE_NOT_FOU

Re: [Qemu-devel] [PATCH v6 07/10] qmp: Add support of "dirty-bitmap" sync mode for drive-backup

2014-11-12 Thread Vladimir Sementsov-Ogievskiy
@@ -317,7 +321,21 @@ static void coroutine_fn backup_run(void *opaque) if (alloced == 0) { continue; } +} else if (job->sync_mode == MIRROR_SYNC_MODE_DIRTY_BITMAP) { +int i, dirty = 0; +for (i = 0

Re: [Qemu-devel] [PATCH v6 04/10] hbitmap: Add hbitmap_copy

2014-11-17 Thread Vladimir Sementsov-Ogievskiy
+ +HBitmap *hbitmap_copy(const HBitmap *bitmap) +{ +int i; +int64_t size; +HBitmap *hb = g_memdup(bitmap, sizeof(struct HBitmap)); + +size = bitmap->size; +for (i = HBITMAP_LEVELS; i-- > 0; ) { +size = MAX((size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1); +hb->l

Re: [Qemu-devel] [PATCH v6 00/10] block: Incremental backup series

2014-11-18 Thread Vladimir Sementsov-Ogievskiy
used for faster sync. Maybe, save some of bitmap levels on timer while vm is running and save the last level on shutdown? CC qemu-devel - ok. Best regards, Vladimir On 18.11.2014 02:46, John Snow wrote: On 11/13/2014 08:54 AM, Vladimir Sementsov-Ogievskiy wrote: Hi I'd just like to st

Re: [Qemu-devel] [PATCH v6 00/10] block: Incremental backup series

2014-11-18 Thread Vladimir Sementsov-Ogievskiy
d? Or sync timer will not be co-routine based? Best regards, Vladimir On 18.11.2014 13:54, Vladimir Sementsov-Ogievskiy wrote: (2) File Format Some standard file magic, which includes: - Some magic byte(s) - Dirty flag. Needed to tell if we can trust this data or not. - The size of the bi

Re: [Qemu-devel] [PATCH v6 00/10] block: Incremental backup series

2014-11-18 Thread Vladimir Sementsov-Ogievskiy
ocking required? Or sync timer will not be co-routine based? Best regards, Vladimir On 18.11.2014 13:54, Vladimir Sementsov-Ogievskiy wrote: (2) File Format Some standard file magic, which includes: - Some magic byte(s) - Dirty flag. Needed to tell if we can trust this data or not. - The siz

[Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.

2014-11-20 Thread Vladimir Sementsov-Ogievskiy
use it here or not is still questionable. Signed-off-by: Vladimir Sementsov-Ogievskiy --- docs/specs/qdb.txt | 132 + 1 file changed, 132 insertions(+) create mode 100644 docs/specs/qdb.txt diff --git a/docs/specs/qdb.txt b/docs/specs/qdb.txt

Re: [Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.

2014-11-20 Thread Vladimir Sementsov-Ogievskiy
Also, it may be better to make this as qcow2 extension. And bitmap will be saved in separate qcow2 file, which will contain only the bitmap(s) and no other data (no disk, no snapshots). Best regards, Vladimir On 20.11.2014 13:34, Vladimir Sementsov-Ogievskiy wrote: QDB file is for storing

Re: [Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.

2014-11-21 Thread Vladimir Sementsov-Ogievskiy
rate it without common procedure with cow or something like this.. Best regards, Vladimir On 20.11.2014 14:36, Stefan Hajnoczi wrote: On Thu, Nov 20, 2014 at 01:41:14PM +0300, Vladimir Sementsov-Ogievskiy wrote: Also, it may be better to make this as qcow2 extension. And bitmap will be saved in

Re: [Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.

2014-11-21 Thread Vladimir Sementsov-Ogievskiy
ent active bitmap and it's snapshot used by backup. Best regards, Vladimir On 20.11.2014 14:36, Stefan Hajnoczi wrote: On Thu, Nov 20, 2014 at 01:41:14PM +0300, Vladimir Sementsov-Ogievskiy wrote: Also, it may be better to make this as qcow2 extension. And bitmap will be saved in sepa

Re: [Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.

2014-11-24 Thread Vladimir Sementsov-Ogievskiy
d not be connected with bitmap file and it's format. Best regards, Vladimir On 21.11.2014 19:55, Stefan Hajnoczi wrote: On Fri, Nov 21, 2014 at 01:27:40PM +0300, Vladimir Sementsov-Ogievskiy wrote: There is a constraint if we want to get live migration for free: The bitmap conten

Re: [Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.

2014-11-25 Thread Vladimir Sementsov-Ogievskiy
can create for itself a bitmap and use it not touching other bitmaps.. Am I wrong? Best regards, Vladimir On 21.11.2014 19:55, Stefan Hajnoczi wrote: On Fri, Nov 21, 2014 at 01:27:40PM +0300, Vladimir Sementsov-Ogievskiy wrote: There is a constraint if we want to get live migration for fre

  1   2   3   4   5   6   7   8   9   10   >