[Qemu-devel] [PATCH 0/7] Allow Valgrind checking all QEMU processes

2019-06-09 Thread Andrey Shinkevich
r the Valgrind tool. Andrey Shinkevich (7): iotests: allow Valgrind checking all QEMU processes iotests: amendment for benchmark output of 039 061 137 iotests: exclude killed processes from running under Valgrind iotests: Valgrind fails to work with nonexistent directory iotests: ext

[Qemu-devel] [PATCH 7/7] iotests: amend QEMU NBD process synchronization

2019-06-09 Thread Andrey Shinkevich
Processes are dying harder under the Valgring. It results in counting the dying process as a newborn one. Make it sure that old NBD job get finished before starting a new one. Suggested-by: Roman Kagan Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/common.nbd | 6 ++ 1 file

[Qemu-devel] [PATCH 2/7] iotests: amendment for benchmark output of 039 061 137

2019-06-09 Thread Andrey Shinkevich
After including the Valgrind into the QEMU processes wrappers in the common.rc script, the output for the tests 039 061 137 has been changed and is to be amended. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/039.out | 30 +- tests/qemu-iotests/061.out | 12

[Qemu-devel] [PATCH 6/7] iotests: extend sleeping time under Valgrind

2019-06-09 Thread Andrey Shinkevich
To synchronize the time when QEMU is running longer under the Valgrind, increase the sleeping time int the test 247. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/247 | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/247 b/tests/qemu-iotests

[Qemu-devel] [PATCH 4/7] iotests: Valgrind fails to work with nonexistent directory

2019-06-09 Thread Andrey Shinkevich
The Valgrind uses the exported variable TMPDIR and fails if the directory does not exist. Let us exclude such a test case from being run under the Valgrind. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/051 | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/qemu-iotests/051 b

[Qemu-devel] [PATCH 1/7] iotests: allow Valgrind checking all QEMU processes

2019-06-09 Thread Andrey Shinkevich
With the '-valgrind' option, let all the QEMU processes be run under the Valgrind tool. The Valgrind own parameters may be set with its environment variable VALGRIND_OPTS, e.g. VALGRIND_OPTS="--leak-check=yes" ./check -qcow2 -valgrind Signed-off-by: Andrey Shinkevich ---

[Qemu-devel] [PATCH 5/7] iotests: extended timeout under Valgrind

2019-06-09 Thread Andrey Shinkevich
As the iotests run longer under the Valgrind, the QEMU_COMM_TIMEOUT is to be increased in the test cases 028, 183 and 192 when running under the Valgrind. Suggested-by: Roman Kagan Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/028 | 6 +- tests/qemu-iotests/183 | 9

[Qemu-devel] [PATCH 3/7] iotests: exclude killed processes from running under Valgrind

2019-06-09 Thread Andrey Shinkevich
The Valgrind tool fails to manage its termination when QEMU raises the signal SIGKILL. Lets exclude such test cases from running under the Valgrind because there is no sense to check memory issues that way. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/039 | 5 + tests/qemu

Re: [Qemu-devel] [PATCH 2/7] iotests: amendment for benchmark output of 039 061 137

2019-06-10 Thread Andrey Shinkevich
On 10/06/2019 12:10, Daniel P. Berrangé wrote: > On Sun, Jun 09, 2019 at 09:35:43PM +0300, Andrey Shinkevich wrote: >> After including the Valgrind into the QEMU processes wrappers in the >> common.rc script, the output for the tests 039 061 137 has been >> changed and is to

Re: [Qemu-devel] [PATCH 1/7] iotests: allow Valgrind checking all QEMU processes

2019-06-10 Thread Andrey Shinkevich
On 10/06/2019 17:24, Eric Blake wrote: > On 6/9/19 1:35 PM, Andrey Shinkevich wrote: >> With the '-valgrind' option, let all the QEMU processes be run under >> the Valgrind tool. The Valgrind own parameters may be set with its >> environment variable VALGRIND_OPTS

[Qemu-devel] [PATCH v2 6/7] iotests: amend QEMU NBD process synchronization

2019-06-11 Thread Andrey Shinkevich
Processes are dying harder under the Valgring. It results in counting the dying process as a newborn one. Make it sure that old NBD job get finished before starting a new one. Suggested-by: Roman Kagan Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/common.nbd | 6 ++ 1 file

[Qemu-devel] [PATCH v2 3/7] iotests: Valgrind fails to work with nonexistent directory

2019-06-11 Thread Andrey Shinkevich
The Valgrind uses the exported variable TMPDIR and fails if the directory does not exist. Let us exclude such a test case from being run under the Valgrind. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/051 | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/qemu-iotests/051 b

[Qemu-devel] [PATCH v2 0/7] Allow Valgrind checking all QEMU processes

2019-06-11 Thread Andrey Shinkevich
r the Valgrind tool. v2: 01: The patch 2/7 of v1 was merged into the patch 1/7, suggested by Daniel. 02: Another patch 7/7 was added to introduce the Valgrind error suppression file into the QEMU project. Andrey Shinkevich (7): iotests: allow Valgrind checking all QEMU processes io

[Qemu-devel] [PATCH v2 1/7] iotests: allow Valgrind checking all QEMU processes

2019-06-11 Thread Andrey Shinkevich
algrind checked file ./.valgrindrc or ~/.valgrindrc like --memcheck:leak-check=no --memcheck:track-origins=yes After including the Valgrind into the QEMU processes wrappers in the common.rc script, the benchmark output for the tests 039 061 137 is to be amended. Signed-off-by: Andrey Shinkevich --- tests/

[Qemu-devel] [PATCH v2 2/7] iotests: exclude killed processes from running under Valgrind

2019-06-11 Thread Andrey Shinkevich
The Valgrind tool fails to manage its termination when QEMU raises the signal SIGKILL. Lets exclude such test cases from running under the Valgrind because there is no sense to check memory issues that way. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/039 | 5 + tests/qemu

[Qemu-devel] [PATCH v2 5/7] iotests: extend sleeping time under Valgrind

2019-06-11 Thread Andrey Shinkevich
To synchronize the time when QEMU is running longer under the Valgrind, increase the sleeping time int the test 247. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/247 | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/qemu-iotests/247 b/tests/qemu-iotests

[Qemu-devel] [PATCH v2 4/7] iotests: extended timeout under Valgrind

2019-06-11 Thread Andrey Shinkevich
As the iotests run longer under the Valgrind, the QEMU_COMM_TIMEOUT is to be increased in the test cases 028, 183 and 192 when running under the Valgrind. Suggested-by: Roman Kagan Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/028 | 6 +- tests/qemu-iotests/183 | 9

[Qemu-devel] [PATCH v2 7/7] iotests: new file to suppress Valgrind errors

2019-06-11 Thread Andrey Shinkevich
lgrind specific format file valgrind.supp to the QEMU project. The file content is extendable for future needs. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/common.rc | 5 - tests/qemu-iotests/valgrind.supp | 31 +++ 2 files changed, 35 insertions

Re: [Qemu-devel] [PATCH 2/2] block-stream: include base into job node list

2019-03-12 Thread Andrey Shinkevich
On 08/03/2019 17:00, Alberto Garcia wrote: > On Thu 21 Feb 2019 04:26:39 PM CET, Andrey Shinkevich wrote: >> The block-stream job needs to own the base node as the limiter >> of the copy-on-read operation. So, the base node is included in >> the job node list (block_job_ad

Re: [PATCH v4 0/7] iotests: Selfish patches

2019-10-14 Thread Andrey Shinkevich
anything back >> then (neither do they really now, but whatever, it’s better >> than a comment)) >> >> - Patch 4: Resolved a conflict because of the change to patch 3 > > Thanks for the reviews, applied to my block branch: > > https://git.xanclic.moe/XanClic/qemu/commits/branch/block > > Max > Good -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 6/6] tests/qemu-iotests: add case for block-stream compress

2019-10-15 Thread Andrey Shinkevich
On 03/10/2019 17:58, Vladimir Sementsov-Ogievskiy wrote: > 02.10.2019 17:22, Andrey Shinkevich wrote: >> Add a test case to the iotest #030 that checks 'compress' option for a >> block-stream job. >> >> Signed-off-by: Andrey Shinkevich >>

[PATCH v3 4/5] block-stream: add compress option

2019-10-15 Thread Andrey Shinkevich
Allow data compression during block-stream job for backup backing chain. Signed-off-by: Andrey Shinkevich --- block/stream.c | 10 -- blockdev.c | 12 +++- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/block/stream.c b/block/stream.c index 5562ccb..25f9324

[PATCH v3 1/5] qcow2: Allow writing compressed data of multiple clusters

2019-10-15 Thread Andrey Shinkevich
ned-off-by: Andrey Shinkevich --- block.c | 12 +- block/io.c| 2 +- block/qcow2.c | 106 ++ block/qcow2.h | 1 + blockdev.c| 4 ++ include/block/block.h | 1 + inc

[PATCH v3 5/5] tests/qemu-iotests: add case for block-stream compress

2019-10-15 Thread Andrey Shinkevich
Add a case to the iotest #030 that tests the 'compress' option for a block-stream job. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/030 | 51 +- tests/qemu-iotests/030.out | 4 ++-- 2 files changed, 52 insertions(+), 3

[PATCH v3 3/5] block: support compressed write for copy-on-read

2019-10-15 Thread Andrey Shinkevich
Support the data compression during block-stream job over a backup backing chain implemented in the following patch 'block-stream: add compress option'. Signed-off-by: Anton Nefedov Signed-off-by: Denis V. Lunev Signed-off-by: Andrey Shinkevich --- block/io.c

[PATCH v3 2/5] tests/qemu-iotests: add case to write compressed data of multiple clusters

2019-10-15 Thread Andrey Shinkevich
Add the test case to the iotest #214 that checks possibility of writing compressed data of more than one cluster size. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/214 | 35 +++ tests/qemu-iotests/214.out | 15 +++ 2 files changed, 50

[PATCH v3 0/5] qcow2: advanced compression options

2019-10-15 Thread Andrey Shinkevich
#x27; option has been introduced on generic block layer as suggested by Roman Kagan. Discussed on the thread ID <1570026166-748566-1-git-send-email-andrey.shinkev...@virtuozzo.com> Andrey Shinkevich (5): qcow2: Allow writing compressed data of multiple clusters tests/qemu

[PATCH v4 3/4] tests/qemu-iotests: add case to write compressed data of multiple clusters

2019-10-16 Thread Andrey Shinkevich
Add the test case to the iotest #214 that checks possibility of writing compressed data of more than one cluster size. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/214 | 35 +++ tests/qemu-iotests/214.out | 15 +++ 2 files changed, 50

[PATCH v4 0/4] qcow2: advanced compression options

2019-10-16 Thread Andrey Shinkevich
ivers, the 'compression' option has been introduced at the block generic layer as suggested by Roman Kagan. Discussed on the email thread with ID <1570026166-748566-1-git-send-email-andrey.shinkev...@virtuozzo.com> Andrey Shinkevich (4): block: support compressed write at ge

[PATCH v4 1/4] block: support compressed write at generic layer

2019-10-16 Thread Andrey Shinkevich
ned-off-by: Andrey Shinkevich --- block.c | 20 +++- block/io.c| 14 ++ block/qcow2.c | 4 blockdev.c| 9 - include/block/block.h | 1 + include/block/block_int.h | 2 ++ qapi/block

[PATCH v4 2/4] qcow2: Allow writing compressed data of multiple clusters

2019-10-16 Thread Andrey Shinkevich
Signed-off-by: Andrey Shinkevich --- block/qcow2.c | 102 ++ 1 file changed, 75 insertions(+), 27 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 6b29e16..9a85d73 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -4156,10

[PATCH v4 4/4] tests/qemu-iotests: add case for block-stream compress

2019-10-16 Thread Andrey Shinkevich
Add a case to the iotest #030 that tests the 'compress' option for a block-stream job. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/030 | 51 +- tests/qemu-iotests/030.out | 4 ++-- 2 files changed, 52 insertions(+), 3

Re: [PATCH v4 3/4] tests/qemu-iotests: add case to write compressed data of multiple clusters

2019-10-20 Thread Andrey Shinkevich
056466703da7f in qcow2_co_pwritev_compressed_task (bs=0x564669143390, offset=393216, bytes=65536, qiov=0x7f4528dddec0, qiov_offset=196608) at block/qcow2.c:4198 #2 0x56466703dc0c in qcow2_co_pwritev_compressed_task_entry (task=0x564669152ac0) at block/qcow2.c:4230 #3 0x5646670a69d0 in aio_task_co (opaque=0x564669152ac0) at block/aio_task.c:45 #4 0x564667147a87 in coroutine_trampoline (i0=1762994976, i1=22086) at util/coroutine-ucontext.c:115 Andrey -- With the best regards, Andrey Shinkevich

[PATCH v5 0/4] qcow2: advanced compression options

2019-10-20 Thread Andrey Shinkevich
ions for many drivers, the 'compression' option has been introduced at the block generic layer as suggested by Roman Kagan. Discussed on the email thread with ID <1570026166-748566-1-git-send-email-andrey.shinkev...@virtuozzo.com> Andrey Shinkevich (4): block: support compre

[PATCH v5 2/4] qcow2: Allow writing compressed data of multiple clusters

2019-10-20 Thread Andrey Shinkevich
Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 102 ++ 1 file changed, 75 insertions(+), 27 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 6b29e16..a1e7279 100644 --- a/block

[PATCH v5 4/4] tests/qemu-iotests: add case for block-stream compress

2019-10-20 Thread Andrey Shinkevich
Add a case to the iotest #030 that tests the 'compress' option for a block-stream job. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/030 | 34 +- tests/qemu-iotests/030.out | 4 ++-- 2 files changed, 35 insertions(+), 3 deletions(-) di

[PATCH v5 3/4] tests/qemu-iotests: add case to write compressed data of multiple clusters

2019-10-20 Thread Andrey Shinkevich
Add the test case to the iotest #214 that checks possibility of writing compressed data of more than one cluster size. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/214 | 45 + tests/qemu-iotests/214.out | 14 ++ 2 files

[PATCH v5 1/4] block: support compressed write at generic layer

2019-10-20 Thread Andrey Shinkevich
ned-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block.c | 20 +++- block/io.c| 13 + block/qcow2.c | 4 blockdev.c| 9 - include/block/block.h | 1 + inc

Re: [PATCH v5 1/4] block: support compressed write at generic layer

2019-10-22 Thread Andrey Shinkevich
On 22/10/2019 12:28, Max Reitz wrote: > On 20.10.19 22:37, Andrey Shinkevich wrote: >> To inform the block layer about writing all the data compressed, we >> introduce the 'compress' command line option. Based on that option, the >> written data will be aligned by

Re: [PATCH v5 1/4] block: support compressed write at generic layer

2019-10-22 Thread Andrey Shinkevich
On 22/10/2019 15:56, Max Reitz wrote: > On 22.10.19 14:23, Vladimir Sementsov-Ogievskiy wrote: >> 22.10.2019 14:31, Max Reitz wrote: >>> On 22.10.19 12:46, Vladimir Sementsov-Ogievskiy wrote: >>>> 22.10.2019 13:21, Andrey Shinkevich wrote: >>>>&g

Re: [PATCH v2 01/22] migration/block-dirty-bitmap: fix dirty_bitmap_mig_before_vm_start

2020-02-18 Thread Andrey Shinkevich
} else { bdrv_dirty_bitmap_enable_successor(b->bitmap); } Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 03/22] migration/block-dirty-bitmap: rename dirty_bitmap_mig_cleanup

2020-02-18 Thread Andrey Shinkevich
opportunity, I would suggest the name like "dirty_bitmap_do_clean_after_saving()" and similar for dirty_bitmap_save_cleanup() "dirty_bitmap_clean_after_saving()". Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 04/22] migration/block-dirty-bitmap: move mutex init to dirty_bitmap_mig_init

2020-02-18 Thread Andrey Shinkevich
if (!migration_object_check(current_migration, &err)) { error_report_err(err); exit(1); I rely on that the mutex initialization is being made in proper time. Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 05/22] migration/block-dirty-bitmap: refactor state global variables

2020-02-18 Thread Andrey Shinkevich
aveVMHandlers savevm_dirty_bitmap_handlers = { void dirty_bitmap_mig_init(void) { -QSIMPLEQ_INIT(&dirty_bitmap_mig_state.dbms_list); -qemu_mutex_init(&finish_lock); +QSIMPLEQ_INIT(&dbm_state.save.dbms_list); +qemu_mutex_init(&dbm_state.load.finish_lock); register_savevm_live("dirty-bitmap", 0, 1, &savevm_dirty_bitmap_handlers, - &dirty_bitmap_mig_state); + &dbm_state); } Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 06/22] migration/block-dirty-bitmap: rename finish_lock to just lock

2020-02-18 Thread Andrey Shinkevich
irty-bitmap", 0, 1, &savevm_dirty_bitmap_handlers, Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 07/22] migration/block-dirty-bitmap: simplify dirty_bitmap_load_complete

2020-02-18 Thread Andrey Shinkevich
* must be) or on merge fail, but merge can't fail when second - * bitmap is empty - */ -assert(ret == s->bitmap && - count == bdrv_get_dirty_count(s->bitmap)); -} -bdrv_dirty_bitmap_unlock(s->bitmap);

Re: [PATCH v2 08/22] migration/block-dirty-bitmap: keep bitmap state for all bitmaps

2020-02-18 Thread Andrey Shinkevich
item; item = g_slist_next(item)) { LoadBitmapState *b = item->data; if (b->bitmap == s->bitmap) { b->migrated = true; +if (s->before_vm_start_handled) { + s->bitmaps = g_slist_remove(s->bitmaps, b); +g_free(b); +} break; } } Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 09/22] migration/block-dirty-bitmap: relax error handling in incoming part

2020-02-18 Thread Andrey Shinkevich
lock(&s->lock); +cancel_incoming_locked(s); +qemu_mutex_unlock(&s->lock); return -EINVAL; } do { +qemu_mutex_lock(&s->lock); + ret = dirty_bitmap_load_header(f, s); if (ret < 0) { +cancel_incoming_locked(s); +

Re: [PATCH v2 10/22] migration/block-dirty-bitmap: cancel migration on shutdown

2020-02-18 Thread Andrey Shinkevich
critical data, and their loss never considered as + * something serious. + */ +dirty_bitmap_mig_cancel_incoming(); } /* For outgoing */ Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 11/22] migration/savevm: don't worry if bitmap migration postcopy failed

2020-02-18 Thread Andrey Shinkevich
ostcopy_state_get() == POSTCOPY_INCOMING_RUNNING && -postcopy_pause_incoming(mis)) { +migrate_postcopy_ram() && postcopy_pause_incoming(mis)) { /* Reset f to point to the newly created channel */ f = mis->from_src_file; goto retry; Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 00/22] Fix error handling during bitmap postcopy

2020-02-18 Thread Andrey Shinkevich
12 +- tests/qemu-iotests/199 | 244 ++ tests/qemu-iotests/199.out | 4 +- 7 files changed, 529 insertions(+), 230 deletions(-) -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 12/22] qemu-iotests/199: fix style

2020-02-18 Thread Andrey Shinkevich
;drive0', name='bitmap') -self.assert_qmp(result, 'return/sha256', sha256); +self.assert_qmp(result, 'return/sha256', sha256) + if __name__ == '__main__': iotests.main(supported_fmts=['qcow2'], supported_cache_modes=['none'], Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 13/22] qemu-iotests/199: drop extra constraints

2020-02-19 Thread Andrey Shinkevich
class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase): if __name__ == '__main__': -iotests.main(supported_fmts=['qcow2'], supported_cache_modes=['none'], - supported_protocols=['file']) +iotests.main(supported_fmts=['qcow2']) R

Re: [PATCH v2 14/22] qemu-iotests/199: better catch postcopy time

2020-02-19 Thread Andrey Shinkevich
;} 1582102676.155691 DST MIGRATION {'status': 'completed'} +if debug: with no usage in the following patches, you can put the whole block of relative code above under the "if debug: section + print('downtime:', downtime) +print('postcopy_time:', postcopy_time) + +# Assert that bitmap migration is finished (check that successor bitmap +# is removed) result = self.vm_b.qmp('query-block') -while len(result['return'][0]['dirty-bitmaps']) > 1: -time.sleep(2) -result = self.vm_b.qmp('query-block') +assert len(result['return'][0]['dirty-bitmaps']) == 1 +# Check content of migrated (and updated by new writes) bitmap result = self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256', node='drive0', name='bitmap') - self.assert_qmp(result, 'return/sha256', sha256) Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 00/22] Fix error handling during bitmap postcopy

2020-02-19 Thread Andrey Shinkevich
On 18/02/2020 23:57, Eric Blake wrote: On 2/18/20 2:02 PM, Andrey Shinkevich wrote: qemu-iotests:$ ./check -qcow2 PASSED (except always failed 261 and 272) Have you reported those failures on the threads that introduced those tests? Not yet unfortunately. I have not investigated the

Re: [PATCH v2 00/22] Fix error handling during bitmap postcopy

2020-02-19 Thread Andrey Shinkevich
On 19/02/2020 16:36, Eric Blake wrote: On 2/19/20 7:25 AM, Andrey Shinkevich wrote: On 18/02/2020 23:57, Eric Blake wrote: On 2/18/20 2:02 PM, Andrey Shinkevich wrote: qemu-iotests:$ ./check -qcow2 PASSED (except always failed 261 and 272) Have you reported those failures on the

Re: [PATCH v2 15/22] qemu-iotests/199: improve performance: set bitmap by discard

2020-02-19 Thread Andrey Shinkevich
27;capability': 'dirty-bitmaps', 'state': True}, @@ -126,8 +135,8 @@ class TestDirtyBitmapPostcopyMigration(iotests.QMPTestCase): self.vm_b_events.append(e_resume) s = 0x8000 -while s < write_size: -self.vm_b.hmp_qemu_io('drive0', 'write %d %d' % (s, chunk)) +while s < discard_size: +self.vm_b.hmp_qemu_io('drive0', 'discard %d %d' % (s, chunk)) s += 0x1 match = {'data': {'status': 'completed'}} Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 16/22] qemu-iotests/199: change discard patterns

2020-02-19 Thread Andrey Shinkevich
e = self.vm_b.event_wait('RESUME') self.vm_b_events.append(e_resume) -s = 0x8000 - while s < discard_size: -self.vm_b.hmp_qemu_io('drive0', 'discard %d %d' % (s, chunk)) -s += 0x1 +apply_discards(self.vm_b, discards2) match = {'data': {'status': 'completed'}} e_complete = self.vm_b.event_wait('MIGRATION', match=match) Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 16/22] qemu-iotests/199: change discard patterns

2020-02-19 Thread Andrey Shinkevich
On 19/02/2020 17:33, Andrey Shinkevich wrote: On 17/02/2020 18:02, Vladimir Sementsov-Ogievskiy wrote: iotest 40 works too long because of many discard opertion. On the same operations At the same time time, postcopy period is very short, in spite of all these efforts. So, let'

Re: [PATCH v2 17/22] qemu-iotests/199: increase postcopy period

2020-02-19 Thread Andrey Shinkevich
, name='bitmap') -self.assert_qmp(result, 'return/sha256', sha256) +assert len(result['return'][0]['dirty-bitmaps']) == nb_bitmaps + +# Check content of migrated bitmaps. Still, don't waste time checking +# every bitmap +for i in range(0, nb_bitmaps, 5): +result = self.vm_b.qmp('x-debug-block-dirty-bitmap-sha256', + node='drive0', name='bitmap{}'.format(i)) +sha256 = discards1_sha256 if i % 2 else all_discards_sha256 +self.assert_qmp(result, 'return/sha256', sha256) if __name__ == '__main__': The updated test passed. Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 19/22] qemu-iotests/199: prepare for new test-cases addition

2020-02-19 Thread Andrey Shinkevich
node='drive0', name='bitmap{}'.format(i)) - sha256 = discards1_sha256 if i % 2 else all_discards_sha256 -self.assert_qmp(result, 'return/sha256', sha256) +sha = self.discards1_sha256 if i % 2 else self.all_discards_sha256 +self.assert_qmp(result, 'return/sha256', sha) if __name__ == '__main__': Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 20/22] qemu-iotests/199: check persistent bitmaps

2020-02-19 Thread Andrey Shinkevich
7;b').add_drive(disk_b) +self.vm_b.launch() check_bitmaps(self.vm_b, nb_bitmaps) # Check content of migrated bitmaps. Still, don't waste time checking Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 21/22] qemu-iotests/199: add early shutdown case to bitmaps postcopy

2020-02-19 Thread Andrey Shinkevich
-Ran 1 tests +Ran 2 tests OK The updated test passed. Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 18/22] python/qemu/machine: add kill() method

2020-02-19 Thread Andrey Shinkevich
onable to number this patch the last but one, that's right before a usage in the patch that follows. Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 21/22] qemu-iotests/199: add early shutdown case to bitmaps postcopy

2020-02-19 Thread Andrey Shinkevich
sts/qemu-iotests/199.out +++ b/tests/qemu-iotests/199.out @@ -1,5 +1,5 @@ -. +.. -- -Ran 1 tests +Ran 2 tests OK -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 22/22] qemu-iotests/199: add source-killed case to bitmaps postcopy

2020-02-19 Thread Andrey Shinkevich
b/tests/qemu-iotests/199.out index fbc63e62f8..8d7e996700 100644 --- a/tests/qemu-iotests/199.out +++ b/tests/qemu-iotests/199.out @@ -1,5 +1,5 @@ -.. +... -- -Ran 2 tests +Ran 3 tests OK The updated test passed. Reviewed-by: Andrey Shinkevich -- With the best regards, Andrey Shinkevich

Re: [PATCH v2 00/22] Fix error handling during bitmap postcopy

2020-02-19 Thread Andrey Shinkevich
On 19/02/2020 17:58, Eric Blake wrote: On 2/19/20 7:52 AM, Andrey Shinkevich wrote: +od: unrecognized option '--endian=big' +Try 'od --help' for more information. +od: invalid -N argument '--endian=big' Yay, same problem for both tests.  Fix common.rc once

Re: [PATCH v12 0/3] qcow2: advanced compression options

2019-12-18 Thread Andrey Shinkevich
Pinging... (please) On 02/12/2019 15:15, Andrey Shinkevich wrote: > The compression filter driver is introduced as suggested by Max. > A sample usage of the filter can be found in the test #214. > Now, multiple clusters can be written compressed. > It is useful for the backup j

[PATCH v9 0/3] qcow2: advanced compression options

2019-11-26 Thread Andrey Shinkevich
check for compression support in bs->file is checked in the compress_open(). Discussed in the email thread with the message ID <1574056144-625164-1-git-send-email-andrey.shinkev...@virtuozzo.com> Andrey Shinkevich (3): block: introduce compress filter driver qcow2: Allow writing

[PATCH v9 3/3] tests/qemu-iotests: add case to write compressed data of multiple clusters

2019-11-26 Thread Andrey Shinkevich
Add the case to the iotest #214 that checks possibility of writing compressed data of more than one cluster size. The test case involves the compress filter driver showing a sample usage of that. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests

[PATCH v9 2/3] qcow2: Allow writing compressed data of multiple clusters

2019-11-26 Thread Andrey Shinkevich
Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.c | 102 ++ 1 file changed, 75 insertions(+), 27 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index

[PATCH v9 1/3] block: introduce compress filter driver

2019-11-26 Thread Andrey Shinkevich
/block/filter-compress.c new file mode 100644 index 000..ef4b12b --- /dev/null +++ b/block/filter-compress.c @@ -0,0 +1,190 @@ +/* + * Compress filter block driver + * + * Copyright (c) 2019 Virtuozzo International GmbH + * + * Author: + * Andrey Shinkevich + * (based on block/copy-on-read.c by

[PATCH v10 1/3] block: introduce compress filter driver

2019-11-28 Thread Andrey Shinkevich
/block/filter-compress.c new file mode 100644 index 000..4aa189b --- /dev/null +++ b/block/filter-compress.c @@ -0,0 +1,166 @@ +/* + * Compress filter block driver + * + * Copyright (c) 2019 Virtuozzo International GmbH + * + * Author: + * Andrey Shinkevich + * (based on block/copy-on-read.c by

[PATCH v10 3/3] tests/qemu-iotests: add case to write compressed data of multiple clusters

2019-11-28 Thread Andrey Shinkevich
Add the case to the iotest #214 that checks possibility of writing compressed data of more than one cluster size. The test case involves the compress filter driver showing a sample usage of that. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz

[PATCH v10 0/3] qcow2: advanced compression options

2019-11-28 Thread Andrey Shinkevich
ndrey.shinkev...@virtuozzo.com> Andrey Shinkevich (3): block: introduce compress filter driver qcow2: Allow writing compressed data of multiple clusters tests/qemu-iotests: add case to write compressed data of multiple clusters block/Makefile.objs| 1 + block/filter-com

[PATCH v10 2/3] qcow2: Allow writing compressed data of multiple clusters

2019-11-28 Thread Andrey Shinkevich
Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2.c | 102 ++ 1 file changed, 75 insertions(+), 27 deletions(-) diff --git a/block/qcow2

Re: [PATCH v10 1/3] block: introduce compress filter driver

2019-12-02 Thread Andrey Shinkevich
On 28/11/2019 17:37, Vladimir Sementsov-Ogievskiy wrote: > 28.11.2019 12:36, Andrey Shinkevich wrote: >> Allow writing all the data compressed through the filter driver. >> The written data will be aligned by the cluster size. >> Based on the QEMU current implementation, tha

[PATCH v11 2/3] qcow2: Allow writing compressed data of multiple clusters

2019-12-02 Thread Andrey Shinkevich
Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2.c | 102 ++ 1 file changed, 75 insertions(+), 27 deletions(-) diff --git a/block/qcow2

[PATCH v11 0/3] qcow2: advanced compression options

2019-12-02 Thread Andrey Shinkevich
the error message on failure of opening the compression driver as suggested by Vladimir. Andrey Shinkevich (3): block: introduce compress filter driver qcow2: Allow writing compressed data of multiple clusters tests/qemu-iotests: add case to write compressed data of multiple clusters

[PATCH v11 1/3] block: introduce compress filter driver

2019-12-02 Thread Andrey Shinkevich
/block/filter-compress.c new file mode 100644 index 000..3e89249 --- /dev/null +++ b/block/filter-compress.c @@ -0,0 +1,168 @@ +/* + * Compress filter block driver + * + * Copyright (c) 2019 Virtuozzo International GmbH + * + * Author: + * Andrey Shinkevich + * (based on block/copy-on-read.c by

[PATCH v11 3/3] tests/qemu-iotests: add case to write compressed data of multiple clusters

2019-12-02 Thread Andrey Shinkevich
Add the case to the iotest #214 that checks possibility of writing compressed data of more than one cluster size. The test case involves the compress filter driver showing a sample usage of that. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz

[PATCH v12 3/3] tests/qemu-iotests: add case to write compressed data of multiple clusters

2019-12-02 Thread Andrey Shinkevich
Add the case to the iotest #214 that checks possibility of writing compressed data of more than one cluster size. The test case involves the compress filter driver showing a sample usage of that. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz

[PATCH v12 0/3] qcow2: advanced compression options

2019-12-02 Thread Andrey Shinkevich
_status_from_backing to _status_from_file (noticed by Vladimir). Andrey Shinkevich (3): block: introduce compress filter driver qcow2: Allow writing compressed data of multiple clusters tests/qemu-iotests: add case to write compressed data of multiple clusters block/Makefile.objs

[PATCH v12 1/3] block: introduce compress filter driver

2019-12-02 Thread Andrey Shinkevich
Allow writing all the data compressed through the filter driver. The written data will be aligned by the cluster size. Based on the QEMU current implementation, that data can be written to unallocated clusters only. May be used for a backup job. Suggested-by: Max Reitz Signed-off-by: Andrey

[PATCH v12 2/3] qcow2: Allow writing compressed data of multiple clusters

2019-12-02 Thread Andrey Shinkevich
Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2.c | 102 ++ 1 file changed, 75 insertions(+), 27 deletions(-) diff --git a/block/qcow2

[Qemu-devel] [PATCH v4 2/2] char-socket: Lock tcp_chr_disconnect() and socket_reconnect_timeout()

2019-08-12 Thread Andrey Shinkevich
8b8191308 in main_loop_should_exit () at vl.c:1885 #16 0x5558b819140d in main_loop () at vl.c:1924 #17 0x5558b8198c84 in main (argc=18, argv=0x7ffc355df3f8, envp=0x7ffc355df490) at vl.c:4665 This patch adds a lock to protect tcp_chr_disconnect() and socket_reconnect_timeout() Signed-o

[Qemu-devel] [PATCH v4 0/2] char-socket: Fix race condition

2019-08-12 Thread Andrey Shinkevich
This fixes a race condition in which the tcp_chr_read() ioc handler can close a connection that is being written to from another thread. v4: The functions qemu_idle_add() and tcp_chr_be_event_closed() were removed because the callback is invoked after the call to object_property_del_all()

[Qemu-devel] [PATCH v4 1/2] main-loop: Fix GSource leak in qio_task_thread_worker()

2019-08-12 Thread Andrey Shinkevich
From: Alberto Garcia After g_source_attach() the GMainContext holds a reference to the GSource, so the caller does not need to keep it. qio_task_thread_worker() is not releasing its reference so the GSource is being leaked since a17536c594bfed94d05667b419f747b692f5fc7f. Signed-off-by: Alberto G

Re: [Qemu-devel] [PATCH] tests: Set read-zeroes on for null-co driver

2019-08-13 Thread Andrey Shinkevich
On 13/08/2019 11:46, Thomas Huth wrote: > On 7/30/19 3:23 PM, Andrey Shinkevich wrote: >> >> >> On 30/07/2019 15:59, Thomas Huth wrote: >>> On 30/07/2019 14.52, Thomas Huth wrote: >>>> On 29/07/2019 14.46, Andrey Shinkevich wrote: >>>>&

Re: [Qemu-devel] [PATCH 0/3] Reduce the number of Valgrind reports in unit tests.

2019-08-13 Thread Andrey Shinkevich
PINGING... On 30/07/2019 19:01, Andrey Shinkevich wrote: > Running unit tests under the Valgrind may help to detect QEMU memory issues > (suggested by Denis V. Lunev). Some of the Valgrind reports relate to the > unit test code itself. Let's eliminate the detected memory issues to e

Re: [Qemu-devel] [PATCH] Fix: fp-test uninitialized member floatX::exp

2019-08-13 Thread Andrey Shinkevich
PINGING... On 30/07/2019 13:13, Andrey Shinkevich wrote: > Not all the paths in the functions, such as f16ToFloatX(), initialize > the member 'exp' of the structure floatX. > > Signed-off-by: Andrey Shinkevich > --- > source/slowfloat.c | 4 > 1 file cha

Re: [Qemu-devel] [PATCH 0/3] Reduce the number of Valgrind reports in unit tests.

2019-08-13 Thread Andrey Shinkevich
On 13/08/2019 15:05, Paolo Bonzini wrote: > On 13/08/19 14:02, Andrey Shinkevich wrote: >> PINGING... > > I thought I had already said I queued the series? > > Paolo > Thank you Paolo. I am clear now. Andrey >> On 30/07/2019 19:01, Andrey Shinkevich wrote: &g

Re: [Qemu-devel] [PATCH v4 2/2] char-socket: Lock tcp_chr_disconnect() and socket_reconnect_timeout()

2019-08-20 Thread Andrey Shinkevich
On 20/08/2019 12:58, Daniel P. Berrangé wrote: > On Mon, Aug 12, 2019 at 06:58:29PM +0300, Andrey Shinkevich wrote: >> From: Alberto Garcia >> >> There's a race condition in which the tcp_chr_read() ioc handler can >> close a connection that is bein

[Qemu-devel] [PATCH] tests: Set read-zeroes on for null-co driver

2019-07-29 Thread Andrey Shinkevich
This patch is to reduce the number of Valgrind report messages about using uninitialized memory with the null-co driver. It helps to filter real memory issues and is the same work done for the iotests with the commit ID a6862418fec4072. Suggested-by: Kevin Wolf Signed-off-by: Andrey Shinkevich

[Qemu-devel] [PATCH] Fix: fp-test uninitialized member floatX::exp

2019-07-30 Thread Andrey Shinkevich
Not all the paths in the functions, such as f16ToFloatX(), initialize the member 'exp' of the structure floatX. Signed-off-by: Andrey Shinkevich --- source/slowfloat.c | 4 1 file changed, 4 insertions(+) diff --git a/tests/fp/berkeley-testfloat-3/source/slowfloat.c b/tests/f

Re: [Qemu-devel] [PATCH] tests: Set read-zeroes on for null-co driver

2019-07-30 Thread Andrey Shinkevich
On 30/07/2019 15:59, Thomas Huth wrote: > On 30/07/2019 14.52, Thomas Huth wrote: >> On 29/07/2019 14.46, Andrey Shinkevich wrote: >>> This patch is to reduce the number of Valgrind report messages about >>> using uninitialized memory with the null-co driver. It help

[Qemu-devel] [PATCH 0/3] Reduce the number of Valgrind reports in unit tests.

2019-07-30 Thread Andrey Shinkevich
Running unit tests under the Valgrind may help to detect QEMU memory issues (suggested by Denis V. Lunev). Some of the Valgrind reports relate to the unit test code itself. Let's eliminate the detected memory issues to ease locating critical ones. Andrey Shinkevich (3): test-throttle

[Qemu-devel] [PATCH 3/3] i386/kvm: initialize struct at full before ioctl call

2019-07-30 Thread Andrey Shinkevich
Not the whole structure is initialized before passing it to the KVM. Reduce the number of Valgrind reports. Signed-off-by: Andrey Shinkevich --- target/i386/kvm.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/target/i386/kvm.c b/target/i386/kvm.c index dbbb137..ed57e31 100644 --- a

[Qemu-devel] [PATCH 1/3] test-throttle: Fix uninitialized use of burst_length

2019-07-30 Thread Andrey Shinkevich
ThrottleState::cfg of the static variable 'ts' is reassigned with the local one in the do_test_accounting() and then is passed to the throttle_account() with uninitialized member LeakyBucket::burst_length. Signed-off-by: Andrey Shinkevich --- tests/test-throttle.c | 2 ++ 1 file

[Qemu-devel] [PATCH 2/3] tests: Fix uninitialized byte in test_visitor_in_fuzz

2019-07-30 Thread Andrey Shinkevich
One byte in the local buffer stays uninitialized, at least with the first iteration, because of the double decrement in the test_visitor_in_fuzz(). This is what Valgrind does not like and not critical for the test itself. So, reduce the number of the memory issues reports. Signed-off-by: Andrey

  1   2   3   4   5   6   7   8   9   >