[Qemu-devel] [PATCH v6 07/11] qcow2: Do not perform potentially damaging repairs

2014-10-20 Thread Max Reitz
lean which should be set to true whenever a fix is rather dangerous or too complicated using the current refcount structures. Another example for this is refcount blocks being referenced more than once. Signed-off-by: Max Reitz Reviewed-by: Benoît Canet --- block/qcow2-refcou

Re: [Qemu-devel] [PATCH v5 01/11] qcow2: Calculate refcount block entry count

2014-10-20 Thread Max Reitz
On 20.10.2014 at 16:25, Kevin Wolf wrote: Am 29.08.2014 um 23:40 hat Max Reitz geschrieben: The size of a refblock entry is (in theory) variable; calculate therefore the number of entries per refblock and the according bit shift (1 << x == entry count) when opening an image. Signed-off-b

[Qemu-devel] [PATCH v6 11/11] iotests: Add test for potentially damaging repairs

2014-10-20 Thread Max Reitz
refblocks beyond the image end by resizing the image accordingly. Add several tests for this as well. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/108 | 141 + tests/qemu-iotests/108.out | 110

Re: [Qemu-devel] [PATCH 1/4] block: add accounting for merged requests

2014-10-20 Thread Max Reitz
x27;, - 'rd_total_time_ns': 'int', 'wr_highest_offset': 'int' } } + 'rd_total_time_ns': 'int', 'wr_highest_offset': 'int', + 'rd_merged': 'int', 'wr_merged': 'int' } } ## # @BlockStats: With the hunk removing the empty line removed (I know that line should not be there, but there's no point in removing it in this patch) and with either 2.2 or 2.3 in block-core.json: Reviewed-by: Max Reitz

Re: [Qemu-devel] [PATCH 2/4] block: introduce bdrv_runtime_opts

2014-10-20 Thread Max Reitz
; - bs->enable_write_cache = !!(flags & BDRV_O_CACHE_WB); /* Open the image, either directly or using a protocol */ I know it's not even your patch originally, but I still don't like spurious empty line removals, whoever might be responsible for them. @@ -1010,6 +1035,8 @@ free_and_fail: g_free(bs->opaque); bs->opaque = NULL; bs->drv = NULL; +fail_opts: +qemu_opts_del(opts); return ret; } With that empty line removal hunk removed: Reviewed-by: Max Reitz

Re: [Qemu-devel] [PATCH 3/4] block: add a knob to disable multiwrite_merge

2014-10-20 Thread Max Reitz
-- a/qmp-commands.hx +++ b/qmp-commands.hx @@ -2104,6 +2104,7 @@ Each json-object contain the following: - "iops_size": I/O size when limiting by iops (json-int) - "detect_zeroes": detect and optimize zero writing (json-string) - Possible values: "off", "on", "unmap" + - "write_merging": enable merging of write requests (json-bool) - "image": the detail of the image, it is a json-object containing the following: - "filename": image file name (json-string) @@ -2181,6 +2182,7 @@ Example: "iops_wr_max": 0, "iops_size": 0, "detect_zeroes": "on", + "write_merging": "true", "image":{ "filename":"disks/test.qcow2", "format":"qcow2", Reviewed-by: Max Reitz

Re: [Qemu-devel] [PATCH 4/4] hw/virtio-blk: add a constant for max number of merged requests

2014-10-20 Thread Max Reitz
simply MAX_MERGE_REQS or, more complicated, MAX_MERGE_REQ_COUNT. But I'm me, and this mail's "From" field says it's not from me. Reviewed-by: Max Reitz

Re: [Qemu-devel] [PATCH 0/4] multiwrite patches for 2.2

2014-10-20 Thread Max Reitz
to disable multiwrite_merge hw/virtio-blk: add a constant for max number of merged requests In addition, I'd like a test for this (just the parameter would be enough). But I don't object to this series without it. Max

Re: [Qemu-devel] [PATCH 0/4] multiwrite patches for 2.2

2014-10-21 Thread Max Reitz
On 2014-10-20 at 22:48, Peter Lieven wrote: Am 20.10.2014 um 17:56 schrieb Max Reitz : On 20.10.2014 at 16:35, Peter Lieven wrote: This adds some preparing patches for upcoming multiwrite modifications. I will leave the dangerous patches for after 2.2 release. Peter Lieven (4): block: add

Re: [Qemu-devel] [PATCH v6 06/11] qcow2: Fix refcount blocks beyond image end

2014-10-21 Thread Max Reitz
On 2014-10-20 at 18:44, Kevin Wolf wrote: Am 20.10.2014 um 16:35 hat Max Reitz geschrieben: If the qcow2 check function detects a refcount block located beyond the image end, grow the image appropriately. This cannot break anything and is the logical fix for such a case. Signed-off-by: Max

[Qemu-devel] [PATCH 0/2] block/vdi: Fix bmap writing error

2014-10-21 Thread Max Reitz
mu to allocate and then write about 2G of data; yes, test 1 in 084 fails for me because qemu cannot allocate 4G for the bmap. In fact, I can only test this once I'm home where I have more RAM available (I made the mistake of activating swap space to test this only once). Max Reitz (2): block/vdi:

[Qemu-devel] [PATCH 1/2] block/vdi: Use {DIV_,}ROUND_UP

2014-10-21 Thread Max Reitz
There are macros for these operations, so make use of them. Signed-off-by: Max Reitz --- block/vdi.c | 9 - 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/block/vdi.c b/block/vdi.c index 9604721..19701ee 100644 --- a/block/vdi.c +++ b/block/vdi.c @@ -407,8 +407,7

[Qemu-devel] [PATCH 2/2] block/vdi: Do not use bdrv_pwrite_sync() for bmap

2014-10-21 Thread Max Reitz
()+bdrv_flush() instead of bdrv_pwrite_sync(). See: https://bugzilla.redhat.com/show_bug.cgi?id=1154940 Signed-off-by: Max Reitz --- block/vdi.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/block/vdi.c b/block/vdi.c index 19701ee..322efcd 100644 --- a/block/vdi.c +++ b

Re: [Qemu-devel] [PATCH 0/4] multiwrite patches for 2.2

2014-10-21 Thread Max Reitz
On 2014-10-21 at 10:01, Peter Lieven wrote: On 21.10.2014 09:06, Max Reitz wrote: On 2014-10-20 at 22:48, Peter Lieven wrote: Am 20.10.2014 um 17:56 schrieb Max Reitz : On 20.10.2014 at 16:35, Peter Lieven wrote: This adds some preparing patches for upcoming multiwrite modifications. I

Re: [Qemu-devel] [PATCH] snapshot: use local variable to bdrv_pwrite_sync L1 table

2014-10-21 Thread Max Reitz
on. g_free(l1_table); is enough. return ret; } The change itself is good, it just needs some polishing. Max

[Qemu-devel] [Bug 1353456] Re: qemu-io: Failure on a qcow2 image with the fuzzed refcount table

2014-10-21 Thread Max Reitz
aio_write failed: Input/output error Thanks for your report, Max -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1353456 Title: qemu-io: Failure on a qcow2 image with the fuzzed refcount table Stat

Re: [Qemu-devel] [PATCH v6 08/11] qcow2: Rebuild refcount structure during check

2014-10-21 Thread Max Reitz
On 2014-10-21 at 11:31, Kevin Wolf wrote: Am 20.10.2014 um 16:35 hat Max Reitz geschrieben: The previous commit introduced the "rebuild" variable to qcow2's implementation of the image consistency check. Now make use of this by adding a function which creates a completely new ref

Re: [Qemu-devel] [PATCH 0/4] multiwrite patches for 2.2

2014-10-21 Thread Max Reitz
On 2014-10-21 at 11:38, Kevin Wolf wrote: Am 21.10.2014 um 11:07 hat Max Reitz geschrieben: On 2014-10-21 at 10:01, Peter Lieven wrote: On 21.10.2014 09:06, Max Reitz wrote: On 2014-10-20 at 22:48, Peter Lieven wrote: Am 20.10.2014 um 17:56 schrieb Max Reitz : On 20.10.2014 at 16:35, Peter

Re: [Qemu-devel] [PATCH v6 07/11] qcow2: Do not perform potentially damaging repairs

2014-10-21 Thread Max Reitz
On 2014-10-21 at 09:52, Kevin Wolf wrote: Am 20.10.2014 um 16:35 hat Max Reitz geschrieben: If a referenced cluster has a refcount of 0, increasing its refcount may result in clusters being allocated for the refcount structures. This may overwrite the referenced cluster, therefore we cannot

Re: [Qemu-devel] [PATCH v6 09/11] qcow2: Clean up after refcount rebuild

2014-10-21 Thread Max Reitz
On 2014-10-21 at 11:59, Kevin Wolf wrote: Am 20.10.2014 um 16:35 hat Max Reitz geschrieben: Because the old refcount structure will be leaked after having rebuilt it, we need to recalculate the refcounts and run a leak-fixing operation afterwards (if leaks should be fixed at all). Signed-off

Re: [Qemu-devel] [PATCH] snapshot: use local variable to bdrv_pwrite_syncL1 table

2014-10-21 Thread Max Reitz
;& l1_table == NULL)" conditional block. Well, okay, I don't know about the callers of qcow2_process_discards(), so they may have set s->cache_discards to true and then expect this function to always call qcow2_process_discards() and set s->cache_discards to false. Okay

[Qemu-devel] [Bug 1354529] Re: qemu-io: Assert failure on the fuzzed qcow2 image

2014-10-21 Thread Max Reitz
** Changed in: qemu Status: New => Fix Committed -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1354529 Title: qemu-io: Assert failure on the fuzzed qcow2 image Status in QEMU: Fix Committ

[Qemu-devel] [Bug 1353456] Re: qemu-io: Failure on a qcow2 image with the fuzzed refcount table

2014-10-21 Thread Max Reitz
** Changed in: qemu Status: New => Fix Committed -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1353456 Title: qemu-io: Failure on a qcow2 image with the fuzzed refcount table Status in QEMU

[Qemu-devel] [Bug 1355738] Re: qemu-img: Killed by SIGTRAP on check of the fuzzed image

2014-10-21 Thread Max Reitz
n the repair function. But this image is broken in a way that there's no real way to fix it. The best we could do is ask the user to use qemu-img convert and then hope for the best. I'll just mark this as fixed. Max ** Changed in: qemu Status: New => Fix Committed -- You

[Qemu-devel] [Bug 1354529] Re: qemu-io: Assert failure on the fuzzed qcow2 image

2014-10-21 Thread Max Reitz
e_clusters failed: Invalid argument qcow2: Marking image as corrupt: Data cluster offset 0xfe00 unaligned (guest offset: 0x2e1000); further corruption events will be suppressed write failed: Input/output error Thanks for your report (and your fuzzer), Max -- You received this bug notification b

[Qemu-devel] [Bug 1355697] Re: qemu-img: Segfault on a fuzzed image with large values of L1/L2 entries

2014-10-21 Thread Max Reitz
Image end offset: 2883584 I'm still working towards the repair function actually doing its job. Thank you for your report, Max -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1355697 Title: qem

[Qemu-devel] [Bug 1355697] Re: qemu-img: Segfault on a fuzzed image with large values of L1/L2 entries

2014-10-21 Thread Max Reitz
s all we can do. Therefore, I'm marking this fixed as well. Max ** Changed in: qemu Status: New => Fix Committed -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1355697 Title: qemu-img: Seg

Re: [Qemu-devel] [PATCH v2 05/11] blockjob: add block_job_defer_to_main_loop()

2014-10-21 Thread Max Reitz
tefan Hajnoczi --- blockjob.c | 45 + include/block/blockjob.h | 19 +++ 2 files changed, 64 insertions(+) Reviewed-by: Max Reitz

Re: [Qemu-devel] [PATCH v2 10/11] block: let commit blockjob run in BDS AioContext

2014-10-21 Thread Max Reitz
/commit.c | 70 -- blockdev.c | 29 2 files changed, 64 insertions(+), 35 deletions(-) Reviewed-by: Max Reitz

Re: [Qemu-devel] [PATCH v2] block: char devices on FreeBSD are not behind a pager

2014-10-21 Thread Max Reitz
level driver that it needs + * to copy the buffer. */ -if ((bs->open_flags & BDRV_O_NOCACHE)) { +if (s->needs_alignement) { if (!bdrv_qiov_is_aligned(bs, qiov)) { type |= QEMU_AIO_MISALIGNED; #ifdef CONFIG_LINUX_AIO With s/alignement/alignment/ and the

Re: [Qemu-devel] [PATCH v2] snapshot: use local variable to bdrv_pwrite_sync L1 table

2014-10-21 Thread Max Reitz
you are modifying the current L1 table, you need to copy the result back to s->l1_table. Sorry for having missed these in my previous review. Max } -if (l1_allocated) -g_free(l1_table); +g_free(l1_table); return ret; }

Re: [Qemu-devel] [Qemu-stable] [PATCH v2] snapshot: use local variable tobdrv_pwrite_sync L1 table

2014-10-21 Thread Max Reitz
here. OK, I will replace g_try_malloc0(ROUND_UP(l1_size2, BDRV_SECTOR_SIZE)) with qemu_try_blockalign(bs, ROUND_UP(l1_size2, BDRV_SECTOR_SIZE)). You can actually omit the ROUND_UP(). There are no accesses beyond l1_table[l1_size - 1], if I'm not mistaken. Max (and we should be usin

Re: [Qemu-devel] [Qemu-stable] [PATCH v2] snapshot: use local variable tobdrv_pwrite_sync L1 table

2014-10-21 Thread Max Reitz
On 2014-10-21 at 15:10, Max Reitz wrote: On 2014-10-21 at 15:08, Zhang Haoyu wrote: Use local variable to bdrv_pwrite_sync L1 table, needless to make conversion of cached L1 table between big-endian and host style. Signed-off-by: Zhang Haoyu --- v1 -> v2: - remove the superflous assignm

Re: [Qemu-devel] [PATCH v6 11/11] iotests: Add test for potentially damaging repairs

2014-10-21 Thread Max Reitz
On 2014-10-21 at 16:12, Kevin Wolf wrote: Am 20.10.2014 um 16:35 hat Max Reitz geschrieben: There are certain cases where repairing a qcow2 image might actually damage it further (or rather, where repairing it has in fact damaged it further with the old qcow2 check implementation). This should

Re: [Qemu-devel] [PATCH v6 09/11] qcow2: Clean up after refcount rebuild

2014-10-21 Thread Max Reitz
On 2014-10-21 at 12:16, Max Reitz wrote: On 2014-10-21 at 11:59, Kevin Wolf wrote: Am 20.10.2014 um 16:35 hat Max Reitz geschrieben: Because the old refcount structure will be leaked after having rebuilt it, we need to recalculate the refcounts and run a leak-fixing operation afterwards (if

Re: [Qemu-devel] [PATCH v6 09/11] qcow2: Clean up after refcount rebuild

2014-10-21 Thread Max Reitz
On 2014-10-21 at 17:11, Kevin Wolf wrote: Am 21.10.2014 um 16:55 hat Max Reitz geschrieben: On 2014-10-21 at 12:16, Max Reitz wrote: On 2014-10-21 at 11:59, Kevin Wolf wrote: Am 20.10.2014 um 16:35 hat Max Reitz geschrieben: Because the old refcount structure will be leaked after having

Re: [Qemu-devel] [PATCH v5 01/11] qcow2: Calculate refcount block entry count

2014-10-21 Thread Max Reitz
On 2014-10-20 at 16:48, Kevin Wolf wrote: Am 20.10.2014 um 16:39 hat Max Reitz geschrieben: On 20.10.2014 at 16:25, Kevin Wolf wrote: Am 29.08.2014 um 23:40 hat Max Reitz geschrieben: The size of a refblock entry is (in theory) variable; calculate therefore the number of entries per refblock

[Qemu-devel] [PATCH] MAINTAINERS: update xtensa boards

2014-10-21 Thread Max Filippov
- fix file names that were changed by the commit b707ab7 hw/xtensa: remove extraneous xtensa_ prefix from file names - mark OpenCores 10/100 Mbit MAC model as maintained. Signed-off-by: Max Filippov --- MAINTAINERS | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a

Re: [Qemu-devel] writing a QEMU block driver

2014-10-22 Thread Max Reitz
least one distribution already having implemented this). If you add a new driver, you either have to distribute its source so people can build qemu themselves with the driver included; or you build qemu yourself and then distribute the binary; or you find your driver good enough to submit it for upstream qemu. Max

[Qemu-devel] [PATCH v7 00/13] qcow2: Fix image repairing

2014-10-22 Thread Max Reitz
008/13:[0007] [FC] 'qcow2: Fix refcount blocks beyond image end' 009/13:[0012] [FC] 'qcow2: Do not perform potentially damaging repairs' 010/13:[0082] [FC] 'qcow2: Rebuild refcount structure during check' 011/13:[0016] [FC] 'qcow2: Clean up after refcount rebuild

[Qemu-devel] [PATCH v7 04/13] qcow2: Split qcow2_check_refcounts()

2014-10-22 Thread Max Reitz
Put the code for calculating the reference counts and comparing them during qemu-img check into own functions. Signed-off-by: Max Reitz Reviewed-by: Benoît Canet Reviewed-by: Kevin Wolf --- block/qcow2-refcount.c | 153 - 1 file changed, 102

[Qemu-devel] [PATCH v7 01/13] block: Add qemu_{, try_}blockalign0()

2014-10-22 Thread Max Reitz
These functions call their non-0-counterparts and then fill the allocated buffer with 0 (if the allocation has been successful). Signed-off-by: Max Reitz --- block.c | 16 include/block/block.h | 2 ++ 2 files changed, 18 insertions(+) diff --git a/block.c b

[Qemu-devel] [PATCH v7 08/13] qcow2: Fix refcount blocks beyond image end

2014-10-22 Thread Max Reitz
If the qcow2 check function detects a refcount block located beyond the image end, grow the image appropriately. This cannot break anything and is the logical fix for such a case. Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 63 ++ 1 file

[Qemu-devel] [PATCH v7 03/13] qcow2: Fix leaks in dirty images

2014-10-22 Thread Max Reitz
When opening dirty images, qcow2's repair function should not only repair errors but leaks as well. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Benoît Canet Reviewed-by: Kevin Wolf --- block/qcow2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/

[Qemu-devel] [PATCH v7 02/13] qcow2: Calculate refcount block entry count

2014-10-22 Thread Max Reitz
The size of a refblock entry is (in theory) variable; calculate therefore the number of entries per refblock and the according bit shift (1 << x == entry count) when opening an image. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.c | 3 +++ block/qcow2.h | 2 ++ 2

[Qemu-devel] [PATCH v7 05/13] qcow2: Use sizeof(**refcount_table)

2014-10-22 Thread Max Reitz
When implementing variable refcounts, we want to be able to easily find all the places in qemu which are tied to a certain refcount order. Replace sizeof(uint16_t) in the check code by sizeof(**refcount_table) so we can later find it more easily. Signed-off-by: Max Reitz --- block/qcow2

[Qemu-devel] [PATCH v7 06/13] qcow2: Pull check_refblocks() up

2014-10-22 Thread Max Reitz
Pull check_refblocks() before calculate_refcounts() so we can drop its static declaration. Signed-off-by: Max Reitz Reviewed-by: Benoît Canet Reviewed-by: Kevin Wolf --- block/qcow2-refcount.c | 102 - 1 file changed, 49 insertions(+), 53

[Qemu-devel] [PATCH v7 07/13] qcow2: Reuse refcount table in calculate_refcounts()

2014-10-22 Thread Max Reitz
We will later call calculate_refcounts multiple times, so reuse the refcount table if possible. Signed-off-by: Max Reitz Reviewed-by: Benoît Canet Reviewed-by: Kevin Wolf --- block/qcow2-refcount.c | 12 +++- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/block/qcow2

[Qemu-devel] [PATCH v7 10/13] qcow2: Rebuild refcount structure during check

2014-10-22 Thread Max Reitz
ructure will be leaked, however. This leak will be dealt with in a follow-up commit. Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 310 - 1 file changed, 304 insertions(+), 6 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-r

[Qemu-devel] [PATCH v7 09/13] qcow2: Do not perform potentially damaging repairs

2014-10-22 Thread Max Reitz
lean which should be set to true whenever a fix is rather dangerous or too complicated using the current refcount structures. Another example for this is refcount blocks being referenced more than once. Signed-off-by: Max Reitz --- block/qcow2-refcou

[Qemu-devel] [PATCH v7 13/13] iotests: Add test for potentially damaging repairs

2014-10-22 Thread Max Reitz
refblocks beyond the image end by resizing the image accordingly. Add several tests for this as well. Signed-off-by: Max Reitz --- tests/qemu-iotests/108 | 141 + tests/qemu-iotests/108.out | 110 +++ tests/qemu-iotests

[Qemu-devel] [PATCH v7 11/13] qcow2: Clean up after refcount rebuild

2014-10-22 Thread Max Reitz
Because the old refcount structure will be leaked after having rebuilt it, we need to recalculate the refcounts and run a leak-fixing operation afterwards (if leaks should be fixed at all). Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 45 + 1

[Qemu-devel] [PATCH v7 12/13] iotests: Fix test outputs

2014-10-22 Thread Max Reitz
rebuild the refcount structure as well. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/039.out | 10 -- tests/qemu-iotests/060.out | 10 -- tests/qemu-iotests/061.out | 18 -- 3 files changed, 28 insertions(+), 10 deletions(-) diff --git a

Re: [Qemu-devel] [PATCH v7 08/13] qcow2: Fix refcount blocks beyond image end

2014-10-22 Thread Max Reitz
On 2014-10-22 at 11:24, Kevin Wolf wrote: Am 22.10.2014 um 10:08 hat Max Reitz geschrieben: If the qcow2 check function detects a refcount block located beyond the image end, grow the image appropriately. This cannot break anything and is the logical fix for such a case. Signed-off-by: Max

Re: [Qemu-devel] [PATCH v7 10/13] qcow2: Rebuild refcount structure during check

2014-10-22 Thread Max Reitz
On 2014-10-22 at 11:14, Kevin Wolf wrote: Am 22.10.2014 um 10:08 hat Max Reitz geschrieben: The previous commit introduced the "rebuild" variable to qcow2's implementation of the image consistency check. Now make use of this by adding a function which creates a completely new ref

Re: [Qemu-devel] [PATCH v3] block: char devices on FreeBSD are not behind a pager

2014-10-22 Thread Max Reitz
unconditionally. Signed-off-by: Roger Pau Monné Reviewed-by: Max Reitz Cc: Kevin Wolf Cc: Stefan Hajnoczi --- Changes since v2: - s/alignement/alignment/. - Don't make needs_alignment a bit field. - Add braces to single statement conditional block. Changes since v1: - Intead of appe

Re: [Qemu-devel] [PATCH v2 02/11] blockdev: acquire AioContext in do_qmp_query_block_jobs_one()

2014-10-22 Thread Max Reitz
On 2014-10-21 at 13:03, Stefan Hajnoczi wrote: Make sure that query-block-jobs acquires the BlockDriverState AioContext so that the blockjob isn't running in another thread while we access its state. Signed-off-by: Stefan Hajnoczi Reviewed-by: Max Reitz --- blockdev.c | 9 +++-- 1

Re: [Qemu-devel] [PATCH v4] snapshot: use local variable to bdrv_pwrite_sync L1 table

2014-10-22 Thread Max Reitz
On 2014-10-22 at 13:59, Gonglei wrote: On 2014/10/22 19:45, Zhang Haoyu wrote: Use local variable to bdrv_pwrite_sync L1 table, needless to make conversion of cached L1 table between big-endian and host style. Signed-off-by: Zhang Haoyu Reviewed-by: Max Reitz --- v3 -> v4: - convert lo

Re: [Qemu-devel] [PATCH v4] snapshot: use local variable to bdrv_pwrite_sync L1 table

2014-10-22 Thread Max Reitz
On 2014-10-22 at 14:01, Max Reitz wrote: On 2014-10-22 at 13:59, Gonglei wrote: On 2014/10/22 19:45, Zhang Haoyu wrote: Use local variable to bdrv_pwrite_sync L1 table, needless to make conversion of cached L1 table between big-endian and host style. Signed-off-by: Zhang Haoyu Reviewed-by

[Qemu-devel] [PATCH v8 00/17] qcow2: Fix image repairing

2014-10-22 Thread Max Reitz
rs' 014/17:[0001] [FC] 'qcow2: Rebuild refcount structure during check' 015/17:[0002] [FC] 'qcow2: Clean up after refcount rebuild' 016/17:[] [--] 'iotests: Fix test outputs' 017/17:[] [--] 'iotests: Add test for potentially damaging repairs' Max Reitz (

[Qemu-devel] [PATCH v8 05/17] qcow2: Use sizeof(**refcount_table)

2014-10-22 Thread Max Reitz
When implementing variable refcounts, we want to be able to easily find all the places in qemu which are tied to a certain refcount order. Replace sizeof(uint16_t) in the check code by sizeof(**refcount_table) so we can later find it more easily. Signed-off-by: Max Reitz --- block/qcow2

[Qemu-devel] [PATCH v8 04/17] qcow2: Split qcow2_check_refcounts()

2014-10-22 Thread Max Reitz
Put the code for calculating the reference counts and comparing them during qemu-img check into own functions. Signed-off-by: Max Reitz Reviewed-by: Benoît Canet Reviewed-by: Kevin Wolf --- block/qcow2-refcount.c | 153 - 1 file changed, 102

[Qemu-devel] [PATCH v8 02/17] qcow2: Calculate refcount block entry count

2014-10-22 Thread Max Reitz
The size of a refblock entry is (in theory) variable; calculate therefore the number of entries per refblock and the according bit shift (1 << x == entry count) when opening an image. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/qcow2.c | 3 +++ block/qcow2.h | 2 ++ 2

[Qemu-devel] [PATCH v8 09/17] qcow2: Let inc_refcounts() return -errno

2014-10-22 Thread Max Reitz
As of a future patch, inc_refcounts() will have to throw errors which are generally signaled by returning -errno. Therefore, let it return an integer which is either 0 for success or -errno and handle the -errno case in all callers. Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 91

[Qemu-devel] [PATCH v8 14/17] qcow2: Rebuild refcount structure during check

2014-10-22 Thread Max Reitz
ructure will be leaked, however. This leak will be dealt with in a follow-up commit. Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 311 - 1 file changed, 305 insertions(+), 6 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-r

[Qemu-devel] [PATCH v8 01/17] block: Add qemu_{, try_}blockalign0()

2014-10-22 Thread Max Reitz
These functions call their non-0-counterparts and then fill the allocated buffer with 0 (if the allocation has been successful). Signed-off-by: Max Reitz --- block.c | 16 include/block/block.h | 2 ++ 2 files changed, 18 insertions(+) diff --git a/block.c b

[Qemu-devel] [PATCH v8 03/17] qcow2: Fix leaks in dirty images

2014-10-22 Thread Max Reitz
When opening dirty images, qcow2's repair function should not only repair errors but leaks as well. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Benoît Canet Reviewed-by: Kevin Wolf --- block/qcow2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/

[Qemu-devel] [PATCH v8 06/17] qcow2: Pull check_refblocks() up

2014-10-22 Thread Max Reitz
Pull check_refblocks() before calculate_refcounts() so we can drop its static declaration. Signed-off-by: Max Reitz Reviewed-by: Benoît Canet Reviewed-by: Kevin Wolf --- block/qcow2-refcount.c | 102 - 1 file changed, 49 insertions(+), 53

[Qemu-devel] [PATCH v8 12/17] qcow2: Fix refcount blocks beyond image end

2014-10-22 Thread Max Reitz
If the qcow2 check function detects a refcount block located beyond the image end, grow the image appropriately. This cannot break anything and is the logical fix for such a case. Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 67 +++--- 1 file

[Qemu-devel] [PATCH v8 08/17] qcow2: Split fail code in L1 and L2 checks

2014-10-22 Thread Max Reitz
Instead of printing out an error message, incrementing check_errors and returning a fixed -errno, just do cleanups and return -ret, with ret set by the code which threw the exception (jumped to the fail label). Also, increment check_errors on error in check_refcounts_l2(). Signed-off-by: Max

[Qemu-devel] [PATCH v8 07/17] qcow2: Use int64_t for in-memory reftable size

2014-10-22 Thread Max Reitz
Use int64_t for the entry count of the in-memory refcount table throughout the check functions. Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 24f297f..a3f4d47

[Qemu-devel] [PATCH v8 15/17] qcow2: Clean up after refcount rebuild

2014-10-22 Thread Max Reitz
Because the old refcount structure will be leaked after having rebuilt it, we need to recalculate the refcounts and run a leak-fixing operation afterwards (if leaks should be fixed at all). Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 45 + 1

[Qemu-devel] [PATCH v8 11/17] qcow2: Reuse refcount table in calculate_refcounts()

2014-10-22 Thread Max Reitz
We will later call calculate_refcounts multiple times, so reuse the refcount table if possible. Signed-off-by: Max Reitz Reviewed-by: Benoît Canet Reviewed-by: Kevin Wolf --- block/qcow2-refcount.c | 12 +++- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/block/qcow2

[Qemu-devel] [PATCH v8 17/17] iotests: Add test for potentially damaging repairs

2014-10-22 Thread Max Reitz
refblocks beyond the image end by resizing the image accordingly. Add several tests for this as well. Signed-off-by: Max Reitz --- tests/qemu-iotests/108 | 141 + tests/qemu-iotests/108.out | 110 +++ tests/qemu-iotests

[Qemu-devel] [PATCH v8 10/17] qcow2: Let inc_refcounts() resize the reftable

2014-10-22 Thread Max Reitz
Now that the refcount table can be passed around by reference, do that for inc_refcounts() (and subsequently check_refcounts_l1() and check_refcounts_l2()) and use it for resizing it when a cluster after the image end is encountered. Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 57

[Qemu-devel] [PATCH v8 13/17] qcow2: Do not perform potentially damaging repairs

2014-10-22 Thread Max Reitz
lean which should be set to true whenever a fix is rather dangerous or too complicated using the current refcount structures. Another example for this is refcount blocks being referenced more than once. Signed-off-by: Max Reitz --- block/qcow2-refcou

[Qemu-devel] [PATCH v8 16/17] iotests: Fix test outputs

2014-10-22 Thread Max Reitz
rebuild the refcount structure as well. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/039.out | 10 -- tests/qemu-iotests/060.out | 10 -- tests/qemu-iotests/061.out | 18 -- 3 files changed, 28 insertions(+), 10 deletions(-) diff --git a

Re: [Qemu-devel] [PATCH v4] snapshot: use local variable to bdrv_pwrite_sync L1 table

2014-10-22 Thread Max Reitz
On 2014-10-22 at 14:21, Gonglei wrote: On 2014/10/22 20:02, Max Reitz wrote: On 2014-10-22 at 14:01, Max Reitz wrote: On 2014-10-22 at 13:59, Gonglei wrote: On 2014/10/22 19:45, Zhang Haoyu wrote: Use local variable to bdrv_pwrite_sync L1 table, needless to make conversion of cached L1

Re: [Qemu-devel] [PATCH v4] snapshot: use local variable to bdrv_pwrite_sync L1 table

2014-10-22 Thread Max Reitz
On 2014-10-22 at 14:30, Gonglei wrote: On 2014/10/22 20:24, Max Reitz wrote: On 2014-10-22 at 14:21, Gonglei wrote: On 2014/10/22 20:02, Max Reitz wrote: On 2014-10-22 at 14:01, Max Reitz wrote: On 2014-10-22 at 13:59, Gonglei wrote: On 2014/10/22 19:45, Zhang Haoyu wrote: Use local

[Qemu-devel] [PATCH v13 01/14] qcow2: Allow "full" discard

2014-10-22 Thread Max Reitz
Normally, discarded sectors should read back as zero. However, there are cases in which a sector (or rather cluster) should be discarded as if they were never written in the first place, that is, reading them should fall through to the backing file again. Signed-off-by: Max Reitz Reviewed-by

[Qemu-devel] [PATCH v13 03/14] qcow2: Optimize bdrv_make_empty()

2014-10-22 Thread Max Reitz
the dirty flag set, which only works for compat=1.1) and creating a trivial refcount structure. If there are snapshots or for compat=0.10, fall back to the simple implementation (discard all clusters). Signed-off-by: Max Reitz --- block/blkdebug.c | 2 + block/qcow2.c | 143

[Qemu-devel] [PATCH v13 12/14] iotests: Add _filter_qemu_img_map

2014-10-22 Thread Max Reitz
As different image formats most probably map guest addresses to different host addresses, add a filter to filter the host addresses out; also, the image filename should be filtered. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- tests/qemu-iotests/common.filter | 7 +++ 1 file

[Qemu-devel] [PATCH v13 02/14] qcow2: Implement bdrv_make_empty()

2014-10-22 Thread Max Reitz
Implement this function by making all clusters in the image file fall through to the backing file (by using the recently extended discard). Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf --- block/qcow2.c | 27 +++ 1 file changed, 27

[Qemu-devel] [PATCH v13 00/14] qemu-img: Implement commit like QMP

2014-10-22 Thread Max Reitz
it like QMP' 009/14:[] [-C] 'qemu-img: Empty image after commit' 010/14:[] [-C] 'qemu-img: Enable progress output for commit' 011/14:[] [-C] 'qemu-img: Specify backing file for commit' 012/14:[] [-C] 'iotests: Add _filter_qemu_img_map'

[Qemu-devel] [PATCH v13 07/14] block/mirror: Improve progress report

2014-10-22 Thread Max Reitz
er in fact desirable. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- block/mirror.c | 34 ++ 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index e8a43eb..2a1acfe 100644 --- a/block/mirror.c +++ b/block/mirror.

[Qemu-devel] [PATCH v13 06/14] iotests: Omit length/offset test in 040 and 041

2014-10-22 Thread Max Reitz
function from iotests.py, the same applies there as well which in turn affects tests 030, 055 and 056. On the other hand, a block job's length does not have to be related to the length of the image file in the first place, so that check was questionable anyway. Signed-off-by: Max Reitz Reviewed-by:

[Qemu-devel] [PATCH v13 04/14] blockjob: Introduce block_job_complete_sync()

2014-10-22 Thread Max Reitz
Implement block_job_complete_sync() by doing the exact same thing as block_job_cancel_sync() does, only with calling block_job_complete() instead of block_job_cancel(). Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf --- blockjob.c | 39

[Qemu-devel] [PATCH v13 08/14] qemu-img: Implement commit like QMP

2014-10-22 Thread Max Reitz
block-commit by using commit_active_start() and then waiting for the block job to finish. Signed-off-by: Max Reitz --- block/Makefile.objs | 3 +- qemu-img.c | 82 - 2 files changed, 64 insertions(+), 21 deletions(-) diff --git a/

[Qemu-devel] [PATCH v13 05/14] blockjob: Add "ready" field

2014-10-22 Thread Max Reitz
ck job may be completed. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- blockjob.c | 3 +++ include/block/blockjob.h | 5 + qapi/block-core.json | 4 +++- 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/blockjob.c b/blockjob.c index a7d57e3..448b9ce 10

[Qemu-devel] [PATCH v13 10/14] qemu-img: Enable progress output for commit

2014-10-22 Thread Max Reitz
Implement progress output for the commit command by querying the progress of the block job. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- qemu-img-cmds.hx | 4 ++-- qemu-img.c | 24 ++-- qemu-img.texi| 2 +- 3 files changed, 25 insertions(+), 5 deletions

[Qemu-devel] [PATCH v13 11/14] qemu-img: Specify backing file for commit

2014-10-22 Thread Max Reitz
Introduce a new parameter for qemu-img commit which may be used to explicitly specify the backing file into which an image should be committed if the backing chain has more than a single layer. Signed-off-by: Max Reitz --- qemu-img-cmds.hx | 4 ++-- qemu-img.c | 24

[Qemu-devel] [PATCH v13 09/14] qemu-img: Empty image after commit

2014-10-22 Thread Max Reitz
After the top image has been committed, it should be emptied unless specified otherwise. Signed-off-by: Max Reitz Reviewed-by: Eric Blake --- qemu-img-cmds.hx | 4 ++-- qemu-img.c | 34 +++--- qemu-img.texi| 6 +- 3 files changed, 38 insertions(+), 6

[Qemu-devel] [PATCH v13 13/14] iotests: Add test for backing-chain commits

2014-10-22 Thread Max Reitz
Add a test for qemu-img commit on backing chains with more than two images. This test also checks whether the top image is emptied (unless this is prevented by specifying either -d or -b) and does therefore not work for qed and vmdk which requires it to be separate from 020. Signed-off-by: Max

[Qemu-devel] [PATCH v13 14/14] iotests: Add test for qcow2's bdrv_make_empty

2014-10-22 Thread Max Reitz
Add a test for qcow2's fast bdrv_make_empty implementation on images without internal snapshots. Signed-off-by: Max Reitz --- tests/qemu-iotests/098 | 79 ++ tests/qemu-iotests/098.out | 45 ++ tests/qemu-iotests/

Re: [Qemu-devel] [PATCH v2 0/3] block: Fix is_allocated() for truncated images

2014-10-22 Thread Max Reitz
On 2014-10-22 at 15:24, Max Reitz wrote: Patch 2: The bdrv_is_allocated() functions may return a number of zero sectors e.g. if a sector beyond the image end has been queried. Respect this case in qemu-io's map implementation so it doesn't run into an infinite loop (https://bugs.lau

[Qemu-devel] [PATCH v2 0/3] block: Fix is_allocated() for truncated images

2014-10-22 Thread Max Reitz
. Therefore, this test does no longer require _filter_qemu_img_map() (because there are no mappings shown) and this series no longer depends on the qemu-img/QMP commit series. Max Reitz (3): block: Respect underlying file's EOF qemu-io: Respect early image end for map iotests:

[Qemu-devel] [PATCH v2 1/3] block: Respect underlying file's EOF

2014-10-22 Thread Max Reitz
: Max Reitz --- block.c | 16 ++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/block.c b/block.c index 773e87e..68dc11d 100644 --- a/block.c +++ b/block.c @@ -3954,13 +3954,25 @@ static int64_t coroutine_fn bdrv_co_get_block_status(BlockDriverState *bs, if (bs

[Qemu-devel] [PATCH v2 3/3] iotests: Add test for map commands

2014-10-22 Thread Max Reitz
Add a test for qemu-img map and qemu-io -c map on truncated files. Signed-off-by: Max Reitz --- tests/qemu-iotests/102 | 64 ++ tests/qemu-iotests/102.out | 10 tests/qemu-iotests/group | 1 + 3 files changed, 75 insertions(+) create

[Qemu-devel] [PATCH v2 2/3] qemu-io: Respect early image end for map

2014-10-22 Thread Max Reitz
bdrv_is_allocated() may report zero clusters which most probably means the image (file) is shorter than expected. Respect this case in order to avoid an infinite loop. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Benoît Canet --- qemu-io-cmds.c | 5 - 1 file changed, 4

Re: [Qemu-devel] [PATCH v2 1/3] block: Respect underlying file's EOF

2014-10-22 Thread Max Reitz
On 2014-10-22 at 15:40, Kevin Wolf wrote: Am 22.10.2014 um 15:24 hat Max Reitz geschrieben: When falling through to the underlying file in bdrv_co_get_block_status(), if it returns that the query offset is beyond the file end (by setting *pnum to 0), return the range to be zero and do not let

<    3   4   5   6   7   8   9   10   11   12   >