Re: [Qemu-block] [PATCH] block: Replace qdict_put() by qdict_put_obj() where appropriate

2018-12-14 Thread Alberto Garcia
On Thu 13 Dec 2018 06:51:54 PM CET, Markus Armbruster wrote: > Patch created mechanically by rerunning: > > $ spatch --sp-file scripts/coccinelle/qobject.cocci \ > --macro-file scripts/cocci-macro-file.h \ > --dir block --in-place > > Signed-off-by: Markus Armbruster Review

Re: [Qemu-block] [Qemu-devel] [PATCH v2 4/8] qcow2-threads: split out generic path

2018-12-14 Thread Vladimir Sementsov-Ogievskiy
14.12.2018 2:28, Paolo Bonzini wrote: > On 11/12/18 17:43, Vladimir Sementsov-Ogievskiy wrote: >> +ThreadPool *pool = aio_get_thread_pool(bdrv_get_aio_context(bs)); >> + >> +while (s->nb_threads >= QCOW2_MAX_THREADS) { >> +qemu_co_queue_wait(&s->thread_task_queue, NULL); >> +} >

Re: [Qemu-block] [PULL 0/6] Qcrypto next patches

2018-12-14 Thread Peter Maydell
On Wed, 12 Dec 2018 at 11:24, Daniel P. Berrangé wrote: > > The following changes since commit bb9bf94b3e8926553290bc9a7cb84315af422086: > > Merge remote-tracking branch > 'remotes/ehabkost/tags/machine-next-pull-request' into staging (2018-12-11 > 19:18:58 +) > > are available in the Git

Re: [Qemu-block] [PATCH v2] qcow2: Assert that refcount block offsets fit in the refcount table

2018-12-14 Thread Alberto Garcia
ping I suppose this one can go in now. On Wed 14 Nov 2018 03:58:57 PM CET, Alberto Garcia wrote: > Refcount table entries have a field to store the offset of the > refcount block. The rest of the bits of the entry are currently > reserved. > > The offset is always taken from the entry using REFT_

Re: [Qemu-block] [PATCH 0/2] Minor fixes for the mirror block job

2018-12-14 Thread Alberto Garcia
ping On Thu 22 Nov 2018 04:00:25 PM CET, Alberto Garcia wrote: > Hi, > > a couple of minor fixes for the mirror block job. I don't think these > bugs can be reproduced at the moment so this shouldn't be 3.1 > material. > > Berto > > Alberto Garcia (2): > mirror: Release the dirty bitmap if mirro

Re: [Qemu-block] [PULL 00/41] Block layer patches

2018-12-14 Thread Peter Maydell
On Wed, 12 Dec 2018 at 13:27, Kevin Wolf wrote: > > The following changes since commit bb9bf94b3e8926553290bc9a7cb84315af422086: > > Merge remote-tracking branch > 'remotes/ehabkost/tags/machine-next-pull-request' into staging (2018-12-11 > 19:18:58 +) > > are available in the Git reposito

Re: [Qemu-block] [PATCH 2/2] avoid TABs in files that only contain a few

2018-12-14 Thread Daniel P . Berrangé
On Thu, Dec 13, 2018 at 11:37:37PM +0100, Paolo Bonzini wrote: > Most files that have TABs only contain a handful of them. Change > them to spaces so that we don't confuse people. > > disas, standard-headers, linux-headers and libdecnumber are imported > from other projects and probably should be

Re: [Qemu-block] [SPAM] [Qemu-ppc] [PATCH 2/2] avoid TABs in files that only contain a few

2018-12-14 Thread Greg Kurz
On Thu, 13 Dec 2018 23:37:37 +0100 Paolo Bonzini wrote: > Most files that have TABs only contain a handful of them. Change > them to spaces so that we don't confuse people. > > disas, standard-headers, linux-headers and libdecnumber are imported > from other projects and probably should be exem

[Qemu-block] [PULL v2 00/42] Block layer patches

2018-12-14 Thread Kevin Wolf
The following changes since commit d8d5fefd8657d4f7b380b3a1533340434b5b9def: Merge remote-tracking branch 'remotes/berrange/tags/qcrypto-next-pull-request' into staging (2018-12-13 18:45:18 +) are available in the Git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream fo

Re: [Qemu-block] [Qemu-devel] Request for clarification on qemu-img convert behavior zeroing target host_device

2018-12-14 Thread De Backer, Fred (Nokia - BE/Antwerp)
>>> Indeed, performance traces are important for issues like this. >>See strace of both FC27 and FC29 attached >Looks like you traced only the main thread. All the I/O is done in other >threads. >These flags would be useful: > >    strace -o log -f -T -tt New straces attached with mentioned flag

Re: [Qemu-block] [Qemu-devel] [PATCH 00/10] Remove deprecated load_image() function

2018-12-14 Thread Peter Maydell
On Fri, 30 Nov 2018 at 15:20, Peter Maydell wrote: > > This patchset removes the load_image() function, which we noted > in a comment as being deprecated in 2008 but were still using > in a handful of places. All the use cases are bugs of some > kind or another, either straightforward "no length c

Re: [Qemu-block] [PATCH 2/2] avoid TABs in files that only contain a few

2018-12-14 Thread Paolo Bonzini
On 14/12/18 11:22, Daniel P. Berrangé wrote: >> crypto/aes.c >> crypto/desrfb.c > > I'd rather like this to be cleaned to finish the job for > crypto/. Ok, will do. Paolo

Re: [Qemu-block] [PATCH] blk: postpone request execution on a context protected with "drained section"

2018-12-14 Thread Denis Plotnikov
On 13.12.2018 15:20, Kevin Wolf wrote: > Am 13.12.2018 um 12:07 hat Denis Plotnikov geschrieben: >> On 12.12.2018 15:24, Kevin Wolf wrote: >>> Am 11.12.2018 um 17:55 hat Denis Plotnikov geschrieben: > Why involve the AioContext at all? This could all be kept at the > BlockBackend level wi

Re: [Qemu-block] [Qemu-devel] [RFC] block: Is name of BlockBackend deprecated with -blockdev parameter?

2018-12-14 Thread Anton Kuchin
I want to make it consistent but I'm note sure I understand all aspects of current state and legacy clients we need to support. The initial idea was to have single function blk_lookup(char* name) and search in all namespaces internally as they are guaranteed to have no intersection. This will

Re: [Qemu-block] [Qemu-devel] Request for clarification on qemu-img convert behavior zeroing target host_device

2018-12-14 Thread Kevin Wolf
Am 14.12.2018 um 11:59 hat De Backer, Fred (Nokia - BE/Antwerp) geschrieben: > >>> Indeed, performance traces are important for issues like this. > >>See strace of both FC27 and FC29 attached > > >Looks like you traced only the main thread. All the I/O is done in other > >threads. > >These flags

Re: [Qemu-block] [Qemu-devel] Request for clarification on qemu-img convert behavior zeroing target host_device

2018-12-14 Thread De Backer, Fred (Nokia - BE/Antwerp)
>Of course, we should also think about the other problem you mentioned, related >to copying a smaller image to a larger block device. Does this require zeroing >the parts after the image or should we leave them alone? > >I'd tend to say that since you're passing the whole block device as a target

Re: [Qemu-block] [Qemu-devel] Request for clarification on qemu-img convert behavior zeroing target host_device

2018-12-14 Thread Richard W.M. Jones
On Fri, Dec 14, 2018 at 12:52:34PM +, De Backer, Fred (Nokia - BE/Antwerp) wrote: > >Of course, we should also think about the other problem you mentioned, > >related to copying a smaller image to a larger block device. Does this > >require zeroing the parts after the image or should we leav

Re: [Qemu-block] [Qemu-devel] Request for clarification on qemu-img convert behavior zeroing target host_device

2018-12-14 Thread Daniel P . Berrangé
On Fri, Dec 14, 2018 at 01:26:59PM +0100, Kevin Wolf wrote: > Am 14.12.2018 um 11:59 hat De Backer, Fred (Nokia - BE/Antwerp) geschrieben: > > >>> Indeed, performance traces are important for issues like this. > > >>See strace of both FC27 and FC29 attached > > > > >Looks like you traced only the

Re: [Qemu-block] [PATCH 2/2] avoid TABs in files that only contain a few

2018-12-14 Thread Peter Maydell
On Thu, 13 Dec 2018 at 22:38, Paolo Bonzini wrote: > > Most files that have TABs only contain a handful of them. Change > them to spaces so that we don't confuse people. > > disas, standard-headers, linux-headers and libdecnumber are imported > from other projects and probably should be exempted

[Qemu-block] [PATCH v4 0/5] qcow2 check improvements part I

2018-12-14 Thread Vladimir Sementsov-Ogievskiy
Hi all! These series are derived from "[PATCH v3 0/8] qcow2 check improvements" (https://lists.gnu.org/archive/html/qemu-devel/2018-10/msg02284.html) Some patches are new, some patches are not included and are postponed to part II. So, here: 01,03 - unchanged 02: handle broken iotest 04-05: new

[Qemu-block] [PATCH v4 3/5] qcow2-refcount: check_refcounts_l2: reduce ignored overlaps

2018-12-14 Thread Vladimir Sementsov-Ogievskiy
Reduce number of structures ignored in overlap check: when checking active table ignore active tables, when checking inactive table ignore inactive ones. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2-refcount.c | 16 +--- 1 file changed, 9 insert

[Qemu-block] [PATCH v4 4/5] qcow2-refcount: check_refcounts_l2: don't count fixed cluster as allocated

2018-12-14 Thread Vladimir Sementsov-Ogievskiy
Do not count a cluster which is fixed to be ZERO as allocated. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index e76085d9aa..8da0e91dd

[Qemu-block] [PATCH v4 5/5] qcow2-refcount: don't mask corruptions under internal errors

2018-12-14 Thread Vladimir Sementsov-Ogievskiy
No reasons for not reporting found corruptions as corruptions in case of some internal errors, especially in case of just failed to fix l2 entry (and in this case, missed corruptions may influence comparing logic, when we calculate difference between corruptions fields of two results) Signed-off-b

[Qemu-block] [PATCH v4 2/5] qcow2-refcount: avoid eating RAM

2018-12-14 Thread Vladimir Sementsov-Ogievskiy
qcow2_inc_refcounts_imrt() (through realloc_refcount_array()) can eat an unpredictable amount of memory on corrupted table entries, which are referencing regions far beyond the end of file. Prevent this, by skipping such regions from further processing. Interesting that iotest 138 checks exactly

[Qemu-block] [PATCH v4 1/5] qcow2-refcount: fix check_oflag_copied

2018-12-14 Thread Vladimir Sementsov-Ogievskiy
Increase corruptions_fixed only after successful fix. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c index 1c63ac244a..b717fdecc6 100644 --- a/block

Re: [Qemu-block] [PATCH 2/2] avoid TABs in files that only contain a few

2018-12-14 Thread Alex Bennée
Paolo Bonzini writes: > Most files that have TABs only contain a handful of them. Change > them to spaces so that we don't confuse people. > > disas, standard-headers, linux-headers and libdecnumber are imported > from other projects and probably should be exempted from the check. > Outside th

Re: [Qemu-block] [PATCH v4 16/18] xen: automatically create XenBlockDevice-s

2018-12-14 Thread Paul Durrant
> -Original Message- [snip] > > + > > +blockdev->auto_iothread = iothread; > > + > > +object_property_set_bool(OBJECT(dev), true, "realized", > &local_err); > > +if (local_err) { > > +error_propagate_prepend(errp, local_err, > > +"initiali

Re: [Qemu-block] 4.2 CI failure on qemu-io tests

2018-12-14 Thread Milan Zamazal
Nir Soffer writes: > On Fri, Dec 7, 2018 at 3:19 PM Milan Zamazal wrote: > >> Hi, I've seen a CI failure on qemu-io tests in ovirt-4.2 branch: >> http://jenkins.ovirt.org/job/vdsm_standard-check-patch/602/ > > > We have several issues: Thank you Nir for looking. > *00:12:02.358* def verify

Re: [Qemu-block] [PATCH 2/2] avoid TABs in files that only contain a few

2018-12-14 Thread Stefan Hajnoczi
On Thu, Dec 13, 2018 at 11:37:37PM +0100, Paolo Bonzini wrote: > Most files that have TABs only contain a handful of them. Change > them to spaces so that we don't confuse people. > > disas, standard-headers, linux-headers and libdecnumber are imported > from other projects and probably should be

Re: [Qemu-block] [PATCH 2/2] avoid TABs in files that only contain a few

2018-12-14 Thread Michael S. Tsirkin
On Thu, Dec 13, 2018 at 11:37:37PM +0100, Paolo Bonzini wrote: > Most files that have TABs only contain a handful of them. Change > them to spaces so that we don't confuse people. > > disas, standard-headers, linux-headers and libdecnumber are imported > from other projects and probably should be

Re: [Qemu-block] [PATCH v4 16/18] xen: automatically create XenBlockDevice-s

2018-12-14 Thread Paul Durrant
> -Original Message- > From: Xen-devel [mailto:xen-devel-boun...@lists.xenproject.org] On Behalf > Of Paul Durrant > Sent: 14 December 2018 14:50 > To: 'Kevin Wolf' > Cc: xen-de...@lists.xenproject.org; Stefano Stabellini > ; qemu-de...@nongnu.org; qemu-block@nongnu.org; > Max Reitz > Sub

Re: [Qemu-block] [PATCH v10 8/9] qcow2: skip writing zero buffers to empty COW areas

2018-12-14 Thread Vladimir Sementsov-Ogievskiy
03.12.2018 13:14, Anton Nefedov wrote: > If COW areas of the newly allocated clusters are zeroes on the backing image, > efficient bdrv_write_zeroes(flags=BDRV_REQ_ALLOCATE) can be used on the whole > cluster instead of writing explicit zero buffers later in perform_cow(). > > iotest 060: > write

Re: [Qemu-block] [PATCH 2/2] avoid TABs in files that only contain a few

2018-12-14 Thread Eric Blake
On 12/13/18 4:37 PM, Paolo Bonzini wrote: Most files that have TABs only contain a handful of them. Change them to spaces so that we don't confuse people. disas, standard-headers, linux-headers and libdecnumber are imported from other projects and probably should be exempted from the check. Out

Re: [Qemu-block] [PATCH] qemu: avoid memory leak while remove disk

2018-12-14 Thread Michael S. Tsirkin
On Fri, Dec 07, 2018 at 09:53:06AM +0800, wangjian wrote: > Memset vhost_dev to zero in the vhost_dev_cleanup function. > This causes dev.vqs to be NULL, so that > vqs does not free up space when calling the g_free function. > This will result in a memory leak. But you can't release vqs > directly

Re: [Qemu-block] [Qemu-devel] [PATCH v2 6/7] iotests: allow pretty-print for qmp_log

2018-12-14 Thread John Snow
On 12/13/18 8:09 AM, Vladimir Sementsov-Ogievskiy wrote: > 13.12.2018 4:50, John Snow wrote: >> If iotests have lines exceeding >998 characters long, git doesn't >> want to send it plaintext to the list. We can solve this by allowing >> the iotests to use pretty printed QMP output that we can ma

[Qemu-block] [PATCH v3 0/7] bitmaps: remove x- prefix from QMP api

2018-12-14 Thread John Snow
Touch up a few last things and remove the x- prefix. V3: - Reworked qmp_log to pretty-print the outgoing command, too [Vladimir] - Modified test to log only bitmap information [Vladimir] - Test disable/enable transaction toggle [Eric] Note that the filter I added is now unused, but I think we

[Qemu-block] [PATCH v3 4/7] iotests.py: don't abort if IMGKEYSECRET is undefined

2018-12-14 Thread John Snow
Instead of using os.environ[], use .get with a default of empty string to match the setup in check to allow us to import the iotests module (for debugging, say) without needing a crafted environment just to import the module. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Vladimir

[Qemu-block] [PATCH v3 1/7] blockdev: abort transactions in reverse order

2018-12-14 Thread John Snow
Presently, we abort transactions in the same order they were processed in. Bitmap commands, though, attempt to restore backup data structures on abort. To that end, though, they need to be aborted in reverse chronological order. Replace the QSIMPLEQ data structure with a QTAILQ one, so we can iter

[Qemu-block] [PATCH v3 2/7] blockdev: n-ary bitmap merge

2018-12-14 Thread John Snow
Especially outside of transactions, it is helpful to provide all-or-nothing semantics for bitmap merges. This facilitates the coalescing of multiple bitmaps into a single target for the "checkpoint" interpretation when assembling bitmaps that represent arbitrary points in time from component bitmap

[Qemu-block] [PATCH v3 6/7] iotests: allow pretty-print for qmp_log

2018-12-14 Thread John Snow
If iotests have lines exceeding >998 characters long, git doesn't want to send it plaintext to the list. We can solve this by allowing the iotests to use pretty printed QMP output that we can match against instead. As a bonus, it's much nicer for human eyes, too. Note that this changes the sort o

[Qemu-block] [PATCH v3 3/7] block: remove 'x' prefix from experimental bitmap APIs

2018-12-14 Thread John Snow
The 'x' prefix was added because I was uncertain of the direction we'd take for the libvirt API. With the general approach solidified, I feel comfortable committing to this API for 4.0. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- blockdev.c

[Qemu-block] [PATCH v3 7/7] iotests: add iotest 236 for testing bitmap merge

2018-12-14 Thread John Snow
New interface, new smoke test. --- tests/qemu-iotests/236 | 124 +++ tests/qemu-iotests/236.out | 200 + tests/qemu-iotests/group | 1 + 3 files changed, 325 insertions(+) create mode 100755 tests/qemu-iotests/236 create mode 100644

[Qemu-block] [PATCH v3 5/7 RESEND] iotests: add filter_generated_node_ids

2018-12-14 Thread John Snow
To mimic the common filter of the same name, but for the python tests. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-