[PATCH v3 1/2] hw/nvme: fix endianess conversion and add controller list

2021-06-14 Thread Gollu Appalanaidu
Add the controller identifiers list CNS 0x13, available list of ctrls in NVM Subsystem that may or may not be attached to namespaces. In Identify Ctrl List of the CNS 0x12 and 0x13 no endian conversion for the nsid field. These two CNS values shows affect when there exists a Subsystem. Added cond

[PATCH v3 2/2] hw/nvme: documentation fix

2021-06-14 Thread Gollu Appalanaidu
In the documentation of the '-detached' param "be" and "not" has been used side by side, fix that. Signed-off-by: Gollu Appalanaidu --- hw/nvme/ctrl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index d513b022c4..21883e4b3c 100644 --- a/hw/

Reporting absent extents in qcow2 image

2021-06-14 Thread Nir Soffer
oVirt started to use now qemu:allocation-depth meta context. In the past, we use base:allocation and reported NBD_STATE_HOLE as a hole, and this broke in qemu 6.0.0. Now we have NBD_STATE_HOLE from base:allocation, and flags == 0 from qemu:allocation-depth. We map flags == 0 to EXTENT_BACKING (1

[PATCH v3 2/2] hw/nvme: default for namespace EUI-64

2021-06-14 Thread Heinrich Schuchardt
On machines with version > 6.0 replace a missing EUI-64 by a generated value. Signed-off-by: Heinrich Schuchardt --- v3: use 52-54-00-00-00-00-00-00 as starting values for generating EUI-64s v2: new patch --- docs/system/nvme.rst | 2 ++ hw/core/machine.c| 1 + hw/nvm

[PATCH v3 0/2] hw/nvme: namespace parameter for EUI-64

2021-06-14 Thread Heinrich Schuchardt
The EUI-64 field is the only identifier for NVMe namespaces in UEFI device paths. Add a new namespace property "eui64", that provides the user the option to specify the EUI-64. v3: use 52-54-00-00-00-00-00-00 as starting values for generating EUI-64s Heinrich Schuchardt (2): hw/

[PATCH v3 1/2] hw/nvme: namespace parameter for EUI-64

2021-06-14 Thread Heinrich Schuchardt
The EUI-64 field is the only identifier for NVMe namespaces in UEFI device paths. Add a new namespace property "eui64", that provides the user the option to specify the EUI-64. Signed-off-by: Heinrich Schuchardt Acked-by: Klaus Jensen --- v3: no change v2: fix typo %s/EUI64/EUI-6

Re: [PATCH v2 3/3] hw/nvme: default for namespace EUI-64

2021-06-14 Thread Heinrich Schuchardt
On 6/14/21 9:23 PM, Heinrich Schuchardt wrote: On 6/14/21 8:31 PM, Klaus Jensen wrote: On Jun 12 01:46, Heinrich Schuchardt wrote: On machines with version > 6.0 replace a missing EUI-64 by a generated value. Signed-off-by: Heinrich Schuchardt --- v2: new patch --- docs/system/nvme.rst |

Re: [PATCH v2 3/3] hw/nvme: default for namespace EUI-64

2021-06-14 Thread Heinrich Schuchardt
On 6/14/21 8:31 PM, Klaus Jensen wrote: On Jun 12 01:46, Heinrich Schuchardt wrote: On machines with version > 6.0 replace a missing EUI-64 by a generated value. Signed-off-by: Heinrich Schuchardt --- v2: new patch --- docs/system/nvme.rst | 2 ++ hw/core/machine.c    | 1 + hw/nvme/ns.c

Re: [PULL 00/13] Misc fixes patches

2021-06-14 Thread Peter Maydell
On Mon, 14 Jun 2021 at 15:18, Daniel P. Berrangé wrote: > > The following changes since commit 894fc4fd670aaf04a67dc7507739f914ff4bacf2: > > Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into > staging (2021-06-11 09:21:48 +0100) > > are available in the Git repository a

Re: [PATCH v2 2/3] hw/nvme: namespace parameter for EUI-64

2021-06-14 Thread Keith Busch
On Sat, Jun 12, 2021 at 01:46:30AM +0200, Heinrich Schuchardt wrote: > +``eui64`` > + Set the EUI-64 of the namespace. This will be reported as a "IEEE Extended > + Unique Identifier" descriptor in the Namespace Identification Descriptor > List. This needs to match Identify Namespace's EUI64 fi

Re: [PATCH v2 2/3] hw/nvme: namespace parameter for EUI-64

2021-06-14 Thread Klaus Jensen
On Jun 14 11:48, Keith Busch wrote: On Sat, Jun 12, 2021 at 01:46:30AM +0200, Heinrich Schuchardt wrote: +``eui64`` + Set the EUI-64 of the namespace. This will be reported as a "IEEE Extended + Unique Identifier" descriptor in the Namespace Identification Descriptor List. This needs to mat

Re: [PATCH] block: fix build waring

2021-06-14 Thread Eric Blake
On Sun, Jun 13, 2021 at 05:38:21PM +0800, Zhiwei Jiang wrote: In the subject, s/waring/warning/ > when i compile this file with some error message > ../block.c: In function ‘bdrv_replace_node_common’: > ../block.c:4903:9: error: ‘to_cow_parent’ may be used > uninitialized in this function [-Werro

Re: [PATCH v2 3/3] hw/nvme: default for namespace EUI-64

2021-06-14 Thread Klaus Jensen
On Jun 12 01:46, Heinrich Schuchardt wrote: On machines with version > 6.0 replace a missing EUI-64 by a generated value. Signed-off-by: Heinrich Schuchardt --- v2: new patch --- docs/system/nvme.rst | 2 ++ hw/core/machine.c| 1 + hw/nvme/ns.c | 9 + hw/nvme/nvme.h

Re: [PATCH v2] hw/nvme: be more careful when deasserting IRQs

2021-06-14 Thread Klaus Jensen
On Jun 14 15:54, Jakub Jermář wrote: An IRQ vector used by a completion queue cannot be deasserted without first checking if the same vector does not need to stay asserted for some other completion queue. To this end the controller structure is extended by a counter of asserted completion queues.

Re: [PATCH] block/nvme: Fix VFIO_MAP_DMA failed: No space left on device

2021-06-14 Thread Michal Prívozník
On 6/14/21 6:03 PM, Philippe Mathieu-Daudé wrote: > On 6/11/21 1:46 PM, Philippe Mathieu-Daudé wrote: >> When the NVMe block driver was introduced (see commit bdd6a90a9e5, >> January 2018), Linux VFIO_IOMMU_MAP_DMA ioctl was only returning >> -ENOMEM in case of error. The driver was correctly handl

Re: [PATCH v2 02/18] qapi: add ModuleInfo schema

2021-06-14 Thread Markus Armbruster
Gerd Hoffmann writes: > Hi, > >> > +# @arch: module architecture. >> >> Semantics? >> >> Should this be enum SysEmuTarget? > > Probably, will check ... > >> > +# @opts: qemu opts implemented by module. >> >> Is this the name of a QemuOptsList? >> >> Since this isn't a list, a module can onl

Re: [PATCH v2 1/3] hw: virt: consider hw_compat_6_0

2021-06-14 Thread Heinrich Schuchardt
On 6/12/21 1:46 AM, Heinrich Schuchardt wrote: virt-6.0 must consider hw_compat_6_0. Fixes: da7e13c00b59 ("hw: add compat machines for 6.1") Signed-off-by: Heinrich Schuchardt Reviewed-by: Cornelia Huck This patch was applied to target-arm.next by Peter. --- v2: add missing Fixes:

Re: [PATCH 3/7] net/rocker: use GDateTime for formatting timestamp in debug messages

2021-06-14 Thread Juan Quintela
Daniel P. Berrangé wrote: > ping: anyone willing to give a review of this one > > On Wed, May 05, 2021 at 11:36:58AM +0100, Daniel P. Berrangé wrote: >> The GDateTime APIs provided by GLib avoid portability pitfalls, such >> as some platforms where 'struct timeval.tv_sec' field is still 'long' >>

Re: [PATCH] block/nvme: Fix VFIO_MAP_DMA failed: No space left on device

2021-06-14 Thread Philippe Mathieu-Daudé
On 6/11/21 1:46 PM, Philippe Mathieu-Daudé wrote: > When the NVMe block driver was introduced (see commit bdd6a90a9e5, > January 2018), Linux VFIO_IOMMU_MAP_DMA ioctl was only returning > -ENOMEM in case of error. The driver was correctly handling the > error path to recycle its volatile IOVA mappi

Re: [PATCH 5/7] io: use GDateTime for formatting timestamp for websock headers

2021-06-14 Thread Dr. David Alan Gilbert
* Daniel P. Berrangé (berra...@redhat.com) wrote: > ping: anyone willing to review this > Reviewed-by: Dr. David Alan Gilbert > On Wed, May 05, 2021 at 11:37:00AM +0100, Daniel P. Berrangé wrote: > > The GDateTime APIs provided by GLib avoid portability pitfalls, such > > as some platforms wher

Re: [PATCH] block/nvme: Fix VFIO_MAP_DMA failed: No space left on device

2021-06-14 Thread Michal Prívozník
On 6/11/21 1:46 PM, Philippe Mathieu-Daudé wrote: > When the NVMe block driver was introduced (see commit bdd6a90a9e5, > January 2018), Linux VFIO_IOMMU_MAP_DMA ioctl was only returning > -ENOMEM in case of error. The driver was correctly handling the > error path to recycle its volatile IOVA mappi

Re: [PATCH v2 02/18] qapi: add ModuleInfo schema

2021-06-14 Thread Gerd Hoffmann
Hi, > > +# @arch: module architecture. > > Semantics? > > Should this be enum SysEmuTarget? Probably, will check ... > > +# @opts: qemu opts implemented by module. > > Is this the name of a QemuOptsList? > > Since this isn't a list, a module can only ever provide one > QemuOptsList. Sure

Re: [PATCH] hw/nvme: be more careful when deasserting IRQs

2021-06-14 Thread Jakub Jermář
On 6/10/21 8:14 PM, Klaus Jensen wrote: On Jun 10 13:46, Jakub Jermář wrote: An IRQ vector used by a completion queue cannot be deasserted without first checking if the same vector does not need to stay asserted for some other completion queue. Signed-off-by: Jakub Jermar --- hw/nvme/ctrl.c |

[PATCH 4/4] iotests/308: Test allow-other

2021-06-14 Thread Max Reitz
We cannot reasonably test the main point of allow-other, which is to allow users other than the current one to access the FUSE export, because that would require access to sudo, which this test most likely will not have. (Also, we would need to figure out some user/group that is on the machine and

[PATCH v2] hw/nvme: be more careful when deasserting IRQs

2021-06-14 Thread Jakub Jermář
An IRQ vector used by a completion queue cannot be deasserted without first checking if the same vector does not need to stay asserted for some other completion queue. To this end the controller structure is extended by a counter of asserted completion queues. To prevent incrementing the counter f

[PATCH 1/4] export/fuse: Add allow-other option

2021-06-14 Thread Max Reitz
Without the allow_other mount option, no user (not even root) but the one who started qemu/the storage daemon can access the export. Allow users to configure the export such that such accesses are possible. When we do pass allow_other, we should also pass default_permissions, because our export c

[PATCH 2/4] export/fuse: Give SET_ATTR_SIZE its own branch

2021-06-14 Thread Max Reitz
In order to support changing other attributes than the file size in fuse_setattr(), we have to give each its own independent branch. This also applies to the only attribute we do support right now. Signed-off-by: Max Reitz --- block/export/fuse.c | 20 +++- 1 file changed, 11 in

[PATCH 3/4] export/fuse: Let permissions be adjustable

2021-06-14 Thread Max Reitz
Allow changing the file mode, UID, and GID through SETATTR. This only really makes sense with allow-other, though (because without it, the effective access mode is fixed to be 0600 (u+rw) with qemu's user being the file's owner), so changing these stat fields is not allowed without allow-other. S

[PATCH 0/4] export/fuse: Allow other users access to the export

2021-06-14 Thread Max Reitz
Hi, With the default mount options, FUSE mounts are not accessible to any users but the one who did the mount, not even to root. To allow such accesses, allow_other must be passed. This is probably useful to some people (it certainly is to me, e.g. when exporting some image as my normal user, an

[PULL 13/13] usb/dev-mtp: use GDateTime for formatting timestamp for objects

2021-06-14 Thread Daniel P . Berrangé
The GDateTime APIs provided by GLib avoid portability pitfalls, such as some platforms where 'struct timeval.tv_sec' field is still 'long' instead of 'time_t'. When combined with automatic cleanup, GDateTime often results in simpler code too. Reviewed-by: Gerd Hoffmann Signed-off-by: Daniel P. Be

[PULL 07/13] softmmu: add trace point when bdrv_flush_all fails

2021-06-14 Thread Daniel P . Berrangé
The VM stop process has to flush outstanding I/O and this is a critical failure scenario that is hard to diagnose. Add a probe point that records the flush return code. Reviewed-by: Connor Kuehl Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé --- softmmu/cpus.c | 7

[PULL 06/13] migration: add trace point when vm_stop_force_state fails

2021-06-14 Thread Daniel P . Berrangé
This is a critical failure scenario for migration that is hard to diagnose from existing probes. Most likely it is caused by an error from bdrv_flush(), but we're not logging the errno anywhere, hence this new probe. Reviewed-by: Connor Kuehl Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Da

[PULL 11/13] migration: use GDateTime for formatting timestamp in snapshot names

2021-06-14 Thread Daniel P . Berrangé
The GDateTime APIs provided by GLib avoid portability pitfalls, such as some platforms where 'struct timeval.tv_sec' field is still 'long' instead of 'time_t'. When combined with automatic cleanup, GDateTime often results in simpler code too. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dan

[PULL 12/13] block: use GDateTime for formatting timestamp when dumping snapshot info

2021-06-14 Thread Daniel P . Berrangé
The GDateTime APIs provided by GLib avoid portability pitfalls, such as some platforms where 'struct timeval.tv_sec' field is still 'long' instead of 'time_t'. When combined with automatic cleanup, GDateTime often results in simpler code too. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berran

[PULL 10/13] block: remove duplicate trace.h include

2021-06-14 Thread Daniel P . Berrangé
Reviewed-by: Connor Kuehl Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé --- block/file-posix.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/block/file-posix.c b/block/file-posix.c index 4189b2bfa6..b3fbb9bd63 100644 --- a/block/file-posix.c +++ b/block/file-posix

[PULL 09/13] block: add trace point when fdatasync fails

2021-06-14 Thread Daniel P . Berrangé
A flush failure is a critical failure scenario for some operations. For example, it will prevent migration from completing, as it will make vm_stop() report an error. Thus it is important to have a trace point present for debugging. Reviewed-by: Connor Kuehl Reviewed-by: Dr. David Alan Gilbert S

[PULL 08/13] block: preserve errno from fdatasync failures

2021-06-14 Thread Daniel P . Berrangé
When fdatasync() fails on a file backend we set a flag that short-circuits any future attempts to call fdatasync(). The first failure returns the true errno, but the later short- circuited calls return a generic EIO. The latter is unhelpful because fdatasync() can return a variety of errnos, includ

[PULL 03/13] docs: document usage of the authorization framework

2021-06-14 Thread Daniel P . Berrangé
The authorization framework provides a way to control access to network services after a client has been authenticated. This documents how to actually use it. Reviewed-by: Marc-André Lureau Signed-off-by: Daniel P. Berrangé --- docs/system/authz.rst | 263 +++

[PULL 02/13] docs: document how to pass secret data to QEMU

2021-06-14 Thread Daniel P . Berrangé
Reviewed-by: Marc-André Lureau Signed-off-by: Daniel P. Berrangé --- docs/system/index.rst | 1 + docs/system/secrets.rst | 162 2 files changed, 163 insertions(+) create mode 100644 docs/system/secrets.rst diff --git a/docs/system/index.rst b/docs/

[PULL 05/13] sasl: remove comment about obsolete kerberos versions

2021-06-14 Thread Daniel P . Berrangé
This is not relevant to any OS distro that QEMU currently targets. Reviewed-by: Marc-André Lureau Signed-off-by: Daniel P. Berrangé --- qemu.sasl | 4 1 file changed, 4 deletions(-) diff --git a/qemu.sasl b/qemu.sasl index abdfc686be..851acc7e8f 100644 --- a/qemu.sasl +++ b/qemu.sasl @@ -

[PULL 04/13] docs: recommend SCRAM-SHA-256 SASL mech instead of SHA-1 variant

2021-06-14 Thread Daniel P . Berrangé
The SHA-256 variant better meats modern security expectations. Also warn that the password file is storing entries in clear text. Reviewed-by: Marc-André Lureau Signed-off-by: Daniel P. Berrangé --- docs/system/vnc-security.rst | 7 --- qemu.sasl| 11 ++- 2 file

[PULL 01/13] docs: add table of contents to QAPI references

2021-06-14 Thread Daniel P . Berrangé
The QAPI reference docs for the guest agent, storage daemon and QMP are all rather long and hard to navigate unless you already know the name of the command and can do full text search for it. A table of contents in each doc will help people locate stuff much more easily. Reviewed-by: Connor Kueh

[PULL 00/13] Misc fixes patches

2021-06-14 Thread Daniel P . Berrangé
The following changes since commit 894fc4fd670aaf04a67dc7507739f914ff4bacf2: Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2021-06-11 09:21:48 +0100) are available in the Git repository at: https://gitlab.com/berrange/qemu tags/misc-fixes-pull-request

Re: [PATCH v2 17/18] modules: check arch and block load on mismatch

2021-06-14 Thread Gerd Hoffmann
Hi, > > I'm not particularly attached to using json for that, it is just that > > we already have infrastructure to parse/serialize structs from/to json > > because we need that for qapi anyway. > > If we can generate json, we could generate .c code which has all the > data statically declared

Re: [PATCH v2 00/18] modules: add metadata database

2021-06-14 Thread Gerd Hoffmann
Hi, > Based on the information you provided, the reason this whole series > exists seems to be to remove the need to update modules in multiple > places. Well, I'm trying to improve the way we handle module meta-data (see other mail just send for details). > Is there a real need to copy over th

Re: [PATCH 2/2] nbd: Add new qemu:joint-allocation metadata context

2021-06-14 Thread Nir Soffer
On Mon, Jun 14, 2021 at 4:56 PM Eric Blake wrote: > > On Sat, Jun 12, 2021 at 02:39:44AM +0300, Nir Soffer wrote: > > Since this change is not simple, and the chance that we also get the dirty > > bitmap included in the result seems to be very low, I decided to check the > > direction of merging m

Re: [PATCH 2/2] nbd: Add new qemu:joint-allocation metadata context

2021-06-14 Thread Eric Blake
On Sat, Jun 12, 2021 at 02:39:44AM +0300, Nir Soffer wrote: > Since this change is not simple, and the chance that we also get the dirty > bitmap included in the result seems to be very low, I decided to check the > direction of merging multiple extents. > > I started with merging "base:allocation

Re: [PATCH v2 17/18] modules: check arch and block load on mismatch

2021-06-14 Thread Daniel P . Berrangé
On Mon, Jun 14, 2021 at 03:44:53PM +0200, Gerd Hoffmann wrote: > Hi, > > > > We can do both. The file is generated and installed as part of the > > > build/install process, and it can be simply used as-is even if some of > > > the modules are missing. > > > > > > It's also possible to update t

Re: [PATCH v2 17/18] modules: check arch and block load on mismatch

2021-06-14 Thread Gerd Hoffmann
Hi, > > We can do both. The file is generated and installed as part of the > > build/install process, and it can be simply used as-is even if some of > > the modules are missing. > > > > It's also possible to update the modinfo.json file in postinstall / > > postuninstall by simply running qem

Re: [PATCH] block: fix build waring

2021-06-14 Thread elish . jiang
gcc version 8.3.1 20190311 (Red Hat 8.3.1-3) (GCC)

Re: [PATCH v2] async: the main AioContext is only "current" if under the BQL

2021-06-14 Thread Paolo Bonzini
On 11/06/21 13:13, Paolo Bonzini wrote: The commit message doesn't specify, but in the buggy case, are we talking about calling aio_co_wake() for a coroutine in the main context specifically, right? Could we have a unit test for this scenario? Yes, that's the scenario.  I will try to write a un

Re: [PATCH v2 00/18] modules: add metadata database

2021-06-14 Thread Claudio Fontana
On 6/10/21 11:54 AM, Gerd Hoffmann wrote: > On Thu, Jun 10, 2021 at 10:32:39AM +0200, Claudio Fontana wrote: >> On 6/10/21 7:57 AM, Gerd Hoffmann wrote: >>> This patch series adds support for module metadata. Here are the pieces >>> of the puzzle: >>> >>> (1) Macros are added to store metadata i

Re: [PATCH v2 17/18] modules: check arch and block load on mismatch

2021-06-14 Thread Claudio Fontana
On 6/10/21 4:03 PM, Gerd Hoffmann wrote: > Hi, > >> Is the JSON file completely static, listing all modules that were built >> regardless of whether they are currently installed, or would it need to >> be refreshed when installing/uninstalling RPMs with modules ? I would >> think we can do the f

Re: [PATCH v2 17/18] modules: check arch and block load on mismatch

2021-06-14 Thread Claudio Fontana
On 6/10/21 2:35 PM, Daniel P. Berrangé wrote: > On Thu, Jun 10, 2021 at 07:57:54AM +0200, Gerd Hoffmann wrote: >> Add module_allow_arch() to set the target architecture. >> In case a module is limited to some arch verify arches >> match and ignore the module if not. >> >> Signed-off-by: Gerd Hoffma

Re: [PATCH v5 06/16] qemu-iotests: delay QMP socket timers

2021-06-14 Thread Emanuele Giuseppe Esposito
On 04/06/2021 11:17, Emanuele Giuseppe Esposito wrote: Attaching gdbserver implies that the qmp socket should wait indefinitely for an answer from QEMU. For Timeout class, create a @contextmanager that switches Timeout with NoTimeout (empty context manager) so that if --gdb is set, no timeout

Re: [PATCH v2 1/2] hw/nvme: fix endianess conversion and add controller list

2021-06-14 Thread Klaus Jensen
On Jun 13 17:29, Gollu Appalanaidu wrote: On Wed, Jun 09, 2021 at 10:22:49PM +0200, Klaus Jensen wrote: On Jun 1 20:32, Gollu Appalanaidu wrote: Add the controller identifiers list CNS 0x13, available list of ctrls in NVM Subsystem that may or may not be attached to namespaces. In Identify Ct

Re: [PATCH v5 00/16] qemu_iotests: improve debugging options

2021-06-14 Thread Emanuele Giuseppe Esposito
On 04/06/2021 11:17, Emanuele Giuseppe Esposito wrote: This series adds the option to attach gdbserver and valgrind to the QEMU binary running in qemu_iotests. It also allows to redirect QEMU binaries output of the python tests to the stdout, instead of a log file. Patches 1-9 introduce the -

[PATCH v5 6/6] blkdebug: protect rules and suspended_reqs with a lock

2021-06-14 Thread Emanuele Giuseppe Esposito
First, categorize the structure fields to identify what needs to be protected and what doesn't. We essentially need to protect only .state, and the 3 lists in BDRVBlkdebugState. Then, add the lock and mark the functions accordingly. Co-developed-by: Paolo Bonzini Signed-off-by: Emanuele Giusepp

[PATCH v5 4/6] blkdebug: do not suspend in the middle of QLIST_FOREACH_SAFE

2021-06-14 Thread Emanuele Giuseppe Esposito
That would be unsafe in case a rule other than the current one is removed while the coroutine has yielded. Keep FOREACH_SAFE because suspend_request deletes the current rule. After this patch, *all* matching rules are deleted before suspending the coroutine, rather than just one. This doesn't affe

[PATCH v5 1/6] blkdebug: refactor removal of a suspended request

2021-06-14 Thread Emanuele Giuseppe Esposito
Extract to a separate function. Do not rely on FOREACH_SAFE, which is only "safe" if the *current* node is removed---not if another node is removed. Instead, just walk the entire list from the beginning when asked to resume all suspended requests with a given tag. Co-developed-by: Paolo Bonzini

[PATCH v5 5/6] block/blkdebug: remove new_state field and instead use a local variable

2021-06-14 Thread Emanuele Giuseppe Esposito
There seems to be no benefit in using a field. Replace it with a local variable, and move the state update before the yields. The state update has do be done before the yields because now using a local variable does not allow the new updated state to be visible by the other yields. Signed-off-by:

[PATCH v5 3/6] blkdebug: track all actions

2021-06-14 Thread Emanuele Giuseppe Esposito
Add a counter for each action that a rule can trigger. This is mainly used to keep track of how many coroutine_yield() we need to perform after processing all rules in the list. Co-developed-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Vladimir Sementsov-Ogievskiy --

[PATCH v5 2/6] blkdebug: move post-resume handling to resume_req_by_tag

2021-06-14 Thread Emanuele Giuseppe Esposito
We want to move qemu_coroutine_yield() after the loop on rules, because QLIST_FOREACH_SAFE is wrong if the rule list is modified while the coroutine has yielded. Therefore move the suspended request to the heap and clean it up from the remove side. All that is left is for blkdebug_debug_event to h

[PATCH v5 0/6] blkdebug: fix racing condition when iterating on

2021-06-14 Thread Emanuele Giuseppe Esposito
When qemu_coroutine_enter is executed in a loop (even QEMU_FOREACH_SAFE), the new routine can modify the list, for example removing an element, causing problem when control is given back to the caller that continues iterating on the same list. Patch 1 solves the issue in blkdebug_debug_resume by

Re: [PATCH v3 0/5] block-copy: make helper APIs thread safe

2021-06-14 Thread Emanuele Giuseppe Esposito
Please discard this thread, I had an issue with git send-email and patch 3-5 are missing. Thank you, Emanuele On 14/06/2021 10:08, Emanuele Giuseppe Esposito wrote: This serie of patches bring thread safety to the smaller APIs used by block-copy, namely ratelimit, progressmeter, co-shared-reso

[PATCH v3 0/5] block-copy: make helper APIs thread safe

2021-06-14 Thread Emanuele Giuseppe Esposito
This serie of patches bring thread safety to the smaller APIs used by block-copy, namely ratelimit, progressmeter, co-shared-resource and aiotask. The end goal is to reduce the usage of AioContexlock in block-copy, by introducing smaller granularity locks thus on making the block layer thread safe.

[PATCH v3 2/5] block-copy: let ratelimit handle a speed of 0

2021-06-14 Thread Emanuele Giuseppe Esposito
From: Paolo Bonzini Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- block/block-copy.c | 28 +++- 1 file changed, 11 insertions(+), 17 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.

[PATCH v3 1/5] ratelimit: treat zero speed as unlimited

2021-06-14 Thread Emanuele Giuseppe Esposito
From: Paolo Bonzini Both users of RateLimit, block-copy.c and blockjob.c, treat a speed of zero as unlimited, while RateLimit treats it as "as slow as possible". The latter is nicer from the code point of view but pretty useless, so disable rate limiting if a speed of zero is provided. Reviewed

Re: [PATCH v3 0/5] block-copy: make helper APIs thread safe

2021-06-14 Thread Emanuele Giuseppe Esposito
On 14/06/2021 10:11, Emanuele Giuseppe Esposito wrote: This serie of patches bring thread safety to the smaller APIs used by block-copy, namely ratelimit, progressmeter, co-shared-resource and aiotask. The end goal is to reduce the usage of AioContexlock in block-copy, by introducing smaller g

[PATCH v3 5/5] co-shared-resource: protect with a mutex

2021-06-14 Thread Emanuele Giuseppe Esposito
co-shared-resource is currently not thread-safe, as also reported in co-shared-resource.h. Add a QemuMutex because co_try_get_from_shres can also be invoked from non-coroutine context. Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/co-shared

[PATCH v3 0/5] block-copy: make helper APIs thread safe

2021-06-14 Thread Emanuele Giuseppe Esposito
This serie of patches bring thread safety to the smaller APIs used by block-copy, namely ratelimit, progressmeter, co-shared-resource and aiotask. The end goal is to reduce the usage of AioContexlock in block-copy, by introducing smaller granularity locks thus on making the block layer thread safe.

[PATCH v3 3/5] blockjob: let ratelimit handle a speed of 0

2021-06-14 Thread Emanuele Giuseppe Esposito
From: Paolo Bonzini Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- blockjob.c | 13 +++-- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/blockjob.c b/blockjob.c index dc1d9e0e46..22e5bb9b1f 100644 --

[PATCH v3 4/5] progressmeter: protect with a mutex

2021-06-14 Thread Emanuele Giuseppe Esposito
Progressmeter is protected by the AioContext mutex, which is taken by the block jobs and their caller (like blockdev). We would like to remove the dependency of block layer code on the AioContext mutex, since most drivers and the core I/O code are already not relying on it. Create a new C file to

[PATCH v3 1/5] ratelimit: treat zero speed as unlimited

2021-06-14 Thread Emanuele Giuseppe Esposito
From: Paolo Bonzini Both users of RateLimit, block-copy.c and blockjob.c, treat a speed of zero as unlimited, while RateLimit treats it as "as slow as possible". The latter is nicer from the code point of view but pretty useless, so disable rate limiting if a speed of zero is provided. Reviewed

[PATCH v3 2/5] block-copy: let ratelimit handle a speed of 0

2021-06-14 Thread Emanuele Giuseppe Esposito
From: Paolo Bonzini Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe Esposito --- block/block-copy.c | 28 +++- 1 file changed, 11 insertions(+), 17 deletions(-) diff --git a/block/block-copy.c b/block/block-copy.

Re: [PATCH v2 02/18] qapi: add ModuleInfo schema

2021-06-14 Thread Markus Armbruster
Gerd Hoffmann writes: > Add QAPI schema for the module info database. > > Signed-off-by: Gerd Hoffmann > --- > qapi/meson.build | 1 + > qapi/modules.json | 36 > qapi/qapi-schema.json | 1 + > 3 files changed, 38 insertions(+) > create mode 1006

[PATCH v4 4/6] block-copy: move progress_set_remaining in block_copy_task_end

2021-06-14 Thread Emanuele Giuseppe Esposito
Moving this function in task_end ensures to update the progress anyways, even if there is an error. It also helps in next patch, allowing task_end to have only one critical section. Reviewed-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Emanuele Giuseppe Esposito --- block/block-copy.c | 6 +

[PATCH v4 6/6] block-copy: atomic .cancelled and .finished fields in BlockCopyCallState

2021-06-14 Thread Emanuele Giuseppe Esposito
By adding acquire/release pairs, we ensure that .ret and .error_is_read fields are written by block_copy_dirty_clusters before .finished is true. The atomic here are necessary because the fields are concurrently modified also outside coroutines. Signed-off-by: Emanuele Giuseppe Esposito --- blo

[PATCH v4 5/6] block-copy: add a CoMutex

2021-06-14 Thread Emanuele Giuseppe Esposito
Add a CoMutex to protect concurrent access of block-copy data structures. This mutex also protects .copy_bitmap, because its thread-safe API does not prevent it from assigning two tasks to the same bitmap region. .finished, .cancelled and reads to .ret and .error_is_read will be protected in the

[PATCH v4 2/6] block-copy: streamline choice of copy_range vs. read/write

2021-06-14 Thread Emanuele Giuseppe Esposito
From: Paolo Bonzini Put the logic to determine the copy size in a separate function, so that there is a simple state machine for the possible methods of copying data from one BlockDriverState to the other. Use .method instead of .copy_range as in-out argument, and include also .zeroes as an addi

[PATCH v4 3/6] block-copy: improve comments of BlockCopyTask and BlockCopyState types and functions

2021-06-14 Thread Emanuele Giuseppe Esposito
As done in BlockCopyCallState, categorize BlockCopyTask and BlockCopyState in IN, State and OUT fields. This is just to understand which field has to be protected with a lock. .sleep_state is handled in the series "coroutine: new sleep/wake API" and thus here left as TODO. Signed-off-by: Emanuele

[PATCH v4 1/6] block-copy: small refactor in block_copy_task_entry and block_copy_common

2021-06-14 Thread Emanuele Giuseppe Esposito
Use a local variable instead of referencing BlockCopyState through a BlockCopyCallState or BlockCopyTask every time. This is in preparation for next patches. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito --- block/block-copy.c | 14 -- 1 file changed, 8 in

[PATCH v4 0/6] block-copy: protect block-copy internal structures

2021-06-14 Thread Emanuele Giuseppe Esposito
This serie of patches aims to reduce the usage of the AioContexlock in block-copy, by introducing smaller granularity locks thus on making the block layer thread safe. This serie depends on my previous serie that brings thread safety to the smaller API used by block-copy, like ratelimit, progress