[PATCH v20 19/21] tests/avocado: s390x cpu topology dedicated errors

2023-04-25 Thread Pierre Morel
Let's test that QEMU refuses to setup a dedicated CPU with low or medium entitlement. Signed-off-by: Pierre Morel --- tests/avocado/s390_topology.py | 51 ++ 1 file changed, 51 insertions(+) diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topolog

[PATCH v20 11/21] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event

2023-04-25 Thread Pierre Morel
When the guest asks to change the polarization this change is forwarded to the upper layer using QAPI. The upper layer is supposed to take according decisions concerning CPU provisioning. Signed-off-by: Pierre Morel --- qapi/machine-target.json | 33 + hw/s390x/cp

[PATCH v2 2/3] pci: pci_add_option_rom(): refactor: use g_autofree for path variable

2023-04-25 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- hw/pci/pci.c | 8 +--- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 4a61c8d24a..a442f8fce1 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2292,7 +2292,7 @@ static void pci_add_option_rom(PCIDev

[PATCH v20 03/21] target/s390x/cpu topology: handle STSI(15) and build the SYSIB

2023-04-25 Thread Pierre Morel
On interception of STSI(15.1.x) the System Information Block (SYSIB) is built from the list of pre-ordered topology entries. Signed-off-by: Pierre Morel --- MAINTAINERS | 1 + include/hw/s390x/cpu-topology.h | 24 +++ include/hw/s390x/sclp.h | 1 + target/s390x/c

[PATCH v20 21/21] tests/avocado: s390x cpu topology query-cpu-polarization

2023-04-25 Thread Pierre Morel
This test verifies that the qmp query query-cpu-polarization reports correctly the polarization when the guest requests a polarization change. Signed-off-by: Pierre Morel --- tests/avocado/s390_topology.py | 33 + 1 file changed, 33 insertions(+) diff --git a/tes

[PATCH v20 05/21] s390x/cpu topology: resetting the Topology-Change-Report

2023-04-25 Thread Pierre Morel
During a subsystem reset the Topology-Change-Report is cleared by the machine. Let's ask KVM to clear the Modified Topology Change Report (MTCR) bit of the SCA in the case of a subsystem reset. Signed-off-by: Pierre Morel Reviewed-by: Thomas Huth --- include/hw/s390x/cpu-topology.h | 1 + targ

[PATCH v20 15/21] tests/avocado: s390x cpu topology polarisation

2023-04-25 Thread Pierre Morel
Polarization is changed on a request from the guest. Let's verify the polarization is accordingly set by QEMU. Signed-off-by: Pierre Morel --- tests/avocado/s390_topology.py | 38 ++ 1 file changed, 38 insertions(+) diff --git a/tests/avocado/s390_topology.py b/t

[PATCH v20 09/21] machine: adding s390 topology to query-cpu-fast

2023-04-25 Thread Pierre Morel
S390x provides two more topology attributes, entitlement and dedication. Let's add these CPU attributes to the QAPI command query-cpu-fast. Signed-off-by: Pierre Morel Reviewed-by: Nina Schoetterl-Glausch Reviewed-by: Thomas Huth --- qapi/machine.json | 9 - hw/core/machine-q

[PATCH v2 1/3] pci: pci_add_option_rom(): improve style

2023-04-25 Thread Vladimir Sementsov-Ogievskiy
Fix over-80 lines and missing curly brackets for if-operators, which are required by QEMU coding style. Signed-off-by: Vladimir Sementsov-Ogievskiy --- hw/pci/pci.c | 20 +++- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/hw/pci/pci.c b/hw/pci/pci.c index def5000

[PATCH v20 18/21] tests/avocado: s390x cpu topology test socket full

2023-04-25 Thread Pierre Morel
This test verifies that QMP set-cpu-topology does not accept to overload a socket. Signed-off-by: Pierre Morel --- tests/avocado/s390_topology.py | 27 +++ 1 file changed, 27 insertions(+) diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py index

[PATCH v20 16/21] tests/avocado: s390x cpu topology entitlement tests

2023-04-25 Thread Pierre Morel
This test takes care to check the changes on different entitlements when the guest requests a polarization change. Signed-off-by: Pierre Morel --- tests/avocado/s390_topology.py | 56 ++ 1 file changed, 56 insertions(+) diff --git a/tests/avocado/s390_topology.py

[PATCH v20 20/21] tests/avocado: s390x cpu topology bad move

2023-04-25 Thread Pierre Morel
This test verifies that QEMU refuses to move a CPU to an unexistant location. Signed-off-by: Pierre Morel --- tests/avocado/s390_topology.py | 25 + 1 file changed, 25 insertions(+) diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py index 37fffc

[PATCH v20 01/21] s390x/cpu topology: add s390 specifics to CPU topology

2023-04-25 Thread Pierre Morel
S390 adds two new SMP levels, drawers and books to the CPU topology. The S390 CPU have specific topology features like dedication and entitlement to give to the guest indications on the host vCPUs scheduling and help the guest take the best decisions on the scheduling of threads on the vCPUs. Let

[PATCH v20 13/21] docs/s390x/cpu topology: document s390x cpu topology

2023-04-25 Thread Pierre Morel
Add some basic examples for the definition of cpu topology in s390x. Signed-off-by: Pierre Morel --- MAINTAINERS| 2 + docs/devel/index-internals.rst | 1 + docs/devel/s390-cpu-topology.rst | 162 +++ docs/system/s390x/cpu-topology.rst | 240

[PATCH v20 04/21] s390x/sclp: reporting the maximum nested topology entries

2023-04-25 Thread Pierre Morel
The maximum nested topology entries is used by the guest to know how many nested topology are available on the machine. Let change the MNEST value from 2 to 4 in the SCLP READ INFO structure now that we support books and drawers. Signed-off-by: Pierre Morel Reviewed-by: Nina Schoetterl-Glausch

[PATCH v20 12/21] qapi/s390x/cpu topology: query-cpu-polarization qmp command

2023-04-25 Thread Pierre Morel
The query-cpu-polarization qmp command returns the current CPU polarization of the machine. Signed-off-by: Pierre Morel --- qapi/machine-target.json | 30 ++ hw/s390x/cpu-topology.c | 14 ++ 2 files changed, 44 insertions(+) diff --git a/qapi/machine-tar

[PATCH v20 00/21] s390x: CPU Topology

2023-04-25 Thread Pierre Morel
Hi, What is new: - Entitlement is now an enum and the default is "auto" Implementation discussions == CPU models -- Since the facility 11, S390_FEAT_CONFIGURATION_TOPOLOGY is already in the CPU model for old QEMU we could not activate it as usual from KVM but ne

[PATCH v20 14/21] tests/avocado: s390x cpu topology core

2023-04-25 Thread Pierre Morel
Introduction of the s390x cpu topology core functions and basic tests. We test the corelation between the command line and the QMP results in query-cpus-fast for various CPU topology. Signed-off-by: Pierre Morel --- MAINTAINERS| 1 + tests/avocado/s390_topology.py | 208 ++

[PATCH v20 08/21] qapi/s390x/cpu topology: set-cpu-topology qmp command

2023-04-25 Thread Pierre Morel
The modification of the CPU attributes are done through a monitor command. It allows to move the core inside the topology tree to optimize the cache usage in the case the host's hypervisor previously moved the CPU. The same command allows to modify the CPU attributes modifiers like polarization e

Re: [PATCH v3 00/20] block: remove aio_disable_external() API

2023-04-25 Thread Stefan Hajnoczi
On Thu, Apr 20, 2023 at 03:39:59PM +0200, Philippe Mathieu-Daudé wrote: > Hi Stefan, > > On 20/4/23 13:37, Stefan Hajnoczi wrote: > > v3: > > - Resend full patch series. v2 was sent in the middle of a git rebase and > > was > >missing patches. [Eric] > > - Apply Reviewed-by tags. > > > Based

Re: [PATCH v3 20/20] aio: remove aio_disable_external() API

2023-04-25 Thread Stefan Hajnoczi
On Thu, Apr 20, 2023 at 03:44:06PM +0200, Philippe Mathieu-Daudé wrote: > On 20/4/23 13:37, Stefan Hajnoczi wrote: > > All callers now pass is_external=false to aio_set_fd_handler() and > > aio_set_event_notifier(). The aio_disable_external() API that > > temporarily disables fd handlers that were

[PATCH v8 0/3] Eliminate multifd flush

2023-04-25 Thread Juan Quintela
Hi In this v8: - rebase over latests Please review. [v7] - Rebased to last upstream - Rename the capability to a property. So we move all the problems that we have on last review dissaper because it is not a capability. So now, it is works as expected. Enabled for old machine types, disable

[PATCH v8 1/3] multifd: Create property multifd-flush-after-each-section

2023-04-25 Thread Juan Quintela
We used to flush all channels at the end of each RAM section sent. That is not needed, so preparing to only flush after a full iteration through all the RAM. Default value of the property is false. But we return "true" in migrate_multifd_flush_after_each_section() until we implement the code in

[PATCH v8 3/3] multifd: Only flush once each full round of memory

2023-04-25 Thread Juan Quintela
We need to add a new flag to mean to flush at that point. Notice that we still flush at the end of setup and at the end of complete stages. Signed-off-by: Juan Quintela --- Add missing qemu_fflush(), now it passes all tests always. --- migration/migration.c | 6 +- migration/migration.h |

[PATCH v8 2/3] multifd: Protect multifd_send_sync_main() calls

2023-04-25 Thread Juan Quintela
We only need to do that on the ram_save_iterate() call on sending and on destination when we get a RAM_SAVE_FLAG_EOS. In setup() and complete() we need to synch in both new and old cases, so don't add a check there. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Signed-off-by:

Re: [PATCH v2 0/3] ROM migration

2023-04-25 Thread Vladimir Sementsov-Ogievskiy
On 25.04.23 19:14, Vladimir Sementsov-Ogievskiy wrote: Hi all! v2: simply ignore romfile on incoming migration when romsize is specified. Here I suggest a way to solve a problem, when we have existing running QEMU with old option ROM of small size and want to migrate to new environment where we

Re: [RFC PATCH v3 01/20] python: update pylint configuration

2023-04-25 Thread Daniel P . Berrangé
On Mon, Apr 24, 2023 at 04:02:29PM -0400, John Snow wrote: > Pylint 2.17.x decided that SocketAddrT was a bad name for a Type Alias for > some > reason. Sure, fine, whatever. > > Signed-off-by: John Snow > --- > python/setup.cfg | 1 + > 1 file changed, 1 insertion(+) Reviewed-by: Daniel P. Be

Re: [PATCH v3 13/20] block/export: rewrite vduse-blk drain code

2023-04-25 Thread Stefan Hajnoczi
On Fri, Apr 21, 2023 at 11:36:02AM +0800, Yongji Xie wrote: > Hi Stefan, > > On Thu, Apr 20, 2023 at 7:39 PM Stefan Hajnoczi wrote: > > > > vduse_blk_detach_ctx() waits for in-flight requests using > > AIO_WAIT_WHILE(). This is not allowed according to a comment in > > bdrv_set_aio_context_commit

Re: [RFC PATCH v3 09/20] tests/docker: add python3-venv dependency

2023-04-25 Thread Daniel P . Berrangé
On Mon, Apr 24, 2023 at 04:02:37PM -0400, John Snow wrote: > Several debian-based tests need the python3-venv dependency as a > consequence of Debian debundling the "ensurepip" module normally > included with Python. > > As mkvenv.py stands as of this commit, Debian requires EITHER: > > (A) setup

Re: [RFC PATCH v3 10/20] tests/vm: Configure netbsd to use Python 3.10

2023-04-25 Thread Daniel P . Berrangé
On Mon, Apr 24, 2023 at 04:02:38PM -0400, John Snow wrote: > NetBSD removes some packages from the Python stdlib, but only > re-packages them for Python 3.10. Switch to using Python 3.10. > > Signed-off-by: John Snow > --- > tests/vm/netbsd | 1 + > 1 file changed, 1 insertion(+) Reviewed-by: D

Re: [RFC PATCH v3 11/20] tests/vm: add py310-expat to NetBSD

2023-04-25 Thread Daniel P . Berrangé
On Mon, Apr 24, 2023 at 04:02:39PM -0400, John Snow wrote: > NetBSD cannot successfully run "ensurepip" without access to the pyexpat > module, which NetBSD debundles. Like the Debian patch, it would be > strictly faster long term to install pip/setuptools, and I recommend > developers at their wor

Re: test-blockjob: intermittent CI failures in msys2-64bit job

2023-04-25 Thread Hanna Czenczek
On 24.04.23 20:32, Vladimir Sementsov-Ogievskiy wrote: On 24.04.23 16:36, Emanuele Giuseppe Esposito wrote: Am 21/04/2023 um 12:13 schrieb Vladimir Sementsov-Ogievskiy: On 17.03.23 15:35, Thomas Huth wrote: On 17/03/2023 11.17, Peter Maydell wrote: On Mon, 6 Mar 2023 at 11:16, Peter Maydell

Re: [RFC PATCH v3 11/20] tests/vm: add py310-expat to NetBSD

2023-04-25 Thread John Snow
On Tue, Apr 25, 2023, 12:45 PM Daniel P. Berrangé wrote: > On Mon, Apr 24, 2023 at 04:02:39PM -0400, John Snow wrote: > > NetBSD cannot successfully run "ensurepip" without access to the pyexpat > > module, which NetBSD debundles. Like the Debian patch, it would be > > strictly faster long term t

Re: test-blockjob: intermittent CI failures in msys2-64bit job

2023-04-25 Thread Vladimir Sementsov-Ogievskiy
On 25.04.23 19:48, Hanna Czenczek wrote: On 24.04.23 20:32, Vladimir Sementsov-Ogievskiy wrote: On 24.04.23 16:36, Emanuele Giuseppe Esposito wrote: Am 21/04/2023 um 12:13 schrieb Vladimir Sementsov-Ogievskiy: On 17.03.23 15:35, Thomas Huth wrote: On 17/03/2023 11.17, Peter Maydell wrote:

Re: [RFC PATCH v3 00/20] configure: create a python venv and ensure meson, sphinx

2023-04-25 Thread Daniel P . Berrangé
On Mon, Apr 24, 2023 at 04:02:28PM -0400, John Snow wrote: > GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/846869409 >(All green, except Python self-tests, see below) > > This patch series creates a mandatory python virtual environment > ("venv") during configure time and uses i

Re: [RFC PATCH v3 00/20] configure: create a python venv and ensure meson, sphinx

2023-04-25 Thread John Snow
On Tue, Apr 25, 2023, 1:17 PM Daniel P. Berrangé wrote: > On Mon, Apr 24, 2023 at 04:02:28PM -0400, John Snow wrote: > > GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/846869409 > >(All green, except Python self-tests, see below) > > > > This patch series creates a mandatory pyt

[PATCH v4 04/20] virtio-scsi: stop using aio_disable_external() during unplug

2023-04-25 Thread Stefan Hajnoczi
This patch is part of an effort to remove the aio_disable_external() API because it does not fit in a multi-queue block layer world where many AioContexts may be submitting requests to the same disk. The SCSI emulation code is already in good shape to stop using aio_disable_external(). It was only

[PATCH v4 07/20] block/export: stop using is_external in vhost-user-blk server

2023-04-25 Thread Stefan Hajnoczi
vhost-user activity must be suspended during bdrv_drained_begin/end(). This prevents new requests from interfering with whatever is happening in the drained section. Previously this was done using aio_set_fd_handler()'s is_external argument. In a multi-queue block layer world the aio_disable_exter

[PATCH v4 00/20] block: remove aio_disable_external() API

2023-04-25 Thread Stefan Hajnoczi
v4: - Remove external_disable_cnt variable [Philippe] - Add Patch 1 to fix assertion failure in .drained_end() -> blk_get_aio_context() v3: - Resend full patch series. v2 was sent in the middle of a git rebase and was missing patches. [Eric] - Apply Reviewed-by tags. v2: - Do not rely on BlockBa

[PATCH v4 13/20] block/export: rewrite vduse-blk drain code

2023-04-25 Thread Stefan Hajnoczi
vduse_blk_detach_ctx() waits for in-flight requests using AIO_WAIT_WHILE(). This is not allowed according to a comment in bdrv_set_aio_context_commit(): /* * Take the old AioContex when detaching it from bs. * At this point, new_context lock is already acquired, and we are now * also ta

[PATCH v4 18/20] virtio-scsi: implement BlockDevOps->drained_begin()

2023-04-25 Thread Stefan Hajnoczi
The virtio-scsi Host Bus Adapter provides access to devices on a SCSI bus. Those SCSI devices typically have a BlockBackend. When the BlockBackend enters a drained section, the SCSI device must temporarily stop submitting new I/O requests. Implement this behavior by temporarily stopping virtio-scs

[PATCH v4 15/20] block/fuse: do not set is_external=true on FUSE fd

2023-04-25 Thread Stefan Hajnoczi
This is part of ongoing work to remove the aio_disable_external() API. Use BlockDevOps .drained_begin/end/poll() instead of aio_set_fd_handler(is_external=true). As a side-effect the FUSE export now follows AioContext changes like the other export types. Signed-off-by: Stefan Hajnoczi --- bloc

[PATCH v4 19/20] virtio: do not set is_external=true on host notifiers

2023-04-25 Thread Stefan Hajnoczi
Host notifiers can now use is_external=false since virtio-blk and virtio-scsi no longer rely on is_external=true for drained sections. Signed-off-by: Stefan Hajnoczi --- hw/virtio/virtio.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/virtio/virtio.c b/hw/virtio/vi

[PATCH v4 20/20] aio: remove aio_disable_external() API

2023-04-25 Thread Stefan Hajnoczi
All callers now pass is_external=false to aio_set_fd_handler() and aio_set_event_notifier(). The aio_disable_external() API that temporarily disables fd handlers that were registered is_external=true is therefore dead code. Remove aio_disable_external(), aio_enable_external(), and the is_external

[PATCH v4 09/20] block: add blk_in_drain() API

2023-04-25 Thread Stefan Hajnoczi
The BlockBackend quiesce_counter is greater than zero during drained sections. Add an API to check whether the BlockBackend is in a drained section. The next patch will use this API. Signed-off-by: Stefan Hajnoczi --- include/sysemu/block-backend-global-state.h | 1 + block/block-backend.c

[PATCH v4 11/20] xen-block: implement BlockDevOps->drained_begin()

2023-04-25 Thread Stefan Hajnoczi
Detach event channels during drained sections to stop I/O submission from the ring. xen-block is no longer reliant on aio_disable_external() after this patch. This will allow us to remove the aio_disable_external() API once all other code that relies on it is converted. Extend xen_device_set_event

[PATCH v4 10/20] block: drain from main loop thread in bdrv_co_yield_to_drain()

2023-04-25 Thread Stefan Hajnoczi
For simplicity, always run BlockDevOps .drained_begin/end/poll() callbacks in the main loop thread. This makes it easier to implement the callbacks and avoids extra locks. Move the function pointer declarations from the I/O Code section to the Global State section in block-backend-common.h. Signe

[PATCH v4 01/20] block-backend: split blk_do_set_aio_context()

2023-04-25 Thread Stefan Hajnoczi
blk_set_aio_context() is not fully transactional because blk_do_set_aio_context() updates blk->ctx outside the transaction. Most of the time this goes unnoticed but a BlockDevOps.drained_end() callback that invokes blk_get_aio_context() fails assert(ctx == blk->ctx). This happens because blk->ctx i

[PATCH v4 08/20] hw/xen: do not use aio_set_fd_handler(is_external=true) in xen_xenstore

2023-04-25 Thread Stefan Hajnoczi
There is no need to suspend activity between aio_disable_external() and aio_enable_external(), which is mainly used for the block layer's drain operation. This is part of ongoing work to remove the aio_disable_external() API. Reviewed-by: David Woodhouse Reviewed-by: Paul Durrant Signed-off-by:

[PATCH v4 12/20] hw/xen: do not set is_external=true on evtchn fds

2023-04-25 Thread Stefan Hajnoczi
is_external=true suspends fd handlers between aio_disable_external() and aio_enable_external(). The block layer's drain operation uses this mechanism to prevent new I/O from sneaking in between bdrv_drained_begin() and bdrv_drained_end(). The previous commit converted the xen-block device to use B

[PATCH v4 05/20] util/vhost-user-server: rename refcount to in_flight counter

2023-04-25 Thread Stefan Hajnoczi
The VuServer object has a refcount field and ref/unref APIs. The name is confusing because it's actually an in-flight request counter instead of a refcount. Normally a refcount destroys the object upon reaching zero. The VuServer counter is used to wake up the vhost-user coroutine when there are n

[PATCH 03/20] block: bdrv/blk_co_unref() for calls in coroutine context

2023-04-25 Thread Kevin Wolf
These functions must not be called in coroutine context, because they need write access to the graph. Signed-off-by: Kevin Wolf --- include/block/block-global-state.h | 3 ++- include/sysemu/block-backend-global-state.h | 5 - block.c | 2 +- b

[PATCH 19/20] block: Mark bdrv_recurse_can_replace() and callers GRAPH_RDLOCK

2023-04-25 Thread Kevin Wolf
This adds GRAPH_RDLOCK annotations to declare that callers of bdrv_recurse_can_replace() need to hold a reader lock for the graph because it accesses the children list of a node. Signed-off-by: Kevin Wolf --- include/block/block-global-state.h | 5 +++-- include/block/block_int-common.h

[PATCH v4 02/20] hw/qdev: introduce qdev_is_realized() helper

2023-04-25 Thread Stefan Hajnoczi
Add a helper function to check whether the device is realized without requiring the Big QEMU Lock. The next patch adds a second caller. The goal is to avoid spreading DeviceState field accesses throughout the code. Suggested-by: Philippe Mathieu-Daudé Reviewed-by: Philippe Mathieu-Daudé Signed-o

[PATCH v4 06/20] block/export: wait for vhost-user-blk requests when draining

2023-04-25 Thread Stefan Hajnoczi
Each vhost-user-blk request runs in a coroutine. When the BlockBackend enters a drained section we need to enter a quiescent state. Currently any in-flight requests race with bdrv_drained_begin() because it is unaware of vhost-user-blk requests. When blk_co_preadv/pwritev()/etc returns it wakes th

[PATCH 11/20] vhdx: Take graph lock for accessing a node's parent list

2023-04-25 Thread Kevin Wolf
This adds GRAPH_RDLOCK annotations to declare that functions accessing the parent list of a node need to hold a reader lock for the graph. As it happens, they already do. Signed-off-by: Kevin Wolf --- block/vhdx.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block

[PATCH v4 16/20] virtio: make it possible to detach host notifier from any thread

2023-04-25 Thread Stefan Hajnoczi
virtio_queue_aio_detach_host_notifier() does two things: 1. It removes the fd handler from the event loop. 2. It processes the virtqueue one last time. The first step can be peformed by any thread and without taking the AioContext lock. The second step may need the AioContext lock (depending on t

[PATCH 16/20] block: Mark BlockDriver callbacks for amend job GRAPH_RDLOCK

2023-04-25 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito This adds GRAPH_RDLOCK annotations to declare that callers of amend callbacks in BlockDriver need to hold a reader lock for the graph. Signed-off-by: Emanuele Giuseppe Esposito Signed-off-by: Kevin Wolf --- include/block/block_int-common.h | 12 ++-- b

[PATCH v4 03/20] virtio-scsi: avoid race between unplug and transport event

2023-04-25 Thread Stefan Hajnoczi
Only report a transport reset event to the guest after the SCSIDevice has been unrealized by qdev_simple_device_unplug_cb(). qdev_simple_device_unplug_cb() sets the SCSIDevice's qdev.realized field to false so that scsi_device_find/get() no longer see it. scsi_target_emulate_report_luns() also ne

[PATCH 04/20] block: Don't call no_coroutine_fns in qmp_block_resize()

2023-04-25 Thread Kevin Wolf
This QMP handler runs in a coroutine, so it must use the corresponding no_co_wrappers instead. Signed-off-by: Kevin Wolf --- blockdev.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/blockdev.c b/blockdev.c index d7b5c18f0a..eb509cf964 100644 --- a/blockdev.c +++ b/block

[PATCH 05/20] test-bdrv-drain: Don't modify the graph in coroutines

2023-04-25 Thread Kevin Wolf
test-bdrv-drain contains a few test cases that are run both in coroutine and non-coroutine context. Running the entire code including the setup and shutdown in coroutines is incorrect because graph modifications can generally not happen in coroutines. Change the test so that creating and destroyin

[PATCH 07/20] graph-lock: Fix GRAPH_RDLOCK_GUARD*() to be reader lock

2023-04-25 Thread Kevin Wolf
GRAPH_RDLOCK_GUARD() and GRAPH_RDLOCK_GUARD_MAINLOOP() only take a reader lock for the graph, so the correct annotation for them to use is TSA_ASSERT_SHARED rather than TSA_ASSERT. Signed-off-by: Kevin Wolf --- include/block/graph-lock.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)

[PATCH 02/20] block: Consistently call bdrv_activate() outside coroutine

2023-04-25 Thread Kevin Wolf
Migration code can call bdrv_activate() in coroutine context, whereas other callers call it outside of coroutines. As it calls other code that is not supposed to run in coroutines, standardise on running outside of coroutines. This adds a no_co_wrapper to switch to the main loop before calling bdr

[PATCH v4 14/20] block/export: don't require AioContext lock around blk_exp_ref/unref()

2023-04-25 Thread Stefan Hajnoczi
The FUSE export calls blk_exp_ref/unref() without the AioContext lock. Instead of fixing the FUSE export, adjust blk_exp_ref/unref() so they work without the AioContext lock. This way it's less error-prone. Suggested-by: Paolo Bonzini Signed-off-by: Stefan Hajnoczi --- include/block/export.h

[PATCH 06/20] graph-lock: Add GRAPH_UNLOCKED(_PTR)

2023-04-25 Thread Kevin Wolf
For some function, parts of their interface is that are called without holding the graph lock. Add a new macro to document this. The macro expands to TSA_EXCLUDES(), which is a relatively weak check because it passes in cases where the compiler just doesn't know if the lock is held. Function point

[PATCH 12/20] mirror: Take graph lock for accessing a node's parent list

2023-04-25 Thread Kevin Wolf
This adds GRAPH_RDLOCK annotations to declare that functions accessing the parent list of a node need to hold a reader lock for the graph. As it happens, they already do. Signed-off-by: Kevin Wolf --- block/mirror.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/mirror

[PATCH 15/20] block: Mark bdrv_co_debug_event() GRAPH_RDLOCK

2023-04-25 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito This adds GRAPH_RDLOCK annotations to declare that callers of bdrv_co_debug_event() need to hold a reader lock for the graph. Unfortunately we cannot use a co_wrapper_bdrv_rdlock, because the function is called by mixed functions that run both in coroutine and no

[PATCH 14/20] block: Mark bdrv_co_get_info() and callers GRAPH_RDLOCK

2023-04-25 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito This adds GRAPH_RDLOCK annotations to declare that callers of bdrv_co_get_info() need to hold a reader lock for the graph. Signed-off-by: Emanuele Giuseppe Esposito Signed-off-by: Kevin Wolf --- include/block/block-io.h | 7 +-- include/block/bloc

[PATCH 10/20] nbd: Mark nbd_co_do_establish_connection() and callers GRAPH_RDLOCK

2023-04-25 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito This adds GRAPH_RDLOCK annotations to declare that callers of nbd_co_do_establish_connection() need to hold a reader lock for the graph. Signed-off-by: Emanuele Giuseppe Esposito Signed-off-by: Kevin Wolf --- block/coroutines.h | 5 +++-- block/nbd.c|

Re: [RFC PATCH v3 00/20] configure: create a python venv and ensure meson, sphinx

2023-04-25 Thread John Snow
On Tue, Apr 25, 2023, 1:22 PM John Snow wrote: > > > On Tue, Apr 25, 2023, 1:17 PM Daniel P. Berrangé > wrote: > >> On Mon, Apr 24, 2023 at 04:02:28PM -0400, John Snow wrote: >> > GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/846869409 >> >(All green, except Python self-tests,

[PATCH 13/20] block: Mark bdrv_co_get_allocated_file_size() and callers GRAPH_RDLOCK

2023-04-25 Thread Kevin Wolf
From: Emanuele Giuseppe Esposito This adds GRAPH_RDLOCK annotations to declare that callers of bdrv_co_get_allocated_file_size() need to hold a reader lock for the graph. Signed-off-by: Emanuele Giuseppe Esposito Signed-off-by: Kevin Wolf --- include/block/block-io.h | 7 +-- incl

[PATCH 01/20] qcow2: Don't call bdrv_getlength() in coroutine_fns

2023-04-25 Thread Kevin Wolf
There is a bdrv_co_getlength() now, which should be used in coroutine context. Signed-off-by: Kevin Wolf --- block/qcow2.h | 4 +++- block/qcow2-refcount.c | 2 +- block/qcow2.c | 19 +-- 3 files changed, 13 insertions(+), 12 deletions(-) diff --git a/block/q

[PATCH 08/20] block: .bdrv_open is non-coroutine and unlocked

2023-04-25 Thread Kevin Wolf
Drivers were a bit confused about whether .bdrv_open can run in a coroutine and whether or not it holds a graph lock. It cannot keep a graph lock from the caller across the whole function because it both changes the graph (requires a writer lock) and does I/O (requires a reader lock). Therefore, i

[PATCH 18/20] block: Mark bdrv_query_block_graph_info() and callers GRAPH_RDLOCK

2023-04-25 Thread Kevin Wolf
This adds GRAPH_RDLOCK annotations to declare that callers of bdrv_query_block_graph_info() need to hold a reader lock for the graph because it accesses the children list of a node. Signed-off-by: Kevin Wolf --- include/block/qapi.h | 7 --- qemu-img.c | 2 ++ 2 files changed, 6 in

[PATCH 20/20] block: Mark bdrv_refresh_limits() and callers GRAPH_RDLOCK

2023-04-25 Thread Kevin Wolf
This adds GRAPH_RDLOCK annotations to declare that callers of bdrv_refresh_limits() need to hold a reader lock for the graph because it accesses the children list of a node. Signed-off-by: Kevin Wolf --- include/block/block-global-state.h | 5 - include/block/block_int-common.h | 3 ++- bl

[PATCH 09/20] nbd: Remove nbd_co_flush() wrapper function

2023-04-25 Thread Kevin Wolf
The only thing nbd_co_flush() does is calling nbd_client_co_flush(). Just use that function directly in the BlockDriver definitions and remove the wrapper. Signed-off-by: Kevin Wolf --- block/nbd.c | 11 +++ 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/block/nbd.c b/bloc

[PATCH 17/20] block: Mark bdrv_query_bds_stats() and callers GRAPH_RDLOCK

2023-04-25 Thread Kevin Wolf
This adds GRAPH_RDLOCK annotations to declare that callers of bdrv_query_bds_stats() need to hold a reader lock for the graph because it accesses the children list of a node. Signed-off-by: Kevin Wolf --- block/qapi.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/bloc

[PATCH v4 17/20] virtio-blk: implement BlockDevOps->drained_begin()

2023-04-25 Thread Stefan Hajnoczi
Detach ioeventfds during drained sections to stop I/O submission from the guest. virtio-blk is no longer reliant on aio_disable_external() after this patch. This will allow us to remove the aio_disable_external() API once all other code that relies on it is converted. Take extra care to avoid atta

Re: [RFC] hw/arm/virt: Provide DT binding generation for PCI eXpander Bridges

2023-04-25 Thread Jonathan Cameron via
On Tue, 25 Apr 2023 09:28:44 +0100 Peter Maydell wrote: > On Mon, 24 Apr 2023 at 22:56, Jonathan Cameron > wrote: > > > > On Mon, 24 Apr 2023 16:45:48 +0100 > > Peter Maydell wrote: > > > On the other hand, having QEMU enumerate PCI devices is *also* a > > > very different model from today, w

[PATCH 00/20] Graph locking, part 3 (more block drivers)

2023-04-25 Thread Kevin Wolf
The first few patches in this series fix coroutine correctness problems that have existed for a while, but only actually start to make things fail in practice with stricter checks that we're going to introduce with the graph locking work. The rest of the series makes sure that the graph lock is he

Re: [RFC PATCH v3 00/20] configure: create a python venv and ensure meson, sphinx

2023-04-25 Thread Daniel P . Berrangé
On Tue, Apr 25, 2023 at 01:22:38PM -0400, John Snow wrote: > On Tue, Apr 25, 2023, 1:17 PM Daniel P. Berrangé > wrote: > > > On Mon, Apr 24, 2023 at 04:02:28PM -0400, John Snow wrote: > > > GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/846869409 > > >(All green, except Python s

[PATCH 1/2] migration/vmstate-dump: Dump array size too as "num"

2023-04-25 Thread Peter Xu
For VMS_ARRAY typed vmsd fields, also dump the number of entries in the array in -vmstate-dump. Without such information, vmstate static checker can report false negatives of incompatible vmsd on VMS_ARRAY typed fields, when the src/dst do not have the same type of array defined. It's because in

[PATCH 0/2] vmstate-static-checker: Fix VMS_ARRAY comparisons

2023-04-25 Thread Peter Xu
I'm doing some machine type checks to make sure nothing breaks for 7.2<->8.0. Along the way I found one false negative report on e1000e using the static checker, turns out to be an issue in the checker itself. The problem is the checker doesn't take VMS_ARRAY into account when comparing with UNUS

[PATCH 2/2] vmstate-static-checker: Recognize "num" field

2023-04-25 Thread Peter Xu
Recognize this field for VMS_ARRAY typed vmsd fields, then we can do proper size matching with previous patch. Note that this is compatible with old -dump-vmstate output, because when "num" is not there we'll still use the old "size" only. Signed-off-by: Peter Xu --- scripts/vmstate-static-chec

Re: [RFC PATCH v3 00/20] configure: create a python venv and ensure meson, sphinx

2023-04-25 Thread Daniel P . Berrangé
On Tue, Apr 25, 2023 at 01:34:52PM -0400, John Snow wrote: > On Tue, Apr 25, 2023, 1:22 PM John Snow wrote: > > > > > > > On Tue, Apr 25, 2023, 1:17 PM Daniel P. Berrangé > > wrote: > > > >> On Mon, Apr 24, 2023 at 04:02:28PM -0400, John Snow wrote: > >> > GitLab CI: https://gitlab.com/jsnow/qem

Re: [PATCH 01/20] qcow2: Don't call bdrv_getlength() in coroutine_fns

2023-04-25 Thread Eric Blake
On Tue, Apr 25, 2023 at 07:31:39PM +0200, Kevin Wolf wrote: > There is a bdrv_co_getlength() now, which should be used in coroutine > context. > > Signed-off-by: Kevin Wolf > --- > block/qcow2.h | 4 +++- > block/qcow2-refcount.c | 2 +- > block/qcow2.c | 19 +

Re: [PATCH 02/20] block: Consistently call bdrv_activate() outside coroutine

2023-04-25 Thread Eric Blake
On Tue, Apr 25, 2023 at 07:31:40PM +0200, Kevin Wolf wrote: > Migration code can call bdrv_activate() in coroutine context, whereas > other callers call it outside of coroutines. As it calls other code that > is not supposed to run in coroutines, standardise on running outside of > coroutines. > >

Re: [PATCH v8 1/3] multifd: Create property multifd-flush-after-each-section

2023-04-25 Thread Peter Xu
On Tue, Apr 25, 2023 at 06:31:12PM +0200, Juan Quintela wrote: > We used to flush all channels at the end of each RAM section > sent. That is not needed, so preparing to only flush after a full > iteration through all the RAM. > > Default value of the property is false. But we return "true" in >

Re: [PATCH v8 0/3] Eliminate multifd flush

2023-04-25 Thread Peter Xu
On Tue, Apr 25, 2023 at 06:31:11PM +0200, Juan Quintela wrote: > Hi > > In this v8: > - rebase over latests One trivial comment for patch 1 on the compat bits, for patch 2-3: Acked-by: Peter Xu Nit: I still think we should just squash the three patches into one, but I'm fine either way. Thank

Re: [RFC PATCH v3 00/20] configure: create a python venv and ensure meson, sphinx

2023-04-25 Thread John Snow
On Tue, Apr 25, 2023 at 2:10 PM Daniel P. Berrangé wrote: > > On Tue, Apr 25, 2023 at 01:34:52PM -0400, John Snow wrote: > > On Tue, Apr 25, 2023, 1:22 PM John Snow wrote: > > > > > > > > > > > On Tue, Apr 25, 2023, 1:17 PM Daniel P. Berrangé > > > wrote: > > > > > >> On Mon, Apr 24, 2023 at 04:

[PATCH] cocoa: Fix warnings about invalid prototype declarations

2023-04-25 Thread Philippe Mathieu-Daudé
Fix the following Cocoa trivial warnings: C compiler for the host machine: cc (clang 14.0.0 "Apple clang version 14.0.0 (clang-1400.0.29.202)") Objective-C compiler for the host machine: clang (clang 14.0.0) [100/334] Compiling Objective-C object libcommon.fa.p/net_vmnet-bridged.m.o net/

[PATCH v3 00/57] tcg: Improve atomicity support

2023-04-25 Thread Richard Henderson
v1: https://lore.kernel.org/qemu-devel/20221118094754.242910-1-richard.hender...@linaro.org/ v2: https://lore.kernel.org/qemu-devel/20230216025739.1211680-1-richard.hender...@linaro.org/ Based-on: 20230424054105.1579315-1-richard.hender...@linaro.org ("[PATCH v3 00/57] tcg: Simplify calls to loa

[PATCH v3 04/57] accel/tcg: Reorg system mode load helpers

2023-04-25 Thread Richard Henderson
Instead of trying to unify all operations on uint64_t, pull out mmu_lookup() to perform the basic tlb hit and resolution. Create individual functions to handle access by size. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 612 +++-

[PATCH v3 11/57] tcg/tci: Use helper_{ld,st}*_mmu for user-only

2023-04-25 Thread Richard Henderson
We can now fold these two pieces of code. Signed-off-by: Richard Henderson --- tcg/tci.c | 89 --- 1 file changed, 89 deletions(-) diff --git a/tcg/tci.c b/tcg/tci.c index 5bde2e1f2e..15f2f8c463 100644 --- a/tcg/tci.c +++ b/tcg/tci.c @@ -292,7

[PATCH v3 05/57] accel/tcg: Reorg system mode store helpers

2023-04-25 Thread Richard Henderson
Instead of trying to unify all operations on uint64_t, use mmu_lookup() to perform the basic tlb hit and resolution. Create individual functions to handle access by size. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 408 + 1 file changed,

[PATCH v3 03/57] accel/tcg: Introduce tlb_read_idx

2023-04-25 Thread Richard Henderson
Instead of playing with offsetof in various places, use MMUAccessType to index an array. This is easily defined instead of the previous dummy padding array in the union. Reviewed-by: Alex Bennée Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- include/exec/cpu-defs.h |

[PATCH v3 06/57] accel/tcg: Honor atomicity of loads

2023-04-25 Thread Richard Henderson
Create ldst_atomicity.c.inc. Not required for user-only code loads, because we've ensured that the page is read-only before beginning to translate code. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 170 +++--- accel/tcg/user-exec.c

[PATCH v3 10/57] accel/tcg: Implement helper_{ld, st}*_mmu for user-only

2023-04-25 Thread Richard Henderson
TCG backends may need to defer to a helper to implement the atomicity required by a given operation. Mirror the interface used in system mode. Signed-off-by: Richard Henderson --- include/tcg/tcg-ldst.h | 6 +- accel/tcg/user-exec.c | 392 - tcg/tcg.c

[PATCH v3 08/57] target/loongarch: Do not include tcg-ldst.h

2023-04-25 Thread Richard Henderson
This header is supposed to be private to tcg and in fact does not need to be included here at all. Signed-off-by: Richard Henderson --- target/loongarch/csr_helper.c | 1 - target/loongarch/iocsr_helper.c | 1 - 2 files changed, 2 deletions(-) diff --git a/target/loongarch/csr_helper.c b/targ

[PATCH v3 01/57] include/exec/memop: Add bits describing atomicity

2023-04-25 Thread Richard Henderson
These bits may be used to describe the precise atomicity requirements of the guest, which may then be used to constrain the methods by which it may be emulated by the host. For instance, the AArch64 LDP (32-bit) instruction changes semantics with ARMv8.4 LSE2, from MO_64 | MO_ATMAX_4 | MO_ATOM_

<    1   2   3   4   5   >