Re: [PATCH v11 3/4] softmmu/dirtylimit: implement virtual CPU throttle

2022-01-21 Thread Hyman Huang
+static void *dirtylimit_thread(void *opaque) +{ +CPUState *cpu; + +rcu_register_thread(); + +while (!qatomic_read(&dirtylimit_quit)) { +sleep(DIRTYLIMIT_CALC_TIME_MS / 1000); Sorry to have not mentioned this: I think we probably don't even need this dirtylimit thread. It

Re: [RESEND] target/riscv: Enable bitmanip Zicbo[m,z,p] instructions

2022-01-21 Thread Atish Patra
On Fri, Jan 21, 2022 at 1:55 AM Christoph Müllner wrote: > > On Tue, Jan 18, 2022 at 9:31 PM Atish Patra wrote: > > > > On Tue, Jan 18, 2022 at 8:48 AM Christoph Muellner > > wrote: > > > > > > The RISC-V base cache management operation ISA extension has been > > > ratified [1]. This patch adds

Re: [PATCH v11 1/4] migration/dirtyrate: refactor dirty page rate calculation

2022-01-21 Thread Hyman Huang
在 2022/1/17 10:19, Peter Xu 写道: On Wed, Jan 05, 2022 at 01:14:06AM +0800, huang...@chinatelecom.cn wrote: From: Hyman Huang(黄勇) + +static void vcpu_dirty_stat_collect(VcpuStat *stat, +DirtyPageRecord *records, +bool sta

Re: [PATCH] hw/remote: Add missing include

2022-01-21 Thread Jag Raman
> On Jan 21, 2022, at 7:33 PM, Philippe Mathieu-Daudé wrote: > > Currently "qemu/error-report.h" is implicitly included, however > if headers in include/ get refactored, we get: > > hw/remote/proxy-memory-listener.c: In function > ‘proxy_memory_listener_commit’: > hw/remote/proxy-memory-lis

[PATCH] hw/remote: Add missing include

2022-01-21 Thread Philippe Mathieu-Daudé via
Currently "qemu/error-report.h" is implicitly included, however if headers in include/ get refactored, we get: hw/remote/proxy-memory-listener.c: In function ‘proxy_memory_listener_commit’: hw/remote/proxy-memory-listener.c:183:9: error: implicit declaration of function ‘error_report’; did yo

[PATCH] target/rx: Remove unused ENV_OFFSET definition

2022-01-21 Thread Philippe Mathieu-Daudé via
Signed-off-by: Philippe Mathieu-Daudé --- target/rx/cpu.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/target/rx/cpu.h b/target/rx/cpu.h index 657db84ef0..58adf9edf6 100644 --- a/target/rx/cpu.h +++ b/target/rx/cpu.h @@ -116,8 +116,6 @@ struct RXCPU { typedef RXCPU ArchCPU; -#define

[PATCH] hw/ppc/vof: Add missing includes

2022-01-21 Thread Philippe Mathieu-Daudé via
vof.h requires "qom/object.h" for DECLARE_CLASS_CHECKERS(), "exec/memory.h" for address_space_read/write(), "exec/address-spaces.h" for address_space_memory and more importantly "cpu.h" for target_ulong. vof.c doesn't need "exec/ram_addr.h". Signed-off-by: Philippe Mathieu-Daudé --- hw/ppc/vof.

[PATCH] target/avr: Correct AVRCPUClass docstring

2022-01-21 Thread Philippe Mathieu-Daudé via
There is no 'vr' field in AVRCPUClass. Likely a copy/paste typo from CRISCPUClass ;) Signed-off-by: Philippe Mathieu-Daudé --- target/avr/cpu-qom.h | 1 - 1 file changed, 1 deletion(-) diff --git a/target/avr/cpu-qom.h b/target/avr/cpu-qom.h index 9fa6989c18..14e5b3ce72 100644 --- a/target/avr

[PATCH] meson: Use find_program() to resolve the entitlement.sh script

2022-01-21 Thread Philippe Mathieu-Daudé via
Using ../configure without any particular option generates 31 targets on Darwin, and meson search for the entitlement.sh script 31 times: Program nm found: YES Program scripts/undefsym.py found: YES (/opt/homebrew/opt/python@3.9/bin/python3.9 /Code/qemu/scripts/undefsym.py) Program scripts/

[PULL 10/17] python/qmp: switch qemu-ga-client to AQMP

2022-01-21 Thread John Snow
Async QMP always raises a "ConnectError" on any connection error which houses the cause in a second exception. We can check if this root cause was python's ConnectionError to determine a fairly similar condition to the original error check here. Signed-off-by: John Snow Reviewed-by: Vladimir Seme

[PULL 16/17] scripts/cpu-x86-uarch-abi: switch to AQMP

2022-01-21 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Daniel P. Berrangé Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Beraldo Leal --- scripts/cpu-x86-uarch-abi.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/cpu-x86-uarch-abi.py b/scripts/cpu-x86-uarch-abi.py inde

[PULL 15/17] scripts/cpu-x86-uarch-abi: fix CLI parsing

2022-01-21 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Daniel P. Berrangé Reviewed-by: Vladimir Sementsov-Ogievskiy --- scripts/cpu-x86-uarch-abi.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/scripts/cpu-x86-uarch-abi.py b/scripts/cpu-x86-uarch-abi.py index 08acc52a81..8963d90f0b 1006

[PULL 17/17] scripts/render-block-graph: switch to AQMP

2022-01-21 Thread John Snow
Creating an instance of qemu.aqmp.ExecuteError is too involved here, so just drop the specificity down to a generic QMPError. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Beraldo Leal --- scripts/render_block_graph.py | 8 +++- 1 file changed, 3 insertion

[PULL 12/17] python/qmp: switch qmp-shell to AQMP

2022-01-21 Thread John Snow
We have a replacement for async QMP, but it doesn't have feature parity yet. For now, then, port the old tool onto the new backend. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy --- python/qemu/aqmp/legacy.py | 3 +++ python/qemu/qmp/qmp_shell.py | 31 +--

[PULL 11/17] python/qmp: switch qom tools to AQMP

2022-01-21 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Beraldo Leal --- python/qemu/qmp/qom.py| 5 +++-- python/qemu/qmp/qom_common.py | 3 ++- python/qemu/qmp/qom_fuse.py | 11 ++- 3 files changed, 11 insertions(+), 8 deletions(-) diff --git a/pyth

[PULL 13/17] python: move qmp utilities to python/qemu/utils

2022-01-21 Thread John Snow
In order to upload a QMP package to PyPI, I want to remove any scripts that I am not 100% confident I want to support upstream, beyond our castle walls. Move most of our QMP utilities into the utils package so we can split them out from the PyPI upload. Signed-off-by: John Snow Reviewed-by: Vlad

[PULL 08/17] python/aqmp: rename AQMPError to QMPError

2022-01-21 Thread John Snow
This is in preparation for renaming qemu.aqmp to qemu.qmp. I should have done this from this from the very beginning, but it's a convenient time to make sure this churn is taken care of. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy --- python/qemu/aqmp/__init__.py | 6 +

[PULL 14/17] python: move qmp-shell under the AQMP package

2022-01-21 Thread John Snow
Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Beraldo Leal --- python/README.rst | 2 +- python/qemu/{qmp => aqmp}/qmp_shell.py | 0 python/setup.cfg | 2 +- scripts/qmp/qmp-shell | 2 +- 4 files chang

[PULL 05/17] python/aqmp: handle asyncio.TimeoutError on execute()

2022-01-21 Thread John Snow
This exception can be injected into any await statement. If we are canceled via timeout, we want to clear the pending execution record on our way out. Signed-off-by: John Snow Reviewed-by: Beraldo Leal Reviewed-by: Vladimir Sementsov-Ogievskiy --- python/qemu/aqmp/qmp_client.py | 8 ++-- 1

[PULL 07/17] python/aqmp: add SocketAddrT to package root

2022-01-21 Thread John Snow
It's a commonly needed definition, it can be re-exported by the root. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Beraldo Leal --- python/qemu/aqmp/__init__.py | 10 +- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/python/qemu/aqmp/__i

[PULL 09/17] python/qemu-ga-client: don't use deprecated CLI syntax in usage comment

2022-01-21 Thread John Snow
Cleanup related to commit ccd3b3b8112b670f, "qemu-option: warn for short-form boolean options". Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Daniel P. Berrangé --- python/qemu/qmp/qemu_ga_client.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff -

[PULL 03/17] python/aqmp: fix docstring typo

2022-01-21 Thread John Snow
Reported-by: Vladimir Sementsov-Ogievskiy Signed-off-by: John Snow Reviewed-by: Beraldo Leal --- python/qemu/aqmp/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/qemu/aqmp/__init__.py b/python/qemu/aqmp/__init__.py index 880d5b6fa7..173556404d 100644 --- a

[PULL 06/17] python/aqmp: copy type definitions from qmp

2022-01-21 Thread John Snow
Copy the remaining type definitions from QMP into the qemu.aqmp.legacy module. Now, users that require the legacy interface don't need to import anything else but qemu.aqmp.legacy wrapper. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Beraldo Leal --- python/q

[PULL 04/17] python/aqmp: add __del__ method to legacy interface

2022-01-21 Thread John Snow
asyncio can complain *very* loudly if you forget to back out of things gracefully before the garbage collector starts destroying objects that contain live references to asyncio Tasks. The usual fix is just to remember to call aqmp.disconnect(), but for the sake of the legacy wrapper and quick, one

[PULL 01/17] python: pin setuptools below v60.0.0

2022-01-21 Thread John Snow
setuptools is a package that replaces the python stdlib 'distutils'. It is generally installed by all venv-creating tools "by default". It isn't actually needed at runtime for the qemu package, so our own setup.cfg does not mention it as a dependency. However, tox will create virtual environments

[PULL 02/17] python: use avocado's "new" runner

2022-01-21 Thread John Snow
The old legacy runner no longer seems to work with output logging, so we can't see failure logs when a test case fails. The new runner doesn't (seem to) support Coverage.py yet, but seeing error output is a more important feature. Signed-off-by: John Snow Reviewed-by: Beraldo Leal Message-id: 20

[PULL 00/17] Python patches

2022-01-21 Thread John Snow
The following changes since commit 5e9d14f2bea6df89c0675df953f9c839560d2266: Merge remote-tracking branch 'remotes/alistair/tags/pull-riscv-to-apply-20220121-1' into staging (2022-01-21 10:31:25 +) are available in the Git repository at: https://gitlab.com/jsnow/qemu.git t

[PATCH] hw/display/tcx: Avoid clearing dirty bitmap in DeviceReset()

2022-01-21 Thread Philippe Mathieu-Daudé via
Commit 2dd285b5f3 ("tcx: make display updates thread safe") converted this model to use the DirtyBitmapSnapshot API, resetting the dirty bitmap in tcx_update_display(). There is no need to do it again in the DeviceReset handler. See more details in commit fec5e8c92b ("vga: make display updates thr

Re: tests/tcg/multiarch/threadcount failing on GitLab

2022-01-21 Thread Philippe Mathieu-Daudé via
On 21/1/22 21:54, John Snow wrote: On Fri, Jan 21, 2022 at 3:53 PM John Snow wrote: lately, I've been seeing this test fail under the clang-user job; see https://gitlab.com/jsnow/qemu/-/jobs/2002782042 make[2]: *** [../Makefile.target:156: run-threadcount] Error 124 make[1]: *** [/builds/jsno

Re: [PATCH 0/2] some simplifications in hw/core/fw-path-provider.c

2022-01-21 Thread Philippe Mathieu-Daudé via
On 21/1/22 21:29, Daniel Henrique Barboza wrote: Hi, These are somes simplifications I found interesting to make while I was investigating Gitlab #720. It isn't clear in the MAINTAINERS file who is responsible for this file, so I'm ccing the QOM maintainers since it's the closest match. Danie

Re: [PATCH] hw/armv7m: Fix broken VMStateDescription

2022-01-21 Thread Philippe Mathieu-Daudé via
On 20/1/22 16:16, Peter Maydell wrote: In commit d5093d961585f02 we added a VMStateDescription to the TYPE_ARMV7M object, to handle migration of its Clocks. However a cut-and-paste error meant we used the wrong struct name in the VMSTATE_CLOCK() macro arguments. The result was that attempting a '

[PATCH 3/3] hw/display/artist: rewrite vram access mode handling

2022-01-21 Thread Sven Schnelle
When writing this code it was assumed that register 0x118000 is the buffer access mode for color map accesses. It turned out that this is wrong. Instead register 0x118000 sets both src and dst buffer access mode at the same time. This required a larger rewrite of the code. The good thing is that b

[PATCH 2/3] hw/display/artist: allow to disable/enable cursor

2022-01-21 Thread Sven Schnelle
Allow to disable/enable the cursor via the cursor control register. Signed-off-by: Sven Schnelle --- hw/display/artist.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/hw/display/artist.c b/hw/display/artist.c index 7956a1a5c3..cfae92d3e8 100644 --- a/hw/display/a

[PATCH 0/3] hw/display/artist: cursor & buffer mode fixes

2022-01-21 Thread Sven Schnelle
Hi List, this set contains two fixes too make the cursor work with HP-UX' X-server, and one large rewrite to make the Linux framebuffer work with artist on hppa. Sven Schnelle (3): hw/display/artist: fix cursor position hw/display/artist: allow to disable/enable cursor hw/display/artist: re

[PATCH 1/3] hw/display/artist: fix cursor position

2022-01-21 Thread Sven Schnelle
Register 0x300200 and 0x300208 seems to be used as scratch register by HP-UX for cursor offset data. It writes a calculated value on X startup, and later reads it back and uses this as offset for all cursor movements. I couldn't figure how this number is calculated, but forcing it to a fixed value

[PATCH] spapr.c: check bus != NULL in spapr_get_fw_dev_path()

2022-01-21 Thread Daniel Henrique Barboza
spapr_get_fw_dev_path() is an impl of FWPathProviderClass::get_dev_path(). This interface is used by hw/core/qdev-fw.c via fw_path_provider_try_get_dev_path() in two functions: - static char *qdev_get_fw_dev_path_from_handler(), which is used only in qdev_get_fw_dev_path_helper() and it's guarded

Re: [PATCH] xen-mapcache: Avoid entry->lock overflow

2022-01-21 Thread Stefano Stabellini
On Fri, 21 Jan 2022, Ross Lagerwall wrote: > In some cases, a particular mapcache entry may be mapped 256 times > causing the lock field to wrap to 0. For example, this may happen when > using emulated NVME and the guest submits a large scatter-gather write. > At this point, the entry map be remapp

Re: [PATCH] python: pin setuptools below v60.0.0

2022-01-21 Thread John Snow
On Fri, Jan 21, 2022 at 2:15 PM Cleber Rosa wrote: > > > John Snow writes: > > > setuptools is a package that replaces the python stdlib 'distutils'. It > > is generally installed by all venv-creating tools "by default". It isn't > > actually needed at runtime for the qemu package, so our own set

tests/tcg/multiarch/threadcount failing on GitLab

2022-01-21 Thread John Snow
lately, I've been seeing this test fail under the clang-user job; see https://gitlab.com/jsnow/qemu/-/jobs/2002782042 make[2]: *** [../Makefile.target:156: run-threadcount] Error 124 make[1]: *** [/builds/jsnow/qemu/tests/tcg/Makefile.qemu:102: run-guest-tests] Error 2 make: *** [/builds/jsnow/qem

[PATCH 29/31] vdpa: Make ncs autofree

2022-01-21 Thread Eugenio Pérez
Simplifying memory management. Signed-off-by: Eugenio Pérez --- net/vhost-vdpa.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 4125d13118..4befba5cc7 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -264,7 +264,8 @@ int

[PATCH 25/31] vdpa: Adapt vhost_vdpa_get_vring_base to SVQ

2022-01-21 Thread Eugenio Pérez
This is needed to achieve migration, so destination can restore its index. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-vdpa.c | 9 + 1 file changed, 9 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 276a559649..887857c177 100644 --- a/hw/virtio/vhos

[PATCH 21/31] util: Add iova_tree_alloc

2022-01-21 Thread Eugenio Pérez
This iova tree function allows it to look for a hole in allocated regions and return a totally new translation for a given translated address. It's usage is mainly to allow devices to access qemu address space, remapping guest's one into a new iova space where qemu can add chunks of addresses. Si

[PATCH 28/31] vdpa: Expose VHOST_F_LOG_ALL on SVQ

2022-01-21 Thread Eugenio Pérez
SVQ is able to log the dirty bits by itself, so let's use it to not block migration. Also, ignore set and clear of VHOST_F_LOG_ALL on set_features if SVQ is enabled. Even if the device supports it, the reports would be nonsense because SVQ memory is in the qemu region. The log region is still all

[PATCH 24/31] vhost: Add vhost_svq_get_last_used_idx

2022-01-21 Thread Eugenio Pérez
This way SVQ queues can be migrated. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.h | 1 + hw/virtio/vhost-shadow-virtqueue.c | 8 2 files changed, 9 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-virtqueue.h index c6f67d6f7

[PATCH 15/31] vdpa: Add vhost_svq_get_num

2022-01-21 Thread Eugenio Pérez
This reports the guest's visible SVQ effective length, not the device's one. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.h | 1 + hw/virtio/vhost-shadow-virtqueue.c | 5 + 2 files changed, 6 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vho

[PATCH 31/31] vdpa: Add x-svq to NetdevVhostVDPAOptions

2022-01-21 Thread Eugenio Pérez
Finally offering the possibility to enable SVQ from the command line. Signed-off-by: Eugenio Pérez --- qapi/net.json| 5 - net/vhost-vdpa.c | 27 --- 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/qapi/net.json b/qapi/net.json index 7fab2e7cd8..d

[PATCH 13/31] vhost: Add vhost_svq_ack_guest_features to shadow vq

2022-01-21 Thread Eugenio Pérez
This combines the previous two feature functions, forwarding the guest ones to the device and setting the transport ones that the SVQ supports with the device. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.h | 3 +++ hw/virtio/vhost-shadow-virtqueue.c | 31 ++

[PATCH 20/31] util: Store DMA entries in a list

2022-01-21 Thread Eugenio Pérez
SVQ need to allocate iova entries, traversing the list looking for holes. GLib offers methods to both transverse the list and look for entries upper than a key since version 2.68. However qemu may need to compile with earlier versions, so we replicate both methods here. Signed-off-by: Eugenio Pér

Re: tests/tcg/multiarch/threadcount failing on GitLab

2022-01-21 Thread John Snow
On Fri, Jan 21, 2022 at 3:53 PM John Snow wrote: > > lately, I've been seeing this test fail under the clang-user job; see > https://gitlab.com/jsnow/qemu/-/jobs/2002782042 > > make[2]: *** [../Makefile.target:156: run-threadcount] Error 124 > make[1]: *** [/builds/jsnow/qemu/tests/tcg/Makefile.qe

[PATCH 30/31] vdpa: Move vhost_vdpa_get_iova_range to net/vhost-vdpa.c

2022-01-21 Thread Eugenio Pérez
Since it's a device property, it can be done in net/. This helps SVQ to allocate the rings in vdpa device initialization, rather than delay that. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-vdpa.c | 15 --- net/vhost-vdpa.c | 32 2 files ch

[PATCH 12/31] vhost: Add vhost_svq_valid_guest_features to shadow vq

2022-01-21 Thread Eugenio Pérez
This allows it to test if the guest has acknowledged an invalid transport feature for SVQ. This will include packed vq layout or event_idx, where the VirtIO device needs help from SVQ. It is not needed at this moment, but since SVQ will not re-negotiate features again with the guest, a failure to

[PATCH 18/31] vhost: Shadow virtqueue buffers forwarding

2022-01-21 Thread Eugenio Pérez
Initial version of shadow virtqueue that actually forward buffers. There is no iommu support at the moment, and that will be addressed in future patches of this series. Since all vhost-vdpa devices use forced IOMMU, this means that SVQ is not usable at this point of the series on any device. For s

[PATCH 22/31] vhost: Add VhostIOVATree

2022-01-21 Thread Eugenio Pérez
This tree is able to look for a translated address from an IOVA address. At first glance it is similar to util/iova-tree. However, SVQ working on devices with limited IOVA space need more capabilities, like allocating IOVA chunks or performing reverse translations (qemu addresses to iova). The al

[PATCH 16/31] vhost: pass queue index to vhost_vq_get_addr

2022-01-21 Thread Eugenio Pérez
Doing that way allows vhost backend to know what address to return. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 7b03efccec..64b955ba0c 100644 --- a/hw/virtio/vhost.c +++ b

[PATCH 08/31] vhost: Add vhost_svq_set_guest_call_notifier

2022-01-21 Thread Eugenio Pérez
This allows the vhost-vdpa device to set SVQ -> guest notifier to SVQ. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.h | 1 + hw/virtio/vhost-shadow-virtqueue.c | 16 2 files changed, 17 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/vi

[PATCH 19/31] utils: Add internal DMAMap to iova-tree

2022-01-21 Thread Eugenio Pérez
So we can store private data not accessible from outside. In this case, we will add intrusive linked list members so we can transverse it for allocation. Signed-off-by: Eugenio Pérez --- util/iova-tree.c | 37 ++--- 1 file changed, 26 insertions(+), 11 deletions(

[PATCH 14/31] virtio: Add vhost_shadow_vq_get_vring_addr

2022-01-21 Thread Eugenio Pérez
It reports the shadow virtqueue address from qemu virtual address space. Since this will be different from the guest's vaddr, but the device can access it, SVQ takes special care about its alignment & lack of garbage data. It assumes that IOMMU will work in host_page_size ranges for that. Signed-

[PATCH 27/31] vdpa: Never set log_base addr if SVQ is enabled

2022-01-21 Thread Eugenio Pérez
Setting the log address would make the device start reporting invalid dirty memory because the SVQ vrings are located in qemu's memory. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-vdpa.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio

[PATCH 06/31] vhost: Route guest->host notification through shadow virtqueue

2022-01-21 Thread Eugenio Pérez
At this moment no buffer forwarding will be performed in SVQ mode: Qemu just forward the guest's kicks to the device. This commit also set up SVQs in the vhost device. Host memory notifiers regions are left out for simplicity, and they will not be addressed in this series. Signed-off-by: Eugenio

[PATCH 26/31] vdpa: Clear VHOST_VRING_F_LOG at vhost_vdpa_set_vring_addr in SVQ

2022-01-21 Thread Eugenio Pérez
Doing so would cause the device to export writes to SVQ addresses, which are not part of the guest's IOVA. Like the previous patch, this is currently not possible since SVQ does not run if the device exports VHOST_VRING_F_LOG. But it's needed to enable migration with SVQ. Signed-off-by: Eugenio P

[PATCH 04/31] vdpa: Add vhost_svq_set_svq_kick_fd

2022-01-21 Thread Eugenio Pérez
This function allows the vhost-vdpa backend to override kick_fd. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.h | 1 + hw/virtio/vhost-shadow-virtqueue.c | 45 ++ 2 files changed, 46 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h

[PATCH 11/31] vhost: Add vhost_svq_valid_device_features to shadow vq

2022-01-21 Thread Eugenio Pérez
This allows SVQ to negotiate features with the device. For the device, SVQ is a driver. While this function needs to bypass all non-transport features, it needs to disable the features that SVQ does not support when forwarding buffers. This includes packed vq layout, indirect descriptors or event i

[PATCH 07/31] vhost: dd vhost_svq_get_svq_call_notifier

2022-01-21 Thread Eugenio Pérez
This allows vhost-vdpa device to retrieve device -> svq call eventfd. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.h | 2 ++ hw/virtio/vhost-shadow-virtqueue.c | 12 2 files changed, 14 insertions(+) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio

[PATCH 0/2] some simplifications in hw/core/fw-path-provider.c

2022-01-21 Thread Daniel Henrique Barboza
Hi, These are somes simplifications I found interesting to make while I was investigating Gitlab #720. It isn't clear in the MAINTAINERS file who is responsible for this file, so I'm ccing the QOM maintainers since it's the closest match. Daniel Henrique Barboza (2): hw/core/fw-path-provider:

[PATCH 10/31] vhost: Route host->guest notification through shadow virtqueue

2022-01-21 Thread Eugenio Pérez
This will make qemu aware of the device used buffers, allowing it to write the guest memory with its contents if needed. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.c | 15 +++ hw/virtio/vhost-vdpa.c | 11 +++ 2 files changed, 26 insertions(+

[PATCH 23/31] vdpa: Add custom IOTLB translations to SVQ

2022-01-21 Thread Eugenio Pérez
Use translations added in VhostIOVATree in SVQ. Only introduce usage here, not allocation and deallocation. As with previous patches, we use the dead code paths of shadow_vqs_enabled to avoid commiting too many changes at once. These are impossible to take at the moment. Signed-off-by: Eugenio Pé

[PATCH 01/31] vdpa: Reorder virtio/vhost-vdpa.c functions

2022-01-21 Thread Eugenio Pérez
vhost_vdpa_set_features and vhost_vdpa_init need to use vhost_vdpa_get_features in svq mode. vhost_vdpa_dev_start needs to use almost all _set_ functions: vhost_vdpa_set_vring_dev_kick, vhost_vdpa_set_vring_dev_call, vhost_vdpa_set_dev_vring_base and vhost_vdpa_set_dev_vring_num. No functional ch

[PATCH 2/2] hw/core/fw-path-provider.c: open code fw_path_provider_get_dev_path()

2022-01-21 Thread Daniel Henrique Barboza
fw_path_provider_get_dev_path() is 2 lines long and it's used only by fw_path_provider_try_get_dev_path(), which checks if FWPathProvider is not NULL before calling it. Open coding fw_path_provider_get_dev_path() into fw_path_provider_try_get_dev_path() simplifies the code inside fw-path-provider.

[PATCH 09/31] vhost-vdpa: Take into account SVQ in vhost_vdpa_set_vring_call

2022-01-21 Thread Eugenio Pérez
Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-vdpa.c | 20 ++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 18de14f0fb..029f98feee 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -687,1

[PATCH 17/31] vdpa: adapt vhost_ops callbacks to svq

2022-01-21 Thread Eugenio Pérez
First half of the buffers forwarding part, preparing vhost-vdpa callbacks to SVQ to offer it. QEMU cannot enable it at this moment, so this is effectively dead code at the moment, but it helps to reduce patch size. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.h | 2 +- hw/

[PATCH 00/31] vDPA shadow virtqueue

2022-01-21 Thread Eugenio Pérez
This series enables shadow virtqueue (SVQ) for vhost-vdpa devices. This is intended as a new method of tracking the memory the devices touch during a migration process: Instead of relay on vhost device's dirty logging capability, SVQ intercepts the VQ dataplane forwarding the descriptors between VM

[PATCH 03/31] vdpa: Add vhost_svq_get_dev_kick_notifier

2022-01-21 Thread Eugenio Pérez
Is needed so vhost-vdpa knows the device's kick event fd. Signed-off-by: Eugenio Pérez --- hw/virtio/vhost-shadow-virtqueue.h | 4 hw/virtio/vhost-shadow-virtqueue.c | 10 +- 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/vir

[PATCH 05/31] vhost: Add Shadow VirtQueue kick forwarding capabilities

2022-01-21 Thread Eugenio Pérez
At this mode no buffer forwarding will be performed in SVQ mode: Qemu will just forward the guest's kicks to the device. Also, host notifiers must be disabled at SVQ start, and they will not start if SVQ has been enabled when the device is stopped. This will be addressed in next patches. Signed-o

[PATCH 02/31] vhost: Add VhostShadowVirtqueue

2022-01-21 Thread Eugenio Pérez
Vhost shadow virtqueue (SVQ) is an intermediate jump for virtqueue notifications and buffers, allowing qemu to track them. While qemu is forwarding the buffers and virtqueue changes, it is able to commit the memory it's being dirtied, the same way regular qemu's VirtIO devices do. This commit only

[PATCH 1/2] hw/core/fw-path-provider: turn fw_path_provider_get_dev_path() static

2022-01-21 Thread Daniel Henrique Barboza
This function is used only by fw_path_provider_try_get_dev_path() in the same file. Signed-off-by: Daniel Henrique Barboza --- hw/core/fw-path-provider.c| 4 ++-- include/hw/fw-path-provider.h | 2 -- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/hw/core/fw-path-provider.c b

Re: Help with ppc e500 tlb usage

2022-01-21 Thread Jesse Millwood
On Fri, 2022-01-21 at 13:17 +0100, Cédric Le Goater wrote: > Hello Jesse, > > ( For some reason, your email never reached my mailbox. In the > future, >    could please also Cc: mailto:qemu-devel@nongnu.org. ) > Absolutely, I wasn't sure how much posting to two mailing lists was accepted. > On

Re: [EXT] - Re: Help with ppc e500 tlb usage

2022-01-21 Thread Jesse Millwood
On Fri, 2022-01-21 at 18:57 +0100, Cédric Le Goater wrote: > CAUTION: This email originated from outside of the organization. > Please do not click links or open attachments unless you recognize > the sender's email address and know the content is safe. > > > > Thanks for the background info Cédr

Re: [PATCH] python: pin setuptools below v60.0.0

2022-01-21 Thread Cleber Rosa
John Snow writes: > setuptools is a package that replaces the python stdlib 'distutils'. It > is generally installed by all venv-creating tools "by default". It isn't > actually needed at runtime for the qemu package, so our own setup.cfg > does not mention it as a dependency. > > However, tox

Re: [PATCH 1/1] target/ppc: fix 'skip KVM' cond in cpu_interrupt_exittb()

2022-01-21 Thread Fabiano Rosas
Daniel Henrique Barboza writes: > cpu_interrupt_exittb() was introduced by commit 044897ef4a22 > ("target/ppc: Fix system lockups caused by interrupt_request state > corruption") as a way to wrap cpu_interrupt() helper in BQL. > > After that, commit 6d38666a8931 ("ppc: Ignore the CPU_INTERRUPT_EX

Re: [PATCH v2] ide: Increment BB in-flight counter for TRIM BH

2022-01-21 Thread John Snow
On Thu, Jan 20, 2022 at 9:23 AM Hanna Reitz wrote: > > When we still have an AIOCB registered for DMA operations, we try to > settle the respective operation by draining the BlockBackend associated > with the IDE device. > > However, this assumes that every DMA operation is associated with an > in

Re: [PATCH] python: pin setuptools below v60.0.0

2022-01-21 Thread John Snow
On Fri, Jan 21, 2022 at 12:54 PM Beraldo Leal wrote: > > On Thu, Jan 20, 2022 at 07:52:21PM -0500, John Snow wrote: > > setuptools is a package that replaces the python stdlib 'distutils'. It > > is generally installed by all venv-creating tools "by default". It isn't > > actually needed at runtim

Re: Help with ppc e500 tlb usage

2022-01-21 Thread Cédric Le Goater
Thanks for the background info Cédric, I wasn't aware of the MMUv2. I see now that in target/ppc/cpu_init.c the e500v2 and e5500 have the same mmu_model, the POWREPC_MMU_BOOKE206. I'd like to be able to set the initial tlb entry size from the platform definition by indicating the size in kib. I

Re: [PATCH] python: pin setuptools below v60.0.0

2022-01-21 Thread Beraldo Leal
On Thu, Jan 20, 2022 at 07:52:21PM -0500, John Snow wrote: > setuptools is a package that replaces the python stdlib 'distutils'. It > is generally installed by all venv-creating tools "by default". It isn't > actually needed at runtime for the qemu package, so our own setup.cfg > does not mention

Re: [PATCH v3 14/16] job.c: use job_get_aio_context()

2022-01-21 Thread Emanuele Giuseppe Esposito
On 21/01/2022 13:33, Emanuele Giuseppe Esposito wrote: On 19/01/2022 11:31, Paolo Bonzini wrote: diff --git a/blockjob.c b/blockjob.c index cf1f49f6c2..468ba735c5 100644 --- a/blockjob.c +++ b/blockjob.c @@ -155,14 +155,16 @@ static void child_job_set_aio_ctx(BdrvChild *c, AioContext *ctx,

Re: [PATCH] ppc/pnv: Fail DMA access if page permissions are not correct

2022-01-21 Thread Cédric Le Goater
On 1/21/22 16:23, Frederic Barrat wrote: If an iommu page has wrong permissions, an error message is displayed, but the access is allowed, which is odd. This patch fixes it. Being curious. How do you generate such errors ? could we have the output ? Thanks, C. Signed-off-by: Frederic Bar

[PATCH v6 29/33] job.h: assertions in the callers of JobDriver funcion pointers

2022-01-21 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- job.c | 9 + 1 file changed, 9 insertions(+) diff --git a/job.c b/job.c index 54db80df66..39bf511949 100644 --- a/job.c +++ b/job.c @@ -381,6 +381,8 @@ void job_ref(Job *job) void job_unref(Job *job) { +assert(qemu_in_main_thread());

[PATCH v6 24/33] block_int-common.h: assertions in the callers of BlockDriver function pointers

2022-01-21 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- block.c| 18 ++ block/create.c | 2 ++ 2 files changed, 20 insertions(+) diff --git a/block.c b/block.c index bad834c86e..448fb9d76f 100644 --- a/block.c +++ b/block.c @@ -526,6 +526,7 @@ static void coroutine_fn bdrv_create_

[PATCH v6 33/33] block.c: assertions to the block layer permissions API

2022-01-21 Thread Emanuele Giuseppe Esposito
Now that we "covered" the three main cases where the permission API was being used under BQL (fuse, amend and invalidate_cache), we can safely assert for the permission functions implemented in block.c Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 12 1 file changed, 12 in

[PATCH v6 26/33] block_int-common.h: assertions in the callers of BdrvChildClass function pointers

2022-01-21 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 9 + 1 file changed, 9 insertions(+) diff --git a/block.c b/block.c index 448fb9d76f..ca16d90627 100644 --- a/block.c +++ b/block.c @@ -1491,6 +1491,7 @@ const BdrvChildClass child_of_bds = { AioContext *bdrv_child_get_parent_aio

Re: [PATCH] tests/qtest/ide-test: Remove bad retry_isa test

2022-01-21 Thread John Snow
On Fri, Jan 21, 2022 at 7:06 AM Thomas Huth wrote: > > The retry_isa test is not doing what it was intended for: The > test_retry_flush() function ignores the machine parameter completely > and thus this test does not get run with the "isapc" machine. > Moreover, in the course of time, the test_re

[PATCH v6 32/33] crypto: delegate permission functions to JobDriver .pre_run

2022-01-21 Thread Emanuele Giuseppe Esposito
block_crypto_amend_options_generic_luks uses the block layer permission API, therefore it should be called with the BQL held. However, the same function is being called by two BlockDriver callbacks: bdrv_amend_options (under BQL) and bdrv_co_amend (I/O). The latter is I/O because it is invoked by

[PATCH v6 20/33] block: rename bdrv_invalidate_cache_all, blk_invalidate_cache and test_sync_op_invalidate_cache

2022-01-21 Thread Emanuele Giuseppe Esposito
Following the bdrv_activate renaming, change also the name of the respective callers. bdrv_invalidate_cache_all -> bdrv_activate_all blk_invalidate_cache -> blk_activate test_sync_op_invalidate_cache -> test_sync_op_activate No functional change intended. Signed-off-by: Emanuele Giuseppe Esposit

Re: [PATCH v4 3/3] meson: generate trace events for qmp commands

2022-01-21 Thread John Snow
On Fri, Jan 21, 2022 at 11:23 AM Vladimir Sementsov-Ogievskiy wrote: > > 1. Add --no-trace-events to suppress trace events generation in some >cases, and make trace events be generated by default. > 2. Add corresponding .trace-events files as outputs in qapi_files >custom target > 3. Defin

[PATCH v6 31/33] include/qemu/job.h: introduce job->pre_run() and use it in amend

2022-01-21 Thread Emanuele Giuseppe Esposito
Introduce .pre_run() job callback. This cb will run in job_start, before the coroutine is created and runs run() in the job aiocontext. Therefore, .pre_run() always runs in the main loop. We can use this function together with clean() cb to replace bdrv_child_refresh_perms in block_crypto_amend_op

[PATCH v6 30/33] include/block/block_int-common.h: introduce bdrv_amend_pre_run and bdrv_amend_clean

2022-01-21 Thread Emanuele Giuseppe Esposito
These two callbacks will be invoked by job callbacks to execute driver-specific code while still being in BQL. In this example, we want the amend JobDriver to execute the permission check (bdrv_child_refresh_perms) currently only done in block/crypto.c block_crypto_amend_options_generic_luks() to a

[PATCH v6 17/33] include/block/snapshot: global state API + assertions

2022-01-21 Thread Emanuele Giuseppe Esposito
Snapshots run also under the BQL lock, so they all are in the global state API. The aiocontext lock that they hold is currently an overkill and in future could be removed. Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- block/snapshot.c

Re: [PATCH v4 1/3] scripts/qapi/gen.py: add FOO.trace-events output module

2022-01-21 Thread John Snow
On Fri, Jan 21, 2022 at 11:22 AM Vladimir Sementsov-Ogievskiy wrote: > > We are going to generate trace events for qmp commands. We should > generate both trace_*() function calls and trace-events files listing > events for trace generator. > > So, add an output module FOO.trace-events for each FO

[PATCH v6 16/33] assertions for blockdev.h global state API

2022-01-21 Thread Emanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito Reviewed-by: Paolo Bonzini Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 3 +++ blockdev.c| 16 2 files changed, 19 insertions(+) diff --git a/block/block-backend.c b/block/block-backend.c index fe56cc1cf4..048b

[PATCH v6 28/33] job.h: split function pointers in JobDriver

2022-01-21 Thread Emanuele Giuseppe Esposito
The job API will be handled separately in another serie. Signed-off-by: Emanuele Giuseppe Esposito --- include/qemu/job.h | 22 ++ 1 file changed, 22 insertions(+) diff --git a/include/qemu/job.h b/include/qemu/job.h index 6e67b6977f..4ea7a4a0cd 100644 --- a/include/qemu/job

[PATCH v6 12/33] block.c: add assertions to static functions

2022-01-21 Thread Emanuele Giuseppe Esposito
Following the assertion derived from the API split, propagate the assertion also in the static functions. Signed-off-by: Emanuele Giuseppe Esposito --- block.c | 47 ++- block/block-backend.c | 3 +++ 2 files changed, 49 insertions(+), 1 del

  1   2   3   >